Aircraft Missile Avoidance Decision-Making Method Based on LSTM and PPO

Abstract

This invention relates to an aircraft-missile evasion decision-making method based on LSTM and PPO. The method includes the following steps: Step 1: Data acquisition; Step 2: Based on the data acquired in the simulation, normalization and feature fusion of the observation information of the aircraft and missile are performed through a Long Short-Term Memory (LSTM) network to establish the temporal features of the aircraft and missile; Step 3: A hierarchical model is established based on the deep deterministic policy algorithm in deep reinforcement learning, and specific flight control is achieved using control levers; Step 4: Client-server communication is established according to the model defined above; Step 5: Steps 1-4 are repeated until the conditions for the PPO algorithm training to end are met, at which point training is terminated. This scheme not only improves the flexibility and real-time performance of decision-making but also reduces the reliance on traditional modeling and expert knowledge, providing a more intelligent solution for modern air combat.

Description

Technical Field

[0001] This invention relates to an aircraft missile avoidance decision-making method based on LSTM and PPO, belonging to the field of aircraft missile avoidance technology. Background Technology

[0002] Escape maneuvers for aircraft to evade enemy missiles are crucial for enhancing fighter jet survivability and have always been a hot topic in air combat research. Modern air combat can be divided into beyond-visual-range (BVR) and close-range (BVR) combat, depending on radar detection range and weapon type. Given limited maneuverability, developing efficient escape maneuver strategies to improve a fighter jet's ability to evade and escape missiles has become a key factor in enhancing air combat survivability.

[0003] Currently, commonly used methods for aircraft to evade missiles include expert system methods, differential game methods, optimal control methods, and model prediction methods. These methods aim to solve for the optimal or suboptimal escape maneuver strategy under the condition of a single missile attack. For example, differential game methods use differential equations to describe the evolution of the missile and the aircraft over time, forming a dynamic system containing time variables; optimal control methods apply optimal control theory or dynamic programming methods to optimize the objective function under given constraints.

[0004] However, these methods each have some significant drawbacks. First, expert system methods heavily rely on the prior knowledge of human experts. When missile or aircraft subsystems change, experts must reanalyze the new subsystems and propose new escape maneuver strategies, a process that is not only time-consuming but may also lead to delayed strategy updates. Second, differential game theory, optimal control, and model prediction methods all depend on explicit and complete mathematical models. Establishing an air combat confrontation model is itself a complex task, typically requiring numerous differential and integral equations to describe the transition laws of the UAV and missile state attributes. When the number of missiles changes, these models often struggle to adapt, significantly increasing the complexity of solving for optimal maneuver strategies. Furthermore, the aircraft evading missile problem involves multiple complex nonlinear systems, and errors are inevitable in the modeling of each subsystem, further increasing the difficulty of solving for aircraft evading missile strategies using the aforementioned methods. In summary, although existing methods can solve the aircraft evading missile problem to some extent, their limitations make it difficult to provide a fast and effective solution in the face of complex and dynamic air combat environments. Therefore, a new approach is urgently needed to address this technical problem. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing an aircraft missile avoidance decision-making method based on LSTM and PPO. This intelligent aircraft avoidance decision-making method based on LSTM and PPO algorithms can quickly adapt to the dynamically changing air combat environment, is highly flexible, and has high decision-making efficiency.

[0006] This invention employs LSTM (Long Short-Term Memory) networks to extract dynamic features of aircraft and missiles, combined with the PPO (Proximal Policy Optimization) algorithm for intelligent decision-making. LSTM effectively captures long-term dependencies in time-series data, enabling the model to understand changes in missile threat in real time. The PPO algorithm, on the other hand, optimizes evasion strategies through adaptive learning, ensuring that decisions remain efficient in constantly changing environments. This innovative approach allows aircraft to quickly formulate efficient evasion strategies in complex combat environments, significantly improving survivability and reducing the risk of being hit. This solution not only enhances the flexibility and real-time nature of decision-making but also reduces reliance on traditional modeling and expert knowledge, providing a more intelligent solution for modern air combat.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: an aircraft missile avoidance decision-making method based on LSTM and PPO, the method comprising the following steps:

[0008] Step 1: Data collection. Obtain relevant data about the aircraft and missiles from the simulation environment, including the aircraft's position, speed, and other information, as well as the missile's position, azimuth, and target distance.

[0009] Step 2: Based on the data collected in the simulation, the observation information of the aircraft and missile is normalized and feature fused using a Long Short-Term Memory (LSTM) network to establish the temporal features of the aircraft and missile.

[0010] Step 3: Establish a hierarchical model based on the Proximal Policy Optimization (PPO) algorithm in deep reinforcement learning. The upper-layer model acts as the "commander" to train a policy and obtain the desired aircraft speed, azimuth, etc.; while the lower-layer model acts as the "executor" to receive the output of the upper-layer model and control the stick to achieve specific flight control.

[0011] Step 4: Based on the model defined above, establish client-server communication. The client is responsible for starting the training of the PPO algorithm, while the server is responsible for starting the simulation game. After the simulation game ends, the server sends data to the client, and the client starts PPO training after receiving the data.

[0012] Step 5: Repeat steps 1-4 above until the conditions for the end of PPO algorithm training are met, then terminate the training.

[0013] Step 1 involves collecting data from the simulation environment to obtain information about the aircraft and missiles in the simulation. Observational information at any time and ,in These represent the azimuth, elevation, x, y, z coordinates, and target distance of the missile, respectively. These represent the aircraft's three coordinates, three velocities, heading angle, and pitch angle, respectively.

[0014] Step 2 is as follows: Since evasion decision-making is a long-term process, the effectiveness of the decision depends on a deep understanding of the complex relationship between the aircraft and the missile. Based on the data collected in the simulation, the observation information of the aircraft and missile is fused using an LSTM network. The multi-dimensional observation data is transformed into a time series format, and the data at each time step is processed, including dynamic features such as the relative position, velocity, acceleration, and warning signals of the aircraft and missile. The input of each time step consists of the features of the aircraft and missile, forming a three-dimensional data structure: time step, number of samples, and number of features. This data is processed by LSTM to automatically extract the long-term relationship features between the aircraft and the missile, providing a basis for subsequent decision-making steps. Ultimately, based on these features, LSTM can provide effective input to the PPO model to formulate the optimal evasion strategy.

[0015] In step 2, the specific implementation steps of LSTM are as follows: The key part of LSTM is a unit controlled by multiple gating mechanisms. For each time step t, LSTM will adjust the current input data... The hidden state of the previous time step and cell state Calculate the current cell state and hidden state This allows LSTM to capture long-term dependencies in time series data. The basic structure of LSTM includes a forget gate. Input gate and output gate The calculation formula for each door is:

[0016]

[0017]

[0018]

[0019] in: and These are the network weights and biases.

[0020] Step 3 involves modeling the avoidance decision as a Markov decision process and establishing a hierarchical model based on the PPO to train the upper-level policy and lower-level control modules respectively. The Markov decision process consists of tuples... Composition. The Markov decision process of the upper-level model is as follows: where S is the state space, composed of the output of the LSTM model; It is the action space, which includes the target velocity, target heading angle, and target pitch angle; It is a reward function, consisting of distance reward, azimuth reward, avoidance reward and final reward; It is the state transition probability; This is the reward discount factor. The Markov decision process of the lower-level model is as follows: the input consists of the output of the upper-level model, namely the target speed, heading angle, and pitch angle; its action space includes the aircraft throttle, rudder, elevator, etc., to achieve precise control of the aircraft; the reward is a control accuracy reward. Through this hierarchical model structure, the upper-level model is responsible for determining the overall trajectory of the aircraft, while the lower-level model, based on the target parameters given by the upper-level model, executes precise control commands to train the aircraft's flight attitude.

[0021] Step 3: The reward function of the upper-level policy model includes: high reward Missile azimuth bonus Bullet distance bonus and avoid rewards ,in The total reward at the end of a game.

[0022]

[0023]

[0024]

[0025]

[0026] .

[0027] In step 3, the reward for the lower-level control model includes a control accuracy reward, specifically:

[0028]

[0029] in: , and These are the weighting coefficients. , and These are the target speed, heading angle, and pitch angle, respectively. , and These are the actual flight speed, heading angle, and pitch angle, respectively.

[0030] Step 4 utilizes a client-server communication mechanism to coordinate the reinforcement learning training process. The client initiates the training of the PPO algorithm, while the server starts simulated games, mimicking the interaction between aircraft and missiles. After each round of simulated games, the server sends the collected observation data and reward information to the client. Upon receiving the data, the client initiates PPO training based on the latest simulation data, updating the policy network. Data transmission and collaborative training are conducted between the client and server via the TCP communication protocol.

[0031] An aircraft missile avoidance decision system based on LSTM and PPO is used to implement the aforementioned aircraft missile avoidance decision method based on LSTM and PPO. The system includes a missile avoidance scenario design module, a feature extraction module, a TCP communication module, and a deep reinforcement learning algorithm training module.

[0032] Specifically as follows:

[0033] The missile evasion scenario design module: In a 1v1 situation, the enemy aircraft is carrying 10 missiles and attacks our aircraft. Our aircraft uses airborne radar, laser radar and other detection tools to detect the missiles launched by the enemy and take reasonable maneuvers to evade them. This includes establishing a dynamic environment model of the aircraft, missiles and their interrelationships. This model can reflect the absolute information of the aircraft and missiles, including position, speed, and the relative position, speed, acceleration and other state parameters between the aircraft and missiles.

[0034] Feature extraction module: Extracts dynamic features from the time-series data of missile evasion using an LSTM network. This module can identify the complex relationship between aircraft and missiles, capture long-term dependencies, and generate feature vectors suitable for decision-making.

[0035] TCP Communication Module: In this module, the client and server communicate via the TCP protocol to collaboratively complete the PPO training and simulation process. The module's task is to ensure smooth data exchange between the simulation environment and the training algorithm, guaranteeing real-time transmission and updates of training data.

[0036] The deep reinforcement learning algorithm training module combines extracted features and applies deep reinforcement learning (PPO) algorithms for intelligent decision-making. This module is divided into two parts: the upper-level policy is responsible for generating the desired aircraft speed, azimuth angle, and other actions; the lower-level policy calls the underlying stick model based on the output of the upper-level policy to generate commands such as throttle and rudder for the aircraft, which is used to generate the optimal avoidance strategy and guide the aircraft to take appropriate maneuvers when facing missile threats, so as to improve the avoidance success rate.

[0037] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the aforementioned aircraft missile avoidance decision-making method based on LSTM and PPO.

[0038] A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the aforementioned LSTM and PPO-based aircraft missile avoidance decision-making method.

[0039] Compared to existing technologies, this invention has the following advantages: 1) In the missile evasion model, this scheme uses LSTM to extract features and designs a hierarchical strategy, which has several significant advantages. First, the use of LSTM enables the model to effectively capture dynamic relationships in time series, thereby more accurately understanding the interaction between the aircraft and the missile. This capability is particularly important because missile threats often change over time, and LSTM can adjust the strategy in real time based on historical data, improving the accuracy of decision-making; 2) Second, the hierarchical strategy design divides the task into upper and lower layers. The upper-layer strategy is responsible for outputting the aircraft's speed, heading angle, and yaw angle. This layer is optimized towards macro-level strategy and can effectively formulate a global escape path. The lower-layer strategy, based on the output of the upper-layer strategy, trains the corresponding control model (such as throttle and rudder) to achieve fine-grained motion control. This hierarchical structure not only improves the interpretability of the system but also makes the strategy more flexible and adaptable, enabling rapid response to different combat scenarios and missile threats. Ultimately, this method achieves significant results. Through a hierarchical strategy design, the aircraft can quickly make effective evasive maneuvers in complex combat environments, successfully increasing the probability of successfully evading missiles. At the same time, the overall control response is smoother, increasing the missile's miss distance, improving the aircraft's ability to evade missiles, and correspondingly enhancing the aircraft's survivability. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0041] To enhance understanding of the present invention, the embodiments will be described in detail below with reference to the accompanying drawings.

[0042] Example 1: See Figure 1 An aircraft missile avoidance decision-making method based on LSTM and PPO, the method includes the following steps:

[0043] Step 1: Data collection. Obtain relevant data about the aircraft and missiles from the simulation environment, including the aircraft's position, speed, and other information, as well as the missile's position, azimuth, and target distance.

[0044] Step 2: Based on the data collected in the simulation, the observation information of the aircraft and missile is normalized and feature fused using a Long Short-Term Memory (LSTM) network to establish the temporal features of the aircraft and missile.

[0045] Step 3: Establish a hierarchical model based on the Proximal Policy Optimization (PPO) algorithm in deep reinforcement learning. The upper-layer model acts as the "commander" to train a policy and obtain the desired aircraft speed, azimuth, etc.; while the lower-layer model acts as the "executor" to receive the output of the upper-layer model and control the stick to achieve specific flight control.

[0046] Step 4: Based on the model defined above, establish client-server communication. The client is responsible for starting the training of the PPO algorithm, while the server is responsible for starting the simulation game. After the simulation game ends, the server sends data to the client, and the client starts PPO training after receiving the data.

[0047] Step 5: Repeat steps 1-4 above until the conditions for the end of PPO algorithm training are met, then terminate the training.

[0048] Step 1 involves collecting data from the simulation environment to obtain information about the aircraft and missiles in the simulation. Observational information at any time and ,in These represent the azimuth, elevation, three-axis coordinates, and target distance of the missile, respectively. These represent the aircraft's three coordinates, three velocities, yaw angle, and pitch angle, respectively.

[0049] Step 2 is implemented as follows: First, the collected aircraft and missile status data (position, velocity, acceleration, warning signals, etc.) are converted into a time series format. The input data for each time step includes the state and relative relationship between the aircraft and missile, forming a three-dimensional data structure (number of time steps, number of samples, number of features). Second, the input layer receives the feature data for each time step. The LSTM unit processes the data for each time step through a gating mechanism (forget gate, input gate, output gate) to capture the long-term dependency relationship between the aircraft and missile. The network outputs feature vectors through hidden layers, reflecting the dynamic relationship between the aircraft and missile. Historical data is used to train the LSTM, and parameters are adjusted through backpropagation to optimize the feature extraction effect, enabling it to accurately capture the interaction between the aircraft and missile. The trained LSTM will output feature vectors, which will be used as input to the subsequent PPO algorithm to formulate the optimal evasion strategy.

[0050] Step 3 involves establishing a deep reinforcement learning network model, modeling the avoidance decision as a Markov decision process, and building a hierarchical model based on PPO to train the upper-layer policy and lower-layer control modules respectively. The Markov decision process consists of tuples... Composition. The Markov decision process of the upper-level model is as follows: where S is the state space, composed of the output of the LSTM model; It is the action space, which includes the target velocity, target heading angle, and target pitch angle; It is a reward function, consisting of distance reward, azimuth reward, avoidance reward and final reward; It is the state transition probability; This is the reward discount factor. The Markov decision process of the lower-level model is as follows: the input consists of the output of the upper-level model, namely the target speed, heading angle, and pitch angle; its action space includes the aircraft throttle, rudder, elevator, etc., to achieve precise control of the aircraft; the reward is a control accuracy reward. Through this hierarchical model structure, the upper-level model is responsible for determining the overall trajectory of the aircraft, while the lower-level model, based on the target parameters given by the upper-level model, executes precise control commands to train the aircraft's flight attitude.

[0051] Step 3: The reward function of the upper-level policy model includes: high reward Missile azimuth bonus Bullet distance bonus and avoid rewards ,in The total reward at the end of a game.

[0052]

[0053]

[0054]

[0055]

[0056] .

[0057] In step 3, the reward for the lower-level control model includes a control accuracy reward, specifically:

[0058]

[0059] in: , and These are the weighting coefficients. , and These are the target speed, heading angle, and pitch angle, respectively. , and These are the actual flight speed, heading angle, and pitch angle, respectively.

[0060] Step 4, TCP communication, is implemented as follows: First, the server creates a TCP socket and binds it to the specified IP address and port, then listens for connection requests from clients. When a client connects, the server accepts the connection and prepares to receive data. Second, the client creates a TCP socket and connects to the server's IP address and port. The client sends request data (such as a training start command) to the server and waits to receive simulation results or status feedback. Next, the server and client exchange data through the TCP socket. The server sends real-time simulation data, and the client starts training or other operations based on this data. Data transmission uses blocking communication to ensure information synchronization. Finally, both parties need to handle network anomalies, such as connection interruptions, ensuring that the connection is properly closed and resources are released when communication ends.

[0061] Example 2: An aircraft missile evasion decision system based on LSTM and PPO, used to implement the aircraft missile evasion decision method based on LSTM and PPO. The system includes a missile evasion scenario design module, a feature extraction module, a TCP communication module, and a deep reinforcement learning algorithm training module.

[0062] Specifically as follows:

[0063] The missile evasion scenario design module: In a 1v1 situation, the enemy aircraft is carrying 10 missiles and attacks our aircraft. Our aircraft uses airborne radar, laser radar and other detection tools to detect the missiles launched by the enemy and take reasonable maneuvers to evade them. This includes establishing a dynamic environment model of the aircraft, missiles and their interrelationships. This model can reflect the absolute information of the aircraft and missiles, including position, speed, and the relative position, speed, acceleration and other state parameters between the aircraft and missiles.

[0064] Feature extraction module: This module extracts dynamic features from the time-series data of missile evasion using a designed LSTM network. It can identify the complex relationship between aircraft and missiles, capture long-term dependencies, and generate feature vectors suitable for decision-making.

[0065] TCP Communication Module: In this module, the client and server communicate via the TCP protocol to collaboratively complete the PPO training and simulation process. The module's task is to ensure smooth data exchange between the simulation environment and the training algorithm, guaranteeing real-time transmission and updates of training data.

[0066] The deep reinforcement learning algorithm training module combines extracted features and applies deep reinforcement learning (PPO) algorithms for intelligent decision-making. This module is divided into two parts: the upper-level policy is responsible for generating the desired aircraft speed, azimuth angle, and other actions; the lower-level policy calls the underlying stick model based on the output of the upper-level policy to generate commands such as throttle and rudder for the aircraft, which is used to generate the optimal avoidance strategy and guide the aircraft to take appropriate maneuvers when facing missile threats, so as to improve the avoidance success rate.

[0067] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. An aircraft missile evasion decision-making method based on LSTM and PPO, characterized in that, The method includes the following steps: Step 1: Data Acquisition. Obtain relevant data on the aircraft and missiles from the simulation environment, including the aircraft's position and speed, and the missile's position, azimuth, and target distance. Step 2: Based on the data collected in the simulation, the observation information of the aircraft and missile is normalized and feature fused using a Long Short-Term Memory (LSTM) network to establish the temporal characteristics of the aircraft and missile. Step 3: Establish a hierarchical model based on the Proximal Policy Optimization (PPO) algorithm in deep reinforcement learning. The upper-layer model is used to train a policy and obtain the desired aircraft speed and azimuth angle; the lower-layer model receives the output of the upper-layer model and uses the control stick to achieve specific flight control. Step 4: Based on the model defined above, establish client-server communication. The client is responsible for starting the PPO algorithm training, while the server is responsible for starting the simulation game. After the simulation game ends, the server sends data to the client. Upon receiving the data, the client starts PPO training. Step 5: Repeat steps 1-4 above until the conditions for the end of PPO algorithm training are met, then terminate the training. Step 1 involves collecting data, obtaining information about the aircraft and missiles in the simulation environment. Observational information at any time and ,in These represent the azimuth, elevation, x, y, z coordinates, and target distance of the missile, respectively. These represent the aircraft's three coordinates, three velocities, heading angle, and pitch angle, respectively. Step 2 is as follows: Based on the data collected in the simulation, the observation information of the aircraft and missile is fused using an LSTM network to convert the multidimensional observation data into a time series format. The data of each time step is processed, including the relative position, velocity, acceleration, and dynamic features of the alarm signal of the aircraft and missile. The input of each time step consists of the features of the aircraft and missile, forming a three-dimensional data structure, namely time step, number of samples, and number of features. The data is processed by LSTM to automatically extract the long-term relationship features between the aircraft and missile, providing a basis for subsequent decision-making steps. Finally, based on these features, LSTM can provide effective input to the PPO model to formulate the optimal evasion strategy. The avoidance decision is modeled as a Markov decision process, and a hierarchical model is built based on PPO to train the upper-level policy and the lower-level control module respectively. The Markov decision process consists of tuples. The Markov decision process of the upper-level model is as follows: where S is the state space, which is composed of the output of the LSTM model; It is the action space, which includes the target velocity, target heading angle, and target pitch angle; It is a reward function, consisting of distance reward, azimuth reward, avoidance reward and final reward; It is the state transition probability; The reward discount factor is used in the Markov decision process of the lower-level model. The input consists of the output of the upper-level model, namely the target speed, heading angle, and pitch angle. Its action space includes the aircraft throttle, rudder, and elevator to achieve precise control of the aircraft. The reward is the control accuracy reward. Through this hierarchical model structure, the upper-level model is responsible for determining the overall trajectory of the aircraft, while the lower-level model executes precise control commands to train the aircraft's flight attitude based on the target parameters given by the upper-level model. The reward function of the upper-level policy model includes: high reward Missile azimuth bonus Bullet distance bonus and avoid rewards ,in The total reward at the end of a game. 。 2. The aircraft missile evasion decision-making method based on LSTM and PPO according to claim 1, characterized in that, The reward for the lower-level control model includes a control accuracy reward, specifically: in: , and These are the weighting coefficients. , and These are the target speed, heading angle, and pitch angle, respectively. , and These are the actual flight speed, heading angle, and pitch angle, respectively.

3. The aircraft missile evasion decision-making method based on LSTM and PPO according to claim 2, characterized in that, The reinforcement learning training process is coordinated through a communication mechanism between the client and the server. The client is responsible for initiating the training of the PPO algorithm, while the server is responsible for initiating simulated games to simulate the interaction between aircraft and missiles. After each round of simulated games, the server sends the collected observation data and reward information to the client. After receiving the data, the client starts PPO training based on the latest simulation data and updates the policy network. The client and the server communicate with each other via TCP protocol for data transmission and collaborative training.

4. An aircraft missile avoidance decision system based on LSTM and PPO, characterized in that, The system is used to implement the aircraft missile avoidance decision-making method based on LSTM and PPO as described in any one of claims 1-3. The system includes a missile avoidance scenario design module, a feature extraction module, a TCP communication module, and a deep reinforcement learning algorithm training module. Specifically as follows: The missile evasion scenario design module simulates a 1v1 situation where an enemy aircraft carries 10 missiles and attacks our aircraft. Our aircraft uses airborne radar and optical missile detection tools to detect the enemy's missiles and take appropriate maneuvers to evade them. This includes establishing a dynamic environment model of the aircraft, missiles, and their interrelationships. This model can reflect the absolute information of the aircraft and missiles, including their positions, speeds, and relative position, speed, and acceleration parameters between the aircraft and missiles. The feature extraction module extracts dynamic features from the time-series data of missile evasion using an LSTM network. This module can identify the complex relationship between aircraft and missiles, capture long-term dependencies, and generate feature vectors suitable for decision-making. TCP Communication Module: In this model, the client and server communicate via the TCP protocol to collaboratively complete the PPO training and simulation process. The task of this module is to ensure smooth data exchange between the simulation environment and the training algorithm, and to guarantee the real-time transmission and updating of training data. The deep reinforcement learning algorithm training module combines extracted features and applies deep reinforcement learning (PPO) algorithms for intelligent decision-making. This module is divided into two parts: the upper-level policy is responsible for generating the desired aircraft speed and azimuth angle maneuvers; the lower-level policy calls the underlying stick model based on the output of the upper-level policy to generate the aircraft's throttle and rudder commands, which are used to generate the optimal avoidance strategy and guide the aircraft to take appropriate maneuvers when facing missile threats, so as to improve the avoidance success rate.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the aircraft missile avoidance decision-making method based on LSTM and PPO as described in any one of claims 1 to 3 above.

6. A computer-readable storage medium storing computer instructions thereon, characterized in that: When executed by the processor, the computer instructions implement the aircraft missile evasion decision-making method based on LSTM and PPO as described in any one of claims 1-3.

Images