Pilot landing operation analysis method and device, electronic equipment and storage medium

Abstract

The invention provides a pilot landing operation analysis method and device, electronic equipment and a storage medium. The pilot landing operation analysis method comprises the steps that external environment parameters and aircraft running state parameters formed in the process that a pilot operates an aircraft to land are obtained; calling a neural network model and inputting the external environment parameters and the aircraft operation state parameters into the neural network model to obtain a state transition matrix at each time point; based on the state transition matrix, hidden states corresponding to the state transition matrix are obtained, so that pilot landing operation analysis is completed, the hidden states are used for representing all states of an attention distribution mode of a pilot, and the hidden states comprise the state without attention loss, the state of attention loss of a yaw angle and the state of attention loss of a height; and any one or more of states in which attention to airspeed is missing. The landing operation of the pilot is accurately and objectively analyzed, so that the risk is conveniently managed and controlled in a targeted manner, and the flight safety level of the pilot is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a pilot landing operation analysis method, device, electronic equipment and storage medium. Background Art

[0002] Landing an aircraft is a complex and difficult task, and more than a quarter of flight safety accidents occur during the landing process, which accounts for only 1% of the flight time. During this stage, facing the changes in the external environment and the aircraft's status, the pilot needs to allocate attention appropriately and perform appropriate operations on the throttle lever position, pitch angle lever position, and roll angle lever position to complete the perception and control of the risk environment and the complex system of the aircraft.

[0003] However, due to limited cognitive resources, pilots sometimes find it difficult to maintain attention on important aircraft operating conditions, such as yaw angle, altitude, and airspeed, and are therefore unable to perceive risks and adjust operations in a timely manner, leading to potential accidents.

[0004] Therefore, finding a pilot landing operation analysis method that can accurately and objectively analyze the pilot's landing operation has become a current research hotspot, so as to facilitate targeted risk control and improve the pilot's flight safety level. Summary of the invention

[0005] The present invention provides a pilot landing operation analysis method, device, electronic equipment and storage medium, which can accurately and objectively analyze the pilot's landing operation, thereby facilitating targeted risk control and improving the pilot's flight safety level.

[0006] The present invention provides a method for analyzing a pilot's landing operation, the method comprising: obtaining external environment parameters and aircraft operating state parameters formed during the pilot's operation of landing an aircraft; calling a pre-trained neural network model, and inputting the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain a corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; based on the state transfer matrix, obtaining a hidden state corresponding to the state transfer matrix to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation mode, and the hidden state includes any one or more of a state without attention loss, a state without attention to a yaw angle, a state without attention to altitude, and a state without attention to airspeed.

[0007] According to a pilot landing operation analysis method provided by the present invention, after obtaining the hidden state corresponding to the state transfer matrix, the method further includes: based on the hidden state, obtaining the observed value of the target operating state parameter formed during the pilot's operation of the aircraft landing corresponding to the hidden state, wherein the target operating state parameter includes any one or more of the yaw angle, altitude and airspeed; based on the observed value of the target operating state parameter, obtaining the predicted action of the pilot during the operation of the aircraft landing; based on the predicted action, determining the flight risk of the pilot during the operation of the aircraft landing.

[0008] According to a pilot landing operation analysis method provided by the present invention, before obtaining the predicted action of the pilot in the process of operating the aircraft to land based on the observed value of the target operating state parameter, the method also includes: determining the optimal control law of the pilot in the process of operating the aircraft to land; obtaining the predicted action of the pilot in the process of operating the aircraft to land based on the observed value of the target operating state parameter specifically includes: obtaining the predicted action of the pilot in the process of operating the aircraft to land based on the observed value of the target operating state parameter and the optimal control law.

[0009] According to a pilot landing operation analysis method provided by the present invention, the optimal control law of the pilot in the process of operating the aircraft to land is determined in the following manner: calling a pre-constructed partially observable Markov decision model corresponding to the pilot's operation of the aircraft to land, wherein the partially observable Markov decision model is modeled based on the partially observable Markov decision process for the pilot's operation of the aircraft to land; obtaining a cost function, wherein the cost function is a linear exponential quadratic function that maintains the aircraft to land at a stable descent rate and to land at a target speed; with the goal of the running trajectory formed by the pilot's operation of the aircraft to land being close to the ideal trajectory and the pilot performing smooth operations during the aircraft landing process, based on the cost function, determining the optimal control law of the pilot in the process of operating the aircraft to land.

[0010] According to a pilot landing operation analysis method provided by the present invention, the partially observable Markov decision model is constructed in the following manner: based on the partially observable Markov decision process, the pilot operating the aircraft landing process is modeled to obtain the partially observable Markov decision model, and the state variables, decision actions, state transition probabilities, observations and cost functions of the partially observable Markov decision model are defined respectively, wherein the state variables are determined based on the aircraft operating state parameters; the decision actions are determined based on the pitch angle pole position and the roll angle pole position; the state transition probabilities are determined based on the system state linear model corresponding to the aircraft operating state parameters; the observations are determined based on the observed values ​​of the target operating state parameters, and a cost function for maintaining the aircraft landing at a stable descent rate and landing at a target speed is constructed.

[0011] According to a pilot landing operation analysis method provided by the present invention, a system state linear model corresponding to the aircraft operating state parameters is constructed in the following manner: based on historical flight rapid recording data, linear modeling processing is performed on each of the aircraft operating state parameters to obtain a system state linear model corresponding to the aircraft operating state parameters, wherein the system state linear model is used to characterize a model formed by the aircraft operating state parameters being affected by any one or more of other aircraft operating state parameters, decision actions, and random items.

[0012] According to a pilot landing operation analysis method provided by the present invention, the neural network model is pre-trained in the following manner: obtaining a training data set, wherein the training data set includes multiple training data, and the training data includes external environment training parameters and aircraft operating status training parameters; based on the external environment training parameters and the aircraft operating status training parameters, obtaining predicted action training data of the pilot in the process of operating the aircraft landing corresponding to the external environment training parameters and the aircraft operating status training parameters; obtaining real action training data of the pilot in the process of operating the aircraft landing corresponding to the external environment training parameters and the aircraft operating status training parameters; determining a loss function based on the mean square error between the predicted action training data and the real action training data; and pre-training the neural network model based on the loss function to obtain the trained neural network model.

[0013] The present invention also provides a pilot landing operation analysis device, the device comprising: an acquisition module, used to acquire external environment parameters and aircraft operating state parameters formed during the pilot's operation of the aircraft landing; a calling module, used to call a pre-trained neural network model, and input the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain the corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; a processing module, used to obtain a hidden state corresponding to the state transfer matrix based on the state transfer matrix, so as to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation mode, and the hidden state includes any one or more of a state without attention loss, a state without attention to the yaw angle, a state without attention to the altitude, and a state without attention to the airspeed.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the pilot landing operation analysis method described in any one of the above methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the pilot landing operation analysis method as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the pilot landing operation analysis method as described in any one of the above is implemented.

[0017] The pilot landing operation analysis method, device, electronic device and storage medium provided by the present invention obtain external environment parameters and aircraft operation state parameters formed during the pilot's operation of the aircraft landing; call a pre-trained neural network model, and input the external environment parameters and the aircraft operation state parameters into the neural network model to obtain a state transfer matrix at each time point; based on the state transfer matrix, obtain a hidden state corresponding to the state transfer matrix, wherein the hidden state is used to characterize each state of the pilot's attention allocation pattern, and the hidden state includes any one or more of a state without attention loss, a state without attention to the yaw angle, a state without attention to the altitude, and a state without attention to the airspeed, thereby completing the analysis of the pilot's attention allocation pattern during the landing operation, and realizing the ability to accurately and objectively analyze the pilot's landing operation, thereby facilitating targeted risk control and improving the pilot's flight safety level. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is one of the flow charts of the pilot landing operation analysis method provided by the present invention.

[0020] Figure 2 This is the second flow chart of the pilot landing operation analysis method provided by the present invention.

[0021] Figure 3 It is a flow chart of determining the optimal control law for a pilot in the process of operating an aircraft to land, provided by the present invention.

[0022] Figure 4 It is a schematic diagram of the process of training the neural network model provided by the present invention.

[0023] Figure 5 It is a structural schematic diagram of the pilot landing operation analysis device provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The pilot landing operation analysis method provided by the present invention proposes to indirectly measure the allocation mode of attention, an important cognitive resource in aircraft operation, through the pilot's operating behavior. Based on decision theory, the hidden state is used to describe the psychological mechanism of the pilot's attention allocation mode switching. The present invention provides a new perspective for indirectly inferring the attention allocation mode, which can be calibrated with the direct measurement results of cognitive load.

[0027] The pilot landing operation analysis method provided by the present invention applies the achievements in sequential decision making, machine learning, and deep learning, adopts the Hidden Markov Model (HMM) to model the transfer mode of the hidden state during the pilot's landing operation, and uses the deep neural network to learn the state transfer matrix of the hidden state from the external environment and aircraft state data. The pilot's hidden state at each time point corresponds to his current attention allocation mode, which in turn determines his actual observation of the aircraft's operating status.

[0028] Figure 1 It is one of the flow charts of the pilot landing operation analysis method provided by the present invention.

[0029] The following will be combined Figure 1 The process of the pilot landing operation analysis method provided by the present invention is described.

[0030] In an exemplary embodiment of the present invention, Figure 1 It can be seen that the pilot landing operation analysis method may include steps 110 to 130, and each step will be introduced below.

[0031] In step 110, the external environment parameters and aircraft operating state parameters formed during the pilot's operation of the aircraft to land are obtained.

[0032] In one embodiment, external environmental parameters formed during the pilot's operation of the aircraft to land may be obtained, such as wind speed, wind direction, temperature, air pressure, and other external environmental parameters, and aircraft operating state parameters formed during the pilot's operation of the aircraft to land may be obtained, such as airspeed, yaw angle, altitude, pitch angle, roll angle, ground speed, descent rate, horizontal wind speed, vertical wind speed, etc. In this embodiment, the external environmental parameters and aircraft operating state parameters are not specifically limited.

[0033] In step 120, a pre-trained neural network model is called, and external environmental parameters and aircraft operating status parameters are input into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain the corresponding state transfer matrix at each time point based on the external environmental parameters and the aircraft operating status parameters.

[0034] In another embodiment, a pre-trained neural network model can be called, and external environment parameters and aircraft operating state parameters can be input into the neural network model, so as to obtain the state transfer matrix at each time point output by the neural network model. The neural network model is pre-trained, and the neural network model can obtain the corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters.

[0035] In step 130, based on the state transfer matrix, the hidden state corresponding to the state transfer matrix is ​​obtained to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize the various states of the pilot's attention allocation pattern, and the hidden state includes any one or more of the state of no attention loss, the state of lack of attention to the yaw angle, the state of lack of attention to the altitude, and the state of lack of attention to the airspeed.

[0036] In another embodiment, a hidden state can be generated based on a state transfer matrix. The hidden state can be used to characterize the various states of the pilot's attention allocation mode, that is, the hidden state describes the psychological mechanism of the pilot's attention allocation mode switching. In one example, the hidden state can include any one or more of a state in which there is no attention loss, a state in which attention to the yaw angle is lost, a state in which attention to the altitude is lost, and a state in which attention to the airspeed is lost. In this embodiment, a neural network model can be used to learn a hidden state sequence, so that changes in attention allocation during flight can be determined.

[0037] In another example, the pilot has four hidden states, which correspond to four attention allocation modes: there is no state of lack of attention, that is, the actual observation values ​​of each operating state are all true states; the state of lack of attention to the yaw angle, at this time, the actual observation values ​​of the altitude and airspeed are true states, and the observation value of the yaw angle is the sum of the true state and normal noise; the state of lack of attention to the altitude, that is, the actual observation values ​​of the yaw angle and airspeed are true states, and the observation value of the altitude is the sum of the true state and normal noise; the state of lack of attention to the airspeed, that is, the actual observation values ​​of the yaw angle and altitude are true states, and the observation value of the airspeed is the sum of the true state and normal noise. Among them, the pilot's attention allocation mode during the landing phase reflects his risk management ability.

[0038] The pilot landing operation analysis method provided by the present invention obtains external environment parameters and aircraft operation state parameters formed during the pilot's operation of the aircraft landing; calls a pre-trained neural network model, and inputs the external environment parameters and the aircraft operation state parameters into the neural network model to obtain a state transfer matrix at each time point; based on the state transfer matrix, obtains a hidden state corresponding to the state transfer matrix, wherein the hidden state is used to characterize each state of the pilot's attention allocation pattern, and the hidden state includes any one or more of a state without attention loss, a state without attention to the yaw angle, a state without attention to the altitude, and a state without attention to the airspeed, thereby completing the analysis of the pilot's attention allocation pattern during the landing operation, and realizing the ability to accurately and objectively analyze the pilot's landing operation, thereby facilitating targeted risk control and improving the pilot's flight safety level.

[0039] Figure 2 This is the second flow chart of the pilot landing operation analysis method provided by the present invention.

[0040] The following will be combined Figure 2 The process of another method of analyzing pilot landing operations is described.

[0041] In another exemplary embodiment of the present invention, Figure 2 It can be seen that after obtaining the hidden state corresponding to the state transfer matrix, the pilot landing operation analysis method can also include steps 210 to 230, and each step will be introduced below.

[0042] In step 210, based on the hidden state, the observed values ​​of the target operating state parameters formed during the pilot's operation of the aircraft landing corresponding to the hidden state are obtained, wherein the target operating state parameters include any one or more of the yaw angle, altitude and airspeed.

[0043] In one embodiment, based on the hidden state, the observed value of the target operating state parameter formed during the landing of the aircraft by the pilot corresponding to the hidden state can be obtained. It should be noted that the target operating state parameter can be considered as the aircraft operating state that needs attention obtained based on the aerodynamic theory analysis, such as yaw angle, altitude, and airspeed. In another example, the target operating state parameter may not be specified, and it can be adjusted accordingly according to the actual situation.

[0044] In another embodiment, the hidden state may be a state in which there is no attention deficit, that is, the actual observation values ​​of each operating state are all true states. In this state, the actual observation values ​​of the yaw angle, altitude, and airspeed are all true states.

[0045] In another embodiment, the hidden state may be a state in which attention to the yaw angle is lost. In this scenario, the actual observed values ​​of the altitude and airspeed are the true state, and the observed value of the yaw angle is the sum of the true state and normal noise, where the mean of the normal noise is 0 and the variance is provided by experts in the field of flight.

[0046] In another embodiment, the hidden state can be a state of lack of attention to altitude. In this scenario, the actual observed values ​​of yaw angle and airspeed are the true state, and the observed value of altitude is the sum of the true state and normal noise, where the mean of normal noise is 0 and the variance is provided by experts in the field of flight.

[0047] In another embodiment, the hidden state may be a state where attention to airspeed is missing, that is, the actual observed values ​​of yaw angle and altitude are true states, and the observed value of airspeed is the sum of the true state and normal noise, where the mean of the normal noise is 0 and the variance is provided by experts in the flight field.

[0048] In step 220, based on the observed values ​​of the target operating state parameters, the predicted actions of the pilot in operating the aircraft to land are obtained.

[0049] In step 230 , based on the predicted action, the flight risk of the pilot in operating the aircraft to land is determined.

[0050] In another embodiment, the pilot's predicted action during the landing of the aircraft can be obtained based on the observed value of the target operating state parameter obtained above, and further, the pilot's flight risk during the landing of the aircraft can be determined based on the predicted action. In this embodiment, the flight risk of the pilot during the landing of the aircraft can be evaluated, thereby laying a foundation for targeted risk control and improving the pilot's flight safety level.

[0051] In another embodiment, since the pilot's last three hidden states (lack of attention to yaw angle, lack of attention to altitude, and lack of attention to airspeed) reflect improper attention allocation in landing operations, the pilot cannot perceive risks and adjust operations in time, leading to potential accidents. Therefore, the appearance of the last three hidden states indicates that the pilot's risk management ability is insufficient, and the first hidden state (no lack of attention) indicates a high level of risk management. The pilot's risk management ability is defined as the sum of the probability of the first hidden state (ideal attention allocation mode) at all time nodes, and the ability of different pilots can be compared.

[0052] In another exemplary embodiment of the present invention, continuing with the above Figure 2 The above embodiment is used as an example for explanation. Before obtaining the predicted action of the pilot in operating the aircraft to land based on the observed value of the target operating state parameter (corresponding to step 220), the pilot landing operation analysis method may further include the following steps: Determine the optimal control law for the pilot to operate the aircraft during landing; Among them, based on the observed values ​​of the target operating state parameters, the predicted actions of the pilot in the process of operating the aircraft to land can be obtained in the following ways: Based on the observed values ​​of the target operating state parameters and the optimal control law, the pilot's predicted actions during the landing of the aircraft are obtained.

[0053] In one embodiment, the optimal control law for the pilot in the process of operating the aircraft to land can be determined. Further, based on the observed values ​​of the target operating state parameters and the optimal control law, the predicted actions of the pilot in the process of operating the aircraft to land can be obtained. It should be noted that the predicted actions are generated based on the pilot's operation on the deviated observed values, so that the flight risk of the pilot in the process of operating the aircraft to land can be effectively determined based on the predicted actions.

[0054] Figure 3 It is a flow chart of determining the optimal control law for a pilot in the process of operating an aircraft to land, provided by the present invention.

[0055] The following will be combined Figure 3 The process of determining the optimal control law for a pilot to operate an aircraft during landing is described.

[0056] In an exemplary embodiment of the present invention, Figure 3 It can be seen that determining the optimal control law for the pilot in operating the aircraft to land may include steps 310 to 330, and each step will be introduced below.

[0057] In step 310, a pre-built partially observable Markov decision model corresponding to the pilot's operation of the aircraft landing process is called, wherein the partially observable Markov decision model is obtained by modeling the pilot's operation of the aircraft landing process based on the partially observable Markov decision process.

[0058] In step 320 , a cost function is obtained, wherein the cost function is a linear exponential quadratic function for maintaining the aircraft to land at a stable descent rate and to land at a target speed.

[0059] In step 330, with the goal of making the operating trajectory formed by the pilot operating the aircraft during the landing process close to the ideal trajectory and the pilot operating the aircraft smoothly during the landing process, the optimal control law for the pilot in operating the aircraft during the landing process is determined based on the cost function.

[0060] In one embodiment, a pre-built partially observable Markov decision model corresponding to the pilot's operation of the aircraft landing process can be called, and further, a cost function is obtained, and the optimal control law of the pilot in the process of operating the aircraft landing is determined based on the cost function, with the goal of making the operating trajectory formed by the pilot's operation of the aircraft landing process close to the ideal trajectory and the pilot performing smooth operation during the aircraft landing process.

[0061] In another embodiment, when the dynamic system is a discrete time linear system and the control target is a quadratic cost function, the optimal action (control input) has a closed-form solution and can be written as a linear function of the state. The solution process is as follows: Substitute the system dynamic equation (which can correspond to the system state linear model) into the Bellman equation, and the optimal solution of the control input at each time node can be obtained through the first-order condition; Substitute the optimal control law into the cost function, and the iterative relationship of the cost function can be written to obtain the discrete time Riccati equation; Solve the Riccati equation (using numerical methods) to obtain the closed-form solution of the optimal action, that is, the optimal control law of the pilot in the process of operating the aircraft landing.

[0062] It should be noted that in the present invention, the optimal control based on POMDP includes two parts: state estimation and implementation operation, so the above closed-form solution cannot be directly applied. However, the present invention focuses on the period of time when the pilot loses attention during the landing process. At this time, the pilot not only cannot observe the true information about the operating state, but also mistakenly regards the biased observation information as the true information, and does not perform state estimation according to the optimal control mode, but directly applies the optimal control law to the observed value.

[0063] In another exemplary embodiment of the present invention, the above-mentioned embodiment is continued to be used as an example for explanation, wherein the partially observable Markov decision model can be constructed in the following manner: Based on the partially observable Markov decision process, the pilot's operation of the aircraft landing process is modeled to obtain a partially observable Markov decision model, and the state variables, decision actions, state transition probabilities, observations, and cost functions of the partially observable Markov decision model are defined respectively, where: Determine state variables based on aircraft operating state parameters; Determine the decision action based on the pitch angle pole position and the roll angle pole position; Determining state transition probabilities based on a system state linear model corresponding to aircraft operating state parameters; Based on the observed value of the target operating state parameter, determining the observation, and Construct a cost function to maintain the aircraft's descent at a steady rate and landing at the target speed.

[0064] In another embodiment, in the pilot landing phase operation concerned by the present invention, the observation of the aircraft operating state is affected by the attention allocation pattern and has the characteristic of partial observability; in addition, the operation has the Markov property (given the current state, the next state is independent of the previous state), can be discretized with a time interval of 1 second, and the environment is random, so it can be modeled as POMDP. In the application process, the pilot behavior can be modeled as a finite-term partially observable Markov decision process (also known as POMDP), and the action, state transition probability, state, observation, and cost function are defined. That is, based on the partially observable Markov decision process, the pilot's operation of the aircraft landing process is modeled to obtain a partially observable Markov decision model, and the state variables, decision actions, state transition probability, observation, and cost function of the partially observable Markov decision model are defined respectively.

[0065] In another embodiment, the present invention focuses on the risks of the aircraft landing phase, during which the pilot needs to start landing from a fixed point while meeting the following requirements: keep the yaw angle unchanged and point in the direction of the runway; make the altitude drop at a constant speed; keep the airspeed dropping at a constant speed. In order to characterize the dynamic system including these three key factors, a total of nine system states are taken into consideration. Therefore, the above-mentioned aircraft operating state parameters can be selected as state variables. That is, based on the aircraft operating state parameters, the state variables are determined.

[0066] In another embodiment, the pilot completes the landing operation by changing the throttle lever position, the pitch angle lever position, and the roll angle lever position. Since the throttle lever position is sometimes automatically calibrated and adjusted, and does not fully reflect the pilot's operation, the pitch angle lever position and the roll angle lever position are selected as the decision variables (actions) in the landing phase. That is, the decision action is determined based on the pitch angle lever position and the roll angle lever position.

[0067] In another embodiment, the pilot's operation during the landing phase meets the POMDP assumption and can be modeled using POMDP. At this time, the discrete time interval in the POMDP is 1 second (the lowest frequency of QAR data), the actions are the pitch angle pole position and the roll angle pole position; the states are speed, yaw angle, altitude, pitch angle, roll angle, ground speed, descent rate, horizontal wind speed, and vertical wind speed; the state transition probability is described by a system dynamics model (corresponding to a system state linear model) that includes actions and states; the observation is the actual observation value based on the attention allocation mode (corresponding to the observation value of the target operating state parameter), and the reward (cost function) is a linear exponential quadratic function (corresponding to the cost function) that simultaneously considers maintaining a stable descent rate and landing at the target speed.

[0068] In another embodiment, during the landing process, the pilot's control objectives include approaching the ideal trajectory and maintaining smooth operation. It is not desirable to deviate the yaw angle, altitude, and airspeed from the requirements, nor is it desirable to operate too much or have a bumpy flight trajectory. Therefore, it is suitable to adopt the LQR cost function modeling. The Q, R, and F matrix coefficients in the cost function should truly reflect the pilot's control objectives, that is, the actions predicted by the cost function should be close to the actual actions. Therefore, in the process of determining the cost function, a data-driven approach can be adopted to find the cost function coefficients that best meet the actual situation in the process of minimizing the mean square error.

[0069] In another embodiment, the parameter estimation can use a grid search algorithm, calculate the optimal control law according to each parameter combination, perform neural network training under each parameter combination (the prediction action is to use the optimal observation law for the actual observation value), and obtain the training error under this parameter combination. Repeat the above operation for each parameter combination, and select the parameter combination corresponding to the minimum training error as the Q, R, and F matrix coefficients. Based on this, the construction of the cost function is completed.

[0070] In another exemplary embodiment of the present invention, the above-mentioned embodiment is continued as an example for explanation, and the system state linear model corresponding to the aircraft operation state parameter can be constructed in the following manner: Based on the historical flight rapid recording data, linear modeling is performed on each aircraft operating state parameter to obtain a system state linear model corresponding to the aircraft operating state parameter, wherein the system state linear model is used to characterize the model formed by the aircraft operating state parameter being affected by any one or more of other aircraft operating state parameters, decision actions, and random items.

[0071] In another embodiment, the system states (corresponding to aircraft operating state parameters) that have an impact on the aircraft operating state that requires attention may be analyzed based on aerodynamic theory. Furthermore, a linear model may be established for each system state, and the model coefficients may be obtained through linear regression and autoregression of more than 4,000 QAR data (corresponding to historical flight quick record data).

[0072] In yet another embodiment, an aerodynamic analysis is performed on the aircraft system to determine that the system state (corresponding to aircraft operating state parameters) that affects the yaw angle, altitude, and airspeed through mechanisms such as generating lift may include nine items: airspeed, yaw angle, altitude, pitch angle, roll angle, ground speed, descent rate, horizontal wind speed, and vertical wind speed.

[0073] Among them, the pilot's actions (control inputs) are the pitch angle stick position and the roll angle stick position. Through the high-order small quantity approximation after Taylor expansion, the system dynamic model can be written as a set of linear equations, each equation describing the mode in which a system state variable is affected by other system states, actions and random terms, where the random terms include other unconsidered factors. The horizontal wind speed and the vertical wind speed are not affected by other variables, and the coefficients can be estimated by autoregression. Through this embodiment, the system state linear model corresponding to the aircraft operating state parameters can be obtained quickly and accurately.

[0074] Figure 4 It is a schematic diagram of the process of training the neural network model provided by the present invention.

[0075] The following will be combined Figure 4 Explain the process of training a neural network model.

[0076] In an exemplary embodiment of the present invention, Figure 4 It can be seen that training the neural network model may include steps 410 to 450, and each step will be introduced below.

[0077] In step 410, a training data set is obtained, wherein the training data set includes a plurality of training data, and the training data includes external environment training parameters and aircraft operation status training parameters.

[0078] In step 420, based on the external environment training parameters and the aircraft operation status training parameters, the pilot's predicted action training data corresponding to the external environment training parameters and the aircraft operation status training parameters during the landing operation of the aircraft is obtained.

[0079] In one embodiment, a training data set may be obtained, wherein the training data set may include multiple training data, and the training data includes external environment training parameters and aircraft operating state training parameters. It should be noted that the external environment training parameters and the aircraft operating state training parameters correspond to the external environment parameters and the aircraft operating state parameters described above, and the two are different in application scenarios.

[0080] Furthermore, based on the external environment training parameters and the aircraft operating status training parameters, the neural network model can be used to obtain the pilot's predicted action training data during the operation of the aircraft landing corresponding to the external environment training parameters and the aircraft operating status training parameters.

[0081] In step 430, actual action training data of the pilot in the process of operating the aircraft to land is obtained, which corresponds to the external environment training parameters and the aircraft operation status training parameters.

[0082] In step 440, a loss function is determined based on the mean square error between the predicted motion training data and the real motion training data.

[0083] In step 450, the neural network model is pre-trained based on the loss function to obtain a trained neural network model.

[0084] In another embodiment, since the training data includes labels, that is, the real action training data of the pilot in the process of operating the aircraft landing corresponding to the external environment training parameters and the aircraft operating status training parameters. During the application process, the labels, that is, the real action training data, can be obtained, and then the loss function is determined based on the mean square error of the predicted action training data and the real action training data. Further, the neural network is trained by minimizing the mean square error, in order to obtain the real impact of the external environment and the aircraft state on the pilot's attention allocation pattern. That is, the neural network model is pre-trained based on the loss function to obtain a trained neural network model, so that the real impact of the external environment and the aircraft state on the pilot's attention allocation pattern can be obtained based on the obtained trained neural network model.

[0085] According to the foregoing description, the pilot landing operation analysis method provided by the present invention obtains the external environment parameters and the aircraft operating state parameters formed during the pilot's operation of the aircraft landing; calls the pre-trained neural network model, and inputs the external environment parameters and the aircraft operating state parameters into the neural network model to obtain the state transfer matrix at each time point; based on the state transfer matrix, obtains the hidden state corresponding to the state transfer matrix, wherein the hidden state is used to characterize each state of the pilot's attention allocation pattern, and the hidden state includes any one or more of the state of no attention loss, the state of lack of attention to the yaw angle, the state of lack of attention to the altitude, and the state of lack of attention to the airspeed, thereby completing the analysis of the pilot's attention allocation pattern during the landing operation, and realizing the ability to accurately and objectively analyze the pilot's landing operation, thereby facilitating targeted risk control and improving the pilot's flight safety level.

[0086] The pilot landing operation analysis device provided by the present invention is described below. The pilot landing operation analysis device described below and the pilot landing operation analysis method described above can be referenced to each other.

[0087] Figure 5 It is a structural schematic diagram of the pilot landing operation analysis device provided by the present invention.

[0088] The following will be combined Figure 5 The structure of the pilot landing operation analysis device is described.

[0089] In an exemplary embodiment of the present invention, Figure 5 It can be seen that the pilot landing operation analysis device may include an acquisition module 510, a calling module 520, and a processing module 530, and each module will be introduced below.

[0090] The acquisition module 510 may be configured to acquire external environment parameters and aircraft operation status parameters formed during the pilot's operation of the aircraft to land; The calling module 520 may be configured to call a pre-trained neural network model, and input the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain a corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; The processing module 530 can be configured to obtain a hidden state corresponding to the state transfer matrix based on the state transfer matrix to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation pattern, and the hidden state includes any one or more of a state in which there is no lack of attention, a state in which there is a lack of attention to the yaw angle, a state in which there is a lack of attention to the altitude, and a state in which there is a lack of attention to the airspeed.

[0091] In an exemplary embodiment of the present invention, the processing module 530 may also be configured to: Based on the hidden state, an observation value of a target operating state parameter formed during the landing process of the pilot operating the aircraft corresponding to the hidden state is obtained, wherein the target operating state parameter includes any one or more of a yaw angle, an altitude, and an airspeed; Based on the observed value of the target operating state parameter, a predicted action of the pilot in operating the aircraft to land is obtained; Based on the predicted action, the flight risk of the pilot in operating the aircraft to land is determined.

[0092] In an exemplary embodiment of the present invention, the processing module 530 may also be configured to: Determining an optimal control law for the pilot in operating the aircraft to land; The processing module 530 may obtain the predicted action of the pilot in the process of landing the aircraft based on the observed value of the target operating state parameter in the following manner: Based on the observed values ​​of the target operating state parameters and the optimal control law, the predicted actions of the pilot in operating the aircraft to land are obtained.

[0093] In an exemplary embodiment of the present invention, the processing module 530 may determine the optimal control law of the pilot in the process of landing the aircraft in the following manner: Calling a pre-built partially observable Markov decision model corresponding to a pilot-operated aircraft landing process, wherein the partially observable Markov decision model is obtained by modeling the pilot-operated aircraft landing process based on the partially observable Markov decision process; Obtaining a cost function, wherein the cost function is a linear exponential quadratic function for maintaining the aircraft to land at a stable descent rate and to land at a target speed; With the goal of making the running trajectory formed by the pilot operating the aircraft during the landing process close to the ideal trajectory and the pilot performing smooth operations during the landing process, the optimal control law of the pilot in the process of operating the aircraft during the landing process is determined based on the cost function.

[0094] In an exemplary embodiment of the present invention, the processing module 530 may determine the partially observable Markov decision model in the following manner: Based on the partially observable Markov decision process, the pilot's operation of the aircraft landing process is modeled to obtain a partially observable Markov decision model, and the state variables, decision actions, state transition probabilities, observations and cost functions of the partially observable Markov decision model are defined respectively, wherein: Determining the state variable based on the aircraft operating state parameter; Determining the decision action based on the pitch angle pole position and the roll angle pole position; determining the state transition probability based on a system state linear model corresponding to the aircraft operating state parameters; determining the observation based on the observed value of the target operating state parameter, and Construct a cost function to maintain the aircraft's descent at a steady rate and landing at the target speed.

[0095] In an exemplary embodiment of the present invention, the processing module 530 may determine the system state linear model corresponding to the aircraft operating state parameter in the following manner: Based on the historical flight rapid recording data, each of the aircraft operating state parameters is linearly modeled to obtain a system state linear model corresponding to the aircraft operating state parameter, wherein the system state linear model is used to characterize a model formed by the aircraft operating state parameter being affected by any one or more of other aircraft operating state parameters, decision actions, and random items.

[0096] In an exemplary embodiment of the present invention, the calling module 520 can implement pre-training to obtain the neural network model in the following manner: Acquire a training data set, wherein the training data set includes a plurality of training data, and the training data includes external environment training parameters and aircraft operation status training parameters; Based on the external environment training parameters and the aircraft operation status training parameters, obtaining the predicted action training data of the pilot in the process of operating the aircraft landing corresponding to the external environment training parameters and the aircraft operation status training parameters; Acquiring real action training data of the pilot in the process of operating the aircraft to land, corresponding to the external environment training parameters and the aircraft operation status training parameters; Determining a loss function based on a mean square error between the predicted action training data and the real action training data; The neural network model is pre-trained based on the loss function to obtain the trained neural network model.

[0097] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor (processor) 610 , a communication interface (Communications Interface) 620 , a memory (memory) 630 and a communication bus 640 , wherein the processor 610 , the communication interface 620 , and the memory 630 communicate with each other through the communication bus 640 . The processor 610 can call the logic instructions in the memory 630 to execute the pilot landing operation analysis method, which includes: obtaining external environment parameters and aircraft operating state parameters formed during the pilot's operation of the aircraft to land; calling a pre-trained neural network model, and inputting the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain the corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; based on the state transfer matrix, obtain a hidden state corresponding to the state transfer matrix to complete the pilot landing operation analysis, wherein the hidden state is used to characterize each state of the pilot's attention allocation mode, and the hidden state includes any one or more of a state without attention loss, a state without attention to the yaw angle, a state without attention to the altitude, and a state without attention to the airspeed.

[0098] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pilot landing operation analysis method provided by the above methods, which includes: obtaining external environment parameters and aircraft operating state parameters formed during the pilot's operation of the aircraft to land; calling a pre-trained neural network model, and inputting the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain a corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; based on the state transfer matrix, a hidden state corresponding to the state transfer matrix is ​​obtained to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation mode, and the hidden state includes any one or more of a state without attention loss, a state of lack of attention to the yaw angle, a state of lack of attention to the altitude, and a state of lack of attention to the airspeed.

[0100] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the pilot landing operation analysis method provided by the above methods, the method comprising: obtaining external environment parameters and aircraft operating state parameters formed during the pilot's operation of the aircraft to land; calling a pre-trained neural network model, and inputting the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain a corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; based on the state transfer matrix, a hidden state corresponding to the state transfer matrix is ​​obtained to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation mode, and the hidden state includes any one or more of a state without attention loss, a state without attention to the yaw angle, a state without attention to the altitude, and a state without attention to the airspeed.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for analyzing a pilot's landing operation, characterized in that: The method comprises: Obtain the external environment parameters and aircraft operating status parameters formed during the pilot's operation of the aircraft landing; Calling a pre-trained neural network model, and inputting the external environment parameters and the aircraft operating state parameters into the neural network model, to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain a corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; Based on the state transfer matrix, a hidden state corresponding to the state transfer matrix is ​​obtained to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation pattern, and the hidden state includes any one or more of a state in which there is no lack of attention, a state in which attention to the yaw angle is lacking, a state in which attention to the altitude is lacking, and a state in which attention to the airspeed is lacking.

2. The pilot landing operation analysis method according to claim 1, characterized in that: After obtaining the hidden state corresponding to the state transfer matrix, the method further includes: Based on the hidden state, an observation value of a target operating state parameter formed during the landing process of the pilot operating the aircraft corresponding to the hidden state is obtained, wherein the target operating state parameter includes any one or more of a yaw angle, an altitude, and an airspeed; Based on the observed value of the target operating state parameter, a predicted action of the pilot in operating the aircraft to land is obtained; Based on the predicted action, the flight risk of the pilot in operating the aircraft to land is determined.

3. The pilot landing operation analysis method according to claim 2, characterized in that: Before obtaining the predicted action of the pilot in operating the aircraft to land based on the observed value of the target operating state parameter, the method further includes: Determining an optimal control law for the pilot in operating the aircraft to land; The step of obtaining the predicted action of the pilot in operating the aircraft to land based on the observed value of the target operating state parameter specifically includes: Based on the observed values ​​of the target operating state parameters and the optimal control law, the predicted actions of the pilot in operating the aircraft to land are obtained.

4. The pilot landing operation analysis method according to claim 3, characterized in that: The optimal control law of the pilot in the process of operating the aircraft to land is determined in the following way: Calling a pre-built partially observable Markov decision model corresponding to a pilot-operated aircraft landing process, wherein the partially observable Markov decision model is obtained by modeling the pilot-operated aircraft landing process based on the partially observable Markov decision process; Obtaining a cost function, wherein the cost function is a linear exponential quadratic function for maintaining the aircraft to land at a stable descent rate and to land at a target speed; With the goal of making the running trajectory formed by the pilot operating the aircraft during the landing process close to the ideal trajectory and the pilot performing smooth operations during the landing process, the optimal control law of the pilot in the process of operating the aircraft during the landing process is determined based on the cost function.

5. The pilot landing operation analysis method according to claim 4, characterized in that: The partially observable Markov decision model is constructed in the following way: Based on the partially observable Markov decision process, the pilot's operation of the aircraft landing process is modeled to obtain a partially observable Markov decision model, and the state variables, decision actions, state transition probabilities, observations and cost functions of the partially observable Markov decision model are defined respectively, wherein: Determining the state variable based on the aircraft operating state parameter; Determining the decision action based on the pitch angle pole position and the roll angle pole position; Determining the state transition probability based on a system state linear model corresponding to the aircraft operating state parameters; determining the observation based on the observed value of the target operating state parameter, and Construct a cost function to maintain the aircraft's descent at a steady rate and landing at the target speed.

6. The pilot landing operation analysis method according to claim 5, characterized in that: The system state linear model corresponding to the aircraft operating state parameters is constructed in the following manner: Based on the historical flight rapid recording data, each of the aircraft operating state parameters is linearly modeled to obtain a system state linear model corresponding to the aircraft operating state parameter, wherein the system state linear model is used to characterize a model formed by the aircraft operating state parameter being affected by any one or more of other aircraft operating state parameters, decision actions, and random items.

7. The pilot landing operation analysis method according to any one of claims 1 to 6, characterized in that: The neural network model is pre-trained in the following way: Acquire a training data set, wherein the training data set includes a plurality of training data, and the training data includes external environment training parameters and aircraft operation status training parameters; Based on the external environment training parameters and the aircraft operation status training parameters, obtaining the predicted action training data of the pilot in the process of operating the aircraft landing corresponding to the external environment training parameters and the aircraft operation status training parameters; Acquiring real action training data of the pilot in the process of operating the aircraft to land, corresponding to the external environment training parameters and the aircraft operation status training parameters; Determining a loss function based on a mean square error between the predicted action training data and the real action training data; The neural network model is pre-trained based on the loss function to obtain the trained neural network model.

8. A pilot landing operation analysis device, characterized in that: The device comprises: An acquisition module is used to acquire external environment parameters and aircraft operation status parameters formed during the pilot's operation of the aircraft landing; A calling module is used to call a pre-trained neural network model, and input the external environment parameters and the aircraft operating state parameters into the neural network model to obtain a state transfer matrix at each time point output by the neural network model, wherein the neural network model is used to obtain a corresponding state transfer matrix at each time point based on the external environment parameters and the aircraft operating state parameters; A processing module is used to obtain a hidden state corresponding to the state transfer matrix based on the state transfer matrix to complete the analysis of the pilot's landing operation, wherein the hidden state is used to characterize each state of the pilot's attention allocation mode, and the hidden state includes any one or more of a state in which there is no lack of attention, a state in which attention to the yaw angle is lacking, a state in which attention to the altitude is lacking, and a state in which attention to the airspeed is lacking.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the pilot landing operation analysis method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pilot landing operation analysis method according to any one of claims 1 to 6 is implemented.

Images