Machine learning based drone operator behavior prediction method and system

Abstract

The application relates to the technical field of unmanned aerial vehicle control and discloses a method and system for predicting the behavior of an unmanned aerial vehicle operator based on machine learning, which comprises the following steps: collecting multi-modal behavior data such as operation instruction sequences, visual fixation trajectories and physiological signals; generating parameter evolution characteristic data through noise suppression and time sequence characteristic processing; constructing an operation mode-behavior response correlation model, analyzing intention correlation and evaluating intention probability; identifying a behavior influence range, coupling operation state modes to generate state action coupling data; and predicting the behavior of an operator and adjusting unmanned aerial vehicle operation parameters based on a machine learning model. The system comprises a scene monitoring module, a data processing module, an intention analysis module, a state evaluation module and a behavior prediction module. The application realizes efficient integration and dynamic prediction of multi-modal data, and improves the intelligent level and response accuracy of unmanned aerial vehicle operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, in particular to a method and system for predicting the behavior of an unmanned aerial vehicle operator based on machine learning. BACKGROUND

[0002] The rapid development of unmanned aerial vehicle technology has led to its widespread application in military, agricultural, and logistics fields. However, during the operation of unmanned aerial vehicles, the behavior of the operator directly affects the efficiency and safety of task execution. Traditional unmanned aerial vehicle control systems mainly rely on manual input from the operator, lacking real-time prediction capabilities for the operator's behavior intentions, leading to system response lag or increased risk of misoperation.

[0003] In existing technologies, some studies attempt to predict operator behavior through single modal data (such as operation instructions or physiological signals), but ignore the synergistic effect of multi-modal data. For example, relying solely on instruction response time analysis cannot fully reflect the cognitive load or visual attention distribution of the operator; while simply relying on heart rate variability makes it difficult to capture the relevance of operation instructions and behavior intentions. In addition, existing methods have deficiencies in noise suppression, time series feature extraction, and multi-parameter coupling analysis, resulting in limited prediction accuracy.

[0004] Another type of research uses machine learning models for behavior prediction, but does not fully consider the dynamic characteristics of the operation scenario. For example, factors such as light intensity and electromagnetic interference in the operation environment can affect data collection quality, and existing technologies lack targeted noise suppression and interference isolation mechanisms. At the same time, the behavior intentions of the operator have time window dependence, and traditional models have difficulty effectively capturing mutation characteristics and key stages in the evolution of intentions.

[0005] Therefore, a method and system are needed that can integrate multi-modal data, suppress noise interference, dynamically analyze behavior intentions, and achieve high-precision prediction, to improve the intelligent level and response efficiency of unmanned aerial vehicle operation. SUMMARY

[0006] The present application aims to provide a method and system for predicting the behavior of an unmanned aerial vehicle operator based on machine learning, to solve the problems raised in the background.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solution: a method and system for predicting the behavior of an unmanned aerial vehicle operator based on machine learning, the method comprising:

[0008] Step A1: Multi-modal behavior data collection for unmanned aerial vehicle operation scenarios, obtaining original behavior data containing operation instruction sequences, visual gaze trajectories, and physiological signals; determining key monitoring dimensions based on the original behavior data, and constructing a multi-source data collection network;

[0009] Step A2: Noise suppression processing is performed on the multi-source data collection network, and real-time operation behavior parameter recording is performed to obtain real-time operation behavior data; time sequence feature processing is performed on the real-time operation behavior data based on instruction response delay, gaze point transfer frequency, and heart rate variability to generate parameter evolution feature data;

[0010] Step A3: An operation mode-behavior response association model is constructed according to the parameter evolution feature data and the original behavior data; behavior intention correlation analysis is performed according to the operation mode-behavior response association model, and time window related intention probability evaluation is performed to generate behavior intention dynamic data;

[0011] Step A4: Operation state association processing is performed according to the behavior intention dynamic data and the real-time operation behavior data to generate operation state dynamic characteristic data; behavior influence range identification is performed based on the operation state dynamic characteristic data to generate behavior influence area data; operation state mode coupling is performed based on instruction priority, physiological load, and intention matching degree according to the behavior influence area data to generate state action coupling data;

[0012] Step A5: A machine learning prediction model is constructed according to the state action coupling data, and operator behavior dynamic prediction is performed using the machine learning prediction model to generate behavior prediction data; operation parameter real-time adjustment is performed based on the behavior prediction data to obtain unmanned aerial vehicle operation control feedback data.

[0013] Preferably, step A1 includes the following steps:

[0014] Step A11: Multi-node data sampling is performed on the unmanned aerial vehicle operation scene to obtain behavior sample data containing operation handle displacement, screen gaze heat map, and electrocardiogram;

[0015] Step A12: The type, duration, and frequency of the operation stage are identified from the behavior sample data to obtain operation stage structure data;

[0016] Step A13: Feature analysis is performed on the behavior sample data based on instruction response time, gaze point dwell time, and heart rate variability according to the operation stage structure data to obtain operation behavior feature data;

[0017] Step A14: Operation environment feature analysis is performed according to the operation behavior feature data to obtain environment feature data, wherein the operation environment feature analysis includes operation scene light intensity monitoring, interference source identification of the behavior sample data, and environment interference influence analysis based on gaze point offset and heart rate fluctuation;

[0018] Step A15: The environment feature data, operation behavior feature data, and operation stage structure data are combined into original behavior data;

[0019] Step A16: Determine the key monitoring dimensions according to the original behavior data, and build a multi-source data acquisition network including instruction sensors, eye tracking devices, and physiological signal collectors.

[0020] Preferably, step A16 includes the following steps:

[0021] Step A161: Perform mutational analysis on the operation phase characteristics according to the original behavior data to obtain phase mutation point data, wherein the mutational analysis specifically identifies the operation phase types and the time points of significant changes in the characteristic indicators;

[0022] Step A162: Evaluate the operation state sensitivity according to the original behavior data to identify periods with large fluctuations in the operation state, and obtain state sensitive period data, wherein the state sensitivity evaluation includes stability analysis based on instruction response time, concentration evaluation based on gaze point dwell time, and load evaluation based on heart rate variability;

[0023] Step A163: Determine the time distribution of the key monitoring dimensions according to the preset operation task requirement data, phase mutation point data, and state sensitive period data, and obtain monitoring dimension arrangement data;

[0024] Step A164: Determine the type of acquisition equipment according to the monitoring dimension arrangement data, and build a multi-source data acquisition network including instruction recording devices, eye tracking devices, and physiological signal acquisition devices.

[0025] Preferably, step A2 includes the following steps:

[0026] Step A21: Perform noise suppression processing based on signal filtering and interference isolation on the multi-source data acquisition network, and record real-time parameters during the operation of the unmanned aerial vehicle, thereby obtaining real-time operation behavior data;

[0027] Step A22: Segment the instruction response delay in the real-time operation behavior data based on time series, thereby establishing a delay-time relationship curve;

[0028] Step A23: Calculate the frequency characteristics and amplitude distribution of the delay fluctuations according to the delay-time relationship curve, and perform correlation analysis of the delay fluctuations and the operation phase, thereby obtaining delay-phase evolution characteristic data;

[0029] Step A24: Establish a relationship curve between the gaze point transfer frequency and the operation phase according to the real-time operation behavior data, and calculate the frequency change rate in different phases, thereby obtaining frequency spatio-temporal evolution characteristic data;

[0030] Step A25: Real-time change record according to heart rate variability in real-time operation behavior data, and correlation feature analysis of heart rate variability and operation stage, so as to obtain variability space-time evolution feature data;

[0031] Step A26: According to delay-stage evolution feature data, frequency space-time evolution feature data and variability space-time evolution feature data, the comprehensive influence analysis of parameter combination on behavior prediction is carried out, and the critical value and typical interval of key parameter combination are identified, so as to generate space-time evolution feature data of parameter combination;

[0032] Step A27: According to space-time evolution feature data, a dynamic evolution model of three-dimensional parameter space is established, and parameter change feature is extracted, so as to obtain parameter evolution feature data.

[0033] Preferably, step A21 comprises the following steps:

[0034] The noise suppression processing based on signal filtering and interference isolation is carried out on the multi-source data acquisition network, and the real-time parameters in the operation process of unmanned aerial vehicle are recorded, so as to obtain real-time operation behavior data; wherein the signal filtering includes low-pass filtering, band-pass filtering and adaptive filtering processing of the collected signal, and the interference isolation includes setting electromagnetic shielding layer, using independent power supply circuit and carrying out anti-interference optimization of installation position of collection equipment.

[0035] Preferably, step A27 comprises the following steps:

[0036] Step A271: According to space-time evolution feature data, a three-dimensional parameter space with instruction response delay, gaze point transfer frequency and heart rate variability as coordinate axes is constructed, so as to obtain parameter space coordinate data;

[0037] Step A272: Based on parameter space coordinate data, parameter change trajectory is established, and parameter motion trajectory curve is constructed through space mapping of time sequence sampling points, so as to obtain parameter trajectory data;

[0038] Step A273: The parameter trajectory data is processed based on the segmentation of operation stage, the parameter change characteristics including change trend, change rate and mutual correlation of different stages are identified, so as to obtain segmented feature data;

[0039] Step A274: According to segmented feature data, a dynamic model of three-dimensional parameter space is constructed, and continuous expression of parameter change is established through time interpolation and numerical fitting, so as to obtain dynamic evolution model data;

[0040] Step A275: The dynamic evolution model data is extracted based on gradient feature, curvature feature and rate feature of parameter change, so as to obtain model feature data;

[0041] Step A276: Based on the model feature data, parameter correlation analysis is performed based on the interaction between parameters and feedback mechanism, so as to obtain parameter correlation data;

[0042] Step A277: According to the model feature data and the parameter correlation data, an index representing the dynamic change rule of the parameters is extracted, so as to generate parameter evolution feature data.

[0043] Preferably, step A3 comprises the following steps:

[0044] Step A31: According to the parameter evolution feature data and the original behavior data, an operation mode-behavior response corresponding table containing the mapping relationship between parameter changes and behavior responses in different operation stages is established, so as to obtain response corresponding data;

[0045] Step A32: The response corresponding data is subjected to data normalization and feature screening processing, and a mathematical model of operation mode-behavior response is established, so as to obtain response model data;

[0046] Step A33: Based on the response model data, machine learning training is performed to construct a nonlinear mapping relationship between operation mode and behavior response, so as to obtain an operation mode-behavior response association model;

[0047] Step A34: According to the operation mode-behavior response association model, behavior intention association analysis including instruction intention derivation, gaze intention recognition and physiological intention analysis is performed, so as to obtain intention association data;

[0048] Step A35: Based on the intention association data, intention probability calculation related to time window is performed, so as to obtain intention probability data, wherein the intention probability calculation includes historical similarity matching of behavior intention and weight distribution analysis of current parameters;

[0049] Step A36: The intention probability data is subjected to key feature and mutation point recognition in the behavior intention evolution process, so as to generate behavior intention dynamic data.

[0050] Preferably, step A4 comprises the following steps:

[0051] Step A41: According to the behavior intention dynamic data and real-time operation behavior data, influence rule analysis of different parameter combinations on operation state is performed, so as to obtain state influence data;

[0052] Step A42: The state influence data is subjected to state performance evaluation based on instruction execution accuracy, gaze target matching degree and physiological load degree, so as to obtain performance evaluation data;

[0053] Step A43: The performance evaluation data is subjected to operation state evolution feature analysis with time, so as to obtain operation state dynamic characteristic data;

[0054] Step A44: Numerical simulation of the behavior influence range according to the operation state dynamic characteristic data, and establishment of a multi-field coupled analysis model including the instruction execution field, the gaze attention field and the physiological load field, so as to obtain influence range data;

[0055] Step A45: Boundary identification and time partitioning of the influence range data, so as to obtain behavior influence area data;

[0056] Step A46: Operation state mode coupling based on instruction priority, physiological load and intention matching degree according to the behavior influence area data, and identification of state field evolution law and critical state characteristics, to generate state action coupling data.

[0057] Preferably, step A5 comprises the following steps:

[0058] Step A51: Feature extraction of the state action coupling data, and construction of a training sample set of a machine learning model, so as to obtain training sample data, wherein the training sample data includes input features and behavior labels;

[0059] Step A52: Construction of a machine learning network structure based on a hybrid architecture of long short-term memory network and attention mechanism according to the training sample data, and model training, so as to obtain a behavior prediction model;

[0060] Step A53: Model prediction accuracy optimization of the behavior prediction model based on the leave-one-out cross-validation method, so as to obtain prediction model data;

[0061] Step A54: Dynamic prediction of the operator behavior using the prediction model data, so as to obtain behavior prediction data, wherein the behavior prediction data includes instruction selection tendency, gaze target transfer and physiological state change;

[0062] Step A55: Key stage and behavior mutation feature identification in the behavior formation process of the behavior prediction data, so as to obtain behavior evolution data, wherein the key stages include task initialization stage, instruction execution stage, state adjustment stage, target correction stage and task completion stage;

[0063] Step A56: Real-time adjustment of operation parameters according to the behavior evolution data for each stage, so as to obtain unmanned aerial vehicle operation control feedback data.

[0064] Preferably, the present application further comprises a machine learning-based unmanned aerial vehicle operator behavior prediction system for executing the machine learning-based unmanned aerial vehicle operator behavior prediction method as described above, the machine learning-based unmanned aerial vehicle operator behavior prediction system comprising:

[0065] A scene monitoring module is configured to collect multi-modal behavior data of a UAV operation scene, to obtain original behavior data including an operation instruction sequence, a visual gaze trajectory and a physiological signal; to determine key monitoring dimensions according to the original behavior data, and to construct a multi-source data collection network;

[0066] A data processing module is configured to perform noise suppression processing on the multi-source data collection network, and to perform real-time recording of operation behavior parameters, to obtain real-time operation behavior data; to perform time sequence feature processing on the real-time operation behavior data based on instruction response delay, gaze point transfer frequency and heart rate variability, to generate parameter evolution feature data;

[0067] An intention analysis module is configured to construct an operation mode-behavior response correlation model according to the parameter evolution feature data and the original behavior data; to perform behavior intention correlation analysis according to the operation mode-behavior response correlation model, and to perform time window related intention probability evaluation, to generate behavior intention dynamic data;

[0068] A state evaluation module is configured to perform operation state correlation processing according to the behavior intention dynamic data and the real-time operation behavior data, to generate operation state dynamic characteristic data; to perform behavior influence range identification based on the operation state dynamic characteristic data, to generate behavior influence area data; to perform operation state mode coupling based on instruction priority, physiological load and intention matching degree according to the behavior influence area data, to generate state action coupling data;

[0069] A behavior prediction module is configured to construct a machine learning prediction model according to the state action coupling data, and to perform operator behavior dynamic prediction using the machine learning prediction model, to generate behavior prediction data; to perform real-time adjustment of operation parameters based on the behavior prediction data, to obtain UAV operation control feedback data.

[0070] Compared with the prior art, the present application has the following advantages:

[0071] The present application integrates operation instructions, visual gaze trajectories and physiological signals through a multi-modal data collection network, comprehensively covers the behavior characteristics of operators, and solves the problem of single modal data prediction limitations. Signal filtering and interference isolation technology significantly reduces the influence of noise and improves data quality, providing a reliable foundation for subsequent analysis. Through time sequence feature processing and three-dimensional parameter space modeling, the evolution law of instruction response delay, gaze point transfer frequency and heart rate variability is dynamically captured, and the spatio-temporal correlation of behavior prediction is enhanced.

[0072] The operation mode-behavior response association model is constructed, and the intention probability evaluation is combined to realize accurate analysis of the operator behavior intention, and the intention mutation and key stage are especially good at identifying. The multi-field coupling analysis model effectively quantifies the behavior influence range, provides a scientific basis for state mode coupling, and improves the adaptability of the system to complex operation scenarios. The hybrid architecture based on long short-term memory network and attention mechanism further optimizes the prediction accuracy, so that the model can learn long-term dependencies and key features.

[0073] The finally generated behavior prediction data can guide the adjustment of unmanned aerial vehicle operation parameters in real time, reduce human operation delay and errors, and significantly improve the task execution efficiency and safety. The present application provides an innovative solution for unmanned aerial vehicle intelligent control, and has wide application prospect and practical value. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 The working principle diagram of the unmanned aerial vehicle operator behavior prediction method based on machine learning described in the present application;

[0075] Figure 2 The construction diagram of the multi-modal data acquisition network;

[0076] Figure 3 The modeling diagram of the three-dimensional parameter space;

[0077] Figure 4 The analysis diagram of the operation state coupling. DETAILED DESCRIPTION

[0078] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0079] Please refer to Figures 1-4 The unmanned aerial vehicle operator behavior prediction method based on machine learning related to the present application is implemented as follows:

[0080] Step A1: multi-modal behavior data acquisition is performed on the unmanned aerial vehicle operation scene to obtain original behavior data including operation instruction sequence, visual gaze trajectory and physiological signal; key monitoring dimensions are determined according to the original behavior data, and a multi-source data acquisition network is constructed;

[0081] Step A2: Noise suppression processing is performed on the multi-source data collection network, and real-time operation behavior parameter recording is performed to obtain real-time operation behavior data; time sequence feature processing is performed on the real-time operation behavior data based on instruction response delay, gaze point transfer frequency, and heart rate variability to generate parameter evolution feature data;

[0082] Step A3: An operation mode-behavior response association model is constructed according to the parameter evolution feature data and the original behavior data; behavior intention correlation analysis is performed according to the operation mode-behavior response association model, and time window related intention probability evaluation is performed to generate behavior intention dynamic data;

[0083] Step A4: Operation state association processing is performed according to the behavior intention dynamic data and the real-time operation behavior data to generate operation state dynamic characteristic data; behavior influence range identification is performed based on the operation state dynamic characteristic data to generate behavior influence area data; operation state mode coupling is performed based on instruction priority, physiological load, and intention matching degree according to the behavior influence area data to generate state action coupling data;

[0084] Step A5: A machine learning prediction model is constructed according to the state action coupling data, and operator behavior dynamic prediction is performed using the machine learning prediction model to generate behavior prediction data; operation parameter real-time adjustment is performed based on the behavior prediction data to obtain unmanned aerial vehicle operation control feedback data.

[0085] Embodiment 1:

[0086] In step A1, multi-node data sampling is performed on the unmanned aerial vehicle operation scene, and this process realizes the synchronous collection of operation instruction sequence, visual gaze trajectory, and physiological signal by means of multiple devices. Specifically, displacement data of the operation handle in three-dimensional space is collected by the instruction sensor, which can reflect the operator's control instruction for the unmanned aerial vehicle, such as the pushing amplitude and direction of the left and right joysticks, etc.; screen gaze heat map is obtained by using eye tracking instrument to record the distribution and movement trajectory of operator's line of sight on the screen during operation, such as whether the operator's gaze point is concentrated in the key parameter display area of height, speed, etc. in the unmanned aerial vehicle flight interface; physiological signal collector is used to collect electrocardiogram data to obtain the operator's heart rate, heart rate variability and other physiological indicators, which can reflect the operator's psychological state and physiological load to a certain extent. Through the above multi-node data sampling, behavior sample data containing operation handle displacement, screen gaze heat map and electrocardiogram are finally obtained.

[0087] The obtained behavior sample data is operated to identify the type, duration and frequency of the operation phase. During the operation of the UAV, multiple different operation phases are usually experienced, such as the take-off phase, the cruising phase, the landing phase, etc. For each phase, its type needs to be determined, for example, the take-off phase is the process of the UAV rising from the ground to a specified height; at the same time, the duration of each phase, i.e. the length of time experienced from the beginning to the end of the phase, is recorded; and the frequency of each phase in the entire operation process is counted. Through such identification processing, operation phase structure data is obtained, which clearly describes the division and related time characteristics of each phase in the entire operation process.

[0088] According to the operation phase structure data, the behavior sample data is analyzed for features based on instruction response time, gaze point dwell time and heart rate variability. In different operation phases, the instruction response time of the operator will be different, for example, in the take-off phase, the operator needs to adjust the operation instruction in time according to the state of the UAV, at this time the instruction response time may be shorter and change more frequently; while in the cruising phase, the state of the UAV is relatively stable, the instruction response time may be relatively longer and more stable. In terms of gaze point dwell time, in the key operation phase, such as the landing phase, the gaze point of the operator may be longer in the relevant display of the landing area to ensure accurate judgment of the landing position and attitude of the UAV; while in the non-key phase, the gaze point dwell time may be shorter and more scattered. Heart rate variability can reflect the physiological load and psychological tension of the operator, in the complex or emergency operation phase, such as when the UAV encounters a sudden situation, the heart rate variability may increase, indicating that the physiological load of the operator increases and the psychological tension increases; while in the normal operation phase, the heart rate variability may be relatively small and stable. Through the analysis of these features, operation behavior feature data is obtained, which describes in detail the various features exhibited by the operation behavior of the operator in different operation phases.

[0089] According to the operation behavior characteristic data, operation environment characteristic analysis is performed to obtain environment characteristic data. The operation environment characteristic analysis includes multiple aspects: first, the light intensity of the operation scene is monitored. The change of light intensity will affect the visual perception of the operator, and further affect the operation behavior. For example, in a strong light environment, the operator may have difficulty observing the state information of the UAV due to screen reflection, resulting in gaze point deviation and instruction response time extension; in a weak light environment, it may also be difficult to identify visually, which may affect the operation. Second, the interference source of the behavior sample data is identified. The interference source may include electromagnetic interference, environmental noise, etc. Electromagnetic interference may affect the normal operation of the instruction sensor and the physiological signal collector, resulting in errors in the collected data; environmental noise may distract the operator's attention, affecting his judgment and execution of the operation instruction. Finally, the influence of environmental disturbance is analyzed based on the gaze point deviation and heart rate fluctuation. By analyzing the degree of gaze point deviation and the amplitude of heart rate fluctuation, the influence of environmental disturbance on the operator's operation behavior can be evaluated. For example, when there is strong electromagnetic interference, the operator may have obvious gaze point deviation, and the heart rate will fluctuate accordingly, thereby affecting the accuracy and stability of the operation.

[0090] The environment characteristic data, operation behavior characteristic data and operation stage structure data are merged into original behavior data. The merged original behavior data integrates environmental factors, operator behavior characteristics and operation stage division in the operation process, providing a comprehensive and rich data basis for subsequent analysis and processing.

[0091] According to the original behavior data, the key monitoring dimensions are determined, and a multi-source data acquisition network including an instruction sensor, an eye tracking instrument and a physiological signal collector is constructed. The determination of the key monitoring dimensions is based on the in-depth analysis of the original behavior data to find out the factors that have important influence on the operator behavior prediction, such as the change rate of the operation handle displacement, the gaze point staying time on the take-off related interface and the heart rate variability in the take-off stage, which may be determined as the key monitoring dimensions. According to these key monitoring dimensions, appropriate acquisition devices are selected, such as the instruction sensor for monitoring the operation handle displacement, the eye tracking instrument for monitoring the gaze trajectory, and the physiological signal collector for monitoring the physiological signal, and these devices are reasonably arranged and connected to construct a multi-source data acquisition network to realize efficient and accurate acquisition of multi-modal behavior data in the UAV operation scene.

[0092] Embodiment 2:

[0093] In step A16, the mutation of the operation phase features is analyzed according to the original behavior data to obtain phase mutation point data. The original behavior data contains multi-dimensional information such as operation instruction sequence, visual gaze trajectory, physiological signal, etc. When analyzing the mutation of the operation phase features, the operation phase type and the change of each feature index need to be considered comprehensively. Specifically, the core of the mutation analysis is to identify the time points when the operation phase type changes and the feature index changes significantly. For example, when the unmanned aerial vehicle switches from the cruising phase to the landing phase, the operation phase type will change significantly, and at this time, the displacement mode of the operation handle, the distribution area of the gaze point on the screen, and the heart rate variability and other feature indexes will also change significantly. In the cruising phase, the displacement of the operation handle may be relatively small and stable, the gaze point is mostly concentrated in the flight parameter display area, and the heart rate variability is in a relatively stable state; while entering the landing phase, the displacement of the operation handle will become more frequent and complex, the gaze point will shift to the landing target area, and the heart rate variability may increase due to the increase in operation difficulty. Through real-time monitoring and analysis of these feature indexes, the time points when the operation phase mutates can be accurately located, and these time points are the phase mutation point data, which are of great significance to understanding the dynamic changes of the operation process.

[0094] The operation state sensitivity is evaluated according to the original behavior data to identify periods with large state fluctuations, thereby obtaining state sensitive period data. The state sensitivity evaluation is mainly based on three aspects: stability analysis based on instruction response time, concentration evaluation based on gaze point residence time, and load evaluation based on heart rate variability. In the stability analysis based on instruction response time, the standard deviation, coefficient of variation and other statistical quantities of instruction response time in different periods are calculated to measure the fluctuation degree. When the fluctuation amplitude of the instruction response time is large, it indicates that the stability of the operation state is poor and is in a sensitive state. For example, when the unmanned aerial vehicle encounters air flow interference, the operator needs to frequently adjust the operation instruction, resulting in an increase in the fluctuation of the instruction response time. The concentration evaluation based on the gaze point residence time is to analyze the residence time distribution of the gaze point in different areas to judge the concentration degree of the operator's attention. When the distribution of the gaze point residence time is scattered or the residence time in the key area is significantly reduced, it indicates that the concentration degree of the operator's attention decreases and the operation state may be in a sensitive period. The load evaluation based on heart rate variability is to use the heart rate variability index to reflect the physiological load of the operator. Generally, the smaller the heart rate variability, the greater the physiological load, and the higher the sensitivity of the operation state. By comprehensively evaluating the results of the three aspects, the periods with large state fluctuations, i.e. the state sensitive period data, can be determined.

[0095] According to the preset operation task requirement data, the stage mutation point data and the state sensitive period data, the time distribution of the key monitoring dimensions is determined, and then the monitoring dimension arrangement data is obtained. The preset operation task requirement data clearly defines the task targets and key operation links of different operation stages. For example, in the unmanned aerial vehicle cargo delivery task, the key task of the take-off stage is stable ascent, and the key task of the landing stage is accurate positioning of the delivery point. The stage mutation point data identifies the time nodes of the operation stage conversion, and the state sensitive period data points out the time period in which the operation state is prone to fluctuation. By combining these three types of data, it can be determined which monitoring dimensions are most critical for accurately reflecting the operator's behavior and operation state in different time intervals. For example, near the stage mutation point and in the state sensitive period, more intensive monitoring of monitoring dimensions such as instruction response time and gaze point transfer frequency may be required. By reasonably planning the distribution of key monitoring dimensions on the time axis, the monitoring dimension arrangement data is obtained, which provides a basis for the subsequent arrangement of data acquisition equipment and the formulation of data acquisition strategies.

[0096] According to the monitoring dimension arrangement data, the type of acquisition equipment is determined, and a multi-source data acquisition network including an instruction recording device, an eye tracking device and a physiological signal acquisition device is constructed. The monitoring dimension arrangement data determines the key indicators that need to be monitored in different time periods, and different monitoring indicators require corresponding acquisition equipment to realize. For example, the monitoring of instruction response time requires a high-precision instruction recording device that can record the issuance time of operation instructions and the response time of the unmanned aerial vehicle in real time; the monitoring of gaze point stay duration and transfer frequency requires an eye tracking device such as a head-mounted eye tracker or a desktop eye tracker to accurately capture the visual gaze trajectory of the operator; the monitoring of heart rate variability requires a physiological signal acquisition device such as an electrocardiograph or a heart rate monitor bracelet, etc. After determining the type of acquisition equipment, it is also necessary to arrange and integrate these devices reasonably according to the characteristics of the operation scene and the monitoring requirements. For example, in the unmanned aerial vehicle ground control station, the instruction recording device can be connected to the control system of the operation handle, the eye tracking device can be installed above the display, and the physiological signal acquisition device can be worn on the operator's body. By organically combining these different types of acquisition equipment into a multi-source data acquisition network, synchronous and real-time acquisition of multi-dimensional data during operation can be achieved, providing rich and accurate data support for subsequent data analysis and behavior prediction.

[0097] Embodiment 3:

[0098] In step A2, noise suppression processing is performed on the multi-source data acquisition network, and real-time parameters during the operation of the UAV are recorded to obtain real-time operation behavior data. The multi-source data acquisition network includes instruction sensors, eye trackers, physiological signal collectors, and other devices. In actual operation scenarios, these devices are easily disturbed by various types of noise. For example, the instruction sensor may be distorted due to electromagnetic environmental influences, the image acquisition of the eye tracker may be disturbed by changes in environmental lighting, and the physiological signal collector may produce motion artifacts due to operator movements. To address these situations, noise suppression processing is divided into signal filtering and interference isolation. The signal filtering stage performs low-pass filtering, band-pass filtering, and adaptive filtering on the collected signals. Low-pass filtering can eliminate high-frequency noise, such as high-frequency interference in circuits. Band-pass filtering can preserve signals in a specific frequency band, for example, preserving the effective frequency band of heart rate signals in physiological signal collection. Adaptive filtering dynamically adjusts parameters based on signal changes to improve filtering effectiveness in complex environments. In terms of interference isolation, electromagnetic shielding layers are used to reduce external electromagnetic interference on the devices, independent power supply lines are used to avoid power crosstalk, and the installation location of the collection device is optimized for anti-interference, such as installing the eye tracker in a location that avoids direct sunlight to ensure stable operation of the device. After noise suppression, real-time parameters such as joystick displacement, gaze point coordinates, and heart rate are recorded to form real-time operation behavior data.

[0099] The instruction response delay in the real-time operation behavior data is segmented based on time series to establish a delay-time relationship curve. Instruction response delay refers to the time interval between the operator issuing an operation instruction and the UAV executing the corresponding action, which exhibits different characteristics in different operation stages. For example, during the UAV takeoff stage, the operator needs to frequently adjust the instruction, and the response delay may be short and fluctuate greatly. During the cruise stage, the instruction is relatively stable, and the delay may be longer and less fluctuating. By time series segmentation, the entire operation process is divided into several time windows, the average response delay in each window is calculated, and a delay-time curve is drawn to intuitively reflect the dynamic changes in instruction response delay.

[0100] The frequency characteristic and amplitude distribution of the delay fluctuation are calculated according to the delay-time curve, and the correlation between the delay fluctuation and the operation stage is analyzed to obtain the delay-stage evolution characteristic data. The frequency characteristic analysis can determine the main frequency component of the delay fluctuation, for example, when encountering airflow interference, the response delay may have periodic fluctuations at a certain frequency; the amplitude distribution calculation can understand the range of the delay fluctuation, such as most of the delay fluctuations are within a certain interval. At the same time, the delay fluctuation is compared with the operation stage structure data to analyze the law of delay fluctuation in different stages. For example, in the landing stage, due to the high precision requirement of operation, the fluctuation amplitude of the command response delay may increase, and the fluctuation frequency may decrease. Through the correlation analysis, the evolution characteristic data of the delay in different operation stages are obtained.

[0101] According to the real-time operation behavior data, the gaze point transfer frequency and operation stage relationship curve is established, and the frequency change rate in different stages is calculated to obtain the frequency space-time evolution characteristic data. The gaze point transfer frequency refers to the number of times the operator's line of sight transfers in different regions of the screen per unit time, which reflects the operator's attention allocation and task switching. When constructing the relationship curve, the time interval is divided according to the operation stage, and the gaze point transfer frequency in each interval is counted. For example, in the target recognition stage, the operator needs to frequently scan different regions, and the transfer frequency is high; while in the command execution stage, the gaze point may be concentrated in a specific region, and the transfer frequency is low. By calculating the frequency change rate between different stages, the dynamic conversion speed of attention allocation can be understood, such as the rising rate of gaze point transfer frequency when switching from the cruising stage to the target search stage, thereby forming the evolution characteristic data of frequency in the time and operation stage dimensions.

[0102] According to the real-time operation behavior data, the heart rate variability is recorded in real time, and its correlation with the operation stage is analyzed to obtain the variability space-time evolution characteristic data. Heart rate variability (HRV) is an important indicator reflecting the activity of the autonomic nervous system and the stress state. In different operation stages, HRV will change due to the psychological load and physiological state change of the operator. For example, during complex operation tasks or emergency situations, HRV may decrease, indicating increased sympathetic nervous excitement and psychological load; while in the stable operation stage, HRV may increase, with the parasympathetic nerve dominating. Real-time recording of the numerical change of HRV, combined with operation stage division, analyzes the mean, standard deviation and other statistical quantities of HRV in each stage, as well as the change trend of HRV when switching stages, such as the decline amplitude and speed of HRV when entering the fault handling stage from the normal operation stage, thereby obtaining the evolution characteristic data of HRV in the time and space dimensions.

[0103] According to the delay-stage evolution characteristic data, the frequency space-time evolution characteristic data and the variability space-time evolution characteristic data, the comprehensive influence of parameter combination on behavior prediction is analyzed, and the critical value and typical interval of key parameter combination are identified, and the space-time evolution characteristic data of parameter combination is generated. The three types of parameters are not independent, but are interrelated and jointly affect the operator's behavior. For example, the increase of instruction response delay may be accompanied by the decrease of gaze point transfer frequency, and the decrease of HRV, which may indicate that the operator is in a state of fatigue or tension. Through multivariate analysis method, the contribution of different parameter combinations to behavior prediction at different operation stages and time points is studied, and it is determined which parameter combination is most critical to the representation of behavior intention. At the same time, the critical value of these key parameter combinations is identified, such as when the instruction response delay exceeds a certain threshold and the HRV is lower than a certain level, the risk of operator's operation failure may increase; and the typical interval, such as in the normal operation stage, the parameter combination is usually distributed in a certain range. Through these analyses, the evolution characteristic data of parameter combination in space-time dimension is generated, providing more comprehensive feature input for subsequent model construction.

[0104] Embodiment 4:

[0105] In step A27, a three-dimensional parameter space with instruction response delay, gaze point transfer frequency and heart rate variability as coordinate axes needs to be constructed according to the space-time evolution characteristic data, so as to obtain parameter space coordinate data. The space-time evolution characteristic data contains the dynamic change information of the three parameters on the operation time axis, for example, in the unmanned aerial vehicle take-off stage, the instruction response delay may fluctuate between 0.5 seconds and 1 second, the gaze point transfer frequency is about 15 to 20 times per minute, and the heart rate variability presents the interval characteristics of 50ms to 80ms. These parameters are respectively taken as the x-axis, y-axis and z-axis of the three-dimensional space, and the parameter value of each time sampling point corresponds to a coordinate point in the space, such as the instruction response delay of 0.8 seconds, the gaze point transfer frequency of 18 times per minute and the heart rate variability of 65ms at a certain time, the coordinates are (0.8, 18, 65). In this way, the parameter change on the time sequence is mapped into the three-dimensional space to form the parameter space coordinate data, so that the dynamic relationship of multiple parameters can be visualized and presented.

[0106] Based on the parameter space coordinate data, a parameter change trajectory is established, and a parameter motion trajectory curve is constructed through the space mapping of time sequence sampling points, so as to obtain parameter trajectory data. In the three-dimensional parameter space, the coordinate points of each time point are sequentially connected in time order to form a continuous trajectory line. For example, in the transition phase from take-off to cruising of the unmanned aerial vehicle, the instruction response delay may gradually increase from 0.8 seconds to 1.2 seconds, the gaze point transfer frequency decreases from 18 times per minute to 10 times per minute, and the heart rate variability increases from 65 ms to 90 ms. These coordinate points are connected to form a trajectory curve from the parameter region of the take-off phase to the parameter region of the cruising phase. Through the morphological analysis of the trajectory curve, the change trend of the parameter with time can be directly observed. For example, the direction of the trajectory reflects the direction of the parameter change, and the density of the trajectory reflects the rate of the parameter change.

[0107] The parameter trajectory data is segmented based on the operation phase, and the parameter change characteristics of different phases are identified to obtain segmented feature data. The operation phases such as take-off, cruising, and landing have different operation tasks and environmental characteristics, and the change characteristics of the parameters in each phase also differ. Taking the cruising phase as an example, the parameter trajectory may exhibit a relatively stable aggregation state, the instruction response delay has a small fluctuation range, the gaze point transfer frequency is maintained at a low level, and the heart rate variability is also relatively stable. In the landing phase, the parameter trajectory may exhibit obvious fluctuations, the change amplitude of the instruction response delay increases, the gaze point transfer frequency significantly increases, and the heart rate variability may decrease. In the segmentation process, the parameter trajectory is divided into sub-trajectories corresponding to the phases according to the operation phase structure data, and the change trend, change rate, and mutual correlation between parameters of each sub-trajectory are analyzed. For example, in the sub-trajectory of the take-off phase, the instruction response delay presents a decreasing trend with time, the change rate is about 0.05 seconds per second, and the gaze point transfer frequency and the heart rate variability present a positive correlation, i.e., when the gaze point transfer frequency increases, the heart rate variability also increases accordingly. These characteristics constitute the segmented feature data.

[0108] According to the segmented characteristic data, a dynamic model of three-dimensional parameter space is constructed, and a continuous expression of parameter variation is established through time interpolation and numerical fitting, so as to obtain dynamic evolution model data. In each operation stage, based on the parameter variation law in the segmented characteristic data, time interpolation method is used to supplement the sparse sampling points, so that the parameter variation is more continuous. For example, if parameter data is collected every 10 seconds in a stage, the parameter estimation value every second can be obtained through linear interpolation or spline interpolation. Then, a mathematical expression of parameter variation with time is constructed by using numerical fitting method, such as polynomial fitting or regression analysis. For example, in the cruise stage, the instruction response delay can be fitted as a constant function plus a small amplitude sinusoidal fluctuation function, to reflect the slight fluctuation around the stable value; the gaze point transfer frequency can be fitted as a linear function, to reflect the slow change trend with time. In this way, the discrete segmented characteristic data is converted into continuous dynamic model, and accurate description of parameter variation is realized.

[0109] The model characteristic data is obtained by performing feature extraction on the dynamic evolution model data based on gradient feature, curvature feature and rate feature of parameter variation. The gradient feature reflects the direction and intensity of parameter variation, for example, the gradient of instruction response delay is positive, indicating that it increases with time, and the greater the absolute value of the gradient, the faster the change speed; the curvature feature describes the bending degree of parameter variation trajectory, and the greater the curvature, the more significant the change in parameter variation direction; the rate feature quantifies the speed of parameter variation, for example, the change frequency of gaze point transfer frequency per second. Taking the landing stage as an example, the gradient of instruction response delay may change from positive to negative at a certain moment, indicating that the change trend is reversed, the curvature reaches a peak value at this point, and the rate feature shows that the change speed reaches the maximum value in this stage, which together constitute the model characteristic data, providing quantitative indicators for in-depth understanding of parameter dynamic variation.

[0110] The parameter correlation analysis is performed based on the model characteristic data, so as to obtain parameter association data. Parameters do not exist independently, but have mutual influence. For example, when the instruction response delay increases, the gaze point transfer frequency may decrease, because the operator needs more time to process information, resulting in slower eye movement; at the same time, the heart rate variability may decrease, reflecting the increase of psychological load of the operator. By calculating the correlation coefficient between parameters or using other correlation analysis methods, the correlation degree and direction between parameters are determined. For example, in complex operation tasks, the instruction response delay and the heart rate variability may have a significant negative correlation, that is, the longer the response delay, the smaller the heart rate variability, indicating that the operator is in a highly nervous state. These parameter association data reveal the cooperative variation law between multiple parameters, providing more abundant information for behavior prediction.

[0111] According to the model feature data and the parameter correlation data, an index representing the dynamic change rule of the parameter is extracted, so as to generate parameter evolution feature data. Considering the model features and the parameter correlations, key indexes that can effectively represent the dynamic change of the parameters are selected. For example, the maximum value of the gradient of the parameter change in each stage, the peak value of the curvature, the average value of the rate, and the like are selected as model feature indexes, and the correlation coefficient and the correlation strength between the parameters are selected as parameter correlation indexes. These indexes are combined into parameter evolution feature data. These feature data not only contain the change rule of a single parameter, but also reflect the interaction between the parameters, and can more comprehensively describe the dynamic evolution process of the operator behavior state, thereby providing deep feature input for subsequent construction of an operation mode-behavior response correlation model and a behavior prediction model.

[0112] Embodiment 5:

[0113] In step A5, feature extraction is performed on the state-action coupling data, and a training sample set of the machine learning model is constructed, so as to obtain training sample data. The state-action coupling data contains the coupling relationship of multi-dimensional information such as instruction priority, physiological load, and intention matching degree in the operation process. For example, in the unmanned aerial vehicle obstacle avoidance operation, the priority of the emergency obstacle avoidance instruction is high, at this time, the physiological load (such as heart rate variability) of the operator may be reduced, and the intention matching degree (such as the matching degree of the gaze point and the obstacle avoidance target) needs to be maintained at a high level. In the feature extraction, valuable features for behavior prediction are extracted from the coupling data, such as the numerical level of the instruction priority, the quantitative index (such as the specific numerical range of the heart rate variability) of the physiological load, and the percentage of the intention matching degree. Taking a specific operation scene as an example, when the unmanned aerial vehicle detects the front obstacle, the system records that the instruction priority at this time is the highest level (set as 5 levels), the heart rate variability of the operator is 40 ms, the intention matching degree is 90%, and the corresponding operator behavior label is “execute emergency obstacle avoidance instruction”. By collecting a large number of similar operation instances, these features are taken as input features, and the corresponding behavior labels (such as “take off”, “cruise”, “avoid obstacle”, and the like) are taken as output, to construct a training sample set, wherein each sample contains the corresponding relationship between the input features and the behavior labels, thereby providing data support for subsequent model training.

[0114] According to the training sample data, a machine learning network structure based on a hybrid architecture of long short-term memory network (LSTM) and attention mechanism is constructed, and model training is performed, so as to obtain a behavior prediction model. The LSTM network is suitable for processing time series data and can capture the long-term dependence of parameters in the operation process. For example, during the long-time cruising of a UAV, the LSTM can remember the operation mode and parameter change trend in the early stage to predict the subsequent behavior. The attention mechanism can make the model pay more attention to key features, such as instruction priority and intention matching degree in the emergency obstacle avoidance scene. In specific construction, the input layer receives the input features (such as instruction response delay, gaze point transfer frequency, heart rate variability, etc.) in the training sample, and after processing by the LSTM layer, different weights are given to the features of different time steps through the attention layer, highlighting the feature influence of key time points. Taking a specific network structure as an example, the input layer includes 10 feature nodes corresponding to 10 input features; the LSTM layer has 128 neurons that can capture complex dependence in time series; the attention layer increases the weight of important features by calculating the weight of each time step feature, such as setting the weight of the instruction priority feature to 0.3 during obstacle avoidance, which is higher than that of other non-key features; the output layer outputs the probability distribution of behavior prediction according to the weighted features, corresponding to different behavior labels (such as “maintain current heading”, “turn to avoid obstacles”, etc.). During model training, a large number of training samples are used to optimize network parameters, adjust the weights of the LSTM layer and the weight coefficients of the attention layer, and minimize the error between the predicted results of the model and the actual behavior labels.

[0115] The behavior prediction model is optimized based on the model prediction accuracy of the leave-one-out cross-validation method, so as to obtain the prediction model data. Leave-one-out cross-validation means that in each validation, n-1 samples are used to train the model and the remaining 1 sample is used for testing, repeated n times (n is the total number of samples) to evaluate the generalization ability of the model. For example, if the training sample set has 1000 samples, 999 samples are taken to train the model each time, and the remaining 1 sample is used for testing, a total of 1000 validations are performed. In this way, overfitting of the model during training can be avoided, and the prediction accuracy of the model on unknown data can be ensured. During optimization, the hyperparameters of the model are adjusted according to the results of cross-validation, such as the number of neurons in the LSTM layer and the weight threshold of the attention mechanism. For example, when it is found that the model has a large error in predicting the “landing” behavior, the attention weight of the landing stage related features can be increased, or the parameters of the LSTM layer can be adjusted to better capture the time series features of the landing stage, so as to obtain the optimized prediction model data and make the model perform more stably in various operation scenarios.

[0116] The operator behavior is dynamically predicted by using the prediction model data, so as to obtain behavior prediction data, wherein the behavior prediction data comprises instruction selection tendency, gaze target transfer and physiological state change. Taking a UAV performing a reconnaissance task as an example, when the model receives current operation parameters (such as instruction response delay of 0.6 seconds, gaze point concentrated in the target area on the right side of the screen, and heart rate variability of 60 ms), through the calculation of the prediction model, the behavior prediction data in the future period of time is output: the probability of the instruction selection tendency being “enlarging the target image” is 70%, the probability of the instruction selection tendency being “switching the reconnaissance view” is 20%, and the probability of the instruction selection tendency being other instructions is 10%; the gaze target transfer prediction is that the transfer is from the current right side area to the parameter display area below, and the transfer time may be in the next 5-8 seconds; and the physiological state change prediction is that the heart rate variability may decrease to about 50 ms in the next 10 seconds, indicating that the operator may increase the physiological load due to the improvement of concentration. The prediction data provides a basis for subsequent operation parameter adjustment, so that the system can predict the behavior intention of the operator in advance.

[0117] Key stages in the behavior formation process and behavior mutation characteristics are identified from the behavior prediction data, so as to obtain behavior evolution data, wherein the key stages comprise a task initialization stage, an instruction execution stage, a state adjustment stage, a target correction stage and a task completion stage. Taking a UAV landing task as an example, the behavior prediction data shows that in the task initialization stage, the instruction selection tendency of the operator is mainly “adjusting the landing height”, and the gaze target is concentrated in the height parameter display area; after entering the instruction execution stage, the instruction selection tendency changes to “controlling the landing speed”, and the gaze target transfers to the landing point image area; in the state adjustment stage, the probability of the instruction selection tendency “correcting the landing angle” may increase due to factors such as wind speed change, and the heart rate variability may also fluctuate; in the target correction stage, the gaze target and the instruction may be adjusted according to real-time feedback; and after the task completion stage, the instruction selection tendency returns to “maintaining hovering” or “shutting down” and the like. In this process, the behavior mutation characteristics may be manifested as sudden changes in the instruction selection tendency, such as in the state adjustment stage, if the wind speed suddenly increases, the probability of the instruction selection tendency “emergency hovering” may suddenly increase from 10% to 60%, at the same time, the gaze target quickly transfers to the wind speed display area, and the heart rate variability rapidly decreases. The identification of these mutation characteristics helps the system to respond to abnormal operation in a timely manner.

[0118] According to the real-time adjustment of the operation parameters in each stage according to the behavior evolution data, the unmanned aerial vehicle operation control feedback data is obtained. Still taking the landing task as an example, in the task initialization stage, according to the instruction tendency of "adjusting the landing height" in the behavior evolution data, the system can set the sensitivity of the height adjustment parameter to a high level in advance, so that the operator's operation is more accurate; in the instruction execution stage, according to the tendency of "controlling the landing speed", the response threshold of the speed control is adjusted to avoid excessive speed fluctuation; when the mutation characteristics of the state adjustment stage are identified (such as the increase of the "emergency hovering" instruction tendency), the system automatically enhances the stability algorithm of the hovering control, and at the same time sends a prompt of abnormal wind speed to the operator; in the target correction stage, according to the gaze target transfer prediction, the display clarity of the target image is optimized; after the task completion stage, according to the "shutdown" tendency of the behavior prediction, the preparation work of system shutdown is done in advance. Through this real-time adjustment, the operation control of the unmanned aerial vehicle is more in line with the behavior intention of the operator, and the safety and efficiency of the operation are improved.

[0119] It should be noted that, in this document, the terms such as first and second are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus including a series of elements includes not only those elements, but also other elements not explicitly listed, or inherent to such a process, method, article or apparatus.

[0120] Although the embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made thereto without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1.A method for predicting behavior of a drone operator based on machine learning, the method comprising: Comprising the following steps: Step A1: Multi-modal behavior data acquisition on unmanned aerial vehicle operation scene, obtaining original behavior data containing operation instruction sequence, visual gaze trajectory and physiological signal; determining key monitoring dimensions according to original behavior data, and constructing multi-source data acquisition network; Step A2: Noise suppression processing on multi-source data acquisition network, and real-time recording of operation behavior parameters, obtaining real-time operation behavior data; Processing the real-time operation behavior data based on instruction response delay, gaze point transfer frequency and heart rate variability to generate parameter evolution feature data; Step A3: Constructing operation mode-behavior response association model according to parameter evolution feature data and original behavior data; behavior intention correlation analysis according to operation mode-behavior response association model, and intention probability evaluation related to time window, generating behavior intention dynamic data; Step A4: Operation state association processing according to behavior intention dynamic data and real-time operation behavior data, generating operation state dynamic characteristic data; behavior influence range identification based on operation state dynamic characteristic data, generating behavior influence area data; Operation state mode coupling based on instruction priority, physiological load and intention matching degree according to behavior influence area data, generating state action coupling data; Step A5: Constructing machine learning prediction model according to state action coupling data, and using machine learning prediction model to predict operator behavior dynamically, generating behavior prediction data; real-time adjustment of operation parameters based on behavior prediction data, obtaining unmanned aerial vehicle operation control feedback data. 2.The method of claim 1, wherein, Step A1 includes the following steps: Step A11: Multi-node data sampling on unmanned aerial vehicle operation scene, obtaining behavior sample data containing operation handle displacement, screen gaze heat map and electrocardiogram; Step A12: Operation stage type, duration and frequency identification on behavior sample data, obtaining operation stage structure data; Step A13: Feature analysis on behavior sample data based on instruction response time, gaze point dwell time and heart rate variability according to operation stage structure data, obtaining operation behavior feature data; Step A14: Operation environment feature analysis according to operation behavior feature data, obtaining environment feature data, wherein environment feature analysis includes operation scene light intensity monitoring, interference source identification on behavior sample data, and environment interference influence analysis based on gaze point offset and heart rate fluctuation; Step A15: Combining environment feature data, operation behavior feature data and operation stage structure data into original behavior data; Step A16: Determining key monitoring dimensions according to original behavior data, and constructing multi-source data acquisition network containing instruction sensor, eye tracker and physiological signal collector. 3.The method of claim 2, wherein, Step A16 includes the following steps: Step A161: Operation stage feature mutation analysis according to original behavior data, obtaining stage mutation point data, wherein mutation analysis is specifically identifying operation stage type and time point of significant change of characteristic index; Step A162: Evaluate the operation state sensitivity based on the original behavior data, identify the period with large fluctuation of operation state, and obtain state sensitive period data, wherein the state sensitivity evaluation includes stability analysis based on instruction response time, concentration evaluation based on gaze point stay time, and load evaluation based on heart rate variability; Step A163: Determine the time distribution of key monitoring dimensions according to the preset operation task requirement data, stage mutation point data and state sensitive period data, and obtain monitoring dimension arrangement data; Step A164: Determine the type of collection equipment according to the monitoring dimension arrangement data, and construct a multi-source data collection network including instruction recording device, eye tracking equipment and physiological signal collection device. 4.The method of claim 3, wherein, Step A2 includes the following steps: Step A21: Perform noise suppression processing based on signal filtering and interference isolation on the multi-source data collection network, and record real-time parameters in the unmanned aerial vehicle operation process, thereby obtaining real-time operation behavior data; Step A22: Segment the instruction response delay in the real-time operation behavior data based on time series, thereby establishing a delay-time relationship curve; Step A23: Calculate the frequency characteristics and amplitude distribution of delay fluctuation according to the delay-time relationship curve, and analyze the correlation between delay fluctuation and operation stage, thereby obtaining delay-stage evolution characteristic data; Step A24: Establish a relationship curve between gaze point transfer frequency and operation stage according to the real-time operation behavior data, and calculate the frequency change rate in different stages, thereby obtaining frequency spatio-temporal evolution characteristic data; Step A25: Record the real-time changes of heart rate variability in the real-time operation behavior data, and analyze the correlation characteristics of heart rate variability and operation stage, thereby obtaining variability spatio-temporal evolution characteristic data; Step A26: Perform comprehensive influence analysis of behavior prediction based on parameter combination according to delay-stage evolution characteristic data, frequency spatio-temporal evolution characteristic data and variability spatio-temporal evolution characteristic data, and identify the critical value and typical interval of key parameter combination, thereby generating spatio-temporal evolution characteristic data of parameter combination; Step A27: Establish a dynamic evolution model of three-dimensional parameter space according to spatio-temporal evolution characteristic data, and extract parameter change characteristics, thereby obtaining parameter evolution characteristic data. 5.The method of claim 4, wherein, Step A21 includes the following steps: Perform noise suppression processing based on signal filtering and interference isolation on the multi-source data collection network, and record real-time parameters in the unmanned aerial vehicle operation process, thereby obtaining real-time operation behavior data; wherein signal filtering includes low-pass filtering, band-pass filtering and adaptive filtering processing of collected signals, and interference isolation includes setting electromagnetic shielding layer, using independent power supply circuit and performing anti-interference optimization of installation position of collection equipment. 6.The method of claim 5, wherein, Step A27 includes the following steps: Step A271: Construct a three-dimensional parameter space with instruction response delay, gaze point transfer frequency and heart rate variability as coordinate axes according to spatio-temporal evolution characteristic data, thereby obtaining parameter space coordinate data; Step A272: Based on the parameter space coordinate data, a parameter change trajectory is established, and a parameter motion trajectory curve is constructed through the spatial mapping of time sequence sampling points, so as to obtain parameter trajectory data; Step A273: The parameter trajectory data is segmented based on the operation stage, and the parameter change characteristics including the change trend, change rate and mutual correlation of different stages are identified, so as to obtain segmented feature data; Step A274: A dynamic model of three-dimensional parameter space is constructed according to the segmented feature data, and a continuous expression of parameter change is established through time interpolation and numerical fitting, so as to obtain dynamic evolution model data; Step A275: The dynamic evolution model data is extracted based on the gradient characteristics, curvature characteristics and rate characteristics of parameter change, so as to obtain model feature data; Step A276: Based on the model feature data, parameter correlation analysis is carried out based on the interaction between parameters and the feedback mechanism, so as to obtain parameter correlation data; Step A277: According to the model feature data and the parameter correlation data, an index representing the dynamic change rule of the parameters is extracted, so as to generate parameter evolution feature data. 7.The method of claim 6, wherein, Step A3 includes the following steps: Step A31: According to the parameter evolution feature data and the original behavior data, an operation mode-behavior response corresponding table containing the mapping relationship between parameter change and behavior response in different operation stages is established, so as to obtain response corresponding data; Step A32: The response corresponding data is normalized and feature screened, and a mathematical model of operation mode-behavior response is established, so as to obtain response model data; Step A33: Based on the response model data, machine learning training is carried out to construct a nonlinear mapping relationship between operation mode and behavior response, so as to obtain an operation mode-behavior response correlation model; Step A34: According to the operation mode-behavior response correlation model, behavior intention correlation analysis including instruction intention deduction, gaze intention identification and physiological intention analysis is carried out, so as to obtain intention correlation data; Step A35: Based on the intention correlation data, intention probability calculation related to time window is carried out, so as to obtain intention probability data, wherein the intention probability calculation includes historical similarity matching of behavior intention and weight distribution analysis of current parameters; Step A36: The intention probability data is processed to identify key features and mutation points in the evolution process of behavior intention, so as to generate behavior intention dynamic data. 8.The method of claim 7, wherein, Step A4 includes the following steps: Step A41: According to the behavior intention dynamic data and real-time operation behavior data, the influence law of different parameter combinations on operation state is analyzed, so as to obtain state influence data; Step A42: The state influence data is evaluated based on the instruction execution accuracy, gaze target matching degree and physiological load degree, so as to obtain performance evaluation data; Step A43: The performance evaluation data is analyzed to obtain the evolution characteristics of the operation state over time, so as to obtain operation state dynamic characteristic data; Step A44: According to the operation state dynamic characteristic data, numerical simulation of the behavior influence range is carried out, and a multi-field coupled analysis model containing instruction execution field, gaze attention field and physiological load field is established, so as to obtain influence range data; Step A45: boundary identification and time partitioning are performed on the influence range data to obtain behavior influence area data; Step A46: operation state mode coupling is performed based on instruction priority, physiological load, and intention matching degree according to the behavior influence area data, and state field evolution law and critical state characteristics are identified to generate state action coupling data. 9.The method of claim 8, wherein, Step A5 includes the following steps: Step A51: feature extraction is performed on the state action coupling data, and a training sample set of a machine learning model is constructed to obtain training sample data, wherein the training sample data includes input features and behavior labels; Step A52: a machine learning network structure based on a hybrid architecture of a long short-term memory network and an attention mechanism is constructed according to the training sample data, and model training is performed to obtain a behavior prediction model; Step A53: model prediction accuracy optimization is performed on the behavior prediction model based on a leave-one-cross-validation method to obtain prediction model data; Step A54: dynamic prediction of operator behavior is performed using the prediction model data to obtain behavior prediction data, wherein the behavior prediction data includes instruction selection tendency, gaze target shift, and physiological state change; Step A55: key stages in the behavior formation process and behavior mutation characteristics are identified from the behavior prediction data to obtain behavior evolution data, wherein the key stages include a task initialization stage, an instruction execution stage, a state adjustment stage, a target correction stage, and a task completion stage; Step A56: real-time adjustment of operation parameters in each stage is performed according to the behavior evolution data to obtain unmanned aerial vehicle operation control feedback data. 10.A system for predicting behavior of a drone operator based on machine learning, the system comprising: The machine learning-based unmanned aerial vehicle operator behavior prediction system for performing the machine learning-based unmanned aerial vehicle operator behavior prediction method of claim 1 comprises: a scene monitoring module for multi-modal behavior data acquisition of unmanned aerial vehicle operation scenes to obtain original behavior data including operation instruction sequences, visual gaze trajectories, and physiological signals; determining key monitoring dimensions according to the original behavior data, and constructing a multi-source data acquisition network; a data processing module for noise suppression processing of the multi-source data acquisition network, and real-time recording of operation behavior parameters to obtain real-time operation behavior data; performing time sequence feature processing based on instruction response delay, gaze point shift frequency, and heart rate variability on the real-time operation behavior data to generate parameter evolution feature data; an intention analysis module for constructing an operation mode-behavior response association model according to the parameter evolution feature data and the original behavior data; performing behavior intention correlation analysis according to the operation mode-behavior response association model, and performing time window-related intention probability evaluation to generate behavior intention dynamic data; a state evaluation module for operation state association processing according to the behavior intention dynamic data and the real-time operation behavior data to generate operation state dynamic characteristic data; performing behavior influence range identification based on the operation state dynamic characteristic data to generate behavior influence area data; performing operation state mode coupling based on instruction priority, physiological load, and intention matching degree according to the behavior influence area data to generate state action coupling data; a state evaluation module for operation state association processing according to the behavior intention dynamic data and the real-time operation behavior data to generate operation state dynamic characteristic data; based on the operation state dynamic characteristic data, behavior influence range identification is performed to generate behavior influence area data; operation state mode coupling is performed based on instruction priority, physiological load, and intention matching degree according to the behavior influence area data to generate state action coupling data; The behavior prediction module is configured to construct a machine learning prediction model according to the state-action coupling data, and to perform dynamic prediction of the operator behavior by using the machine learning prediction model to generate behavior prediction data; and to perform real-time adjustment of the operation parameters based on the behavior prediction data to obtain unmanned aerial vehicle operation control feedback data.

Images