Unmanned aerial vehicle pilot hand control normative real-time evaluation method based on posture evaluation

Abstract

The present application provides a kind of real-time evaluation method of unmanned aerial vehicle pilot hand control specification based on attitude evaluation, the present application utilizes multi-sensor array to collect hand three-dimensional coordinates and joint angle, joystick displacement and other attitude data, through real-time filtering, inverse kinematics calculation and standardization processing, task-oriented dynamic causal diagram is constructed by fusing flight task parameters, and the input features segmented by sliding window drive lightweight Transform network to extract operation intent, efficiently realize the dynamic adjustment of causal node weight and action prediction.Abnormal action identified based on causal dependence chain is attributed to analysis, and context-sensitive correction suggestions are generated in combination with knowledge base, and variational autoencoder is introduced to generate enhanced samples with biomechanical constraints.The present application effectively improves the accuracy of unmanned aerial vehicle control anomaly recognition and the real-time performance of error correction feedback.The present application can model the logical dependence and time evolution relationship between typical control actions in real time.

Description

Technical Field

[0001] This invention relates to the field of UAV operation attitude recognition and causal reasoning technology, and in particular to a real-time evaluation method for the standardization of UAV pilot hand operation based on attitude assessment. Background Technology

[0002] Current drone pilot hand control compliance assessment systems generally adopt a technology approach based on action recognition and template matching. They capture hand posture data streams and use conventional temporal neural networks or deep learning models to classify and score operational actions. Mainstream solutions typically rely on static compliance templates, comparing actual action sequences frame-by-frame with a standard library and providing a macroscopic "pass / fail" judgment when inconsistencies are found. These systems are primarily used for basic training assessment and habit formation, and have some application in actual flight training and combat simulations, but mainly focus on the surface-level compliance judgment of actions.

[0003] As drone control skills and application scenarios become increasingly complex, the industry is gradually exploring more granular methods for action sequence modeling and abnormal behavior detection, such as using convolutional neural networks, temporal self-attention mechanisms, and graph neural networks to improve the accuracy of temporal action recognition and prediction. Meanwhile, some cutting-edge research is attempting to introduce knowledge-based error attribution and feedback mechanisms, hoping to further identify the error type and potential causes beyond simply identifying "where the error occurred." However, existing solutions often model causal relationships using static or weakly dynamic structures, limited to predefined operational process templates or finite state transition rules. For dynamically changing operational intentions and chain-like abnormal actions during flight missions, it is difficult to achieve real-time and accurate error attribution. Most products and systems remain at the level of single-step action evaluation and common error correction suggestions generation, lacking in-depth reasoning and personalized feedback for complex causal chains.

[0004] Traditional technologies are primarily suitable for training novice drone pilots, and are well-suited for standardized operations and basic error recognition. However, when faced with complex flight controls involving multiple stages and tasks, especially in dynamic situations with strong coupling between continuous actions or rapid intention switching, existing technologies reveal the following shortcomings: Existing action error attribution systems typically rely on fixed templates for matching, which cannot dynamically capture changes in control intent and chain reactions of actions that actually occur during flight missions. They lack the ability to adapt to dynamic environments and real-time intent evolution, resulting in the inability to locate the root cause in a timely manner when new or mixed errors occur. Causal relationship modeling generally adopts a static graph structure, which ignores the chain of anomalies caused by factors such as command rhythm and slight deviations in initial posture during actual operation. It is easy to simplify complex anomalies to "non-standard operation" or "incorrect sequence", and fails to achieve multi-level causal path tracing and decomposition. The corrective suggestions generated by existing systems are mostly in template language, with limited pertinence and interpretability. They cannot provide personalized and understandable feedback based on the current flight situation, action context, and operation history, making it difficult to meet the needs of precise training and behavior correction. Traditional methods cannot effectively identify or attribute rare or newly emerging error patterns, which greatly reduces the system's generalization ability and training value, affecting the improvement of drone pilots' actual skills and operational safety in complex scenarios. Summary of the Invention

[0005] This invention provides a real-time evaluation method for the standardization of hand control by drone pilots based on attitude assessment, aiming to solve the problems existing in the prior art mentioned in the background section.

[0006] The technical solution of this invention is: a real-time evaluation method for the standardization of hand operation by UAV pilots based on attitude evaluation, specifically including the following steps: S1: Collect the hand posture data stream of the drone pilot; S2: Construct a dynamic cause-effect graph model based on the standard action template library and flight mission parameters; S3: Perform sliding window segmentation on the pose data stream, and after normalization, input it into a lightweight Transformer network to generate high-level feature vectors containing the semantics of the operation intention; S4: Use high-level feature vectors as input conditions for the dynamic causal graph model to dynamically adjust the causal weights between nodes; S5: Perform forward propagation computation based on the updated dynamic causal graph to generate the probability distribution of the predicted action sequence, compare it with the real action sequence, and identify abnormal action nodes with deviations exceeding the threshold. S6: For the identified abnormal action nodes, perform reverse attribution analysis and output a causal dependency chain containing the initial deviation node and the chain of error nodes; S7: Map the causal dependency chain to the error correction knowledge base, generate context-sensitive correction suggestions based on the predefined rule engine, and convert them into voice or text feedback through the natural language processing module; S8: Construct a variational autoencoder based on flight state context parameters to perform latent spatial encoding on hand posture data and generate abnormal motion enhancement samples that conform to biomechanical constraints.

[0007] Furthermore, the attitude data stream includes the three-dimensional spatial coordinates of each key joint of the driver's hand, joint angle vectors, and joystick displacement. Kalman filtering noise reduction was performed on the three-dimensional spatial coordinates of key joints of the driver's hand to obtain the filtered joint position sequence; Then, the inverse kinematics algorithm is used to calculate the angle vectors of each finger joint; The joint angle vector and the joystick displacement data are processed in time synchronization to form a multidimensional hand posture data stream.

[0008] Furthermore, in step S2, the standard action template library is layered according to flight mission domain, action level, feature dimension, and associated attributes. The hand posture data stream controlled by an experienced pilot is used as the standard template, and a unique ID is assigned to each template. After being classified according to the layered architecture, the template is entered into the standard action template library and called through a multi-dimensional composite index, including template ID, mission domain, action level, flight altitude, speed, attitude angle, and operation intention code.

[0009] Furthermore, in step S3, the normalized pose feature matrix is ​​input into the lightweight Transformer encoder module. The Transformer encoder module includes a multi-head self-attention mechanism and a position encoding layer. It performs context-aware feature aggregation based on the pose features within the local time window to generate a low-latency temporal feature embedding vector. The embedded vector is subjected to semantic enhancement processing of intent. Cosine similarity is calculated based on a pre-trained standard operation intent template library. Intent labels with matching degree higher than the threshold are selected to generate enhanced feature vectors that fuse semantic labels. An operation intent representation sequence is constructed based on the generated enhanced feature vectors.

[0010] Furthermore, in step S6), the reverse attribution analysis includes calculating the anomaly propagation gradient vector and locating the root cause path leading to the current error. The reverse causal attribution analysis process uses the backpropagation algorithm and Softmax normalization to trace the influence of the abnormal action node back to the predecessor node, performs a depth-first traversal of the dynamic causal graph, outputs the causal dependency chain, and combines it with rules for validity verification.

[0011] Furthermore, in step S7), the correction suggestion includes an error type identifier, a physical cause description, and a sequence of corrective actions. When generating the error correction suggestion, a multilayer perceptron model is used to classify the error type based on the causal dependency chain structure features. The physical cause and the corrective action sequence adapted to the current flight context are obtained through the error correction knowledge base, and finally fed back in the form of text or voice.

[0012] The beneficial technical effects of this invention are as follows: 1) This invention significantly improves the deep semantic understanding of hand manipulation action sequences by introducing a collaborative mechanism between the operation intention recognition submodule and the dynamic causal graph model. 2) This invention utilizes a lightweight temporal neural network to extract high-level semantic representations from continuous posture data and uses them as the basis for dynamically adjusting the weights of causal graph nodes, enabling the system to model the logical dependencies and temporal evolution relationships between typical control actions in real time. 3) This invention continuously updates the causal adjacency matrix through an online learning mechanism. The system can adapt to changes in operation modes at different flight mission stages and accurately identify fundamental error sources such as chain reactions caused by initial attitude deviations or disordered action rhythms, which greatly enhances the interpretability of the evaluation results and the teaching guidance value. 4) This invention constructs a feedback generation system based on causal tracing and context awareness, which significantly improves the intelligence and practicality of human-computer interaction. It maps the identified key error nodes to a predefined error correction knowledge base, performs contextual reasoning in combination with the current flight state context, and uses a rule engine to generate natural language suggestions with physical meaning and operational orientation. 5) To improve the model's generalization ability to rare error scenarios, this invention introduces a variational autoencoder generation mechanism based on pilot historical behavior data. Under the premise of ensuring biomechanical rationality, it synthesizes diverse abnormal action sequences for training set enhancement, thereby effectively alleviating the problem of insufficient model robustness under small sample conditions and further ensuring the stable operation of the system in real complex environments. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention; Figure 2 This is a schematic diagram of the process of generating high-level feature vectors using a Transformer network according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the process of dynamically adjusting the causal weights between nodes in an embodiment of the present invention. Detailed Implementation

[0014] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0015] The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the invention. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0016] like Figure 1 As shown in the figure, this embodiment provides a real-time evaluation method for the standardization of hand operation by UAV pilots based on attitude evaluation, which specifically includes the following steps: S1: Collect the drone pilot's hand posture data stream, which includes the three-dimensional spatial coordinates of each key joint of the pilot's hand, joint angle vectors, and joystick displacement; the sampling frequency is set to 60Hz and a timestamp is added. S2: Based on the standard action template library and flight mission parameters, a dynamic cause-effect graph model is constructed. The model includes push / pull stick action nodes, yaw adjustment nodes, and composite operation nodes. The connection weights between nodes are initialized according to historical standard operation data. S3: Perform sliding window segmentation on the attitude data stream obtained in step S1. Each time window is 200ms long and overlaps by 50ms. After normalization, it is input into a lightweight Transformer network to generate a high-level feature vector containing the semantics of the operation intention. S4: Use the high-level feature vector generated in step S3 as the input condition of the dynamic causal graph model, and dynamically adjust the causal weights between nodes. Specifically, calculate the cosine similarity between the current feature vector and the standard intent template, and update the adjacency matrix element values ​​according to the exponential decay function. S5: Perform forward propagation calculation based on the dynamic causal graph updated in step S4 to generate the probability distribution of the predicted action sequence, and compare it with the real action sequence collected in step S1 to identify abnormal action nodes with deviations exceeding the threshold Δ. S6: For the abnormal action nodes identified in step S5, perform reverse attribution analysis: calculate the abnormal propagation gradient vector, locate the root cause path leading to the current error, and output the causal dependency chain containing the initial deviation node and the chain error nodes. S7: Map the causal dependency chain obtained in step S6 to the error correction knowledge base, generate context-sensitive correction suggestions based on the predefined rule engine, the suggestions include error type identifiers, physical cause descriptions and correction operation sequences, and convert them into voice / text feedback through the natural language processing module; S8: Construct a variational autoencoder based on flight state context parameters, perform latent spatial encoding on the hand posture data collected in step S1, generate abnormal motion enhancement samples that conform to biomechanical constraints, and use them to iteratively optimize the parameters of the dynamic causal graph model.

[0017] In this embodiment, step S1, which involves collecting the drone pilot's hand gesture data stream, specifically includes the following steps: S1.1: Based on a drone simulator or a real flight control platform, deploy a hand attitude sensor array, which includes an inertial measurement unit (IMU) and an optical tracking module to obtain the three-dimensional spatial coordinates of each key joint of the pilot's hand in order to obtain raw spatial trajectory data; Based on the operating environment of a drone simulator or a real flight control platform, a sensor type that matches the hand posture data acquisition requirements is selected as the input condition. The sensor array includes an inertial measurement unit (IMU) and an optical tracking module. The two are combined to simultaneously acquire the hand's acceleration and angular velocity information as well as spatial coordinate information, thereby realizing the acquisition of posture data in different dimensions.

[0018] S1.2: Perform Kalman filtering noise reduction on the three-dimensional spatial coordinate data obtained in S1.1 to eliminate the influence of sensor noise on the attitude estimation accuracy, and output the filtered joint position sequence; S1.3: Based on the filtered joint position sequence output by S1.2, the inverse kinematics algorithm is used to calculate the angle vector of each finger joint to obtain a high-dimensional angular feature representation of the hand posture; S1.4: Configure a joystick displacement sensor on the flight control console to collect the displacement data of the joystick in the X, Y, and Z axes, and convert it into digital signals through an analog-to-digital converter circuit to obtain the physical behavior mapping of the operation input; S1.5: Perform time synchronization processing on the joint angle vector output by S1.3 and the lever displacement data output by S1.4. Based on a unified clock source, add a timestamp to each frame of data to form a multi-dimensional hand posture data stream with time sequence identification.

[0019] S1.6: Perform data packaging and format standardization operations on the timestamped hand pose data stream generated in S1.5, encapsulating it into a unified structured data packet to adapt to the data interface requirements of the subsequent sliding window segmentation and feature extraction modules.

[0020] S1.7: Based on the structured data packets output by S1.6, a data buffer queue is constructed and a 60Hz sampling frequency control logic is set to ensure that the data stream is continuously output at a fixed frequency, so as to maintain the real-time performance of the system and the continuous stability of data acquisition.

[0021] In this embodiment, in step S2, a dynamic cause-effect graph model is constructed based on the standard action template library and flight mission parameters. The model includes push / pull stick action nodes, yaw adjustment nodes, and composite operation nodes. The connection weights between nodes are initialized according to historical standard operation data. Specifically, the steps are as follows: S2.1: Adapt the standard action template library to the mission context based on flight mission parameters. The flight mission parameters include flight altitude, speed, attitude angle and mission type. The standard action template library contains a multi-dimensional set of standard control actions. A subset of action templates suitable for the current flight phase is selected through a mission parameter matching mechanism to construct a mission-oriented set of action nodes. In this embodiment, the standard action template library is layered according to flight mission domain, action level, feature dimension, and associated attributes. The hand posture data stream controlled by an experienced pilot is used as the standard template, and a unique ID is assigned to each template. After being classified according to the layered architecture, the template is entered into the standard action template library and called through a multi-dimensional composite index, including template ID, mission domain, action level, flight altitude, speed, attitude angle, and operation intention code.

[0022] S2.2: Perform action semantic parsing on the task-oriented action node set obtained in S2.1, and extract the logical dependencies of each action node based on the manipulation semantic graph. The logical dependencies include time sequence constraints, control channel coupling relationships and operation intention correlations, so as to form the initial node connection topology of the causal graph.

[0023] S2.3: Initialize the initial connection topology generated in S2.2 with causal weights based on the historical standard operation dataset. The historical standard operation dataset contains the hand posture and stick displacement sequences of multiple pilots in standard flight missions. The conditional probability distribution between each action node is calculated by a statistical causal inference algorithm, thereby generating the initial value of the causal adjacency matrix.

[0024] Furthermore, a statistical causal inference algorithm is used to calculate the conditional probabilities between each action node. The dataset is discretized using the conditional probability formula to unify the action states of each node into a finite set. The state transition frequencies between each node are statistically analyzed to obtain the joint probability distribution matrix. By correcting the partial derivative of causal probability using Bayes' theorem, extreme conditional probabilities caused by sample sparsity can be eliminated.

[0025] Furthermore, by aligning the aforementioned conditional probability matrix with the nodes in the topological structure, initial values ​​for the causal adjacency matrix are generated, with each matrix element corresponding to the causal edge weights and mapped to intervals. Inside.

[0026] S2.4: Bind the causal adjacency matrix generated in S2.3 to the task-oriented action node set in a graph structure to construct a dynamic causal graph model with weight labels. The dynamic causal graph model is represented in the form of a graph structure, where nodes represent control action units, edges represent causal dependencies between actions, and weights represent causal strength, so as to support subsequent dynamic updates based on intent features. S2.5: The dynamic causal graph model constructed in S2.4 is validated and optimized in terms of graph structure. The forward propagation process of the causal graph is simulated based on the action execution sequence in the standard action template library. The consistency between the simulated output and the actual standard action sequence is compared. The weights of the action paths with deviations exceeding the threshold are retrained to improve the initial prediction accuracy of the causal graph model.

[0027] In this embodiment, in step S3, the pose data stream acquired in S1 is segmented using a sliding window, with each time window being 200ms long and overlapping by 50ms. After normalization, the data is input into a lightweight Transformer network to generate high-level feature vectors containing operational intent semantics, such as... Figure 2 As shown, the specific steps include the following: S3.1: Perform sliding window segmentation processing on the raw hand posture data stream acquired in S1. The data stream includes a three-dimensional coordinate sequence, joint angle vectors, and joystick displacement. The time window length is set to 200ms and 50ms overlap is set to capture the local temporal features of continuous control actions.

[0028] S3.2: Perform normalization processing on the data of each time window after the segmentation in S3.1. Perform Z-score standardization based on the mean and standard deviation of each dimension feature to eliminate the influence of the difference in the dimensions of different sensors on the subsequent network modeling, and obtain the standardized attitude feature matrix.

[0029] S3.3: Input the standardized pose feature matrix obtained in S3.2 into the lightweight Transformer encoder module. The module includes a multi-head self-attention mechanism and a position encoding layer. It performs context-aware feature aggregation based on the pose features within the local time window to generate a low-latency temporal feature embedding vector. S3.4: Perform intent semantic enhancement processing on the embedded vectors output by S3.3, calculate cosine similarity based on the pre-trained standard operation intent template library, and filter those with a matching degree higher than a threshold. The intent label is used to generate an enhanced feature vector that integrates semantic labels, thereby improving the ability to understand the operation target; Furthermore, vector normalization is used to ensure the consistency of the dimensions of each feature vector before similarity calculation, and a normalized embedding vector set is obtained.

[0030] Furthermore, a batch similarity calculation method is used to perform matrix multiplication between the embedded vector and the template library matrix, generating a similarity vector of length N, as shown in the following formula: , in, For embedding vectors, For template vector, This represents the similarity value.

[0031] Furthermore, by setting a matching threshold Conditional filtering is performed using a Boolean mask generation method, selecting those elements in the similarity vector that are greater than a certain threshold. The template index position is identified as the matching intent tag index set, and the matching tag set is obtained.

[0032] For example, in a drone training task, the input is a 128-dimensional embedding vector output by a lightweight Transformer encoder. The standard operational intent template library contains 50 templates, each a 128-dimensional vector, and a matching threshold. The similarity coefficient was set to 0.85. After normalization, matrix multiplication was used to calculate the similarity, generating a similarity vector of length 50. Assuming the similarity of the 6th, 15th, 22nd, and 35th templates is higher than 0.85, the matching tag index set is {6, 15, 22, 35}. The corresponding intent tag encoding has a total dimension of 64, and after vector concatenation, the enhanced feature vector has a dimension of 192. This enhanced feature vector significantly improves the node weight matching accuracy during subsequent dynamic causal graph weight adjustment. The system's intent recognition accuracy during the task phase is greatly improved. In training validation, it can stably and in real-time determine the pilot's action intent and support the generation of targeted correction suggestions.

[0033] S3.5: Construct an operation intent representation sequence based on the enhanced feature vector generated in S3.4. The representation sequence is arranged in chronological order and appended with task stage identifiers. It serves as the action-driven input for the dynamic causal graph model and is used to adjust the causal weights between nodes in real time to support subsequent action sequence prediction and anomaly identification.

[0034] In this embodiment, in step S4, the high-level feature vector generated in S3 is used as the input condition of the dynamic causal graph model to dynamically adjust the causal weights between nodes, such as... Figure 3 As shown, the specific steps include the following: S4.1: Obtain the high-level feature vector and standard intent template library output by S3. The high-level feature vector is a 128-dimensional temporal encoding vector output by the lightweight Transformer network. The standard intent template library contains typical operation intent templates predefined based on flight mission parameters. Each template is in the form of a 128-dimensional vector to support subsequent intent matching calculation. S4.2: Based on the high-level feature vector and standard intent template obtained in S4.1, calculate the cosine similarity between the current operation intent and each template to quantify the degree of matching between the current operation behavior and the predefined intent, and output a similarity vector with dimension N, where N is the number of intent templates in the template library, which is used as the input basis for subsequent dynamic weight adjustment. S4.3: Based on the similarity vector output by S4.2, the adjacency matrix elements in the dynamic causal graph are updated with weights using an exponential decay function. The exponential decay function takes similarity as input and outputs a weight adjustment factor, which strengthens the connection between action nodes in the adjacency matrix that have a high degree of matching with the current intent, and weakens it otherwise, thereby realizing the dynamic reconstruction of the causal graph structure. Using an exponential decay function Construct a weight adjustment factor to achieve the matching degree The mapping is to edge weight gain, and the function form is: , In the formula, For matching degree; This is the attenuation coefficient.

[0035] Furthermore, by element-wise matrix multiplication, the weight adjustment factor matrix is ​​multiplied by the corresponding elements of the current adjacency matrix to obtain the updated adjacency matrix. The formula is: ; in, To update the previous adjacency matrix, For the adjustment factor matrix, the symbols are... This represents the Hadamard product.

[0036] Furthermore, by judging the distribution gradient of the weight adjustment factor on each side, matching degrees higher than a threshold are identified. The nodes are subjected to edge weight enhancement constraints, which increase the connection strength to the weighted sum of the initial value and the gain value.

[0037] Furthermore, exponential decay updates are performed on edges with matching degrees below a threshold, which reduces the weights between nodes with irrelevant actions to weaken the causal influence of non-target paths and ensure that the causal graph structure is closer to the real state driven by the current task operation intent.

[0038] S4.4: Based on the adjacency matrix updated in S4.3, and combined with the initial causal weights of the action nodes in the dynamic causal graph and the current flight mission parameters, the causal propagation path is recalculated to generate an updated causal dependency graph, which contains the real-time causal intensity distribution between each action node, providing structured input for subsequent action prediction. S4.5: Normalize the causal dependency graph generated in S4.4 to eliminate the impact of uneven weight distribution among nodes on subsequent prediction modules, and obtain a standardized dynamic causal graph model, which serves as the input model for action sequence prediction and anomaly identification in S5, thereby improving the system's adaptability to flight mission context and evaluation accuracy.

[0039] In this embodiment, in step S5, forward propagation calculation is performed based on the dynamic causal graph updated in S4 to generate the probability distribution of the predicted action sequence, and it is compared with the real action sequence collected in S1 to identify abnormal action nodes with deviations exceeding the threshold Δ. Specifically, the steps include the following: S5.1: Based on the dynamic causal graph model updated by S4, forward propagation calculation is performed on the operational intent feature vector of the current flight mission phase to generate the probability distribution of the expected action node in the next moment. S5.2: Perform Softmax normalization on the probability distribution output by S5.1 to obtain the normalized probability value of each candidate action node, which is used to represent the prediction confidence of the current causal graph for the next action. Using the predicted action node probability distribution matrix output by S5.1 as input, the Softmax normalization algorithm (parameter: temperature coefficient T) is used to convert the original unnormalized score vector into a probability value vector.

[0040] Furthermore, the exponential activation value of each candidate node is calculated using an exponential function, employing the following formula: , in, For the first The raw scores of each candidate action node. For temperature coefficient, Here, M represents the normalized probability value; M is the number of candidate nodes. Global normalized coefficients are generated by summing the denominators, where the summation term in the denominator must cover all... The exponential activation values ​​of each candidate node are used to ensure that the sum of the probability values ​​is 1.

[0041] For example, in the scenario of evaluating the hovering attitude stability of a drone, for Each candidate action node (push-off fine-tuning, pull-off fine-tuning, left yaw correction, right yaw correction), temperature coefficient Set to 0.8, the original score vector output by S5.1 is [2.0, 1.0, 0.5, -0.5]. Using a maximum value shifting process, subtracting the maximum value of 2.0 from each element yields [0.0, -1.0, -1.5, -2.5]. Substituting into the Softmax formula: dividing each element by T=0.8 and taking the exponent, we get the exponent activation values ​​[1.0, 0.2865, 0.1534, 0.0439]; the sum of the denominators is 1.4838, and the normalized probability vector is [0.674, 0.193, 0.103, 0.029]. This result shows that the causal graph predicts the next action most likely as a push lever adjustment with a confidence level of 0.674, followed by a pull lever adjustment with a confidence level of 0.193. This distribution provides a quantitative basis for prediction of downstream S5.3 action alignment and deviation calculation, significantly improving the effectiveness of the anomaly identification module in attitude stability tasks.

[0042] S5.3: After the real action sequence collected in S1 is segmented by a sliding time window, it is aligned node by node with the predicted action probability distribution output in S5.2 to construct the action deviation vector between prediction and reality; S5.4: Perform L2 norm calculation on the action deviation vector generated in S5.3 to quantify the overall deviation between the current action node and the prediction result, and compare it with the preset threshold Δ.

[0043] S5.5: Based on the comparison results of S5.4, identify action nodes whose deviation exceeds the threshold Δ, mark them as abnormal action nodes, and output their node identifier, deviation value, and timestamp information to the subsequent attribution analysis module.

[0044] In this embodiment, in step S6, for the abnormal action nodes identified in S5, reverse attribution analysis is performed: the abnormal propagation gradient vector is calculated, the root causal path leading to the current error is located, and the causal dependency chain containing the initial deviation node and the chain of error nodes is output. Specifically, the steps are as follows: S6.1: Based on the abnormal action nodes identified in S5 and their corresponding deviation vectors, an abnormal propagation gradient model is constructed. The model uses the backpropagation algorithm to perform sensitivity analysis on the node connection weights in the dynamic causal graph in order to quantify the influence of each predecessor node on the current abnormal node. S6.2: Perform normalization processing on the anomaly propagation gradient model constructed in S6.1, and use the Softmax function to normalize the influence weights of each predecessor node to obtain a standardized causal contribution distribution vector. S6.3: Based on the causal contribution distribution vector obtained in S6.2, perform causal path backtracking operation, use the depth-first search algorithm to traverse the dynamic causal graph, identify the causal dependency chain from the initial deviation node to the current abnormal node, so as to form a set of causal paths containing key error propagation paths; S6.4: Perform semantic parsing on the set of causal paths identified in S6.3, extract the action type identifier, time offset and causal strength value of the nodes in each path, so as to generate structured causal dependency chain description information; S6.5: Based on the structured causal dependency chain description information generated in S6.4, combined with the context parameters of the flight mission phase, the causal chain validity verification operation is performed. A rule-based reasoning mechanism is used to eliminate false causal paths caused by environmental interference or sensor noise, so as to output the final causal dependency chain result.

[0045] In this embodiment, in step S7, the causal dependency chain obtained in S6 is mapped to the error correction knowledge base, and context-sensitive correction suggestions are generated according to the predefined rule engine. The suggestions include error type identifiers, physical cause descriptions, and correction operation sequences, which are then converted into voice / text feedback by the natural language processing module. Specifically, the steps include the following: S7.1: Perform topological structure parsing on the causal dependency chain output by S6, extract the directed edge connection relationship between the initial deviation node and the chain error node, and identify the error propagation path based on graph structure semantics to obtain the structural feature vector of the causal dependency graph. S7.2: Perform error pattern classification on the structural feature vector of the causal dependency graph. Use a pre-trained multilayer perceptron model to classify the feature vector to identify whether the current error belongs to the posture deviation class, action sequence class, execution rhythm class or compound error class, and output the error type identifier. S7.3: Based on the error type identifier and flight state context parameters, perform a multi-dimensional index query on the error correction knowledge base. The flight state context parameters include altitude, speed, and attitude angle. The error correction knowledge base stores the mapping relationship between error types and physical causes to extract the corresponding physical cause description text. S7.4: Based on the physical cause description text and the operation sequence template in the standard action template library, perform semantic alignment and path planning to generate a corrected operation sequence that conforms to the current flight state constraints. The corrected operation sequence includes action type, execution order and time window constraint parameters. S7.5: Perform natural language generation processing on the corrected operation sequence, convert the structured operation sequence into a natural language description based on the pre-trained sequence-to-sequence model, and select the output format according to the feedback channel type to generate corrected suggestion feedback in the form of speech or text. In this embodiment, in step S8, a variational autoencoder is constructed based on flight state context parameters to perform latent spatial encoding on the hand posture data collected in S1, generating abnormal action enhancement samples that conform to biomechanical constraints, which are used to iteratively optimize the parameters of the dynamic causal graph model. Specifically, the steps include the following: S8.1: Based on flight state context parameters, construct a context-aware variational autoencoder architecture. The flight state context parameters include flight altitude, speed, attitude angle and joystick displacement, which are used to generate potential hand action representations that match the current flight phase, so as to improve the scene adaptability of sample generation. S8.2: Perform normalization processing on the hand posture data collected in S1. The hand posture data includes a three-dimensional coordinate sequence, joint angle vectors and lever displacement, to obtain a standardized input vector, which is used as the training input of the variational autoencoder to improve the model convergence efficiency. S8.3: The variational autoencoder is used to perform latent space encoding on the normalized hand pose data. The mean vector and variance vector are generated through the encoder network to construct the latent distribution representation of hand movements for subsequent sample generation and enhancement. S8.4: Based on the latent distribution representation and combined with flight state context parameters, a sampling operation is performed to generate abnormal action enhancement samples with biomechanical constraints to simulate typical erroneous actions that the pilot may make under specific flight conditions, thereby expanding the diversity of the training set; S8.5: The generated abnormal action enhancement samples are fed back to the dynamic causal graph model. Based on the action sequence and causal dependency of the abnormal samples, parameter fine-tuning and adjacency matrix updates are performed to improve the model's ability to identify and attribute rare errors, thereby optimizing the generalization performance of the overall evaluation system.

[0046] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

[0047] Unless otherwise defined, the technical or scientific terms used herein should be understood in their ordinary sense by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” “third,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the elements or objects preceding “comprising” or “including” encompass the elements or objects listed following “comprising” or “including” and their equivalents, and do not exclude other elements or objects. The “multiple” involved in the embodiments of this invention refers to two or more. A and / or B indicate three possibilities: A; B; and A and B.

[0048] The above description is merely an exemplary embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A real-time evaluation method for the standardization of hand control by UAV pilots based on attitude assessment, characterized in that, Includes the following steps: S1: Collect the hand posture data stream of the drone pilot; S2: Based on the standard action template library and flight mission parameters, construct a dynamic cause-effect graph model; wherein, the standard action template library is layered according to flight mission domain, action level, feature dimension, and associated attributes, with the hand posture data stream controlled by senior pilots as the standard template, and each template is assigned a unique ID, and is entered into the standard action template library after being classified according to the layered architecture. S3: Perform sliding window segmentation on the pose data stream, and after normalization, input it into a lightweight Transformer network to generate high-level feature vectors containing the semantics of the operation intention; S4: Use high-level feature vectors as input conditions for the dynamic causal graph model to dynamically adjust the causal weights between nodes; S5: Perform forward propagation computation based on the updated dynamic causal graph to generate the probability distribution of the predicted action sequence, compare it with the real action sequence, and identify abnormal action nodes with deviations exceeding the threshold. S6: For the identified abnormal action nodes, perform reverse attribution analysis and output a causal dependency chain containing the initial deviation node and the chain of error nodes; S7: Map the causal dependency chain to the error correction knowledge base, generate context-sensitive correction suggestions based on the predefined rule engine, and convert them into voice or text feedback through the natural language processing module; S8: Construct a variational autoencoder based on flight state context parameters to perform latent spatial encoding on hand posture data and generate abnormal motion enhancement samples that conform to biomechanical constraints.

2. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S1, the attitude data stream includes the three-dimensional spatial coordinates of each key joint of the driver's hand, the joint angle vector, and the displacement of the control stick. Kalman filtering noise reduction was performed on the three-dimensional spatial coordinates of key joints of the driver's hand to obtain the filtered joint position sequence; Then, the inverse kinematics algorithm is used to calculate the angle vectors of each finger joint; The joint angle vector and the joystick displacement data are processed in time synchronization to form a multidimensional hand posture data stream.

3. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S2, the flight mission parameters include flight altitude, speed, attitude angle and mission type. The standard action template library contains a multi-dimensional set of standard control actions. A subset of action templates suitable for the current flight phase is selected through a mission parameter matching mechanism to construct a mission-oriented set of action nodes.

4. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 3, characterized in that, Perform action semantic parsing on the set of task-oriented action nodes, extract the logical dependencies of each action node based on the manipulation semantic graph, and form the initial node connection topology of the causal graph. The initial connection topology is initialized with causal weights based on the historical normal operation dataset. The conditional probability distribution between each action node is calculated by a statistical causal inference algorithm to generate the initial value of the causal adjacency matrix. The causal adjacency matrix is ​​bound to the task-oriented action node set in a graph structure to construct a dynamic causal graph model with weight labels, and the dynamic causal graph model is verified and optimized in terms of graph structure. The dynamic causal graph model is represented in the form of a graph structure, where nodes represent manipulation action units, edges represent causal dependencies between actions, and weights represent causal strength.

5. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S3, the normalized pose feature matrix is ​​input into the lightweight Transformer encoder module. The Transformer encoder module includes a multi-head self-attention mechanism and a position encoding layer. It performs context-aware feature aggregation based on the pose features within the local time window to generate a low-latency temporal feature embedding vector. The embedded vector is subjected to semantic enhancement processing of intent. Cosine similarity is calculated based on a pre-trained standard operation intent template library. Intent labels with matching degree higher than the threshold are selected to generate enhanced feature vectors that fuse semantic labels. An operation intent representation sequence is constructed based on the generated enhanced feature vectors.

6. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S4, the high-level feature vector and standard intent template library obtained in step S3 are obtained. The high-level feature vector is the temporal encoding vector output by the lightweight Transformer network, and the standard intent template library contains typical operation intent templates predefined based on flight mission parameters.

7. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 6, characterized in that, In step S4, the details are as follows: Based on high-level feature vectors and standard intent templates, the cosine similarity between the current operation intent and each template is calculated to quantify the degree of matching between the current operation behavior and the predefined intent, and a similarity vector is output. Based on the similarity vector, the adjacency matrix elements in the dynamic causal graph are updated using an exponential decay function with weights. Based on the updated adjacency matrix, and combined with the initial causal weights of the action nodes in the dynamic causal graph and the current flight mission parameters, the causal propagation path is recalculated to generate an updated causal dependency graph. The causal dependency graph is normalized to obtain a standardized dynamic causal graph model.

8. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S5, based on the updated dynamic causal graph model, forward propagation calculation is performed on the operational intent feature vector of the current flight mission phase to generate the probability distribution of the expected action node at the next moment. Softmax normalization is performed on the probability distribution to obtain the normalized probability value of each candidate action node, which is used to represent the prediction confidence of the current causal graph for the next action. After the collected real action sequences are segmented by a sliding time window, they are aligned node by node with the predicted action probability distribution to construct the action deviation vector between the prediction and the actual action. The L2 norm is calculated on the action deviation vector to quantify the overall degree of deviation between the current action node and the prediction result, and then compared with a preset threshold. Based on the comparison results, action nodes with deviations exceeding the threshold are identified and marked as abnormal action nodes.

9. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S6), the reverse attribution analysis includes calculating the anomaly propagation gradient vector and locating the root cause path leading to the current error. The reverse causal attribution analysis process uses the backpropagation algorithm and Softmax normalization to trace the influence of the abnormal action node back to the predecessor node, performs a depth-first traversal of the dynamic causal graph, outputs the causal dependency chain, and combines it with rules for validity verification.

10. The real-time evaluation method for the standardization of UAV pilot hand control based on attitude evaluation according to claim 1, characterized in that, In step S7), the correction suggestion includes an error type identifier, a physical cause description, and a sequence of corrective actions. When generating the error correction suggestion, a multilayer perceptron model is used to classify the error type based on the causal dependency chain structure features. The physical cause and the corrective action sequence adapted to the current flight context are obtained through the error correction knowledge base, and finally fed back in the form of text or voice.

Images