Unmanned aerial vehicle delivery authentication method and device based on implicit gait behavior

Abstract

The application discloses a kind of unmanned aerial vehicle delivery authentication method and device based on implicit gait behavior, and user places order through mobile phone delivery application program.Reaching specified place, unmanned aerial vehicle hovers in the air, informs user its arrival, then establishes key protection communication channel with user mobile phone.User hand-held mobile phone goes to specified position.Unmanned aerial vehicle confirms the real-time position of user by sharing GPS data, and searches user using camera.At the same time, unmanned aerial vehicle captures user video and processes in real time, extracts the time series of user body key point.At the same time, the mobile phone of user constantly collects time series data from its IMU sensor.Then, the two data are exchanged between unmanned aerial vehicle and user mobile phone, so as to start mutual identity authentication process.The application utilizes hand movement mode when user walks to unmanned aerial vehicle to realize mutual identity authentication, without additional hardware or specific identity authentication operation, and shows strong resistance to various fraud attacks.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of information security, and relates to a UAV delivery two-way authentication method and device, in particular to a UAV delivery authentication method and device based on implicit gait behavior. BACKGROUND

[0002] Unmanned aerial vehicle (UAV) delivery is the use of unmanned aerial vehicles to deliver and pick up packages, which has the advantages of being fast and efficient, cost-saving, having a small environmental impact, and improving logistics coverage, and has become a trend in the development of the logistics industry. Although UAV delivery brings many conveniences, its security and privacy protection in the delivery process still face serious challenges, especially how to effectively verify the identity of users and UAVs.

[0003] To solve the above technical problems, existing authentication schemes are mainly focused on the following two directions:

[0004] (1) Feature-based authentication technology: The research direction is committed to achieving this by analyzing and identifying unique signal features emitted by UAVs. These features can be the sound of the UAV flying, the gait behavior of the user, or other quantifiable feature indicators. Combined with modern deep learning algorithms, it can adapt to different environmental and operating conditions, thereby improving the security and reliability of the UAV delivery process.

[0005] Chinese patent document No. CN118447856A, published on August 6, 2024, describes a UAV identity authentication method based on acoustic features. The invention collects sound samples from a specific microphone through a data acquisition module and samples and evaluates them at preset time intervals. The collected sound samples are divided into training and test sets after data enhancement. A feature extraction module is used to extract digital features from the sound samples to form a feature set. Then, an authentication model based on a CNN-Transformer hybrid network is constructed, the training set is trained to obtain the authentication model, and finally the test set is tested to achieve UAV identity authentication. This method uses a deep learning structure to significantly improve the accuracy and reliability of UAV identity authentication, and can effectively identify and analyze the sound signals emitted by the UAV. However, this technology has the following problems: First, this authentication method relies on a specific microphone for sound collection, which may result in environmental noise affecting data collection and thus affecting the accuracy and reliability of acoustic features. In complex environments, external interference may cause inconsistencies in sound samples, thereby reducing the performance of the authentication model. Second, this authentication method requires effective authentication within a relatively short distance, which may limit its application in large-scale or open spaces. The short authentication distance may not meet the needs of some practical scenarios, affecting user experience and the universality of the application.

[0006] (2) Wireless communication authentication technology: The research direction is committed to exploring various wireless communication means to ensure the accurate verification of the identity of the UAV. It mainly includes Bluetooth signals, WIFI signals and other detectable signals. The research focuses on the applicability of different communication protocols, combined with factors such as signal strength, transmission delay and data packet integrity, to meet the needs of different environments and application scenarios.

[0007] Chinese patent document number CN117295062A, published on December 26, 2023, describes a UAV identity ID remote identification system and method based on Bluetooth 5.0. The invention collects data in authorized and unauthorized areas, writes UAV authentication information in the remote ID device to make area judgments. In the unauthorized area, the first public key is used to encrypt and broadcast the second identity information; in the authorized area, the third identity information is encrypted and broadcasted using the authentication public key. This technology has the following problems: First, the authentication method relies on Bluetooth 5.0 technology for identity recognition, which is limited by the propagation characteristics of Bluetooth signals. In environments with strong signal interference or many obstacles, the effective range of Bluetooth signals may be reduced, affecting the reliability and real-time performance of the authentication. Second, the authentication system needs to combine multiple components (such as Bluetooth devices, area judgment modules, etc.), which may increase the overall complexity of the system. In addition, the cost of maintaining and updating these components may be high, affecting the sustainable development of the system.

[0008] At the same time, existing authentication schemes are mostly focused on one-way authentication of users. For example, through scanning of user mobile phone two-dimensional code, one-time digital token, fingerprint and face recognition, etc. One-way authentication technology scheme verifies the identity of the recipient, etc. These one-way authentication schemes have a significant problem: they only verify the user's identity, ignoring the possibility that a malicious UAV may impersonate a legitimate sender UAV to steal packages. In addition, attackers may use fake or relay attacks to take over or interfere with the delivery UAV, which may pose a security risk to the system. For example, in the two-dimensional code, digital token or biometric identification scheme, a malicious UAV can intercept the authentication data sent by the user through a relay attack and replay it, resulting in user information leakage, package hijacking, etc. SUMMARY

[0009] To solve the problem of identity authentication in the UAV delivery process and ensure the safety and reliability of delivery, a UAV delivery authentication method and device based on implicit gait behavior are provided.

[0010] The technical solution adopted by the method of the present invention is: a UAV delivery authentication method based on implicit gait behavior, comprising the following steps:

[0011] Step 1: Collect IMU sensor data and user's time series gait data when the user and the drone perform mutual authentication;

[0012] Step 2: Normalize and denoise the IMU sensor data and the time series gait data;

[0013] Step 3: Perform spatio-temporal consistency conversion on the IMU sensor data and the time series gait data processed in Step 2;

[0014] Step 4: Extract time domain and frequency domain features from the IMU sensor data and the time series gait data processed in Step 3;

[0015] Step 5: Input the time domain and frequency domain features into a one-class classifier to perform consistency authentication between the IMU sensor data and the hand key point data;

[0016] Convert the mobile phone IMU sensor data into Euler angle vectors and combine it with accelerometer data to represent the user's gait behavior data; then input it into the feature extractor and one-class classifier for gait recognition authentication;

[0017] Step 6: If both authentications pass, mutual authentication is successful; otherwise, the drone adjusts its position according to the distance from the user and returns to Step 1 to repeat the process until the maximum number of attempts is reached; if it still fails to authenticate, the user is rejected.

[0018] As a preferred embodiment, in Step 1, when the user and the drone perform mutual authentication, the user's mobile phone collects IMU sensor data; the drone's camera records the user's gait information, performs human pose detection, identifies and matches the hand key points of the user holding the mobile phone, and extracts the time series data of the key points, thereby converting the visual-based gait information into time series-based gait data.

[0019] As a preferred embodiment, in Step 2, the IMU sensor data is down-sampled to match the sampling rate of the time series data of the key points.

[0020] As a preferred embodiment, in Step 2, an adaptive cutoff frequency filtering method is used to denoise the IMU sensor data and the time series gait data; first, the fast Fourier transform is used to analyze the frequency spectrum characteristics of the time series to determine the frequency range of the energy principal components, and then the high-pass frequency and low-pass frequency of the filter are set to the upper and lower limits of the frequency range, respectively.

[0021] As a preferred embodiment, in Step 2, adaptive discrete cosine transform and multi-joint cooperative Kalman filtering are applied to adaptively filter out abnormal data caused by occlusion;

[0022] The adaptive discrete cosine transform utilizes a discrete cosine transform to eliminate signal abnormalities caused by body occlusion; in order to realize dynamic adjustment of reserved discrete cosine transform coefficients , an entropy-based adaptive adjustment method is adopted to calculate K , ; wherein is an initial reserved coefficient, is an adaptive adjustment parameter, is an information entropy value of gait data, is a length of gait data;

[0023] The multi-joint cooperative Kalman filter integrates the information of a plurality of adjacent key points into a state vector for the data processed by the adaptive discrete cosine transform, so as to realize smoothing processing of a single joint motion trajectory.

[0024] As a preferred, in step 3, a clock synchronization protocol is run, and a Kalman filter is combined to process the fluctuation of network delay, estimate the trend of clock offset change, and perform space-time consistency conversion on the IMU sensor data processed in step 2 and the time series-based gait data.

[0025] The specific implementation includes the following sub-steps:

[0026] Step 3.1: Fusion of user IMU sensor data , wherein respectively represent the coordinate data of the accelerometer, gyroscope and magnetometer in the mobile phone coordinate system , for the data at a certain time , the coordinate data of the accelerometer, gyroscope and magnetometer in the mobile phone coordinate system is recorded as , , ; based on the above data, the direction quaternion and the azimuth angle, pitch angle and roll angle are obtained;

[0027] The specific implementation includes the following sub-steps:

[0028] (1) Calculate the quaternion at the initial time;

[0029] The initial attitude angle is calculated from the accelerometer and magnetometer data: , , ;

[0030] Then, the angles are converted into the initial quaternion :

[0031] ;

[0032] (2) Accelerometer and magnetometer data normalization and magnetic field direction calculation;

[0033] For any time , starting from the quaternion of the previous time, the current attitude is calculated; first, the accelerometer data and magnetometer data at the current time are normalized; through normalization, the acceleration vector of unit length is obtained; the magnetometer data is normalized to obtain the magnetic field vector of unit length;

[0034] (3) Using normalized magnetometer data and the current quaternion , the direction of the earth's magnetic field under the current attitude is calculated;

[0035] First, the magnetic field direction vector is calculated through quaternion multiplication; where is the conjugate of the quaternion , and represents the multiplication of quaternions; the obtained vector describes the components of the magnetic field direction under the current quaternion attitude ; through , the horizontal component and the vertical component of the magnetic field are extracted, and the final magnetic field direction vector is calculated; the vector describes the direction of the earth's magnetic field under the current attitude;

[0036] (4) Use gradient descent algorithm to correct the quaternion; the correction vector is calculated as follows:

[0037] ;

[0038] (5) Calculate the Jacobian matrix , which provides the partial derivative of each quaternion component with respect to the objective function, to guide the correction process of the quaternion;

[0039] ;

[0040] (6) Use the Jacobian matrix and the correction vector to calculate the correction step , and further obtain the normalized correction step ;

[0041] (7) Calculate the rate of change of the quaternion;

[0042] The rate of change of the quaternion caused by the gyroscope data is ; then calculate the total rate of change of the quaternion , where is an adjustment parameter used to control the response speed of the algorithm;

[0043] (8) Integrate to get the new quaternion;

[0044] Using the sampling period , calculate the update amount of the quaternion estimate in each iteration , further obtain the normalized new quaternion , and finally obtain the direction quaternion ;

[0045] (9) Calculate the rotation angle, i.e. azimuth, pitch and roll, from the new quaternion:

[0046] ;

[0047] Obtain the time series of the rotation angle ;

[0048] Step 3.2: Multiply the acceleration vector in the phone coordinate system by the direction quaternion on the left and by the quaternion conjugate on the right to complete the conversion to the world coordinate system: ;

[0049] Step 3.3: Calculate the motion speed of the hand according to the IMU sensor data; set the initial speed at the initial time to be zero, then at time , the formula for calculating the motion speed of the hand is: ; where represents the speed of the hand at time , represents the acceleration of the hand at time , is the sampling rate of the accelerometer, is the time step;

[0050] Step 3.4: Perform spatial consistency processing on the IMU sensor data; the main frequency components of the IMU sensor data include gravity change , user walking component and hand motion component , and the velocity is decomposed into ; an adaptive cutoff filtering method is used to dynamically adjust the filtering threshold to eliminate and and then the hand key point velocity is normalized to obtain the hand velocity in the body coordinate system The hand key point velocity is compared with the velocity of the corresponding hand key point in the body posture coordinate system to ensure the spatial consistency of the comparison data.

[0051] In processing the data collected by the unmanned aerial vehicle, the hand key point position is mapped to the body-centered coordinate system, and the velocity of the corresponding key point is calculated; after the hand key point position information is converted to the human-centered coordinate system, the velocity of the hand key point is obtained by calculating the change of the hand key point position between two consecutive frames and dividing by the time interval.

[0052] As preferred, in step 4, 12 features are selected from the time domain and frequency domain, and the Fisher score of all features is calculated to screen out the most discriminative features; finally, four time domain features and two frequency domain features are selected, and the processed data sample forms a six-dimensional feature vector.

[0053] As preferred, in step 5, the feature extractor is composed of sequentially connected input single-channel convolutional layers, ResNet and bidirectional LSTM network; the ResNet includes four residual blocks, each block contains two convolutional layers, a batch normalization layer is integrated after each convolutional layer to speed up model convergence, and a ReLU activation function is applied to introduce nonlinearity; the output of the last residual block is converted into a fixed-size feature vector through a global average pooling layer; the output of the ResNet and the hidden state of the bidirectional LSTM network are spliced in the feature dimension, and finally a fully connected layer and a Softmax layer are added for multi-class classification.

[0054] As preferred, in step 5, the one-class classifier is a model trained through supervised learning; in the training process, a training data set containing labels is used, which is composed of sensor data of different users, and each sample is attached with a label identifying different users to guide the model to learn how to distinguish users.

[0055] In the training process, features are extracted from the gait data of the user, the model is used for feature analysis and extraction of the gait data, the extracted features are input into the supervised learning classifier for forward propagation to obtain the prediction result, and the label data of the user is compared with the prediction result; the cross-entropy loss function is used to optimize the model, the gradient is calculated for back propagation, and the model weights are updated step by step to improve the recognition accuracy of the model for "legitimate user" and "unauthorized user" samples.

[0056] ​After pre-training is completed, a transfer learning method is used to delete the supervised learning classifier of the model, retain the model weight of the forward layer as a feature extractor, put a small amount of unlabeled data into the feature extractor for feature extraction in a real user registration scene, and use the extracted features to train a single-class classifier, thereby completing the registration of the user authentication model.

[0057] The technical scheme of the device of the application is: an unmanned aerial vehicle delivery authentication device based on implicit gait behavior, comprising:

[0058] One or more processors;

[0059] A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the implicit gait behavior-based unmanned aerial vehicle delivery authentication method.

[0060] Compared with the prior art, the beneficial effects of the application include:

[0061] 1. High accuracy: the application can achieve efficient user and unmanned aerial vehicle mutual authentication by utilizing the unique hand movement features in the user's gait, showing extremely high accuracy, ensuring the effectiveness and security of authentication.

[0062] 2. Strong robustness: the application exhibits superior robustness under various environmental conditions, can adapt to different light, shielding and various device configurations, and ensures the reliability in actual application.

[0063] 3. Effective resistance to attacks: the application can effectively resist various types of attacks, including wireless relay, device hijacking and imitation, etc., ensuring the secure communication between the user and the unmanned aerial vehicle and improving the security of the overall system.

[0064] 4. User friendliness: the design of the application reduces user intervention in the authentication process, and the user only needs to naturally walk towards the unmanned aerial vehicle, improving the user experience and making the authentication process more smooth and intuitive.

[0065] 5. Wide application potential: since the application does not rely on additional hardware and can achieve authentication at a relatively long distance, the application has wide application prospects and is suitable for various unmanned aerial vehicle logistics scenarios, promoting the safe development of the industry. BRIEF DESCRIPTION OF DRAWINGS

[0066] The technical scheme of the application is further illustrated below using examples and specific embodiments. In addition, some drawings are also used in the process of explaining the technical scheme. For those skilled in the art, other drawings and the intent of the application can also be obtained from these drawings without creative labor.

[0067] Figure 1 Method flowchart of the embodiment of the present application;

[0068] Figure 2 Single-class classifier training flowchart of the embodiment of the present application;

[0069] Figure 3 Consistency verification, gait authentication based on IMU and ROC curve of the overall performance of the method in the experiment of the present embodiment;

[0070] Figure 4 Influence of the hovering height and authentication distance of the unmanned aerial vehicle on the consistency verification of the present application in the experiment of the present embodiment. DETAILED DESCRIPTION

[0071] In order to facilitate those skilled in the art to understand and implement the present application, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0072] In order to overcome the problems existing in the related art, the present embodiment provides a method and device for unmanned aerial vehicle delivery authentication based on implicit gait behavior. The user places an order through a delivery application on his / her mobile phone. After arriving at the designated location, the unmanned aerial vehicle hovers in the air and notifies the user of its arrival through SMS, telephone, etc. Then, a key-protected communication channel is established with the user's mobile phone. The user walks to the designated location with the mobile phone in hand. The unmanned aerial vehicle confirms the real-time position of the user by sharing GPS data and searches for the user using a camera. In this process, the unmanned aerial vehicle captures the user's video and processes it in real time to extract the time series of the user's body key points. At the same time, the user's mobile phone continuously collects time series data from its IMU sensor. Then, the two data are exchanged between the unmanned aerial vehicle and the user's mobile phone, thereby starting the mutual identity authentication process.

[0073] See Figure 1 The method for unmanned aerial vehicle delivery authentication based on implicit gait behavior provided by the present embodiment includes the following steps:

[0074] Step 1: When the user and the unmanned aerial vehicle perform mutual authentication, the user's mobile phone collects IMU sensor data; the unmanned aerial vehicle camera records the user's gait information, performs human posture detection, identifies and matches the hand key points of the user holding the mobile phone, and extracts the time series data of the key points, thereby converting the visual-based gait information into time series-based gait data;

[0075] The data of the IMU sensor includes time-series gait behavior information, such as the data of the accelerometer, gyroscope, and magnetometer, while the video data records the visual gait information of the user. By performing human pose detection on the video data, the key points of the user's hand holding the phone are identified and matched, and the time-series data of the key points are extracted, thereby converting the visual-based gait information into time-series gait information. The hand movement speed information of the user holding the phone is used as an important basis for verifying the consistency of the two sides of the data.

[0076] In the registration phase, the user needs to hold the smart phone and walk forward at a normal pace according to his daily habits. The system collects the inertial measurement unit (IMU) data of the user in real time. During this process, the sensors of the smart phone will record the output data of the accelerometer, gyroscope, and magnetometer to obtain the gait characteristics of the user. In order to ensure the comprehensiveness of the data, the user needs to complete data collection at different walking paths and speeds, such as straight walking and slight turning.

[0077] In the user authentication process, in addition to collecting IMU data, the system also needs to use the camera of the unmanned aerial vehicle to record video of the user and obtain the visual gait information of the user. At this time, the IMU data and video data will be collected synchronously to form time-series data for subsequent consistency verification. In this phase, a time synchronization protocol needs to be used, and a Kalman filter is used to solve the problem of network delay fluctuation to estimate the trend of clock offset change. In the video collection segment, the YOLOv7 target detection algorithm is used to detect the phone and compare it with the center coordinates of the user to obtain the left and right hand information of the user holding the phone.

[0078] By extracting the time-series data of the user's body key points, the system can convert the visual gait information into spatio-temporal gait information, and use the hand speed holding the phone as an important indicator for consistency verification between the IMU data and the video data. The system can only retain the time-series data of the user's body key points and discard the original video data, thereby protecting the user's privacy during data transmission.

[0079] Step 2: Normalize and denoise the IMU sensor data and time-series gait data;

[0080] In one embodiment, a wavelet denoising algorithm is used to process the data to improve the accuracy of the data and minimize noise interference. Wavelet denoising can effectively filter out noise components unrelated to user motion, thereby improving the overall quality of the time-series data.

[0081] Considering that the video-derived keypoint time series data is sampled at 60Hz, while the IMU data is sampled at 100Hz, the IMU data needs to be down-sampled to match the sampling rate of the keypoint sequence, and the data needs to be normalized separately.

[0082] Since a user typically exhibits about 1 to 3 gait cycles within a second, the corresponding dominant frequency range is 1 to 3 Hz. In an embodiment, a Butterworth filter is used for denoising. To optimize the filter parameters, a Fast Fourier Transform (FFT) is used to analyze the spectral characteristics of the time series data to determine the frequency range where the dominant energy components are concentrated. Then, the high-pass and low-pass frequencies of the filter are set to the upper and lower limits of the range.

[0083] When the UAV camera is at a high vertical angle or significantly deviated from the horizontal angle, the user's hand can be occluded, resulting in inaccurate estimated keypoint positions. In an embodiment, by applying an Adaptive Discrete Cosine Transform (ADCT) and a Multi-Joint Cooperative Kalman Filter (MJCKF), these occlusion-induced anomalies can be adaptively filtered out.

[0084] In the ADCT process, DCT is used to eliminate signal anomalies caused by body occlusion. To achieve dynamic adjustment of the number of retained DCT coefficients , in an embodiment, an entropy-based adaptive adjustment method is used, where is the initial retained coefficient, is the adaptive adjustment parameter, is the information entropy value of the gait data, is the length of the data.

[0085] Although the ADCT process effectively suppresses abnormal fluctuations in the keypoint position sequence, it can introduce signal distortion and edge effects. To alleviate this problem, in an embodiment, a Multi-Joint Cooperative Kalman Filter (MJCKF) method is used, which integrates the information of multiple adjacent keypoints into the state vector. In this way, not only the smoothing of individual joint motion trajectories is achieved, but also the coordination between multiple joints is preserved. With this cooperative filtering strategy, the stability and accuracy of the filter are significantly improved, effectively reducing the signal distortion introduced by ADCT, while ensuring the consistency and accuracy of the joint motion data.

[0086] Step 3: Perform spatio-temporal consistency conversion on the IMU sensor data and time series-based gait data processed in Step 2;

[0087] To ensure the time consistency of human key point data and IMU data, in an embodiment, a runtime clock synchronization protocol is run, and a Kalman filter is combined to process fluctuations in network delay, estimate the trend of changes in clock offset, and perform spatiotemporal consistency conversion on IMU sensor data and time series-based gait data after step 2 processing; this synchronization mechanism can maintain precise time consistency between devices, effectively avoiding data mismatch problems caused by time deviation.

[0088] The user key point motion information captured by the UAV camera is referenced in a human-centered coordinate system. The motion information measured by the IMU sensor is referenced in a mobile phone-centered coordinate system. During the process of the user approaching the UAV while holding the mobile phone, the mobile phone will continuously rotate with the user's hand. Directly using the motion information measured by the IMU sensor will cause a large deviation from the user key point motion information captured by the camera due to the continuous change of the mobile phone coordinate system. To eliminate this deviation, the IMU data needs to be first converted from the mobile phone coordinate system to the earth coordinate system. The specific implementation includes the following sub-steps:

[0089] Step 3.1: Fusion of user IMU sensor data , wherein represents the accelerometer, gyroscope, and magnetometer coordinate data in the mobile phone coordinate system for a certain time . The coordinate data of the accelerometer, gyroscope, and magnetometer in the mobile phone coordinate system are denoted as , , ; based on the above data, the direction quaternion and the azimuth angle, pitch angle, and roll angle are obtained.

[0090] The specific implementation includes the following sub-steps:

[0091] (1) Calculate the initial quaternion .

[0092] The initial attitude angle is calculated from the accelerometer and magnetometer data: , , .

[0093] Then, these angles are converted into the initial quaternion , and the specific expression is:

[0094] .

[0095] (2) Normalization of accelerometer and magnetometer data and calculation of magnetic field direction. For any time from the previous time's quaternion Start calculating the current attitude. To ensure the accuracy of attitude calculation, first need to normalize the accelerometer data and magnetometer data at the current time.

[0096] Through normalization, get the acceleration vector of unit length Similarly, normalize the magnetometer data to get the magnetic field vector of unit length The normalized acceleration vector and magnetic vector ensure the consistency and accuracy of each direction measurement in attitude calculation.

[0097] (3) Using normalized magnetometer data and the current quaternion , you can calculate the direction of the earth's magnetic field under the current attitude. First, through the quaternion multiplication, calculate the magnetic field direction vector Here, is the conjugate of the quaternion , indicates the multiplication of the quaternion. The resulting vector describes the component of the magnetic field direction under the current quaternion attitude.

[0098] Through , extract the horizontal component and vertical component of the magnetic field, and calculate the final magnetic field direction vector This vector describes the direction of the earth's magnetic field under the current attitude, which is used for subsequent attitude correction and update.

[0099] (4) After obtaining the initial quaternion and magnetic field direction, use the gradient descent algorithm to correct the quaternion. The correction vector is calculated as follows, which is used to adjust the estimated direction of the sensor to compensate for the deviation caused by sensor error or external interference:

[0100] ;

[0101] (5) Next, calculate the Jacobian matrix , which provides the partial derivative of each quaternion component with respect to the objective function, effectively guiding the correction process of the quaternion:

[0102] ;

[0103] (6) Use the Jacobian matrix and correction vector to calculate the correction step The normalized correction step size is further obtained. .

[0104] (7) Calculate the rate of change of the quaternion. First, the rate of change of the quaternion caused by the gyroscope data is... Then, the total quaternion change rate is ,in These are adjustable parameters used to control the algorithm's response speed; for a mobile phone's IMU sensor, The typical value is 0.041.

[0105] (8) Integrate to obtain a new quaternion. Sampling period This determines the amount of quaternion estimation update in each iteration. Furthermore, a new normalized quaternion is obtained. The final directional quaternion is .

[0106] (9) The rotation angles, namely azimuth, pitch, and roll, can be calculated using the new quaternions:

[0107] ;

[0108] This allows us to obtain the time series of the rotation angle. .

[0109] Step 3.2: Convert the acceleration vector in the phone's coordinate system Left-multiply by the direction quaternion respectively and right-multiplication of quaternion conjugate Complete the transformation to the world coordinate system: ;

[0110] Step 3.3: Calculate the hand's movement velocity based on IMU sensor data; assuming the initial velocity at the initial moment is zero, then at time... Calculate hand movement speed The formula is: ;in Indicates the hand in time The speed of time, Indicates the hand in time acceleration at time, It is the sampling rate of the accelerometer. It is the time step;

[0111] Step 3.4: Perform spatial consistency processing on the IMU sensor data; the main frequency components of the IMU sensor data include gravity variations. User walking component and hand movement components ,speed decomposed into ; using the adaptive cutoff filtering method described in step 2, dynamically adjusting the filtering threshold to eliminate and , and then normalizing the hand key point velocity to obtain the hand velocity in the body coordinate system ; then compare with the velocity of the corresponding hand key point in the body posture coordinate system to ensure the spatial consistency of the comparison data;

[0112] In processing the data collected by the UAV, the hand key point position is mapped to the body-centered coordinate system, and the velocity of the corresponding key point is calculated; after converting the hand key point position information to the human-centered coordinate system, the velocity of the hand key point is obtained by calculating the change of the hand key point position between two consecutive frames and dividing by the time interval.

[0113] Step 4: Extract time domain and frequency domain features from the IMU sensor data and time series-based gait data after processing in step 3;

[0114] In an embodiment, time domain and frequency domain features are extracted from user hand motion data, which are derived from IMU data and key point time series. By initially selecting 12 features from the time domain and frequency domain range, and by calculating the Fisher score of all features, the most discriminative features are selected. Finally, four time domain features and two frequency domain features are selected, and the processed data samples form a six-dimensional feature vector.

[0115] Step 5: Input the time domain and frequency domain features into the one-class classifier to perform consistency authentication between the IMU sensor data and the hand key point data;

[0116] The mobile phone IMU sensor data is converted into an Euler angle vector, combined with the accelerometer data, and used to represent the user's gait behavior data; then input into the feature extractor and one-class classifier for gait recognition authentication;

[0117] In an embodiment, for consistency verification between the UAV and the user, after feature extraction, in order to effectively verify the consistency between the IMU data and the hand key point data, the feature vector is input into the one-class classifier for model training. The OC-SVM (One-Class SVM) single classifier is used, and its hyperparameters are optimized through grid search. It needs to be emphasized that the consistency verification model only needs to be pre-trained once, and then it can be directly applied to unobserved subjects for authentication.

[0118] For gait recognition authentication, first, the phone IMU data is converted into Euler angle vectors through quaternion calculation, and combined with accelerometer data to represent the user's gait behavior. Then, a deep learning-based feature extractor is constructed using transfer learning. Specifically, the processed data is used to train the base model as a feature extractor. After the pre-trained model is completed, the classification layer of the model is removed and a single-class classifier is connected. In the subsequent user authentication process, only the single-class classifier needs to be adjusted, while the pre-trained base model remains unchanged.

[0119] In the gait feature extraction process, accelerometer and gyroscope data are used to extract user gait features. According to the accelerometer, gyroscope and magnetometer data, the quaternion is calculated, and then the corresponding rotation angles can be calculated: roll angle (Roll, ), pitch angle (Pitch, ) and yaw angle (Yaw, ). Finally, the acceleration and rotation angle time series data are obtained .

[0120] In one embodiment, a deep learning-based feature extractor is constructed using transfer learning. First, the base model is pre-trained on gait representation data collected from different participants, and then the pre-trained model is adapted as a general feature extractor to capture individual user gait features. A hybrid model architecture combining ResNet and LSTM networks is used, and the model design includes four residual blocks, each containing two convolutional layers. After each convolutional layer, a batch normalization (BN) layer is integrated to speed up model convergence, and a ReLU activation function is applied to introduce nonlinearity. The output of the last residual block is converted into a fixed-size feature vector through a global average pooling layer. To be compatible with the feature vector generated after feature selection, a single-channel convolutional layer is added before the ResNet residual block. In addition, a bidirectional LSTM network with 128 hidden units is also designed, and the input feature vector is reshaped into a three-dimensional tensor representing batch size, time step and feature dimension. To integrate the features of ResNet and LSTM, the output of ResNet and the hidden state of LSTM are concatenated in the feature dimension, and finally a fully connected layer and a Softmax layer are added for multi-class classification.

[0121] Step 6: If both authentications pass, mutual authentication is successful; otherwise, the drone determines whether to adjust its position based on the distance from the user, returns to step 1 and repeats the process until the maximum number of attempts is reached; if the authentication still fails, the user is rejected.

[0122] In one embodiment, in order to effectively verify the consistency between the IMU data and the hand key point data, the consistency feature vector extracted in step 5 is input into the OC-SVM classifier for decision-making. At the same time, the gait features extracted by the feature extractor in step 5 are input into the OC-SVM classifier for user authentication. When the results of consistency verification and gait authentication both pass the classifier, it is determined that the user belongs to a legitimate user; otherwise, the user is rejected.

[0123] See Figure 2 In one embodiment, the one-class classifier is a model trained through supervised learning. The model is specifically used to identify legitimate users and distinguish unauthorized users. The training process includes user label data and gait data.

[0124] In the training process, the present application uses training data sets containing labels, which are composed of sensor data from different users. Each sample is attached with a label, and the label values are 0, 1, 2, 3,..., which represent different users respectively. These labels are used to guide the model to learn how to distinguish users.

[0125] The present application adopts supervised learning algorithms, especially convolutional neural network (CNN) and recurrent neural network (RNN) models based on deep learning. These models are trained through label data, so that the model can identify the movement patterns and features of different users. During the training process, a standard classification loss function (such as cross-entropy loss function) is used to optimize the model to reduce the error between the predicted label and the actual label.

[0126] In the training process, features are extracted from the gait data of users, and models such as convolutional neural network and recurrent neural network are used to analyze and extract features from the gait data. The extracted features are input into the supervised learning classifier (MLP layer and softmax) for forward propagation to obtain the prediction result, and the user's label data is compared with the prediction result. The cross-entropy loss function is used to optimize the model, and the gradient is calculated for back propagation to gradually update the model weights and improve the recognition accuracy of the model for "legitimate users" and "unauthorized users" samples.

[0127] After pre-training, the present application uses the idea of transfer learning to remove the supervised learning classifier (MLP layer and softmax) of the model and retain the model weights of the forward layer as a feature extractor. In the real user registration scenario, a small amount of unlabeled data is put into the feature extractor for feature extraction, and the extracted features are used to train the one-class classifier, thereby completing the registration of the user authentication model. In this way, the present application can successfully generate an independent identity authentication model for each user using a small amount of positive class samples, achieving efficient user registration and authentication.​

[0128] This embodiment also provides a drone delivery authentication device based on implicit gait behavior, including:

[0129] One or more processors;

[0130] A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the UAV delivery authentication method based on implicit gait behavior.

[0131] The invention will be further illustrated below through specific experiments.

[0132] Following approval from the IRB (Integrity Board), this experiment recruited 31 participants aged 22 to 45, comprising 20 men and 11 women, including undergraduate students, graduate students, and faculty members. Prior to data collection, all participants received a detailed explanation of the research objectives and signed informed consent forms. During data collection, participants held smartphones and walked towards a fixed drone flight path. The drone hovered in front of the participants, collecting video data in real time. Sensors built into the smartphones (accelerometers, gyroscopes, etc.) continuously recorded sensor data related to the participants' movements. The drone captured the positions of key points on the participants' bodies through video and processed the video using a real-time attitude estimation algorithm to extract key point coordinates and generate time-series data. Simultaneously, the smartphones continuously recorded IMU (Inertial Measurement Unit) sensor data. In each data collection session, five pairs of IMU and KeyPoint time-series data were extracted from the participants' smartphones and the drone devices. Ultimately, all session data comprised a dataset containing 65,650 pairs of time-series data.

[0133] In this experiment, the following metrics were used to evaluate the performance of the system. False Acceptance Rate (FAR) represents the proportion of unauthorized user samples that are falsely identified as legitimate users, used to evaluate the resistance of the system to attacks. False Rejection Rate (FRR) represents the proportion of legitimate user samples that are falsely rejected as unauthorized users, used to evaluate the usability of the system. Equal Error Rate (EER) refers to the value when FAR and FRR are equal, a lower EER value indicates a higher reliability of the authentication system. Balanced Accuracy (BAC) refers to the average of True Acceptance Rate (TAR) and True Rejection Rate (TRR), where TAR represents the proportion of correctly identified legitimate user samples, and TRR represents the proportion of correctly identified unauthorized user samples. BAC is used to evaluate the performance of models trained on unbalanced data. The Receiver Operating Characteristic (ROC) curve is used to describe the relationship between TAR and FAR at different thresholds, and the Area Under the ROC Curve (AUC) is used as an indicator of the overall performance of the model, with an AUC value close to 100% indicating a stronger ability of the system to distinguish between legitimate users and unauthorized users.

[0134] To evaluate the overall performance of the present application, the experimental data set was used to train the model and evaluate its effectiveness. Under one authentication attempt, the consistency verification module, the IMU-based gait authentication module, and the overall performance of the present application were tested. The corresponding ROC curves are shown in Figure 3 The AUC of the consistency verification module is 99.83%, and the EER is 0.54%, while the AUC of the IMU-based gait authentication module is 99.98%, and the EER is 0.26%. The overall performance of the present application shows an AUC close to 100%, and the EER is 0.09%. The ROC curve shows that both modules show a high TAR close to 100%, while the FAR is relatively inferior. These results show that the present application performs well in real-world scenarios.

[0135] In addition, the experiment studied the influence of the hovering height of the UAV and the authentication distance on the consistency verification of the present application. As shown in Figure 4 When the hovering height of the UAV is 3 m or 4 m, the effect of consistency verification is almost the same. As the hovering height increases further, the BAC shows a slight decrease. In addition, when the authentication distance exceeds 24 meters, the BAC of consistency verification begins to gradually decrease due to the decrease in the accuracy of locating the key points of the user's body. Therefore, the experiment suggests that the hovering height of the UAV be set to 4 meters, and mutual authentication be initiated at a distance of 24 meters.

[0136] The application provides a UAV bidirectional identity authentication method and device based on implicit gait behavior. In the process of a user holding a mobile phone and walking towards a UAV, the user's hand posture change data is collected by the inertial sensor built in the smart phone and the UAV camera at the same time, so that bidirectional authentication of the user and the UAV is realized. Compared with the prior art, the method does not need additional hardware support, can realize efficient authentication in a long-distance environment of more than 18 meters, and does not need additional authentication actions, and the user experience is friendly. Gait is a unique dynamic feature determined by the human skeletal structure and behavior habits, which is difficult to be forged and imitated. The inertial sensor (such as an accelerometer and a gyroscope) commonly integrated in a smart phone can efficiently and low-costly capture gait data, and the UAV delivery bidirectional authentication by using implicit gait features is not only convenient and reliable, but also can effectively resist imitation attacks and device hijacking attacks.

[0137] The application improves the security of the UAV delivery system, reduces potential risks caused by insufficient identity authentication, and provides a more convenient and secure delivery experience for users.

[0138] It should be understood that the above-described embodiments are part of the embodiments of the application, rather than all the embodiments. In addition, the technical features in each embodiment or single embodiment provided by the application can be combined with each other to form a feasible technical solution, and such combination is not restricted by the sequence of steps and / or structure composition mode, but should be based on the realization by the ordinary skilled in the art, when the combination of technical solutions appears contradictory or unfeasible, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope of the application.

[0139] It should be understood that the above description of the preferred embodiments is more detailed, and therefore should not be considered as a limitation on the scope of patent protection of the application, and the ordinary skilled in the art can make substitutions or modifications under the inspiration of the application without departing from the scope of protection claimed by the application, all of which fall within the protection scope of the application, and the protection scope of the application should be subject to the appended claims.

Claims

1. A drone delivery authentication method based on implicit gait behavior, characterized in that, Includes the following steps: Step 1: When the user and the drone perform two-way authentication, collect IMU sensor data and the user's time-series gait data; Step 2: Normalize and denoise the IMU sensor data and time-series-based gait data; Step 3: Perform spatiotemporal consistency transformation on the IMU sensor data and time-series-based gait data processed in Step 2; The specific implementation includes the following sub-steps: Step 3.1: Fusing User IMU Sensor Data ,in These represent the accelerometer, gyroscope, and magnetometer in the phone's coordinate system, respectively. Coordinate data; for a certain moment Data from the accelerometer, gyroscope, and magnetometer in the phone's coordinate system Coordinate data is denoted as , , ; Based on the above data, we obtain the directional quaternion. and azimuth Pitch angle and roll angle ; (1) Calculate the quaternion at the initial time. ; The initial attitude angle was calculated from accelerometer and magnetometer data: , , ; Then convert these angles into initial quaternions. : ； (2) Normalization of accelerometer and magnetometer data and calculation of magnetic field direction; For any time From the quaternion of the previous moment Begin calculating the current attitude; first, use the accelerometer data at the current moment. and magnetometer data Normalization is performed; through normalization, the acceleration vector per unit length is obtained. Normalize the magnetometer data to obtain the magnetic field vector per unit length. ; (3) Use normalized magnetometer data and the current quaternion Calculate the direction of the Earth's magnetic field under the current attitude; First, the direction vector of the magnetic field is calculated using quaternion multiplication. ;in, It is a quaternion conjugate, Represents quaternion multiplication; the resulting vector This describes the components of the magnetic field direction under the current quaternion attitude. ;pass Extracting the horizontal component of the magnetic field and vertical components And calculate the final magnetic field direction vector. ; vector It describes the direction of the Earth's magnetic field in its current orientation; (4) Correct the quaternion using the gradient descent algorithm; correction vector The calculation is as follows: ； (5) Calculate the Jacobian matrix It provides partial derivatives of each quaternion component with respect to the objective function to guide the quaternion correction process; ； (6) Using the Jacobian matrix and correction vector Calculate the correction step size The normalized correction step size is further obtained. ; (7) Calculate the rate of change of the quaternion; The rate of change of the quaternion caused by the gyroscope data is firstly... Then calculate the total quaternion rate of change. ,in These are adjustment parameters used to control the response speed of the algorithm; (8) Integrating yields a new quaternion; Using sampling period Calculate the update amount of the quaternion estimate in each iteration. Furthermore, a new normalized quaternion is obtained. ; (9) Using the new quaternions, the rotation angles, namely the azimuth, pitch, and roll angles, are calculated: ； The time series q of the direction quaternion and the time series of the rotation angle are obtained. ; Step 3.2: Convert the acceleration vector in the phone's coordinate system Left multiplication respectively and right-hand multiplication Complete the transformation to the world coordinate system: ; Step 3.3: Calculate the hand's movement velocity based on IMU sensor data; assuming the initial velocity at the initial moment is zero, then at time... Calculate hand movement speed The formula is: ;in Indicates the hand in time The speed of time, Indicates the hand in time acceleration at time, It is the sampling rate of the accelerometer. It is the time step; Step 3.4: Perform spatial consistency processing on the IMU sensor data; the frequency components of the IMU sensor data include gravity variations. User walking component and hand movement components ,speed Decomposed into An adaptive cutoff filtering method is used to dynamically adjust the filter threshold and eliminate... and Then, the speed of key hand points is normalized to obtain the body coordinate system. The speed of the hand; then The velocity of the corresponding hand key points in the body posture coordinate system is compared to ensure spatial consistency of the comparison data. When processing data collected by drones, the positions of key hand points are mapped to a coordinate system centered on the human body, and the velocity of the corresponding key points is calculated. After converting the position information of key hand points to a coordinate system centered on the human body, the velocity of the key hand points is obtained by calculating the change in the position of key hand points between two consecutive frames and dividing by the time interval. Step 4: Extract time-domain and frequency-domain features from the IMU sensor data and time-series-based gait data processed in Step 3; Step 5: Input the time-domain and frequency-domain features into a single-class classifier to verify the consistency between the IMU sensor data and the hand key point data; The mobile phone IMU sensor data is converted into Euler angle vectors and combined with accelerometer data to characterize the user's gait behavior data; then it is input into a feature extractor and a single-class classifier for gait recognition and authentication. Step 6: If both authentications pass, mutual authentication is successful; otherwise, the drone determines whether to adjust its position based on the distance to the user, returns to Step 1, and repeats the process until the maximum number of attempts is reached; if authentication still fails, the user is rejected.

2. The UAV delivery authentication method based on implicit gait behavior according to claim 1, characterized in that: In step 1, when the user and the drone perform two-way authentication, the user's mobile phone collects IMU sensor data; the drone's camera records the user's gait information, performs human posture detection, identifies and matches key points of the user's hand holding the mobile phone, and extracts time-series data of the key points, thereby converting vision-based gait information into time-series-based gait data.

3. The UAV delivery authentication method based on implicit gait behavior according to claim 1, characterized in that: In step 2, the IMU sensor data is downsampled to match the sampling rate of the time series data of the key points.

4. The UAV delivery authentication method based on implicit gait behavior according to claim 1, characterized in that: In step 2, an adaptive cutoff frequency filtering method is used to denoise the IMU sensor data and the time series-based gait data. First, the spectral characteristics of the time series are analyzed using fast Fourier transform to determine the frequency range where the principal energy components are located. Then, the high-pass frequency and low-pass frequency of the filter are set as the upper and lower limits of this frequency range, respectively.

5. The UAV delivery authentication method based on implicit gait behavior according to claim 1, characterized in that: In step 2, abnormal data caused by occlusion is adaptively filtered out by applying adaptive discrete cosine transform and multi-joint cooperative Kalman filtering; The adaptive discrete cosine transform utilizes the discrete cosine transform to eliminate signal anomalies caused by body occlusion; in order to achieve dynamic adjustment of the retained discrete cosine transform coefficients... An entropy-based adaptive adjustment method is used to calculate... K , ;in These are the initial retention coefficients. It is an adaptive parameter adjustment. The information entropy value of gait data. The length of the gait data; The multi-joint cooperative Kalman filter integrates the information of several adjacent key points into the state vector for the data processed by adaptive discrete cosine transform, thereby achieving smooth processing of the motion trajectory of a single joint.

6. The UAV delivery authentication method based on implicit gait behavior according to claim 1, characterized in that: In step 4, 12 features were initially selected from the time and frequency domains, and the Fisher scores of all features were calculated to filter out the most discriminative features. Finally, four time-domain features and two frequency-domain features were selected, and the processed data samples formed a six-dimensional feature vector.

7. The UAV delivery authentication method based on implicit gait behavior according to claim 1, characterized in that: In step 5, the feature extractor consists of sequentially connected single-channel inputs. The system consists of convolutional layers, a ResNet, and a bidirectional LSTM network. The ResNet includes four residual blocks, each containing two convolutional layers. A batch normalization layer is integrated after each convolutional layer to accelerate model convergence, and a ReLU activation function is applied to introduce non-linearity. The output of the last residual block is transformed into a fixed-size feature vector through a global average pooling layer. The output of the ResNet is concatenated with the hidden state of the bidirectional LSTM network along the feature dimension. Finally, a fully connected layer and a Softmax layer are added for multi-class classification.

8. The UAV delivery authentication method based on implicit gait behavior according to any one of claims 1-7, characterized in that: In step 5, the single-class classifier is a model trained through supervised learning. During the training process, a labeled training dataset is used, which consists of sensor data from different users. Each sample is labeled to identify different users, which guides the model to learn how to distinguish users. During training, features are extracted from the user's gait data. The model is used to perform temporal analysis and feature extraction on the gait data. The extracted features are then input into a supervised learning classifier for forward propagation to obtain prediction results. The user's label data is compared with the prediction results. The cross-entropy loss function is used to optimize the model. The gradient is calculated for backward propagation to gradually update the model weights and improve the model's accuracy in identifying "legitimate users" and "unauthorized users" samples. After pre-training, a transfer learning method is used to remove the supervised learning classifier from the model and retain the model weights of the feedforward layer as a feature extractor. In a real user registration scenario, a small amount of unlabeled data is fed into the feature extractor for feature extraction, and the extracted features are used to train a single-class classifier to complete the user authentication model registration.

9. A drone delivery authentication device based on implicit gait behavior, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the UAV delivery authentication method based on implicit gait behavior as described in any one of claims 1 to 8.

Images