An unmanned aerial vehicle airborne real-time estimation method for a ground target motion state

Abstract

The application discloses a kind of unmanned aerial vehicle airborne real-time ground target motion state estimation method, belong to unmanned aerial vehicle technical field, including the following steps, S1, input image, and identify locking ground target object from image, S2, using KCF algorithm, the detection result of S1 is tracked dynamically target, and the line of sight angle of unmanned aerial vehicle to target is solved, S3, establish target position detection model, the relative coordinate of unmanned aerial vehicle and target is obtained according to line of sight angle, S4, construct adaptive Kalman filtering framework based on autoregressive prediction model, S5, according to adaptive Kalman filtering framework and the process noise covariance after updating, obtain final prediction value, and utilize real-time innovation sequence dynamic adjustment process noise covariance, process noise covariance is updated, the application adopts above-mentioned method, can effectively enhance the accuracy of the estimation of unmanned aerial vehicle to target motion state.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method for estimating the motion state of UAV-borne real-time ground targets. Background Technology

[0002] During operations, UAVs often need to interact collaboratively with ground targets. However, in the absence of communication links, the observation data acquired by their onboard sensors are easily affected by significant noise. Furthermore, traditional target recognition methods often rely on manual feature extraction (such as template matching and SIFT feature point detection) or deep learning-based target detection algorithms, but these algorithms suffer from decreased detection accuracy and insufficient generalization ability in complex environments. For target tracking, traditional Kalman filtering (KF) and its extended forms (EKF, UKF) are prone to state divergence when dealing with nonlinear motion models. While particle filtering (PF) has a certain degree of nonlinear adaptability, its computational complexity increases linearly with the number of particles, making it difficult to meet the real-time requirements of UAVs with limited onboard computing power (usually embedded GPUs or low-power CPUs). Summary of the Invention

[0003] The purpose of this invention is to provide a method for estimating the motion state of ground targets on an airborne UAV, which can effectively enhance the accuracy of UAV estimation of target motion state.

[0004] To achieve the above objectives, the present invention provides a method for estimating the motion state of a UAV-borne real-time ground target, comprising the following steps:

[0005] S1. Input an image and identify and lock onto ground target objects from the image;

[0006] S2. Using the KCF algorithm, dynamic target tracking is performed on the detection results of S1, and the line-of-sight angle of the UAV to the target is calculated;

[0007] S3. Establish a target position detection model and obtain the relative coordinates between the UAV and the target based on the line-of-sight angle;

[0008] S4. Construct an adaptive Kalman filter framework based on an autoregressive prediction model;

[0009] S5. Based on the adaptive Kalman filter framework and the updated process noise covariance, the final predicted value is obtained, and the process noise covariance is updated by dynamically adjusting the process noise covariance using the real-time innovative sequence.

[0010] Preferably, the process of identifying and locking ground objects from the image in S1 is as follows:

[0011] S11. Input image data into the backbone network;

[0012] S12 and YOLOv4 use CSPDarknet53 in the backbone network to reduce gradient redundancy through cross-stage partial connections, as follows:

[0013] X out =Concat(X) base ,f(X Split ));

[0014] Where X out The output feature map, X, represents the final feature representation after cross-stage partial connection processing. Split X represents the segmentation portion of the input feature map. base This represents the part of the input feature map that has not undergone convolution processing. Concat(·) represents the concatenation operation, and f(·) represents the convolution operation.

[0015] S13. Perform spatial pyramid pooling at the end of the backbone to obtain the features of the input image. The process is as follows:

[0016] F spp =Concat(max Pool(F) in ,k i ))(k i ∈{5,9,13});

[0017] Where F in Represents the input feature map, k i This represents the kernel size, maxPool(·) represents the max pooling operation, and F spp Indicates the features of the output image;

[0018] S14. Input all image features into the neck network. YOLOv4 uses the PANet structure to fuse them through a bidirectional feature pyramid to obtain the final fused features, and then inputs them into the head network.

[0019] S15. After the head network receives the final fused features, it uses the CIoU loss function to calculate the error between the predicted value and the true value. The process is as follows:

[0020]

[0021] in This represents the CIoU loss function, where IoU represents the wall ratio of the overlapping area to the total area, ρ represents the Euclidean distance between the center points of the predicted box and the ground truth box, and b pred b represents the center coordinates of the prediction box. gt α represents the center coordinates of the ground truth box, c represents the diagonal length of the smallest bounding box that simultaneously covers both the predicted and ground truth boxes, α represents the weighting coefficient of the aspect ratio penalty term, and v represents the aspect ratio consistency measure.

[0022] S16. After training, use 1×1 convolution to adjust the number of channels, output the predicted parameters of each anchor box, obtain the final detection result, and lock the ground target object. The process is as follows:

[0023] Pred i =Conv 1×1 (P i );

[0024] Where P i Pred represents the fused feature obtained from the i-th layer in the neck network. i Represents the prediction parameters for the i-th anchor box, Conv 1×1 (·) represents a 1×1 convolution.

[0025] Preferably, the process of obtaining the final fusion feature in S14 is as follows:

[0026] S141. After upsampling the high-level semantic features in the image features, they are concatenated with the mid-level features. The process is as follows:

[0027]

[0028] Where F top F represents the feature concatenation result of the top-down path. high F represents high-level semantic features in image features. mid This represents the mid-level features in the image features, UpSample(·) represents the upsampling operation, and Conv(·) represents the convolution operation;

[0029] S142. After downsampling the shallow detail features in the image features, they are concatenated with the mid-level features. The process is as follows:

[0030]

[0031] Where F bottom F represents the feature concatenation result of the bottom-up path. low Represents shallow detail features; DownSample(·) indicates a downsampling operation.

[0032] S143, F top and F bottom The fusion process is as follows:

[0033] F fusion =Concat(F top ,F bottom );

[0034] Where F fusion This indicates the fusion result.

[0035] Preferably, the process of S2 is as follows:

[0036] S21. Employ the kernelized correlation filter algorithm and combine it with the prediction parameters obtained in S1 to perform dynamic target tracking of the target object;

[0037] S22. Determine the projected coordinates of the target object in the camera's normalized coordinate system. The process is as follows:

[0038]

[0039] Where (x′,y′) represents the normalized coordinates of the target in the camera plane, (u,v) represents the pixel coordinates of the target in the image, (u0,v0) represents the coordinates of the principal point of the image, and f represents the camera focal length;

[0040] S23. Based on the pinhole camera model and normalized coordinates, calculate the azimuth and pitch angles of the target object's line of sight. The process is as follows:

[0041]

[0042] Where θ c φ represents the azimuth angle of the target's line of sight in camera coordinates. c This represents the target's pitch angle in camera coordinates.

[0043] Preferably, the process of dynamic target tracking of the target object in S21 is as follows:

[0044] S211. Based on the initial position of the target obtained in S1, take the image patch containing the target and extract its features;

[0045] S212. Divide the features in S211 into background and target object to obtain the initial filter, and then set the filter coefficients... The features are defined as the initial template and the filter coefficients. The calculation process is as follows:

[0046]

[0047] in Let λ represent the expected response spectrum, and λ represent the regularization coefficient. This is the diagonalized frequency domain representation of the kernel matrix;

[0048] S213. Extract an image patch from the image at the next time step. The position of the image patch is the same as the position of the image patch extracted in S211. Extract the features at this time as candidate regions.

[0049] S214. The Gaussian kernel function measures the similarity h between the candidate region and the template. gauss (x,x'), the process is as follows:

[0050]

[0051] Where x represents the template feature, x' represents the candidate region feature, and σ represents the kernel function bandwidth parameter;

[0052] S215, according to h gauss (x,x') yields the response map, and the maximum and minimum values ​​of the response map are obtained. The tracking quality is dynamically assessed, and the displacement of the object in the image is calculated using the displacement of the point with the largest response. The new target position is obtained, the template features are recalculated, the filter is updated, and the process returns to S213. The dynamic assessment of tracking quality is as follows:

[0053]

[0054] Where APCE represents the average peak correlation energy, R max and R min Represent the maximum and minimum values ​​of the response plot, respectively, R i,j This represents the response intensity of the pixel in the i-th row and j-th column of the response graph.

[0055] Preferably, the process of establishing the target position detection model and obtaining the relative coordinates in S3 is as follows:

[0056] S31. Establish the body coordinate system ox b y b z b The x-axis is forward along the longitudinal axis of the UAV, the y-axis is perpendicular to the longitudinal plane of the UAV and points to the right, and the z-axis is determined according to the Cartesian coordinate system rules.

[0057] S32. Based on the positional relationship between the UAV and the target object, determine the relative coordinates of the target object in the UAV coordinate system. The process is as follows:

[0058] X b =r cosβcosα;

[0059] Y b =r cosβsinα;

[0060] Z b = -r sinβ;

[0061] Where (X) b ,Y b Z b ) represents the position of the target object in the body coordinate system, and α represents the shadow cast by the line connecting the loitering munition and the target to x. b y b plane and y b The angle between the axes, β, represents the projection of the line connecting the loitering munition and the target onto the x-axis. b z b plane with x bThe angle between the drone and the target is defined as follows: β is positive downwards, and r represents the straight-line distance between the drone and the target.

[0062] S33. Transform the target position in the body coordinate system to the inertial frame, and obtain the relative coordinates of the UAV and the target in the inertial frame. The process is as follows:

[0063]

[0064]

[0065] Where R x (φ) represents the roll rotation matrix about the x-axis of the aircraft, R y (θ) represents the pitch and rotation matrix about the y-axis of the aircraft, R z (ψ) represents the yaw rotation matrix around the z-axis of the aircraft, R represents the total transformation matrix from the body coordinate system to the inertial coordinate system, and φ, θ and ψ represent the attitude angles of the UAV.

[0066] S34. The relative coordinates of the UAV and the target in the inertial frame are obtained using the following formula:

[0067]

[0068] Where (x) i ,y i ,z i ) represents the relative coordinates of the UAV and the target in the inertial frame.

[0069] Preferably, the process of S4 is as follows:

[0070] S41. Based on the polynomial approximation characteristics of vehicle motion state, the autoregressive prediction model is embedded into the Kalman filter, and the state vector is defined as the historical position sequence. The process is as follows:

[0071]

[0072] in Let r represent the state vector, which represents the position measurements taken over the past M sampling times. k Indicates the current position, r k-1 This indicates the position at the previous time step, and M represents the order of the autoregressive prediction model.

[0073] S42. Using Weierstrass's theorem, the vehicle trajectory is modeled as an Nth-order polynomial, and the AR prediction equation is established. The process is as follows:

[0074]

[0075] Where r k+1-m r represents the location information over the past N time points. k+1h represents the predicted position at the next moment. m h represents the coefficients of the AR model. m The following conditions must be met:

[0076]

[0077] S43. By constructing the Vandermonde matrix and constraint vectors, the AR prediction equation is transformed into a convex quadratic programming closed form, as follows:

[0078]

[0079] Where u k =[h1 h2 … h M ] T P k-1|k-1 Let A denote the state covariance matrix, and b denote the Vandermonde matrix, and b denote the constraint vector.

[0080] S44. Solve the closed-form convex quadratic programming problem using the Lagrange multiplier method. The process is as follows:

[0081]

[0082] in The column vector representing the coefficients of the AR model is used to obtain the adaptive Kalman filter framework based on the autoregressive prediction model.

[0083] Preferably, the process of obtaining the final predicted value in S5 is as follows:

[0084] S51. Predict the current state based on the state estimate from the previous time step and the control input at the current time step, as follows:

[0085]

[0086] Where x k|k-1 F represents the predicted state value at the current moment. k|k-1 Let x represent the state transition matrix. k-1|k-1 This represents the state estimate at the previous moment;

[0087] S52. Estimate the covariance matrix based on the state at the previous time step, and predict the covariance matrix for the current time step. The process is as follows:

[0088]

[0089] Where P k|k-1 Let P represent the covariance matrix of the current state prediction. k-1|k-1 Let Q represent the state estimation covariance matrix at the previous time step. k-1This represents the process noise covariance matrix at the previous time step;

[0090] S53. Determine the degree of correction of the measured value to the state estimate, as follows:

[0091]

[0092] Where K k Let H represent the Kalman gain matrix, H represent the observation matrix, and R represent the Kalman gain matrix. k This represents the measurement noise covariance matrix at the current moment;

[0093] S54. Correct the state prediction value using the measurement value at the current time to obtain the state estimate value at the current time. The process is as follows:

[0094]

[0095] Where x k|k z represents the state estimate at the current time. k This represents the measurement value at the current moment;

[0096] S55. Update the state estimation covariance matrix, the process is as follows:

[0097]

[0098] Where P k|k This represents the state estimation covariance matrix at the current moment;

[0099] S56. Utilizing real-time innovative sequences to dynamically adjust process noise covariance Q k The process is as follows:

[0100]

[0101] Where d k Represents the real-time innovation sequence, z k Represents the observed value. This represents the covariance of the innovation samples.

[0102] Therefore, the present invention provides a real-time ground target motion state estimation method for UAVs with the above-mentioned structure. It uses a YOLO-based target recognition model that balances speed and accuracy, and combines adaptive Kalman filtering to reduce high noise caused by UAV motion. It also maintains accuracy without significantly increasing complexity.

[0103] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0104] Figure 1This is an overall flowchart of a method for estimating the motion state of a UAV-borne real-time ground target according to the present invention;

[0105] Figure 2 This is a diagram of process S1 of a method for estimating the motion state of a UAV-borne real-time ground target according to the present invention.

[0106] Figure 3 This is a schematic diagram of the body coordinate system and the relative relationship between the UAV and the target object in the UAV-borne real-time ground target motion state estimation method of the present invention. Detailed Implementation

[0107] Example

[0108] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0109] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0110] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0111] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed when in use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0112] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0113] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0114] like Figures 1-3 As shown, this invention provides a method for estimating the motion state of a UAV-borne real-time ground target, such as... Figure 1 The steps shown are as follows:

[0115] S1. Input an image, and identify and lock ground target objects from the image based on the YOLOv4 model, such as... Figure 2 As shown, Figure 2 In this context, P3, P4, P5, F3, F4, and F5 represent different levels of output within the same layer, representing feature maps of shallow, medium, and deep layers.

[0116] S11. Input image data into the backbone network;

[0117] S12 and YOLOv4 use CSPDarknet53 in the backbone network to reduce gradient redundancy through cross-stage partial connections, as follows:

[0118]

[0119] Where X out The output feature map, X, represents the final feature representation after cross-stage partial connection processing. Split X represents the segmentation portion of the input feature map. base This represents the part of the input feature map that has not undergone convolution processing. Concat(·) represents the concatenation operation, and f(·) represents the convolution operation.

[0120] S13. Perform spatial pyramid pooling at the end of the backbone to concatenate multi-scale feature maps, enhancing the model's adaptability to targets at different scales and obtaining the features of the input image. The process is as follows:

[0121]

[0122] Where F in Represents the input feature map, k iThis represents the kernel size, maxPool(·) represents the max pooling operation, and F spp Indicates the features of the output image;

[0123] S14. All image features are input into the neck network. The neck network is used to fuse features at different levels to enhance multi-scale detection capabilities. YOLOv4 uses the PANet structure to fuse features through a bidirectional feature pyramid. The bidirectional feature pyramid achieves top-down and bottom-up fusion to obtain the final fused features, which are then input into the head network.

[0124] S141. After upsampling the high-level semantic features in the image features, they are concatenated with the mid-level features. The process is as follows:

[0125] F top =UpSample(F high )⊕Conv(F mid );

[0126] Where F top F represents the feature concatenation result of the top-down path. higt F represents high-level semantic features in image features. mid This represents the mid-level features in the image features, UpSample(·) represents the upsampling operation, and Conv(·) represents the convolution operation;

[0127] S142. After downsampling the shallow detail features in the image features, they are concatenated with the mid-level features. The process is as follows:

[0128] F bottom =DownSample(F low )⊕Conv(F mid );

[0129] Where F bottom F represents the feature concatenation result of the bottom-up path. low Representing shallow detail features, DownSample(·) indicates a downsampling operation, i.e., F low Space dimensions and F mid Complete the corresponding task;

[0130] S143, F top and F bottom The fusion process is as follows:

[0131] F fusion =Concat(F top ,F bottom );

[0132] Where F fusion This indicates the fusion result.

[0133] S15. The head network is responsible for generating the final detection result. After receiving the final fused features, the head network uses the CIoU loss function to calculate the error between the predicted value and the true value. This loss is then backpropagated through the neural network to update the network weights. The process is as follows:

[0134]

[0135] in This represents the CIoU loss function, used to measure the difference between the predicted bounding box and the ground truth bounding box. IoU represents the wall ratio of the overlapping area to the total area, ρ represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, and b... pred b represents the center coordinates of the prediction box. gt α represents the center coordinates of the ground truth box, c represents the diagonal length of the smallest bounding box (smallest closed rectangle) that simultaneously covers both the predicted and ground truth boxes, α represents the weighting coefficient of the aspect ratio penalty term, and v represents the aspect ratio consistency measure.

[0136] S16. After training, use 1×1 convolution to adjust the number of channels, output the predicted parameters of each anchor box, obtain the final detection result, and lock the ground target object. The process is as follows:

[0137] Pred i =Conv 1×1 (P i );

[0138] Where P i Pred represents the fused feature obtained from the i-th layer in the neck network. i Represents the prediction parameters for the i-th anchor box, Conv 1×1 (·) represents a 1×1 convolution.

[0139] S2. The KCF algorithm is adopted because the UAV is far away from the target, making identification difficult and long-term accurate tracking hard. Therefore, the KCF algorithm is used to perform dynamic target tracking on the detection results of S1, improve the robustness of the system, and calculate the line-of-sight angle of the UAV to the target.

[0140] S21. Employ the kernelized correlation filter algorithm and combine it with the prediction parameters obtained in S1 to perform dynamic target tracking of the target object;

[0141] S211. Based on the initial position of the target obtained in S1, take the image patch containing the target and extract its features;

[0142] S212. Divide the features in S211 into background and target object to obtain the initial filter, and then set the filter coefficients... The features are defined as the initial template and the filter coefficients. The calculation process is as follows:

[0143]

[0144] in Let λ represent the expected response spectrum, and λ represent the regularization coefficient. This is the diagonalized frequency domain representation of the kernel matrix;

[0145] S213. Extract an image patch from the image at the next time step. The position of the image patch is the same as the position of the image patch extracted in S211. Extract the features at this time as candidate regions.

[0146] S214. The Gaussian kernel function measures the similarity h between the candidate region and the template. gauss (x,x'), the process is as follows:

[0147]

[0148] Where x represents the template feature, x' represents the candidate region feature, and σ represents the kernel function bandwidth parameter;

[0149] S215, according to h gauss (x,x') yields the response map, and the maximum and minimum values ​​of the response map are obtained. The tracking quality is dynamically assessed, and the displacement of the object in the image is calculated using the displacement of the point with the largest response. The new target position is obtained, the template features are recalculated, the filter is updated, and the process returns to S213. The dynamic assessment of tracking quality is as follows:

[0150]

[0151] Where APCE represents the average peak correlation energy, R max and R min Represent the maximum and minimum values ​​of the response plot, respectively, R i,j This represents the response intensity of the pixel in the i-th row and j-th column of the response graph.

[0152] S22. Determine the projected coordinates of the target object in the camera's normalized coordinate system. The process is as follows:

[0153]

[0154] Where (x′,y′) represents the normalized coordinates of the target in the camera plane, (u,v) represents the pixel coordinates of the target in the image, (u0,v0) represents the coordinates of the principal point of the image (usually the image center), and f represents the camera focal length;

[0155] S23. Based on the pinhole camera model and normalized coordinates, calculate the azimuth and pitch angles of the target object's line of sight. The process is as follows:

[0156]

[0157] Where θ c φ represents the line-of-sight azimuth angle of the target in camera coordinates (the horizontal angle around the camera's optical axis). c This represents the target's pitch angle (vertical angle) in camera coordinates.

[0158] S3. Establish a target position detection model and obtain the relative coordinates between the UAV and the target based on the line-of-sight angle;

[0159] S31, such as Figure 3 As shown, establish the body coordinate system ox. b y b z b The x-axis points forward along the longitudinal axis of the UAV, the y-axis points to the right perpendicular to the longitudinal plane of the UAV, and the z-axis is determined according to the Cartesian coordinate system rules. b Indicates the location of the drone, P t Representing the target substance, l = P t -P b Indicates the line connecting bullets and targets;

[0160] S32. Based on the positional relationship between the UAV and the target object, determine the relative coordinates of the target object in the UAV coordinate system. The process is as follows:

[0161] X b =r cosβcosα;

[0162] Y b =r cosβsinα;

[0163] Z b = -r sinβ;

[0164] Where (X) b ,Y b Z b ) represents the position of the target object in the body coordinate system, and α represents the shadow cast by the line connecting the loitering munition and the target to x. b y b plane and y b The angle between the axes, β, represents the projection of the line connecting the loitering munition and the target onto the x-axis. b z b plane with x b The angle between the camera coordinate system and the UAV body coordinate system is defined as β downwards as positive (when the camera coordinate system and the UAV body coordinate system coincide, α and β have the same meaning as θ and φ mentioned above, but φ is in the opposite direction to β), and r represents the straight-line distance between the UAV and the target;

[0165] S33. Transform the target position in the body coordinate system to the inertial frame, and obtain the relative coordinates of the UAV and the target in the inertial frame. The rotation matrix process from the body coordinate system to the inertial frame is as follows:

[0166]

[0167] Where R x (φ) represents the roll rotation matrix about the x-axis of the aircraft, R y (θ) represents the pitch and rotation matrix about the y-axis of the aircraft, R z (ψ) represents the yaw rotation matrix around the z-axis of the aircraft, R represents the total transformation matrix from the body coordinate system to the inertial coordinate system, and φ, θ and ψ represent the attitude angles of the UAV.

[0168] S34. The relative coordinates of the UAV and the target in the inertial frame are obtained using the following formula:

[0169]

[0170] Where (x) i ,y i ,z i ) represents the relative coordinates of the UAV and the target in the inertial frame.

[0171] S4. Construct an adaptive Kalman filter framework based on an autoregressive (AR) prediction model;

[0172] S41. Based on the polynomial approximation characteristics of vehicle motion state, the autoregressive prediction model is embedded into the Kalman filter, and the state vector is defined as the historical position sequence. The process is as follows:

[0173]

[0174] in Let r represent the state vector, which represents the position measurements taken over the past M sampling times. k Indicates the current position, r k-1 This indicates the position at the previous time step, and M represents the order of the autoregressive prediction model.

[0175] S42. Using Weierstrass's theorem, the vehicle trajectory is modeled as an Nth-order polynomial, and the AR prediction equation is established. The process is as follows:

[0176]

[0177] Where r k+1-m r represents the location information over the past N time points. k+1 h represents the predicted position at the next moment. m h represents the coefficients of the AR model. m The following conditions must be met:

[0178]

[0179] S43. By constructing the Vandermonde matrix and constraint vectors, the AR prediction equation is transformed into a convex quadratic programming closed form, as follows:

[0180]

[0181] Where u k =[h1 h2 … h M ] T P k-1|k-1 Let A denote the state covariance matrix, and b denote the Vandermonde matrix, and b denote the constraint vector.

[0182] S44. Solve the closed-form convex quadratic programming problem using the Lagrange multiplier method. The process is as follows:

[0183]

[0184] in The column vector representing the coefficients of the AR model is used to obtain the adaptive Kalman filter framework based on the autoregressive prediction model.

[0185] S5. Based on the adaptive Kalman filter framework and the updated process noise covariance, the final predicted value is obtained, and the process noise covariance is updated by dynamically adjusting the process noise covariance using the real-time innovative sequence.

[0186] S51. Predict the current state based on the state estimate from the previous time step and the control input at the current time step, as follows:

[0187]

[0188] Where x k|k-1 F represents the predicted state value at the current moment. k|k-1 Let x represent the state transition matrix. k-1|k-1 This represents the state estimate at the previous moment;

[0189] S52. Estimate the covariance matrix based on the state at the previous time step, and predict the covariance matrix for the current time step. The process is as follows:

[0190]

[0191] Where P k|k-1 Let P represent the covariance matrix of the current state prediction. k-1|k-1 Let Q represent the state estimation covariance matrix at the previous time step. k-1 This represents the process noise covariance matrix at the previous time step;

[0192] S53. Determine the degree of correction of the measured value to the state estimate, as follows:

[0193]

[0194] Where K k Let H represent the Kalman gain matrix, H represent the measurement matrix (i.e., the observation matrix), and R represent the observation matrix. k This represents the measurement noise covariance matrix at the current moment;

[0195] S54. Correct the state prediction value using the measurement value at the current time to obtain the state estimate value at the current time. The process is as follows:

[0196]

[0197] Where x k|k z represents the state estimate at the current time. k This represents the measurement value at the current moment;

[0198] S55. Update the state estimation covariance matrix, the process is as follows:

[0199]

[0200] Where P k|k This represents the state estimation covariance matrix at the current moment;

[0201] S56. Utilizing real-time innovative sequences to dynamically adjust process noise covariance Q k The process is as follows:

[0202]

[0203] Where d k Represents the real-time innovation sequence, z k Represents the observed value. This represents the covariance of the innovation samples.

[0204] The above steps can be used to complete a full process from target identification and locking, coordinate system transformation, and target motion state estimation.

[0205] Therefore, the present invention provides a real-time ground target motion state estimation method for UAVs with the above-mentioned structure. It uses a YOLO-based target recognition model that balances speed and accuracy, and combines adaptive Kalman filtering to reduce high noise caused by UAV motion. It also maintains accuracy without significantly increasing complexity.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for estimating the motion state of a UAV-borne real-time ground target, characterized in that, Includes the following steps: S1. Input an image and identify and lock onto ground target objects from the image; S2. Using the KCF algorithm, dynamic target tracking is performed on the detection results of S1, and the line-of-sight angle of the UAV to the target is calculated; S3. Establish a target position detection model and obtain the relative coordinates between the UAV and the target based on the line-of-sight angle; S4. Construct an adaptive Kalman filter framework based on an autoregressive prediction model, as follows: S41. Based on the polynomial approximation characteristics of vehicle motion state, the autoregressive prediction model is embedded into the Kalman filter, and the state vector is defined as the historical position sequence. The process is as follows: ； in Represents the state vector, representing the past. The position measurement value at each sampling time, Indicates the current position. Indicates the position at the previous moment. Indicates the order of the autoregressive prediction model; S42. Using Weierstrass's theorem, the vehicle trajectory is modeled as an Nth-order polynomial, and the AR prediction equation is established. The process is as follows: ； in Indicates the past Location information at any given time This represents the predicted position at the next moment. Represents the coefficients of the AR model. The following conditions must be met: ； ； S43. By constructing the Vandermonde matrix and constraint vectors, the AR prediction equation is transformed into a convex quadratic programming closed form, as follows: ； in , Represents the state covariance matrix. Represents the Vandermonde matrix. Represents the constraint vector; S44. Solve the closed-form convex quadratic programming problem using the Lagrange multiplier method. The process is as follows: ； in The column vector representing the coefficients of the AR model is used to obtain the adaptive Kalman filter framework based on the autoregressive prediction model. S5. Based on the adaptive Kalman filter framework and the updated process noise covariance, the final predicted value is obtained, and the process noise covariance is updated by dynamically adjusting the process noise covariance using the real-time innovative sequence.

2. The method for estimating the motion state of a UAV-borne real-time ground target according to claim 1, characterized in that, The process of identifying and locking ground objects from the image in S1 is as follows: S11. Input image data into the backbone network; S12 and YOLOv4 use CSPDarknet53 in the backbone network to reduce gradient redundancy through cross-stage partial connections, as follows: ； in This represents the output feature map, the final feature representation after cross-stage partial connection processing. This represents the segmentation portion of the input feature map. This represents the portion of the input feature map that has not undergone convolution processing. This indicates a splicing operation. This is a convolution operation; S13. Perform spatial pyramid pooling at the end of the backbone to obtain the features of the input image. The process is as follows: ； in Indicates the input feature map, Indicates the kernel size. This indicates a max pooling operation. Indicates the features of the output image; S14. Input all image features into the neck network. YOLOv4 uses the PANet structure to fuse them through a bidirectional feature pyramid to obtain the final fused features, and then inputs them into the head network. S15. After the head network receives the final fused features, it adopts... The loss function is used to calculate the error between the predicted and the actual values ​​using a weighted average. The process is as follows: ； in express loss function The wall ratio representing the overlapping area to the total area. This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. Indicates the coordinates of the center of the prediction box. Represents the coordinates of the center of the true bounding box. This represents the diagonal length of the smallest bounding box that simultaneously covers both the predicted and ground truth boxes. This represents the weighting coefficient of the aspect ratio penalty term. This indicates a measure of aspect ratio consistency. S16. After training, use 1×1 convolution to adjust the number of channels, output the predicted parameters of each anchor box, obtain the final detection result, and lock the ground target object. The process is as follows: ； in The first one obtained in the neck network Layer fusion characteristics, Indicates the first Predicted parameters for each anchor box. This represents a 1×1 convolution.

3. The method for estimating the motion state of a UAV-borne real-time ground target according to claim 2, characterized in that, The process of obtaining the final fusion feature in S14 is as follows: S141. After upsampling the high-level semantic features in the image features, they are concatenated with the mid-level features. The process is as follows: ； in This represents the feature concatenation result of the top-down path. This represents high-level semantic features in image features. This represents mid-level features in image features. Indicates an upsampling operation. Indicates the convolution operation; S142. After downsampling the shallow detail features in the image features, they are concatenated with the mid-level features. The process is as follows: ； in This represents the feature concatenation result of a bottom-up path. Indicates shallow detail features. Indicates a downsampling operation; S143, will and The fusion process is as follows: ； in This indicates the fusion result.

4. The method for estimating the motion state of a UAV-borne real-time ground target according to claim 3, characterized in that, The process of S2 is as follows: S21. Employ the kernelized correlation filter algorithm and combine it with the prediction parameters obtained in S1 to perform dynamic target tracking of the target object; S22. Determine the projected coordinates of the target object in the camera's normalized coordinate system. The process is as follows: ； ； in Represents the normalized coordinates of the target in the camera plane. This represents the pixel coordinates of the target in the image. Represents the coordinates of the principal point in the image. Indicates the camera's focal length; S23. Based on the pinhole camera model and normalized coordinates, calculate the azimuth and pitch angles of the target object's line of sight. The process is as follows: ； ； in This represents the azimuth angle of the target's line of sight in camera coordinates. This represents the target's pitch angle in camera coordinates.

5. The method for estimating the motion state of a UAV-borne real-time ground target according to claim 4, characterized in that, The process of dynamic target tracking of the target object in S21 is as follows: S211. Based on the initial position of the target obtained in S1, take the image patch containing the target and extract its features; S212. Divide the features in S211 into background and target object to obtain the initial filter, and then set the filter coefficients... The features are defined as the initial template and the filter coefficients. The calculation process is as follows: ； in Indicates the expected response spectrum. Represents the regularization coefficient. This is the diagonalized frequency domain representation of the kernel matrix; S213. Extract an image patch from the image at the next time step. The position of the image patch is the same as the position of the image patch extracted in S211. Extract the features at this time as candidate regions. S214. The Gaussian kernel function measures the similarity between the candidate region and the template. The process is as follows: ； in Indicates template features, Indicates the characteristics of the candidate region. This represents the kernel function bandwidth parameter; S215, according to The response map is obtained, along with its maximum and minimum values. Tracking quality is dynamically assessed, and the object's displacement in the image is calculated using the displacement of the point with the largest response. The new target location is then obtained, template features are recalculated, the filter is updated, and the process returns to S213. The dynamic assessment of tracking quality is as follows: ； in Indicates the average peak correlation energy. and These represent the maximum and minimum values ​​of the response graph, respectively. Indicates the first in the response graph line, number The response intensity of each column pixel.

6. The method for estimating the motion state of a UAV-borne real-time ground target according to claim 5, characterized in that, The process of establishing a target location detection model in S3 and obtaining the relative coordinates is as follows: S31. Establish the body coordinate system ,in The axis moves forward along the longitudinal axis of the drone. The axis is perpendicular to the drone's longitudinal plane, pointing to the right, and is determined according to the Cartesian coordinate system rules. axis; S32. Based on the positional relationship between the UAV and the target object, determine the relative coordinates of the target object in the UAV coordinate system. The process is as follows: ； ； ； in This indicates the position of the target object in the machine's coordinate system. This indicates the line of sight between the loitering munition and the target. plane and The included angle of the axis, This indicates the line of sight between the loitering munition and the target. plane and The included angle, and define Downward is positive. This indicates the straight-line distance between the drone and the target; S33. Transform the target position in the body coordinate system to the inertial frame, and obtain the relative coordinates of the UAV and the target in the inertial frame. The process is as follows: ； ； ； ； in Indicates wrapping around the body The roll rotation matrix of the axis, Indicates wrapping around the body The pitch and rotation matrix of the axis. Indicates wrapping around the body The yaw rotation matrix of the axis. This represents the total transformation matrix from the body coordinate system to the inertial coordinate system. , and Indicates the attitude angle of the drone; S34. The relative coordinates of the UAV and the target in the inertial frame are obtained using the following formula: ； in This represents the relative coordinates between the UAV and the target in the inertial frame.

7. The method for estimating the motion state of a UAV-borne real-time ground target according to claim 6, characterized in that, The process of obtaining the final predicted value in S5 is as follows: S51. Predict the current state based on the state estimate from the previous time step and the control input at the current time step, as follows: ； in This represents the predicted state value at the current moment. Represents the state transition matrix. This represents the state estimate at the previous moment; S52. Estimate the covariance matrix based on the state at the previous time step, and predict the covariance matrix for the current time step. The process is as follows: ； in This represents the covariance matrix of the current state prediction. This represents the state estimation covariance matrix at the previous time step. This represents the process noise covariance matrix at the previous time step; S53. Determine the degree of correction of the measured value to the state estimate, as follows: ； in Represents the Kalman gain matrix. Represents the observation matrix. This represents the measurement noise covariance matrix at the current moment; S54. Correct the state prediction value using the measurement value at the current time to obtain the state estimate value at the current time. The process is as follows: ； in This represents the estimated state value at the current moment. This represents the measurement value at the current moment; S55. Update the state estimation covariance matrix, the process is as follows: ； in This represents the state estimation covariance matrix at the current moment; S56. Utilizing real-time innovative sequences to dynamically adjust process noise covariance The process is as follows: ； ； ； in Indicates a real-time innovation sequence. Represents the observed value. This represents the covariance of the innovation samples.

Images