A lightweight recognition and tracking method for ground target terminal guidance

Abstract

The present invention discloses a lightweight recognition and tracking method for terminal guidance of ground targets, which relates to the fields of artificial intelligence and unmanned aerial vehicle control technology. S1: Obtain a first-person perspective image through an onboard camera; S2: Input the obtained image into a recognition network, which infers the first-person perspective image to obtain ground target information; S3: Construct a value function based on the confidence level of the integrated object attributes and the object's motion state, and compare it with a preset threshold. If the value function is greater than the set threshold, the identified target is considered to be a strike target, and the tracking module is activated; S4: The tracking module tracks the strike target and calculates the relative line of sight angle of the strike target relative to the position of the fuselage based on the position of the strike target in the image, and transmits the result to the guidance module to control the aircraft to fly toward the strike target, thereby achieving a terminal-guided strike on the target. The present invention solves the problems of the large computing power requirements of the UAV visual terminal guidance neural network, which is difficult to deploy on edge devices and causes misidentification.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and unmanned aerial vehicle (UAV) control technology, and in particular to a lightweight identification and tracking method for terminal guidance of ground targets. Background Art

[0002] As intelligent combat systems integrating multiple technologies in modern local information warfare, drones (UAVs) can conduct reconnaissance, surveillance, and suppression over target areas, and can autonomously output guidance signals from their seekers based on battlefield dynamics. Network collaboration, miniaturization, intelligence, and low cost are key trends in their future development. The guidance process for UAVs involves the seeker acquiring target information and applying normal overload according to a designed guidance rate to steer the aircraft along the desired trajectory. Early guidance modes were relatively simple. With the iterative upgrades of individual guidance technologies and the increasing complexity of operational requirements, modern guidance technologies are becoming increasingly complex and intelligent, with an emphasis on mitigating the impact of electromagnetic interference on sensor performance. However, challenges remain, such as the high computing power requirements of UAV visual terminal guidance neural networks, the difficulty of deploying them on edge devices, and the potential for misidentification.

[0003] Therefore, it is necessary to provide a lightweight identification and tracking method for ground target terminal guidance to solve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to provide a lightweight recognition and tracking method for terminal guidance of ground targets, which solves the problems of high computing power requirements of UAV visual terminal guidance neural networks, difficulty in deployment on edge devices, and misidentification. Through the neural network method, target features can be extracted in multiple dimensions, and the recognition algorithm can reduce the missed detection rate of recognition, greatly improving the accuracy and robustness of recognition and tracking.

[0005] To achieve the above object, the present invention provides a lightweight identification and tracking method for ground target terminal guidance, comprising the following steps:

[0006] S1: Obtain first-person perspective images through the onboard camera;

[0007] S2: The obtained image is input into the recognition network, which infers the first-person view image to obtain ground target information. The target information includes whether there is a suspected target and the confidence level of the recognized target;

[0008] S3: A value function is constructed by integrating the object attributes and the object's motion state through confidence, and compared with a preset threshold. If the value function is greater than the threshold, the identified target is considered to be a strike target and the tracking module is activated;

[0009] S4: The tracking module tracks the target and calculates the relative sight angle of the target relative to the fuselage position based on the target's position in the image, and transmits it to the guidance module to control the aircraft to fly towards the target and achieve terminal guidance strike on the target.

[0010] Preferably, in step S2, the recognition network adopts an improved YOLOV8 network, including network pruning, value function and weight quantization; network pruning visualizes the preferred input of each layer of neurons by maximizing activation AM, and represents the learned features with a synthetic input pattern that causes maximum activation of neurons.

[0011] Preferably, the basic algorithm for maximizing activation is considered to synthesize a pattern image to maximize the activation of the target neuron. The process is expressed as:

[0012] x=argmaxh m,n (θ,x)

[0013] where h m,n (θ, x) represents the activation of a given convolutional filter n in a given layer m in the network, and θ represents the weight parameters of the network;

[0014] Starting from a given layer m of the network, the activation maximization process is divided into the following three stages:

[0015] Phase 1: Set a random pixel value image as the starting point for activation calculation;

[0016] Phase 2: Fix the network parameters and calculate the gradient with respect to the noisy image through backpropagation

[0017] The third stage: in the gradient Guided by the direction, iterate each pixel to maximize neuron activation;

[0018] In the third stage, Where α represents the gradient ascent step size;

[0019] In order to stabilize the numerical calculation, normalization is performed by the following formula:

[0020]

[0021] Where ε<0.001, to prevent the denominator from being zero;

[0022] By calculation The average value of is used to indicate the importance of the convolution filter;

[0023]

[0024] Where K is the iteration step size and L is the dimension of the convolution filter.

[0025] Preferably, in step S3, a value function V is constructed by integrating the speed of the target, the target azimuth angle, the recognized category, and the confidence level. The value function is as follows:

[0026] V = αV v + βV θ + γV c + δV Z

[0027] Where: α, β, γ, δ are the weights of the speed of the target, the target azimuth angle, the recognized category, and the confidence level respectively. α, β, γ, δ take values of 0.2, 0.2, 0.3, 0.3 respectively. V v , V θ , V θ , V Z respectively represent the speed of the target, the target azimuth angle, the recognized category, and the confidence level;

[0028] Among them, V v = 1 - e a·v , where: V v is the speed value of the target, v is the speed of the air-ground target, a is a constant coefficient, and the value is -0.005;

[0029]

[0030] Where: V θ is the azimuth angle value of the target, θ is the azimuth angle of the target, and the value range is 0° to 180°;

[0031]

[0032] Where: V c is the category value of the target, a + b + c = 1, a > b > c;

[0033] V Z = C

[0034] Where: V Z is the confidence level value of the target, C is the recognized confidence level, 0 < C < 1;

[0035] From the above formula, 0 < V < 4 can be obtained. If there is only one target in the field of view and when the value function V of the target > 2.5, the recognized target is considered as the strike target. When there are multiple targets in the field of view, the target with a value function value greater than the threshold 2.5 and the highest score is selected as the strike target.<00001​Preferably, weight quantization includes PTQ quantization, which starts from a pre-trained model and is quantized using a quantization dataset, which is a subset of the training dataset. During the quantization process, the dynamic range of weights and activation values ​​is calculated, the quantization parameters are determined, and the model is finally quantized using the quantization parameters. Weight quantization converts activation values ​​and weights from FP32 to INT8 through linear mapping.

[0037] Preferably, in step S4, the final output result of the recognition network is H = [x, y, w, h, Value], where x, y represent the center coordinates of the bounding box in the image coordinate system, w, h represent the width and height of the bounding box relative to the entire pixel, and Value is a value function that integrates confidence, target attributes, and motion state; the tracking module uses an 8-dimensional state matrix [u, v, r, h, u', v', r', h'] as a direct observation model of the target, where (u, v) is the center coordinate of the bounding box, (r, h) are the aspect ratio and height of the bounding box, respectively, and u', v', r', h' are the first-order derivatives of the first four values. The following formula is used to predict parameters such as the target frame position and velocity of the current frame:

[0038]

[0039] P t|t =P t|t-1 +K t ·H t ·P t|t-1

[0040]

[0041] in is the prior prediction value of the system state at time t based on the system parameters at time t-1, P t|t-1 is the prior prediction error, X t|t is the posterior prediction value of the system state after the update at time t, K t is the Kalman gain, Z t is the observed value, H t is the transfer matrix; the motion state matrix at that moment is obtained; after tracking is achieved, the sight angle of the target relative to itself is calculated according to the tracking frame information [x, y, w, h]:

[0042] Cam=K -1 I

[0043] α=tan -1 (Cam[0],Cam[2])

[0044] β=-1*tan -1 (Cam[1],Cam[2])

[0045] In the above formula, K represents the camera intrinsic parameter, i is the pixel coordinate of the target [x, y, 1], the 1 in [x, y, 1] is the supplementary dimension, α is the horizontal viewing angle, and β is the vertical viewing angle.

[0046] Preferably, in step S4, during the guidance process of the guidance module, the angular velocity of the UAV velocity vector is proportional to the angular velocity of the target line, and the guidance equation is as follows:

[0047]

[0048]

[0049] where a n is the acceleration perpendicular to the instantaneous velocity vector of the UAV, N is the proportional constant, which is a dimensionless integer value between 3 and 5, is the rate of change of the sight angle, V is the closing speed; integrate both sides to get the pitch angle

[0050]

[0051] The roll angle is obtained by the following formula:

[0052] λ=cos(-φ)λ h +sin(φ)λ v

[0053]

[0054] where λ h is the body line of sight deflection angle, λ v is the body line of sight deflection angle, φ d It is the lateral roll angle. After obtaining the pitch angle and roll angle, the target guidance is achieved.

[0055] Therefore, the present invention adopts the above-mentioned lightweight identification and tracking method for terminal guidance of ground targets, which has the following beneficial effects:

[0056] (1) The present invention simplifies the network structure through network pruning and weight quantization technology, reducing the network's dependence on computing power.

[0057] (2) The present invention can extract target features in multiple dimensions through the neural network method, and can reduce the missed detection rate of recognition through the recognition algorithm, greatly improving the accuracy and robustness of recognition and tracking.

[0058] (3) The present invention introduces a value function, which comprehensively considers recognition confidence, target attributes and motion status when identifying the target, and can manually end the strike, reducing the false detection rate and enhancing the security of the algorithm.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of a lightweight identification and tracking method for terminal guidance of ground targets according to the present invention;

[0061] Figure 2 This is a framework diagram of the UAV terminal guidance strike process of the present invention;

[0062] Figure 3 is a flow chart of network pruning of the present invention;

[0063] Figure 4 It is a flow chart of the model weight quantization of the present invention. DETAILED DESCRIPTION

[0064] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0065] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0066] The words “include” or “comprising” and similar words used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms “inside”, “outside”, “upper”, “lower”, etc. is based on the orientation or position relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly stipulated and limited, the terms such as “attachment” should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral whole; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0067] like Figure 1 As shown, the present invention provides a lightweight identification and tracking method for ground target terminal guidance, comprising the following steps:

[0068] S1: Obtain first-person perspective images through the onboard camera;

[0069] S2: The obtained image is input into the recognition network, which infers the first-person view image to obtain ground target information. The target information includes whether there is a suspected target and the confidence level of the recognized target;

[0070] In step S2, the recognition network uses a modified YOLOV8 network, including network pruning, value functions, and weight quantization. Neural network pruning techniques, which remove side branches that have minimal impact on the results, effectively enhance network simplicity and reduce computing power requirements while maintaining accuracy. Network pruning uses activation maximization (AM) to visualize the preferred inputs of neurons in each layer, representing the learned features as a synthetic input pattern that maximizes neuron activation.

[0071] The basic algorithm for maximizing activation is considered to synthesize a pattern image to maximize the activation of the target neuron. The process is expressed as:

[0072] x=argmaxh m,n (θ,c)

[0073] where h m,n (θ, x) represents the activation of a given convolutional filter n in a given layer m in the network, and θ represents the weight parameters of the network;

[0074] Starting from a given layer m of the network, the activation maximization process is divided into the following three stages:

[0075] Phase 1: Set a random pixel value image as the starting point for activation calculation;

[0076] Phase 2: Fix the network parameters and calculate the gradient with respect to the noisy image through backpropagation

[0077] The third stage: in the gradient Guided by the direction, iterate each pixel to maximize neuron activation;

[0078] In the third stage, Where α represents the gradient ascent step size;

[0079] In order to stabilize the numerical calculation, normalization is performed by the following formula:

[0080]

[0081] Where ε<0.001 is a very small amount to prevent the denominator from being zero;

[0082] By calculation The average value of is used to indicate the importance of the convolution filter;

[0083]

[0084] Where K is the iteration step size and L is the dimension of the convolution filter. The AM method can be a good indicator of the working of the convolution filter. A large AM gradient indicates that the filter has a strong influence on the network and should be retained. A filter with zero AM gradient has zero effect and does not learn anything from the dataset. It can be safely pruned without much loss of accuracy.

[0085] The YOLOV8 recognition network after network pruning can greatly reduce the network's demand for computing power, such as Figure 3 As shown in Figure 3, the AM method also reduces the impact of network pruning on recognition accuracy and improves recognition efficiency.

[0086] S3: A value function is constructed by integrating the object attributes and the object's motion state through confidence, and compared with a preset threshold. If the value function is greater than the threshold, the identified target is considered to be a strike target and the tracking module is activated;

[0087] In step S3, a value function V is constructed by integrating the target speed, target azimuth, recognized category, and confidence. The value function is as follows:

[0088] V=αV v +βV θ +γV c +δV Z

[0089] Where: α, β, γ, δ are the target speed, target azimuth, recognition category and confidence weight respectively. α, β, γ, δ are 0.2, 0.2, 0.3, 0.3 respectively. V v ,V θ ,V θ ,V Z Represent the target's speed, target azimuth, recognized category and confidence respectively;

[0090] Among them, V v =1-e a·v , where: V v is the target's velocity value, v is the velocity of the air-ground target, and a is a constant coefficient with a value of -0.005;

[0091]

[0092] Where: V θ is the azimuth value of the target, θ is the azimuth angle of the target, and its value range is 0°~180°;

[0093]

[0094] Where: V cThe category value for the target, where a + b + c = 1, a > b > c;

[0095] V Z = C

[0096] In the formula: V Z is the confidence value for the target, C is the confidence of recognition, 0 < C < 1;

[0097] From the above formula, 0 < V < 4 can be obtained. If there is only one target in the field of view and when the value function V of the target > 2.5, it is considered that the recognized target is the strike target. When there are multiple targets in the field of view, the target with a value function value greater than the threshold 2.5 and the highest score is selected as the strike target.

[0098] Weight quantization includes PTQ quantization, and the process is as Figure 4 shown. Starting from the pre-trained model, a quantization dataset is used for quantization. The quantization dataset is a subset of the training dataset; during the quantization process, the dynamic ranges of weights and activation values are calculated to determine the quantization parameters and finally the model is quantized using the quantization parameters. Weight quantization converts activation values and weights from FP32 to INT8 through a linear mapping method. At the same time, in order to avoid bit-width waste caused by simple quantization methods, which may lead to a decrease in accuracy. While ensuring that the accuracy meets the conditions, a larger model simplification and performance improvement can be obtained.

[0099] S4: The tracking module tracks the strike target and calculates the relative line-of-sight angle of the strike target with respect to the position of the fuselage based on the position of the strike target in the image, and transmits it to the guidance module to control the aircraft to fly towards the strike target, achieving terminal guidance and strike on the target. In step S4, the final output result of the recognition network is H = [x, y, w, h, Value], where x and y represent the center coordinates of the bounding box in the image coordinate system, w and h represent the width and height of the bounding box relative to the entire pixel, and Value is the value function that synthesizes confidence, target attributes, and motion state; the tracking module uses an 8-dimensional state matrix [u, v, r, h, u', v', r', h'] as the direct observation model of the target, where (u, v) are the center coordinates of the bounding box, (r, h) are the aspect ratio and height of the bounding box respectively, and u', v', r', h' are the first-order derivatives of the first four values. The following formula is used to predict parameters such as the target frame position and speed in the current frame:

[0100]

[0101] P t|t = P t|t-1 + K t ·H t ·P t|t-1

[0102]

[0103] in is the prior prediction value of the system state at time t based on the system parameters at time t-1, P t|t-1 is the prior prediction error, X t|t is the posterior prediction value of the system state after the update at time t, K t is the Kalman gain, Z t is the observed value, H t is the transfer matrix; the motion state matrix at that moment is obtained; after tracking is achieved, the sight angle of the target relative to itself is calculated according to the tracking frame information [x, y, w, h]:

[0104] Cam=K -1 I

[0105] α=tan -1 (Cam[0],Cam[2])

[0106] β=-1*tan -1 (Cam[1],Cam[2])

[0107] In the above formula, K represents the camera intrinsic parameter, I is the pixel coordinate of the target [x, y, 1], the 1 in [x, y, 1] is the supplementary dimension, α is the horizontal viewing angle, and β is the vertical viewing angle.

[0108] In step S4, during the guidance process of the guidance module, the angular velocity of the UAV velocity vector is proportional to the angular velocity of the target line. The guidance equation is as follows:

[0109]

[0110]

[0111] where a n is the acceleration perpendicular to the instantaneous velocity vector of the UAV, N is the proportional constant, which is a dimensionless integer value between 3 and 5, is the rate of change of the sight angle, V is the closing speed; integrate both sides to get the pitch angle

[0112]

[0113] The roll angle is obtained by the following formula:

[0114] λ=cos(-φ)λ h +sin(φ)λ v

[0115]

[0116] where λ h is the body line of sight deflection angle, λv is the body line of sight deflection angle, φ d It is the lateral roll angle. After obtaining the pitch angle and roll angle, the target guidance is achieved.

[0117] Therefore, the present invention adopts the above-mentioned lightweight recognition and tracking method for ground target terminal guidance, which solves the problem that the UAV visual terminal guidance neural network has high computing power requirements and is difficult to deploy on edge devices and causes misidentification.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A lightweight identification and tracking method for ground target terminal guidance, characterized by: It includes the following steps: S1: Obtain an image from the first perspective through an on-board camera; S2: Input the obtained image into the recognition network. The recognition network infers the first-perspective image to obtain ground target information. The target information includes whether there is a suspected target and the confidence of the recognized target; S3: The confidence constructs a value function by integrating object attributes and object motion states, and compares it with a preset threshold. If the value function is greater than the set threshold, the recognized target is considered a strike target, and the tracking module is activated; S4: The tracking module tracks the strike target and calculates the relative line-of-sight angle of the strike target relative to the fuselage position based on the position of the strike target in the image, and transmits it to the guidance module to control the aircraft to fly towards the strike target, realizing the terminal guidance strike on the target; In step S2, the recognition network uses an improved YOLOV8 network, including network pruning, value function, and weight quantization; Network pruning visualizes the preferred input of each layer of neurons through activation maximization (AM), and represents the learned features with a synthetic input pattern that maximizes the activation of neurons; In step S3, the value function is constructed by integrating the target speed, target azimuth, recognized category and confidence , the value function is as follows: Where: are the target speed, target azimuth, recognized category and confidence weight respectively, The values ​​are 0.2, 0.2, 0.3, 0.3 respectively. Represent the target's speed, target azimuth, recognized category and confidence respectively; in, , where: is the target velocity value, is the speed of the air-to-ground target, is a constant coefficient with a value of -0.005; Where: is the azimuth value of the target, is the azimuth of the target, and its value range is ; ； Where: is the category value of the target, ; Where: is the confidence value of the target, C is the confidence of recognition, 0 <C<1; From the above formula, it can be obtained that 0 < V < 4. If there is only one target in the field of view and the value function V of the target > 2.5, the recognized target is considered a strike target. When there are multiple targets in the field of view, the target with a value function value greater than the threshold 2.5 and the highest score is selected as the strike target.

2. The terminally guided lightweight identification and tracking method for ground targets according to claim 1, characterized in that: The basic algorithm of activation maximization is regarded as synthesizing a pattern image to maximize the activation of target neurons, and the process is expressed as: in represents the activation of a given convolutional filter n in a given layer m in the network, Represents the weight parameters of the network; Starting from a given layer m of the network, the activation maximization process is divided into the following three stages: The first stage: Set a random pixel value image as the starting point for activation calculation; Phase 2: Fix the network parameters and calculate the gradient with respect to the noisy image through backpropagation ; The third stage: in the gradient Guided by the direction, iterate each pixel to maximize neuron activation; In the third stage, , where represents the gradient ascent step size; To stabilize numerical calculations, normalization is performed through the following formula: in ; By calculation The average value of is used to indicate the importance of the convolution filter; in is the iteration step length, is the dimension of the convolution filter.

3. The terminally guided lightweight identification and tracking method for ground targets according to claim 2, characterized in that: Weight quantization includes post-training quantization (PTQ). Starting from the pre-trained model, a quantization dataset is used for quantization. The quantization dataset is a subset of the training dataset; Quantization During the process, the dynamic ranges of weights and activation values are calculated to determine the quantization parameters and finally the model is quantized using the quantization parameters. Weight quantization converts activation values and weights from FP32 to INT8 through a linear mapping method.

4. The terminally guided lightweight identification and tracking method for ground targets according to claim 3, characterized in that: In step S4, the final output of the recognition network is ,in Represents the center coordinates of the bounding box in the image coordinate system, represents the width and height of the bounding box relative to whole pixels, It is a value function that integrates confidence, target attributes and motion state; the tracking module uses an 8-dimensional state matrix , as the direct observation model of the target, where are the center coordinates of the bounding box, are the aspect ratio and height of the bounding box, respectively, is the first-order derivative of the first four values, and the following formula is used to predict the target frame position and velocity parameters of the current frame: in is the prior prediction value of the system state at time t based on the system parameters at time t-1, is the prior prediction error, is the posterior prediction value of the system state after the update at time t, is the Kalman gain, is the observed value, is the transfer matrix; get the motion state matrix at that moment; After tracking is achieved, according to the tracking frame information Calculate the target's sight angle relative to itself: In the above formula Represents the camera internal parameters, The pixel coordinates of the target , The 1 is the supplementary dimension, is the horizontal sight angle, is the vertical sight angle.

5. The terminal-guided lightweight identification and tracking method for ground targets according to claim 4, characterized in that: In step S4, during the guidance process of the guidance module, the rotational angular velocity of the UAV speed vector is proportional to the rotational angular velocity of the target line. The guidance equation is as follows: in is the acceleration perpendicular to the instantaneous velocity vector of the UAV, is the proportionality constant, a dimensionless integer value between 3 and 5, is the rate of change of sight angle, is the closing speed; Integrate both sides to get the pitch angle : The roll angle is obtained through the following formula: in is the body line of sight deflection angle, is the body line of sight deflection angle, It is the lateral roll angle. After obtaining the pitch angle and roll angle, the target guidance is achieved.

Images