Lightweight identification and tracking method for ground target terminal guidance

Abstract

The invention discloses a ground target terminal guidance-oriented lightweight identification and tracking method, and relates to the technical field of artificial intelligence and unmanned aerial vehicle control, and the method comprises the steps: S1, obtaining an image of a first visual angle through an airborne camera; s2, inputting the obtained image into a recognition network, and reasoning the first view angle image by the recognition network to obtain ground target information; s3, constructing a value function by integrating the object attribute and the object motion state according to the confidence degree, comparing the value function with a preset threshold value, if the value function is greater than the preset threshold value, determining that the recognition target is a striking target, and starting a tracking module; s4, the tracking module tracks the striking target and calculates the relative sight angle of the striking target relative to the fuselage position according to the position of the striking target in the image, the relative sight angle is transmitted to the guidance module, the aircraft is controlled to fly to the striking target, and terminal guidance striking of the target is achieved. The method solves the problem that the unmanned aerial vehicle visual terminal guidance neural network is difficult to deploy at edge equipment and misrecognize due to large computing power demand.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and unmanned aerial vehicle control technology, and in particular to a terminal-guided lightweight identification and tracking method for ground targets. Background Art

[0002] As an intelligent combat system integrating multiple technologies in modern local information warfare, UAVs can conduct reconnaissance, surveillance and suppression over target areas and output guidance signals from the seeker autonomously according to the battlefield situation. Network collaboration, small-scale intelligence and low cost are the main trends of their future development. The guidance process of UAVs is that the seeker obtains target information and generates normal overload according to the designed guidance rate to make it fly along the expected trajectory. The early guidance mode was relatively simple. With the iterative upgrade of each single guidance technology and the increasing complexity of combat requirements, modern guidance technology is constantly tending towards compounding and intelligence, and is committed to eliminating the impact of electromagnetic interference on sensor performance. However, there are still problems such as large computing power requirements for UAV visual terminal guidance neural networks, difficulty in deployment on edge devices, and misidentification.

[0003] Therefore, it is necessary to provide a lightweight identification and tracking method for ground target terminal guidance to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a lightweight recognition and tracking method for terminal guidance of ground targets, which solves the problem that the UAV visual terminal guidance neural network has a large demand for computing power and is difficult to deploy on edge devices and causes misidentification. Through the neural network method, target features can be extracted in multiple dimensions, and the recognition algorithm can reduce the missed detection rate of recognition, greatly improving the accuracy and robustness of recognition and tracking.

[0005] To achieve the above object, the present invention provides a terminally guided lightweight identification and tracking method for ground targets, comprising the following steps:

[0006] S1: Obtain first-person perspective images through the onboard camera;

[0007] S2: Input the obtained image into the recognition network, and the recognition network infers the first-person view image to obtain ground target information, including whether there is a suspected target and the confidence of the recognized target;

[0008] S3: The value function is constructed by integrating the object attributes and the object motion state through the confidence, and compared with the preset threshold. If the value function is greater than the set threshold, the identified target is considered to be a strike target, and the tracking module is turned on;

[0009] S4: The tracking module tracks the target and calculates the relative sight angle of the target relative to the position of the fuselage according to the position of the target in the image, and transmits it to the guidance module to control the aircraft to fly towards the target and achieve terminal guidance strike on the target.

[0010] Preferably, in step S2, the recognition network adopts an improved YOLOV8 network, including network pruning, value function and weight quantization; network pruning visualizes the preferred input of each layer of neurons by maximizing activation AM, and represents the learned features with a synthetic input pattern that causes maximum activation of neurons.

[0011] Preferably, the basic algorithm for maximizing activation is considered to synthesize a pattern image to maximize the activation of the target neuron, and the process is expressed as:

[0012] x=argmaxh m,n (θ,x)

[0013] where h m,n (θ, x) represents the activation of a given convolutional filter n in a given layer m in the network, and θ represents the weight parameters of the network;

[0014] Starting from a given layer m of the network, the activation maximization process is divided into the following three stages:

[0015] Phase 1: Set a random pixel value image as the starting point for activation calculation;

[0016] Phase 2: Fix the network parameters and calculate the gradient relative to the noisy image through back propagation

[0017] Stage 3: In the gradient Guided by the direction, iterate each pixel to maximize neuron activation;

[0018] In the third stage, Where α represents the gradient ascent step size;

[0019] In order to stabilize the numerical calculation, normalization is performed by the following formula:

[0020]

[0021] Where ε<0.001, to prevent the denominator from being zero;

[0022] By calculation The average value of is used to indicate the importance of the convolution filter;

[0023]

[0024] Where K is the iteration step size and L is the dimension of the convolution filter.

[0025] Preferably, in step S3, a value function V is constructed by integrating the speed of the target, the target azimuth angle, the recognized category and the confidence level. The value function is as follows:

[0026] V = αV v + βV θ + γV c + δV Z

[0027] Where: α, β, γ, δ are the weights of the speed of the target, the target azimuth angle, the recognized category and the confidence level respectively. α, β, γ, δ take the values of 0.2, 0.2, 0.3, 0.3 respectively. V v , V θ , V θ , V Z represent the speed of the target, the target azimuth angle, the recognized category and the confidence level respectively;

[0028] Among them, V v = 1 - e a·v , where: V v is the speed value of the target, v is the speed of the air-ground target, and a is a constant coefficient with a value of -0.005;

[0029]

[0030] Where: V θ is the azimuth angle value of the target, θ is the azimuth angle of the target, and the value range is 0° to 180°;

[0031]

[0032] Where: V c is the category value of the target, a + b + c = 1, a > b > c;

[0033] V Z = C

[0034] Where: V Z is the confidence level value of the target, C is the recognized confidence level, 0 < C < 1;

[0035] From the above formula, 0 < V < 4 can be obtained. If there is only one target in the field of view and when the value function V of the target > 2.5, the recognized target is considered as the strike target. When there are multiple targets in the field of view, the target with a value function value greater than the threshold 2.5 and the highest score is selected as the strike target.

[0036] Preferably, weight quantization includes PTQ quantization, which starts from a pre-trained model and is quantized using a quantization data set, which is a subset of the training data set. During the quantization process, the dynamic range of weights and activation values ​​is calculated, the quantization parameters are determined, and the model is finally quantized using the quantization parameters. Weight quantization converts activation values ​​and weights from FP32 to INT8 by linear mapping.

[0037] Preferably, in step S4, the final output result of the recognition network is H=[x, y, w, h, Value], where x, y represent the center coordinates of the bounding box in the image coordinate system, w, h represent the width and height of the bounding box relative to the entire pixel, and Value is a value function of the comprehensive confidence, target attributes and motion state; the tracking module uses an 8-dimensional state matrix [u, v, r, h, u', v', r', h'] as a direct observation model of the target, where (u, v) are the center coordinates of the bounding box, (r, h) are the aspect ratio and height of the bounding box, respectively, and u', v', r', h' are the first-order derivatives of the first four values, and the following formula is used to predict the parameters such as the target frame position and speed of the current frame:

[0038]

[0039] P t|t =P t|t-1 +K t ·H t ·P t|t-1

[0040]

[0041] in is the prior prediction value of the system state at time t based on the system parameters at time t-1, P t|t-1 is the prior prediction error, X t|t is the posterior prediction value of the system state after updating at time t, K t is the Kalman gain, Z t is the observed value, H t is the transfer matrix; the motion state matrix at that moment is obtained; after tracking is realized, the sight angle of the target relative to itself is calculated according to the tracking frame information [x, y, w, h]:

[0042] Cam=K -1 I

[0043] α=tan -1 (Cam[0],Cam[2])

[0044] β=-1*tan -1 (Cam[1],Cam[2])

[0045] In the above formula, K represents the camera intrinsic parameter, i is the pixel coordinate of the target [x, y, 1], 1 in [x, y, 1] is the supplementary dimension, α is the horizontal line of sight angle, and β is the vertical line of sight angle.

[0046] Preferably, in step S4, during the guidance process of the guidance module, the angular velocity of the UAV velocity vector is proportional to the angular velocity of the target line, and the guidance equation is as follows:

[0047]

[0048]

[0049] where a n is the acceleration perpendicular to the instantaneous velocity vector of the UAV, N is the proportional constant, which is a dimensionless integer value of 3-5, is the rate of change of sight angle, V is the closing speed; integrate both sides to get the pitch angle

[0050]

[0051] The roll angle is obtained by the following formula:

[0052] λ=cos(-φ)λ h +sin(φ)λ v

[0053]

[0054] where λ h is the stereo line of sight deflection angle, λ v is the body line of sight deflection angle, φ d It is the lateral rolling angle. After obtaining the pitch angle and rolling angle, the target guidance is achieved.

[0055] Therefore, the present invention adopts the above-mentioned lightweight identification and tracking method for terminal guidance of ground targets, which has the following beneficial effects:

[0056] (1) The present invention simplifies the network structure through network pruning and weight quantization technology, reducing the network's dependence on computing power.

[0057] (2) The present invention can extract target features in multiple dimensions through the neural network method, and can reduce the missed detection rate of recognition through the recognition algorithm, thereby greatly improving the accuracy and robustness of recognition and tracking.

[0058] (3) The present invention introduces a value function, which comprehensively considers recognition confidence, target attributes and motion status when identifying the target, and can manually end the strike, thereby reducing the false detection rate and enhancing the security of the algorithm.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of a lightweight identification and tracking method for terminal guidance of ground targets of the present invention;

[0061] Figure 2 This is a framework diagram of the UAV terminal guidance strike process of the present invention;

[0062] Figure 3 is a flow chart of network pruning of the present invention;

[0063] Figure 4 It is a flow chart of the model weight quantization of the present invention. DETAILED DESCRIPTION

[0064] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0065] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0066] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0067] like Figure 1 As shown, the present invention provides a lightweight identification and tracking method for terminal guidance of ground targets, comprising the following steps:

[0068] S1: Obtain first-person perspective images through the onboard camera;

[0069] S2: Input the obtained image into the recognition network, and the recognition network infers the first-person view image to obtain ground target information, including whether there is a suspected target and the confidence of the recognized target;

[0070] In step S2, the recognition network uses an improved YOLOV8 network, including network pruning, value function and weight quantization; subtracting some side branches that have little impact on the results through neural network pruning technology can effectively enhance the simplicity of the network and reduce the network's demand for computing power without losing network accuracy. Network pruning visualizes the preferred input of each layer of neurons by maximizing the activation AM, and represents the learned features with a synthetic input pattern that causes the maximum activation of neurons.

[0071] The basic algorithm for maximizing activation is considered to synthesize a pattern image to maximize the activation of the target neuron. The process is expressed as:

[0072] x=argmaxh m,n (θ,c)

[0073] where h m,n (θ, x) represents the activation of a given convolutional filter n in a given layer m in the network, and θ represents the weight parameters of the network;

[0074] Starting from a given layer m of the network, the activation maximization process is divided into the following three stages:

[0075] Phase 1: Set a random pixel value image as the starting point for activation calculation;

[0076] Phase 2: Fix the network parameters and calculate the gradient relative to the noisy image through back propagation

[0077] Stage 3: In the gradient Guided by the direction, iterate each pixel to maximize neuron activation;

[0078] In the third stage, Where α represents the gradient ascent step size;

[0079] In order to stabilize the numerical calculation, normalization is performed by the following formula:

[0080]

[0081] Among them, ε<0.001 is a very small amount to prevent the denominator from being zero;

[0082] By calculation The average value of represents the importance of the convolution filter;

[0083]

[0084] Where K is the iteration step size and L is the dimension of the convolution filter. The AM method can be used as a good representation of the convolution filter's work. A large AM gradient indicates that the filter has a great influence on the network and needs to be retained; a filter with zero AM gradient has zero effect and will not learn anything from the dataset, and can be safely pruned without losing too much accuracy.

[0085] The YOLOV8 recognition network after network pruning can greatly reduce the network's demand for computing power, such as Figure 3 As shown in the figure, the AM method also reduces the impact of network pruning on recognition accuracy and improves the recognition efficiency.

[0086] S3: The value function is constructed by integrating the object attributes and the object motion state through the confidence, and compared with the preset threshold. If the value function is greater than the set threshold, the identified target is considered to be a strike target, and the tracking module is turned on;

[0087] In step S3, the value function V is constructed by integrating the target speed, target azimuth, recognized category and confidence. The value function is as follows:

[0088] V=αV v +βV θ +γV c +δV Z

[0089] Where: α, β, γ, δ are the target speed, target azimuth, recognized category and confidence weights, α, β, γ, δ are 0.2, 0.2, 0.3, 0.3, V v ,V θ ,V θ ,V Z They represent the target’s speed, target azimuth, recognized category and confidence respectively;

[0090] Among them, V v =1-e a·v , where: V v is the speed value of the target, v is the speed of the air target, a is a constant coefficient, and its value is -0.005;

[0091]

[0092] Where: V θ is the azimuth value of the target, θ is the azimuth of the target, and its value range is 0°~180°;

[0093]

[0094] Where: V cThe category value of the target, where a + b + c = 1, a > b > c;

[0095] V Z = C

[0096] In the formula: V Z is the confidence value of the target, C is the confidence of recognition, and 0 < C < 1;

[0097] From the above formula, it can be obtained that 0 < V < 4. If there is only one target in the field of view and when the value function V of the target > 2.5, it is considered that the recognized target is the target to be struck. When multiple targets appear in the field of view, the target with a value function value greater than the threshold 2.5 and the highest score is selected as the target to be struck.

[0098] Weight quantization includes PTQ quantization, and the process is as Figure 4 shown. Starting from the pre-trained model, a quantization dataset is used for quantization. The quantization dataset is a subset of the training dataset; during the quantization process, the dynamic ranges of weights and activation values are calculated to determine the quantization parameters and finally the model is quantized using the quantization parameters. Weight quantization converts activation values and weights from FP32 to INT8 through a linear mapping method. At the same time, in order to avoid bit-width waste caused by simple quantization methods, which may lead to a decrease in accuracy. While ensuring that the accuracy meets the conditions, a greater model simplification and performance improvement can be achieved.

[0099] S4: The tracking module tracks the target to be struck and calculates the relative line-of-sight angle of the target to be struck with respect to the fuselage position based on the position of the target to be struck in the image, and transmits it to the guidance module to control the aircraft to fly towards the target to be struck, realizing the terminal guidance strike on the target. In step S4, the final output result of the recognition network is H = [x, y, w, h, Value], where x and y represent the center coordinates of the bounding box in the image coordinate system, w and h represent the width and height of the bounding box relative to the entire pixel, and Value is the value function that combines confidence, target attributes, and motion states; the tracking module uses an 8-dimensional state matrix [u, v, r, h, u', v', r', h'] as the direct observation model of the target, where (u, v) are the center coordinates of the bounding box, (r, h) are the aspect ratio and height of the bounding box respectively, and u', v', r', h' are the first-order derivatives of the first four values. The following formula is used to predict parameters such as the target frame position and speed in the current frame:

[0100]

[0101] P t|t = P t|t-1 + K t ·H t ·P t|t-1

[0102]

[0103] in is the prior prediction value of the system state at time t based on the system parameters at time t-1, P t|t-1 is the prior prediction error, X t|t is the posterior prediction value of the system state after updating at time t, K t is the Kalman gain, Z t is the observed value, H t is the transfer matrix; the motion state matrix at that moment is obtained; after tracking is realized, the sight angle of the target relative to itself is calculated according to the tracking frame information [x, y, w, h]:

[0104] Cam=K -1 I

[0105] α=tan -1 (Cam[0],Cam[2])

[0106] β=-1*tan -1 (Cam[1],Cam[2])

[0107] In the above formula, K represents the camera intrinsic parameter, I is the pixel coordinate of the target [x, y, 1], 1 in [x, y, 1] is the supplementary dimension, α is the horizontal line of sight angle, and β is the vertical line of sight angle.

[0108] In step S4, during the guidance process of the guidance module, the angular velocity of the UAV velocity vector is proportional to the angular velocity of the target line, and the guidance equation is as follows:

[0109]

[0110]

[0111] where a n is the acceleration perpendicular to the instantaneous velocity vector of the UAV, N is the proportional constant, which is a dimensionless integer value of 3-5, is the rate of change of sight angle, V is the closing speed; integrate both sides to get the pitch angle

[0112]

[0113] The roll angle is obtained by the following formula:

[0114] λ=cos(-φ)λ h +sin(φ)λ v

[0115]

[0116] where λ h is the stereo line of sight deflection angle, λv is the body line of sight deflection angle, φ d It is the lateral rolling angle. After obtaining the pitch angle and rolling angle, the target guidance is achieved.

[0117] Therefore, the present invention adopts the above-mentioned lightweight recognition and tracking method for terminal guidance of ground targets, which solves the problem that the UAV visual terminal guidance neural network has a large demand for computing power and is difficult to deploy on edge devices and causes misidentification.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A lightweight identification and tracking method for terminal guidance of ground targets, characterized by: The following steps are involved: S1: Obtain first-person perspective images through the onboard camera; S2: Input the obtained image into the recognition network, and the recognition network infers the first-person view image to obtain ground target information, including whether there is a suspected target and the confidence of the recognized target; S3: The confidence integrates the object attributes and the object motion state to construct a value function, which is compared with the preset threshold. If the value function is greater than the set threshold, the identified target is considered to be a strike target and the tracking module is turned on. S4: The tracking module tracks the target and calculates the relative sight angle of the target relative to the position of the fuselage according to the position of the target in the image, and transmits it to the guidance module to control the aircraft to fly towards the target and achieve terminal guidance strike on the target.

2. The method for terminally guided ground target identification and tracking according to claim 1, characterized in that: In step S2, the recognition network adopts an improved YOLOV8 network, including network pruning, value function and weight quantization; network pruning visualizes the preferred input of each layer of neurons through activation maximization AM, and represents the learned features with a synthetic input pattern that causes maximum activation of neurons.

3. The terminal-guided lightweight identification and tracking method for ground targets according to claim 2 is characterized in that: The basic algorithm for maximizing activation is considered to synthesize a pattern image to maximize the activation of the target neuron. The process is expressed as: x=argmaxh m，n (θ,x) where h m，n (θ, x) represents the activation of a given convolutional filter n in a given layer m in the network, and θ represents the weight parameters of the network; Starting from a given layer m of the network, the activation maximization process is divided into the following three stages: Phase 1: Set a random pixel value image as the starting point for activation calculation; The second stage: fix the network parameters and calculate the gradient relative to the noisy image through back propagation Stage 3: In the gradient Guided by the direction, iterate each pixel to maximize neuron activation; In the third stage, Where α represents the gradient ascent step size; In order to stabilize the numerical calculation, normalization is performed by the following formula: Where ε<0.001; By calculation The average value of is used to indicate the importance of the convolution filter; Where K is the iteration step size and L is the dimension of the convolution filter.

4. The terminal-guided lightweight identification and tracking method for ground targets according to claim 3 is characterized in that: In step S3, the value function V is constructed by integrating the target speed, target azimuth, recognized category and confidence. The value function is as follows: V=αV v +βV θ +γV c +δV Z Where: α, β, γ, δ are the target speed, target azimuth, recognized category and confidence weights, α, β, γ, δ are 0.2, 0.2, 0.3, 0.3, V v , V θ , V θ , V Z They represent the target’s speed, target azimuth, recognized category and confidence respectively; Among them, V v =1-e a·v , where: V v is the speed value of the target, v is the speed of the air target, a is a constant coefficient, and its value is -0.005; Where: V θ is the azimuth value of the target, θ is the azimuth of the target, and its value range is 0°180°; Where: V c is the category value of the target, a+b+c=1, a>b>c; In Z =C Where: V Z is the confidence value of the target, C is the confidence of recognition, 0<C<1; From the above formula, we can get 0<V<4. If there is only one target in the field of view and the target's value function V>2.5, the identified target is considered to be the strike target. When multiple targets appear in the field of view, the target with a value function value greater than the threshold of 2.5 and the highest score is selected as the strike target.

5. The terminal-guided lightweight identification and tracking method for ground targets according to claim 4 is characterized in that: Weight quantization includes PTQ quantization, which starts from the pre-trained model and uses a quantized dataset for quantization. The quantized dataset is a subset of the training dataset. Quantification In the process, the dynamic range of weights and activation values ​​is calculated, the quantization parameters are determined, and finally the quantization parameters are used to quantize the model. Weight quantization converts activation values ​​and weights from FP32 to INT8 through linear mapping.

6. The terminal-guided lightweight identification and tracking method for ground targets according to claim 5, characterized in that: In step S4, the final output result of the recognition network is H = [x, y, w, h, Value], where x, y represent the center coordinates of the bounding box in the image coordinate system, w, h represent the width and height of the bounding box relative to the entire pixel, and Value is the value function of the comprehensive confidence, target attributes and motion state; the tracking module uses an 8-dimensional state matrix [u, v, r, h, u′, v′, r′, h′] as a direct observation model of the target, where (u, v) is the center coordinate of the bounding box, (r, h) are the aspect ratio and height of the bounding box, u′, v′, r′, h′ are the first-order derivatives of the first four values, and the following formula is used to predict the target frame position and speed and other parameters of the current frame: P t|t =P t|t-1 +K t ·H t ·P t|t-1 in is the prior prediction value of the system state at time t based on the system parameters at time t-1, P t|t-1 is the prior prediction error, X t|t is the posterior prediction value of the system state after updating at time t, K t is the Kalman gain, Z t is the observed value, H t is the transfer matrix; get the motion state matrix at that moment; After tracking is achieved, the sight angle of the target relative to itself is calculated based on the tracking frame information [x, y, w, h]: Cam=K -1 ·I α=tan -1 (Cam[0],Cam[2]) β=-1*tan -1 (Cam[1],Cam[2]) In the above formula, K represents the camera intrinsic parameter, I is the pixel coordinate of the target [x, y, 1], 1 in [x, y, 1] is the supplementary dimension, α is the horizontal line of sight angle, and β is the vertical line of sight angle.

7. A terminally guided lightweight identification and tracking method for ground targets according to claim 6, characterized in that: In step S4, during the guidance process of the guidance module, the angular velocity of the UAV velocity vector is proportional to the angular velocity of the target line, and the guidance equation is as follows: where a n is the acceleration perpendicular to the instantaneous velocity vector of the UAV, N is the proportional constant, which is a dimensionless integer value of 3-5, is the rate of change of sight angle, V is the closing speed; integrating both sides, we get the pitch angle θ d : The roll angle is obtained by the following formula: λ=cos(-φ)λ h +sin(φ)λ v where λ h is the stereo line of sight deflection angle, λ v is the body line of sight deflection angle, φ d It is the lateral rolling angle. After obtaining the pitch angle and rolling angle, the target guidance is achieved.

Images