Four-axis aircraft target area positioning visual attack method based on attention mechanism

Abstract

The invention belongs to the technical field of unmanned aerial vehicles, and particularly provides a quadrotor target area positioning visual attack method based on an attention mechanism in order to solve the problem that in the prior art, an attacker makes an unmanned aerial vehicle misjudge a target object, and consequently the unmanned aerial vehicle tracks mistakenly and even crashes. Comprising the following steps: S1, screening out image frames with high contribution to an attack effect in a video stream based on an attention mechanism, screening out the image frames, and generating a thermodynamic diagram by using a Grad-CAM + + method, a focus in the thermodynamic diagram being the position of a target in the image frames; s2, taking the thermodynamic diagram as a mask, and fusing the thermodynamic diagram with an original image frame to generate an adversarial sample; and S3, injecting the confrontation sample into the video stream data of the unmanned aerial vehicle, so that the unmanned aerial vehicle cannot detect the target, and the normal execution of the tracking task is interfered. According to the method provided by the invention, the L2, the SSIM and the PSNR are respectively improved by 69.39%, 64.01% and 44.55% in the aspects of L2, SSIM and PSNR. In an experiment of performing visual attack on the TElo of the four-axis aircraft, the TElo tracking task can fail only by processing 4.25% of the video stream data.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a visual attack method for positioning a target area of ​​a quadcopter based on an attention mechanism. Background Art

[0002] Quadcopters are widely used in agricultural plant protection, smart monitoring, unmanned logistics, emergency rescue, and formation flying. They have unique advantages such as vertical take-off and landing, hovering in the air, and flying close to the ground, and can replace humans to perform complex tasks. However, in recent years, drone security issues have frequently occurred. The most famous incident is that hackers successfully used firmware vulnerabilities to control drones, which has attracted widespread attention from the society. At the same time, drone illegal flights and privacy leaks are common, which seriously disrupt social order. These security risks have prompted researchers to study drones from the perspective of attackers and defenders, explore vulnerabilities and propose solutions. With the increasing application of drones in multiple fields, the role of visual sensors has become increasingly prominent, but researchers often only focus on improving the performance of visual sensors and ignore their safety. Therefore, the security research of quadcopter visual sensors is urgent and necessary.

[0003] The visual sensor is one of the core components of the quadcopter. Its main function is to capture real-time video stream data of the surrounding environment. By inputting these video stream data into the deep learning network model, the system can perform various advanced visual tasks, such as target tracking, object detection, scene segmentation, visual obstacle avoidance, and autonomous navigation. However, although deep learning technology has greatly improved the performance of visual sensors, it also brings potential safety hazards. Specific neural network models may have vulnerabilities, allowing attackers to use generated adversarial samples to attack the model in a targeted manner. The adversarial samples are carefully designed, and small changes are enough to mislead the deep learning model and cause the system to make wrong judgments. In scenarios with high decision sensitivity requirements, adversarial samples may seriously affect the decision-making system. For example, in the process of drone tracking targets, attackers can use generated adversarial samples to deceive the target detection model, causing the drone to mistake non-target objects in the surrounding environment as target objects, or interfere with the drone so that it cannot correctly identify obstacles or dangerous objects in the surrounding environment. This attack may cause the drone to perform incorrect tracking tasks or even cause an unexpected collision and crash. Summary of the invention

[0004] Based on the above research background and existing technical problems, the present invention proposes a visual attack method for target area positioning of a quadcopter based on an attention mechanism. This method aims to fill the research gap in the security of the visual perception field of drones. At present, the research on adversarial sample attacks in the field of drones is still in its infancy, especially adversarial attack methods for drone visual tracking tasks have not been fully explored. As a high-speed moving aircraft, the adversarial sample attack of the quadcopter must be completed before the ground control station receives the video stream and responds to ensure the timeliness and effectiveness of the attack. This requires that the attack method must be highly real-time and able to generate and apply adversarial samples in a short time to quickly respond to the behavior of the drone. At the same time, since the operator can remotely observe the video collected by the drone, the adversarial sample attack must be carried out without arousing the operator's suspicion.

[0005] Therefore, the generated adversarial samples must be as similar as possible to normal image frames to avoid being discovered by the operator. To this end, the attack method needs to minimize the changes to the image while maintaining the attack effect, and maintain the normality of the drone's behavior to the greatest extent.

[0006] In order to achieve the above technical effects, the present invention provides a quadcopter target area positioning visual attack method based on an attention mechanism, comprising:

[0007] S1: Based on the attention mechanism, image frames with high contribution to the attack effect are selected from the video stream, and the Grad-CAM++ method is used to generate a heat map. The focus of the heat map is the position of the target in the image frame;

[0008] S2: Use the heat map as a mask and fuse it with the original image frame to generate adversarial samples;

[0009] S3: Inject adversarial samples into drone video stream data, making it impossible for the drone to detect the target and interfering with its normal tracking task.

[0010] Preferably, S1 comprises:

[0011] S1.1: For the input video stream data I, first take the first frame image as the query image I q , and then use the attention scoring function of the attention mechanism to score the subsequent image frames, which is the weight; if the weight of the image frame exceeds the threshold I th , then generate the heat map of the image frame and update the query image I q ; Otherwise, skip the image frame and continue to process subsequent frames by scoring and calculating weights, so as to filter out image frames in the video stream that contribute most to the attack effect;

[0012] S1.2: Input the filtered image frames into the Grad-CAM++ algorithm to obtain the heat map L (c) Grad-CAM++ :

[0013]

[0014] Among them, ReLU is the activation function, k∈{1,2,3,..,K} represents the feature map channel, α k It is the product of the gradient of the kth channel relative to category c and the global average pooling result of the channel.

[0015] Preferably, in S1.1, it is assumed that the image frame sequence I of a certain time period collected by the quadcopter is I={I 0 ,I 1 ,I 2 ,···,I n}, I m ∈I represents the mth image frame, I q ∈I represents the query image frame, score(I q ,I m ) is the attention scoring function:

[0016]

[0017] Among them, the attention scoring function score(I q ,I m ) measure the query image frame I q With the mth image frame I m The correlation between them.

[0018] Preferably, S2 is specifically:

[0019] Heat map L (c) Grad-CAM++ As a mask, P th represents the threshold imposed on the mask, and the adversarial sample X is expressed as:

[0020]

[0021] Among them, I (T) represents an image based on T iterations of the PGD algorithm, I r represents the original image frame, F(L (c) Grad-CAM++ ,P th ) represents the calculation formula of the mask.

[0022] Preferably, in S2, if the heat map L (c) Grad-CAM++ The value of the pixel position (r, c) in the image frame is greater than or equal to the threshold P th, the mask value is 1; otherwise it is 0;

[0023] F(L (c) Grad-CAM++ ) is expressed as follows:

[0024]

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) The present invention proposes an adversarial sample screening method based on the attention mechanism. Based on the similarity of consecutive image frames, image frames that contribute more to the attack effect in the video stream are screened out as attack samples, thereby reducing the computing resources required for the attack and improving the real-time performance of the method.

[0027] (2) The present invention proposes a Mask-PGD adversarial sample local attack method. The sensitive area in the image frame is located by the heat map generated by Grad-CAM++, and the heat map is used as a mask to superimpose the noise map generated by PGD to generate a perturbed sample, so that the designated target can be attacked more accurately and the concealment of the attack is improved.

[0028] (3) The present invention has carried out performance evaluation and physical verification. Experimental results show that compared with the traditional method, the proposed method has achieved 69.39%, 64.01% and 44.55% improvement in L2, SSIM and PSNR respectively. In the experiment of visual attack on the quadcopter Tello, only 4.25% of the video stream data needs to be processed to make the Tello tracking task fail. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0030] In the attached picture:

[0031] Figure 1 It is the visual attack process of the present invention;

[0032] Figure 2 Generate a framework diagram for adversarial examples;

[0033] Figure 3 For the quadcopter platform Tello;

[0034] Figure 4 Comparison of the attack effects against adversarial samples;

[0035] Figure 5Figure 1 shows the attack effect: (a) the original image collected by Tello; (b) the image processed by the target detection model; (c) the heat map generated based on the Grad-CAM++ method; (d) the heat map of the next frame generated based on the Grad-CAM++ method; (e) the output result of the target detection model; (f) the output result of the next frame of the target detection model;

[0036] Figure 6 This is a graph showing the flight altitude, yaw angle, pitch angle, and roll angle of the quadcopter changing with time. DETAILED DESCRIPTION

[0037] The following is combined with Figure 1 -Attached Figure 6 The preferred embodiments of the present invention are described. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0038] Example:

[0039] The quadcopter uses the camera on board to identify and track specific targets. The process is generally divided into four key steps:

[0040] First, the quadcopter collects video stream data of the surrounding environment and transmits it to the ground control station terminal. Then, the terminal's detection model D analyzes the image frame and obtains the target selection box (x top-left ,y top-left ,x bottom-right ,x bottom-right ); Secondly, the target selection box information is input into the controller C to calculate the attitude angle (ψ, θ, φ) and three-axis speed (v x ,v y ,v z ); Finally, the terminal transmits control commands to the quadcopter to enable it to continue tracking the target.

[0041] Therefore, if the attacker injects adversarial samples into the video stream of the quadcopter in step 1, so that the terminal cannot recognize the position of the target selection box in step 2, the controller in step 3 will generate incorrect attitude angle and three-axis velocity output, causing the quadcopter to be unable to continue tracking the target in step 4. At this point, the attacker has successfully caused the quadcopter to fail in the tracking task. Figure 1 shown.

[0042] The quadcopter is equipped with a camera and processor that enables it to identify and track specific targets. This visual tracking process is usually divided into three key steps: first, target detection, by using computer vision algorithms to detect targets in the video stream and obtain the target's selection box position information in the image frame; then, based on this selection box position information, the attitude angle and speed of the drone are adjusted to ensure that the aircraft maintains a suitable distance and direction from the target; finally, the drone tracks the target flight based on the adjusted information, which may involve real-time calculations of automatic flight control algorithms and attitude adjustment mechanisms.

[0043] However, if the attacker injects adversarial samples into the drone’s video stream in the first stage, such as adding interference or distortion, so that the drone cannot accurately identify the target selection box position information, this will hinder the drone’s ability to accurately adjust its attitude angle and speed in the second stage, causing the drone to be unable to continue tracking the target. The attacker thus successfully causes the drone to fail in the tracking task. The process of this quadcopter visual attack is as follows: Figure 1 shown.

[0044] In addition to the properties of the adversarial samples themselves, the attack location is also an important consideration for the attack effect of adversarial samples. At present, the center mapping method is mainly used for adversarial attacks, but the adversarial samples generated by this method are less effective, and inefficient and insufficient in generalization in practical applications. In contrast, the convolutional neural network visualization method can generate target feature heat maps, which accurately reflect the model's interest in different areas. Therefore, choosing to attack in areas of high model interest can achieve better and more stable attack effects.

[0045] Therefore, this embodiment proposes a quadcopter target area positioning visual attack method based on an attention mechanism, including:

[0046] S1: Based on the attention mechanism, image frames with high contribution to the attack effect are selected from the video stream, and the Grad-CAM++ method is used to generate a heat map. The focus of the heat map is the position of the target in the image frame;

[0047] Grad-CAM++ (Gradient-weighted Class Activation Mapping Plus Plus) is a method for visualizing sensitive regions of feature maps in deep learning models. It generates more accurate class activation maps by combining gradient information and feature map weights. Compared to Grad-CAM, Grad-CAM++ uses second-order gradient information to generate more refined heat maps by weighted combination of first-order and second-order gradients. Therefore, the Grad-CAM++ method can be used to locate targets in image frames, and by introducing small perturbations to these key areas, the model can make incorrect predictions.

[0048] However, for the visual attack of quadcopters, if the Grad-CAM++-based heat map generation method is used to process all image frame sequences, a large amount of computing resources will be consumed, thus affecting the real-time performance of the visual attack. Therefore, in order to effectively shorten the time of the quadcopter visual attack, the image frame sequence is first screened, and only those image frames that contribute more to the attack effect are processed. In the case of limited computing resources, the attention mechanism can be used to prioritize important image frames. The core of this mechanism is to calculate the correlation between image frames and determine the association between the query image frame and all image frames in the image frame sequence through a scoring function.

[0049] S1.1: For the input video stream data I, first take the first frame image as the query image I q , and then use the attention scoring function of the attention mechanism to score the subsequent image frames, which is the weight; if the weight of the image frame exceeds the threshold I th , then generate the heat map of the image frame and update the query image I q ; Otherwise, skip the image frame and continue to process subsequent frames by scoring and calculating weights, so as to filter out image frames in the video stream that contribute most to the attack effect;

[0050] Assume that the image frame sequence I collected by the quadcopter in a certain period of time is I = {I 0 ,I 1 ,I 2 ,···,I n}, I m ∈I represents the mth image frame, I q ∈I represents the query image frame, score(I q ,I m ) is the attention scoring function:

[0051]

[0052] Among them, the attention scoring function score(I q ,I m) measure the query image frame I q With the mth image frame I m The correlation between them.

[0053] Select I m The probability p m It is expressed as:

[0054]

[0055] By calculating these probabilities, it is possible to determine which image frames have higher importance in visual attacks, and thus prioritize these image frames to reduce the consumption of computing resources.

[0056] S1.2: Input the filtered image frame of S1.1 into the target tracking model to obtain the output feature map A of the last convolutional layer (k) , k represents the feature map channel, and the feature map size is H×W; secondly, the gradient information related to the output category c is calculated by back propagation y c is the output score of the detection model for category c; in order to calculate the channel importance weight, local gradient information and global average pooling are introduced; for each channel, the product of its gradient relative to category c and the global average pooling result of the channel is calculated, that is:

[0057]

[0058] The feature map A (k) The corresponding channel importance weight α k Channel-by-channel multiplication is performed to obtain the weighted feature map, and the ReLU activation function is applied to remove negative interference.

[0059] Input the filtered image frames into the Grad-CAM++ algorithm to obtain the thermal map L (c) Grad-CAM++ :

[0060]

[0061] Among them, ReLU is the activation function, k∈{1,2,3,..,K} represents the feature map channel, α k It is the product of the gradient of the kth channel relative to category c and the global average pooling result of the channel.

[0062] S2: Use the heat map as a mask and fuse it with the original image frame to generate adversarial samples;

[0063] PGD ​​(Projected Gradient Descent) is an adversarial sample attack algorithm for deep learning models. It generates adversarial samples by iteratively fine-tuning the original samples to deceive the detection results of deep learning models. Compared with algorithms such as FGSM (Fast Gradient Sign Method) and BIM (Basic Iterative Method), PGD has a higher success rate and robustness. PGD optimizes adversarial samples through continuous iterations to make them more effective in deceiving the target model. In contrast, FGSM and BIM are greatly affected by the locality of the gradient, which may result in a lower success rate of the generated adversarial sample attacks. In each iteration, PGD performs a projection operation to restrict the adversarial samples to the allowed input space, thereby generating higher quality adversarial samples. However, FGSM and BIM may generate inputs beyond a reasonable range during the iteration process, reducing the quality of the adversarial samples.

[0064] In the initialization phase of PGD, it is necessary to determine the initial image frame I (0) , the number of iterations T and the step size α of each iteration. In the iterative update phase, for each iteration t∈{1,2,···,T}, the following steps are performed: First, the gradient of the current image frame is calculated in represents the partial derivative of I, J is the loss function, y true is the correct label corresponding to the image frame. Then the gradient is denoised, and a certain disturbance δ is superimposed in the direction of each gradient to obtain the noisy gradient The noisy gradient is then clipped to ensure that its norm does not exceed ∈ to limit the perturbation range of the image frame. The clipping function is expressed as The clipped gradient is Finally, according to the clipped gradient, the gradient descent method is used to update the current image frame. The update formula of the image frame is The sign function returns the sign of the input to ensure movement in the gradient direction. After T iterations, the final noise map I is obtained. (T) .

[0065]

[0066] The noise map generated by PGD is a global perturbation of the entire image frame, which can cause the detection of all targets in the image frame to fail and is easily detected by the deep learning model, thereby reducing the success rate of visual attacks. In order to improve the success rate and concealment of adversarial attacks, attackers can use heat maps to determine attack-sensitive areas in the image frame, which are the key part of the target detection model's decision-making. By selectively applying small perturbations to these sensitive areas while keeping other areas in their original state, even if the target detection model analyzes the entire image, it is difficult to detect these local perturbations. Therefore, it is possible to locate the attack target more accurately and attack the model in a more covert way.

[0067] Heat map L (c) Grad-CAM++ As a mask, P th represents the threshold imposed on the mask, and the adversarial sample X is expressed as:

[0068]

[0069] Among them, I (T) represents an image based on T iterations of the PGD algorithm, I r represents the original image frame, F(L (c) Grad-CAM++ ,P th ) represents the calculation formula of the mask.

[0070] In S2, if the heat map L (c) Grad-CAM++ The value of the pixel position (r, c) in the image frame is greater than or equal to the threshold P th , the mask value is 1; otherwise it is 0;

[0071] F(L (c) Grad-CAM++ ) is expressed as follows:

[0072]

[0073] S3: Inject adversarial samples into the drone video stream data, so that the drone cannot detect the target and interferes with its normal tracking task. This part belongs to the existing technology and will not be described here.

[0074] Simulation experiment:

[0075] 1. Experimental Platform

[0076] The experiment used the quadcopter Tello as the visual attack target. Developed by DJI, the Tello drone is equipped with a 5-megapixel HD camera that can shoot 720p resolution videos and 5MP photos. The bottom infrared camera, barometer, inertial measurement unit and other sensor equipment enable Tello to achieve stable hovering, precise control and perception of the external environment during flight. Tello communicates with the laptop (ground station) through the UDP protocol to transmit flight data such as video streaming data and control commands.

[0077] 2. Experimental Evaluation Metrics

[0078] In order to evaluate the effectiveness and concealment of the method in this paper in launching visual adversarial sample attacks on drones, indicators such as L2 norm, structural similarity index, and peak signal-to-noise ratio are used to measure the attack effect.

[0079] L2 norm: reflects the difference between the adversarial sample Iadv and the original sample Iori at the pixel level, that is, the size of the disturbance introduced by the attack. The smaller the L2 norm, the higher the similarity between the adversarial sample and the original sample, the smaller the disturbance caused by the attack, which means the more concealed the attack is. The L2 norm calculation formula is as follows:

[0080]

[0081] Among them, I ori (i, j) represents the pixel value of the original image at position (i, j), I adv (i,j) represents the pixel value of the adversarial sample at position (i,j).

[0082] Structural Similarity Index (SSIM): It is used to measure the degree of distortion of adversarial samples relative to the original samples. Compared with the traditional pixel difference metric, SSIM not only considers factors such as image brightness, contrast, and structure, but also considers the characteristics of human visual perception. The SSIM value range is [0,1]. The larger the value, the smaller the image distortion, the more difficult it is for the human eye to detect, and the higher the concealment. The SSIM calculation formula is as follows:

[0083]

[0084] Where μori and μadv are the means of the original image and the adversarial sample. σ2ori and σ2adv are the variances of the original image and the adversarial sample. σori,adv is the covariance of the original image and the adversarial sample. c1 and c2 are constants used to stabilize the denominator.

[0085] Peak Signal-to-Noise Ratio (PSNR): An indicator used to measure image quality, which can quantify the degree of quality loss between the original image and the image after some processing (such as compression, distortion, etc.). The higher the PSNR value, the smaller the image distortion is, and more original information is retained, so the concealment of the adversarial sample is higher. The calculation formula of PSNR is as follows:

[0086]

[0087] Where L is the maximum pixel value (usually 255) and MSE is the mean square error.

[0088] 3 Adversarial Sample Attack Experiment

[0089] The experiment uses ResNet50 as the basic model and the ILSVRC2012 dataset as the experimental object. In order to verify the effectiveness of the proposed method in adversarial attacks, the method in this paper is compared with FastGradient Sign Method, Momentum Iterative Method, and Projected GradientDescent. In order to ensure the fairness of the experiment, we will keep the parameter settings of all adversarial sample attack methods the same. Among them, CrossEntropyLoss is selected as the loss function, the perturbation size is set to 0.1, the number of iterations is set to 20, the input dimension range is [0,1], and the image size is 244x244.

[0090] Verify whether the attack effect of Grad-CAM++ thermal center is better than that of non-thermal center. Use a fixed-size window on the original image and slide it with a certain step size. Add noise in each window. Calculate the confidence of the image after adding noise and the Euclidean distance between the window center and the thermal center. The results are as follows Figure 3 As shown in the figure, the data above the dotted line indicates an unsuccessful attack, and the data below the dotted line indicates a successful attack. The closer the noise window is to the thermal center, the better the attack effect. The farther the noise window is from the thermal center, the worse the attack effect. Therefore, Grad-CAM++ attacks better in the thermal center than in the non-thermal center. Adding noise closer to the thermal center will more effectively disrupt the model's confidence in the image, making the attack more successful. On the contrary, adding noise farther from the thermal center is relatively less effective because the model pays less attention to these areas and the impact of noise is smaller.

[0091] Explore the impact of the perturbation ratio on the attack effect. Draw a circle on the original image with the thermal center as the center, and add noise inside the circle. By gradually increasing the radius of the circle, calculate the confidence of the image after adding noise and the ratio of noise to the image size. The results are as follows Figure 5 As shown in the figure, the area above the dotted line indicates an unsuccessful attack, and the area below the dotted line indicates a successful attack. The larger the area occupied by the noise, the greater the impact on the model's confidence, making the attack more likely to succeed. Increasing the area of ​​the noise is equivalent to perturbing a larger range of image features, making it more difficult for the model to extract accurate features from it for judgment. Therefore, choosing an appropriate perturbation ratio and using the thermal center as the attack point can achieve efficient attacks while keeping the perturbation extremely small, and the introduced perturbation is almost imperceptible to the human eye.

[0092] Figure 4 Comparison of the attack effects of different adversarial samples. The first column shows the original input image, and the second to fourth columns show the effects of the attacks using the FSGM, MIM, and PGD methods. The fifth column is the mask image generated by Grad-CAM++, and the sixth column shows the attack effect of our method. It can be clearly seen from Table 1 that our method performs better in adversarial attacks. FGSM, MIM, and PGD usually require the introduction of large perturbations to change the classification results of the image, while our method can achieve efficient attacks while keeping the perturbations extremely small. Through the mask image generated by Grad-CAM++, we can further select local areas of the image as attack targets. Compared with traditional adversarial attack methods, the perturbations introduced by our method are almost imperceptible to the human eye, and only a very small amount of perturbations need to be added to achieve the purpose of the attack, meeting the requirements of attack concealment.

[0093] The evaluation index results of FSGM, MIM, PGD and our method on three different images are listed in Table 1. Our method shows the best performance in L2, SSIM and PSNR indicators. Compared with PGD, our method improves L2, SSIM and PSNR by 69.39%, 64.01% and 44.55% respectively, which shows that our method can minimize the difference between adversarial samples and original images, thereby better preserving the quality of original images.

[0094] Table 1: Comparison between this method and other methods

[0095]

[0096] 4 Target Tracking Visual Attack Experiment

[0097] The quadcopter Tello is used to perform pedestrian tracking tasks to verify whether the proposed method can successfully carry out visual attacks on drones while maintaining real-time and concealment. The YOLOv7 algorithm is used for drone target detection. YOLO (You Only Look Once) is an efficient end-to-end target detection algorithm. Its core idea is to complete the target detection task in a single forward propagation. The algorithm divides the image into grid cells, predicts the bounding box and object category in each cell, and uses multi-level feature maps to detect targets of different sizes. It has good target detection capabilities. Considering the real-time requirements, YOLOv7 is selected as the target detection algorithm. It is a lightweight variant of YOLO with higher detection accuracy and faster speed, which meets the real-time requirements of drones. The output of the YOLOv7 model includes the detected object bounding box, category, and confidence score.

[0098] In the experiment, the ground control station terminal established a connection with Tello via Wi-Fi and used the UDP protocol to control its takeoff, hovering, movement, and landing. The terminal receives real-time video streaming data and displays it on the terminal in the form of image frames, allowing the operator to monitor the drone environment in real time and make corresponding controls and decisions. The target detection model on the terminal is used to identify pedestrians in the video frame and obtain their pixel position information in order to generate flight instructions and send them to Tello's attitude and speed control module. Therefore, an attacker can invade the ground control station terminal and replace the video frames collected by the drone with adversarial samples, thereby achieving a visual attack on the drone.

[0099] If the target is displaced, the position of the selection box on the image frame will also change accordingly. Therefore, the flight control can be simplified and feedback control of the selection box position change based on PID can be adopted. The core of PID control is to calculate the deviation e(t) between the given value and the actual output value, and then calculate the control amount u(t) according to the linear combination relationship of proportional-integral-differential to change the control amount to control the controlled object. Its mathematical expression is:

[0100]

[0101] By calculating the deviation (wdiff, hdiff) between the center point of the selected pixel of the target being tracked and the center point of the pixel of the current image frame, and taking the deviation as the error parameter, the PID controller is used to calculate the corresponding attitude angle and three-axis acceleration of the quadcopter at the next moment, so as to achieve the displacement of the quadcopter in the front, back, left, right, and up and down directions, thereby tracking the target. Figure 6 shown.

[0102] The attacker's goal is to make the drone unable to correctly identify pedestrians, thus causing the drone to fail to track pedestrians. The attack method includes the following steps: First, the Grad-CAM++ method based on the attention mechanism is used to generate a heat map to locate sensitive areas in the image frame. Secondly, the pixel values ​​of the area with brightness values ​​higher than the threshold Pth in the heat map are set to 1, and the pixel values ​​of other areas are set to 0, and a mask operation is performed with the PGD noise map to generate a perturbed sample. Finally, the perturbed sample is fused with the original image to generate an adversarial sample and injected into the drone video stream. In this way, the video stream received by the drone will contain images with visual perturbations, causing errors in the drone's pedestrian recognition algorithm, thereby affecting its flight path for tracking pedestrians.

[0103] Figure 5 Images from a visual attack on Tello are shown. Figure 5 (a) The original image collected by Tello is 640x480 pixels in size and transmitted to the ground control station terminal through the network. After being processed by the target detection model, Figure 5 As shown in (b), the pedestrian is successfully identified with a confidence rate of 0.95, and the two interference targets are also detected. Figure 5 (c) and Figure 5 (d) is a heat map generated based on the Grad-CAM++ method, which is used to locate the sensitive area of ​​the target in the image. The two heat maps show three heat centers, one of which is located at the location of the pedestrian, and the other two are located at the locations of the interference targets. In the drone image frame, the tracking target is usually located near the center of the image, so we selected the heat center near the center of the image as the attack point and applied local interference. After the attack processing, Figure 5 (e) and Figure 5 (f) shows the output of the target detection model. Due to the local interference imposed on the pedestrian’s position, the target detection model cannot correctly identify the pedestrian, causing the drone to fail when continuing to perform the tracking task. This shows that by injecting adversarial samples into the drone video stream data, its normal target tracking task can be effectively interfered with.

[0104] During the experiment, the flight altitude, yaw angle, pitch angle and roll angle of the quadcopter Tello changed with time as shown in the figure below: Figure 6As shown. Tello quickly took off vertically from the ground and rose to a height of about 60 cm. During the takeoff phase, due to Tello's own power adjustment, the yaw angle, pitch angle and roll angle fluctuated slightly. After Tello hovered, the video stream target detection system was started. The target object began to move forward on the left side of the video screen, triggering Tello's tracking program. At this time, the roll angle changed first and adjusted to the left to follow the target's movement direction; then, the pitch angle was gradually adjusted to make Tello move in the direction of the target's advance. At the 50-second mark, the attacker launched a visual attack, and Tello lost its target recognition ability and could only maintain a hovering state. Finally, when Tello did not receive a control command for a long time and triggered the protection program, it began to land.

[0105] In order to verify the real-time performance of the algorithm, we randomly selected 30 seconds of video stream data collected by Tello, which contains 800 frames of images. Using the adversarial sample generation method based on the attention mechanism, only 34 frames of images were processed, and the processing time was 0.47 seconds. In contrast, if the attention mechanism is not used, each frame of the image needs to be processed to generate adversarial samples, which takes a total of 11.25 seconds. In actual flight, as the number of image frames increases, the number of adversarial samples that need to be processed becomes smaller and smaller. As shown in Table 2, as the number of received images increases, the proportion of adversarial samples that need to be generated for the attack continues to decrease. The experimental results show that when processing a large amount of video stream data, the use of the attention mechanism can help us more effectively identify image frames that contribute more to the attack effect, thereby generating adversarial samples in a targeted manner and reducing the time cost of calculation and processing.

[0106] Table 2: Comparison of processing frame numbers

[0107]

[0108] The present invention screens out image frames with higher contribution to the attack in the drone video stream based on the attention mechanism, and introduces the deep learning interpretable model Gard-CAM++ to impose local interference on the image frames, so as to achieve the purpose of attacking the drone with less disturbance cost. By comparing the experimental results with the existing classic methods FSGM, MIM, and PGD, the effect of this method is more significant. In the experiment of visual attack on the quadcopter Tello, the attention mechanism can reasonably discard some picture frames that have little effect on the attack results, saving a lot of computing resources. Therefore, in the experiment, only 4.25% of the video stream data needs to be processed to make Tello unable to recognize pedestrians without affecting the detection level of other targets, resulting in the failure of drone target tracking.

[0109] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A visual attack method for target area positioning of a quadcopter based on an attention mechanism, characterized by: include: S1: Based on the attention mechanism, image frames with high contribution to the attack effect are selected from the video stream, and the Grad-CAM++ method is used to generate a heat map. The focus of the heat map is the position of the target in the image frame; S2: Use the heat map as a mask and fuse it with the original image frame to generate adversarial samples; S3: Inject adversarial samples into drone video stream data, making it impossible for the drone to detect the target and interfering with its normal tracking task.

2. The method for visual attack of a quadcopter target area positioning based on an attention mechanism according to claim 2, characterized in that: S1 includes: S1.1: For the input video stream data I, first take the first frame image as the query image I q , and then use the attention scoring function of the attention mechanism to score the subsequent image frames, and the score is the weight; If the weight of the image frame exceeds the threshold I th , then generate the heat map of the image frame and update the query image I q ; Otherwise, skip the image frame and continue to process subsequent frames by scoring and calculating weights, so as to filter out image frames in the video stream that contribute most to the attack effect; S1.2: Input the filtered image frames into the Grad-CAM++ algorithm to obtain the heat map L (c) Grad-CAM++ : Among them, ReLU is the activation function, k∈{1,2,3,..,K} represents the feature map channel, α k It is the product of the gradient of the kth channel relative to category c and the global average pooling result of the channel.

3. The quadcopter visual attack method based on Grad-CAM++ and Mask-PGD as described in claim 2, characterized in that: In S1.1, it is assumed that the image frame sequence I = {I0, I1, I2, ···, I n }, I m ∈I represents the mth image frame, I q ∈I represents the query image frame, score(I q ,I m ) is the attention scoring function: Among them, the attention scoring function score(I q ,I m ) measure the query image frame I q With the mth image frame I m The correlation between them.

4. The method for visual attack of quadcopter target area positioning based on attention mechanism according to claim 3 is characterized in that: S2 is specifically: Heat map L (c) Grad-CAM++ As a mask, P th represents the threshold imposed on the mask, and the adversarial sample X is expressed as: Among them, I (T) represents an image based on T iterations of the PGD algorithm, I r Represents the original image frame, F(L (c) Grad-CAM++ ,P th ) represents the calculation formula of the mask.

5. The method for visual attack of quadcopter target area positioning based on attention mechanism according to claim 4 is characterized in that: In S2, if the heat map L (c) Grad-CAM++ The value of the pixel position (r, c) in the image frame is greater than or equal to the threshold P th , the mask value is 1; otherwise it is 0; F(L (c) Grad-CAM++ ) is expressed as follows:

Images