A high-precision locking and striking method for UAVs in complex backgrounds

Abstract

The present invention discloses a high-precision locking and striking method for drones under complex backgrounds, which belongs to the field of drone strike technology. The method is specifically as follows: S1: drone target acquisition, S2: inputting the target slice acquired in S1 into the weak point extraction network, if the weak point coordinates are successfully extracted, selecting the coordinates with the highest confidence as the strike point, if the weak point coordinates are not successfully extracted, determining that the extraction has failed, and re-running the target detection algorithm to obtain a new target slice; S3: target tracking network initialization, S4: dynamic adjustment of the focal length of the visual device, S5: drone target continuous tracking, S6: striking the weak points of the drone target, S7: completing the drone strike of the target. The high-precision locking and striking method for drones under complex backgrounds of the present invention can intelligently lock drone targets with high precision under complex backgrounds, and can improve the comprehensive treatment effect of the laser damage system.

Description

Technical Field

[0001] The present invention specifically relates to a high-precision locking and striking method for a UAV under complex background, and belongs to the technical field of UAV striking. Background Art

[0002] In recent years, drone technology has developed rapidly, and consumer-grade drones have been widely used, such as low-altitude photogrammetry, power line inspections, public security inspections, and personal photography by enthusiasts. As a hard-kill method for drones, high-energy laser damage systems use target detection and tracking technology and high-energy laser beam far-field control technology to achieve continuous high-energy burning of drone targets, thereby achieving physical damage to drone targets. This method is silent, accurate and efficient, and has no electromagnetic pollution to the environment. However, this method has very high requirements for target tracking accuracy. Existing damage systems are prone to off-target phenomena in complex backgrounds (such as buildings, trees, mountains, thick clouds, etc.) and when there is large fire coverage, which makes damage more difficult or even fails. Summary of the Invention

[0003] To solve the above problems, the present invention proposes a high-precision locking and striking method for UAVs under complex backgrounds, which can intelligently lock UAV targets with high precision under complex backgrounds and improve the comprehensive treatment effect of the laser damage system.

[0004] The present invention provides a high-precision locking and striking method for drones under complex backgrounds, and the method is specifically as follows:

[0005] S1: UAV target acquisition, after receiving the external lock tracking command, the visual equipment captures the tracking scene image in real time. In order to make the target imaging details clear and convenient for algorithm processing, the target imaging scale is required to be large enough. The resolution is no less than 100 pixels by 100 pixels, and the scene image is sent to the intelligent processing unit through the image acquisition card. The intelligent processing unit detects the drone target through the target detection algorithm. If the drone target is successfully detected, the detection result with the highest confidence is selected as the tracking target. If the drone target is not successfully detected, the detection is determined to have failed, and the target detection algorithm is used to continue to detect the scene image sent by the image acquisition card. The drone target detected by the target detection algorithm is the target slice cropped by the detection frame.

[0006] S2: Input the target slice obtained in S1 into the weak point extraction network. If the weak point coordinates are successfully extracted, the coordinates with the highest confidence are selected as the strike point. If the weak point coordinates are not successfully extracted, the extraction is considered a failure and the target detection algorithm is re-run to obtain a new target slice.

[0007] S3: Initialize the target tracking network. Use the weak point coordinates of the target detection box and its corresponding target slice to initialize the target tracking network. The target tracking network is initialized.

[0008] S4: Dynamically adjust the focal length of the visual device to maintain the size consistency of the collected on-site images; measure the distance in real time through the rangefinder equipped with the visual device, and adjust the camera focal length of the visual device in real time according to the distance measurement results;

[0009] S5: The drone continuously tracks the target, and the target tracking network continuously updates the coordinates of the target bounding box and the coordinates of the weak points on the fuselage;

[0010] S6: Strike the weak points of the drone target. Upon receiving an external laser strike command, the servo turntable and quick-reflection mirror deflect in response to the drone target continuously tracked by S5. At the same time, the laser beam passes through the quick-reflection mirror and continuously burns the drone's weak points.

[0011] S7. Complete the target drone strike. When the intelligent processing unit detects that the characteristic map response value corresponding to the target center point position is less than 0.01, it is confirmed that the target is lost, the tracking and strike are stopped, and the target tracking network is re-initialized; when the stop light emission and cancel tracking instructions are received, the intelligent processing unit stops the target tracking network initialization operation, and then controls the servo turntable and the fast reflection mirror to stop deflection, completing the target drone strike; the high-precision locking and strike method of drones under complex backgrounds of the present invention realizes rapid and intelligent extraction of drone target weaknesses by constructing a deep convolution model algorithm; at the same time, a twin network model algorithm is constructed to achieve continuous locking and tracking of drone targets, and ensure that the laser strike point is always focused on the weak points of the drone target, thereby achieving rapid disposal of the drone target.

[0012] The high-precision locking and striking method for drones under complex backgrounds of the present invention requires tracking and processing of images through a GPU intelligent processing device when processing on-site images. In order to ensure processing efficiency, the tracking camera video stream data on the tracking turntable can be acquired with low latency through an image acquisition card, and the delay is controlled within 20ms. When the intelligent processing unit is operating, the image processing can be completed within 15ms, and the target's miss distance information can be converted into angle control information, and the miss distance information can be tracked and pushed to the servo control system and the quick reflex mirror system in real time. The servo control system drives the servo turntable to respond quickly; the servo turntable realizes tracking of the drone target, ensuring that the drone target is located in the center of the tracking video image; the quick reflex mirror system drives the quick reflex mirror to respond quickly; and the high-energy laser beam pointing is controlled in real time to ensure that the laser strike point is stably focused on the weak point of the drone target body.

[0013] Furthermore, the camera focal length of the visual device is adjusted in real time according to the distance measurement result as follows: the camera focal length value at the time of initialization of the known target tracking network and ranging values , then the focal length value and ranging values The conversion relationship is as follows:

[0014] Formula 1;

[0015] Since the camera focal length value at the initialization moment of the target tracking network and ranging values is a known condition, and the distance value It can be obtained in real time through the rangefinder. According to formula 1, the focal length value can be calculated ; and according to the focal length value Adjust the camera focus of the vision device in real time.

[0016] Furthermore, the target detection algorithm is a YOLO series algorithm; if yolov5 is used for detection, in the detection result, if the confidence level is greater than 0.5, it is assumed that the target is successfully detected.

[0017] Furthermore, the working process of the vulnerability extraction network is as follows:

[0018] The target slice obtained by S1 is scaled to a size of 64*64*3, and then input into the backbone network for feature extraction. In order to improve the video processing speed and achieve the effective frame rate, the backbone network selects lightweight convolutional neural networks such as MobileNetV3 or GhostNet. The output of the backbone convolutional neural network is an 8*8*3 feature map; the backbone network outputs an 8*8*3 feature map, which is used to extract the weak points of the fuselage; according to actual combat experience, the rotor of the drone is easily penetrated by high-energy lasers, and hitting the rotor can cause the drone to fall quickly, so the rotor of the drone can be selected as the weak point of the fuselage. Since the weak point extraction is affected by factors such as camera imaging effects, background interference, and foreground occlusion, the weak point extraction network can obtain and output a set of recommended rankings of the weak point positions of the fuselage. When the weak point extraction network is trained, the training evaluation and model optimization are performed through the loss function. The loss function is defined as follows:

[0019] Formula 2;

[0020] As shown in Formula 2, during the training phase, the data annotation points are regressed by calculating the loss function. Specifically, during the training phase, the input image is divided into 8*8 grids. When a data annotation point falls into a certain grid, this grid is responsible for predicting this annotation point. The output feature map generates 8*8 groups of prediction values. Each group of prediction values contains x, y coordinates and confidence conf, where Table grid coordinates, when the grid contains annotation points , otherwise 0, On the contrary, Represents the weight parameter when calculating coordinate loss, Represents the weight parameter when calculating the confidence loss if the grid does not contain the annotation point. represents the coordinate prediction value, represents the true value of the coordinate, represents the confidence prediction value, Represents the true confidence value when the grid contains the annotation point , otherwise it is 0; in the inference stage, the weak points of the input image are predicted. In the inference stage, the backbone network can predict the weak points of the input image and obtain 64 sets of prediction values. The loss function is used for evaluation. When the prediction confidence is When the prediction results are considered credible, the prediction results are mapped back to the initial image slice coordinate values and sorted from large to small according to the confidence level; finally, a set of recommended rankings of fuselage weak point locations is output.

[0021] Furthermore, the target tracking network can achieve continuous tracking of the drone target and continuous pointing to the key point area on the drone. The target tracking network is specifically as follows:

[0022] S51: Establish a twin network with the same structure as the weak point extraction network, and set the frame number of the acquired image to , and the image tracking box size ; Initialize the twin network, the initial frame is known , target bounding box and target key points, the initial frame is preprocessed based on the target bounding box and target key point coordinates, the initial frame preprocessing includes cropping and scaling, and a 32*32*3 key point frame and a 127*127*3 template frame are obtained; in order to avoid the violent jitter of the tracking target center point due to the continuous change of the target tracking frame size during target tracking, the focal length of the visual device is dynamically adjusted to maintain the size consistency of the collected scene image; that is, the prerequisite requirement for the use of the target tracking network, the scale of the tracked target image will not change significantly, and a fixed-size tracking frame size is used. To limit;

[0023] S52: Input the key point frame and template frame obtained in S51 into the backbone network of the twin network. The backbone network of the twin network uses a 5-layer convolutional network to output feature maps of 2*2*256 and 14*14*256 respectively. and ;

[0024] S53: Start target tracking, knowing the previous frame The target bounding box coordinates of the current frame are based on this coordinate position Perform preprocessing, including cropping and scaling, to obtain a 303*303*3 search frame;

[0025] S54: Input the search frame obtained in S53 into the backbone network of the twin network and output a 36*36*256 feature map ; When the weak point of the target is continuously affected by the laser focus, it will produce fire cover. At this time, if the same backbone network parameters are used for feature extraction of the key point frame, template frame and search frame, it is difficult for the backbone network to learn the similarity between the two features of whether the target is covered by fire or not. Therefore, during model training, the key point frame, template frame and search frame need to learn different backbone network parameters respectively; therefore, when extracting the features of the key point frame, template frame and search frame, they are completed respectively through the backbone network of their respective twin networks;

[0026] S55: and The input is sent to the KeyPoint branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 27*27*1 response map representing the similarity between different positions and the initial keypoint frame;

[0027] S56: respectively and Input to the Classification branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 17*17*1 response map, representing whether the target information is contained at different positions;

[0028] S57: and The input is sent to the Centerness branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 17*17*1 response map, which represents the probability of the target center at different positions in the image.

[0029] S58: First multiply the elements of the Classification branch response map and the Centerness branch response map, then take the location of the maximum response value, map it back to the original map coordinates, and get the final target center point coordinates. ;

[0030] S59: Map the KeyPoint branch response map back to the original image coordinates, knowing the previous frame The target key point coordinates and the previous frame Coordinate offset between center point coordinates ,by For the search center, The search radius is obtained, and the coordinates of the location with the largest response value in the search area are obtained to obtain the final key point prediction result. The key point prediction result includes the bounding box coordinates and the coordinate values of the weak point of the fuselage.

[0031] Furthermore, the deflection response of the servo turntable and the fast reflection mirror is specifically as follows:

[0032] Calculate the servo miss distance based on the camera parameters of the visual device, the target center point coordinates, and the target weak point coordinates and the miss distance of the fast-reflecting mirror , servo miss distance and the miss distance of the fast-reflecting mirror This is the correction amount that the servo turntable and the quick-reflection mirror need to make; the servo turntable and the quick-reflection mirror respond with deflection, so that the servo turntable Continuously track the target center, the fast mirror Continue to accurately point to the drone's weak points.

[0033] Compared with the existing technology, the high-precision locking and striking method of drones under complex backgrounds of the present invention can achieve rapid disposal of drone targets by quickly and intelligently extracting the weak points of drone targets, continuously locking and tracking drone targets, and ensuring that the laser striking point is always focused on the weak points of drone targets. It can realize intelligent locking and striking of drone targets in complex backgrounds such as buildings, trees, mountains, and large fire coverage during strikes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a block diagram of the overall workflow module of the UAV high-precision locking and striking method of the present invention.

[0035] Figure 2 The figure is a schematic diagram of the overall workflow of the high-precision locking and striking method for unmanned aerial vehicles of the present invention.

[0036] Figure 3 This is a schematic diagram of the network workflow for extracting weaknesses of the present invention.

[0037] Figure 4 This is a schematic diagram of the connection structure of each module of the target tracking network of the present invention.

[0038] Figure 5 Schematic diagram of the target tracking network workflow of the present invention. DETAILED DESCRIPTION

[0039] Example 1:

[0040] like Figures 1 to 5 The high-precision locking and striking method for drones in complex backgrounds is shown as follows:

[0041] S1: UAV target acquisition, after receiving the external lock tracking command, the visual equipment captures the tracking scene image in real time. In order to make the target imaging details clear and convenient for algorithm processing, the target imaging scale is required to be large enough. The resolution is no less than 100 pixels by 100 pixels, and the scene image is sent to the intelligent processing unit through the image acquisition card. The intelligent processing unit detects the drone target through the target detection algorithm. If the drone target is successfully detected, the detection result with the highest confidence is selected as the tracking target. If the drone target is not successfully detected, the detection is determined to have failed, and the target detection algorithm is used to continue to detect the scene image sent by the image acquisition card. The drone target detected by the target detection algorithm is the target slice cropped by the detection frame.

[0042] S2: Input the target slice obtained in S1 into the weak point extraction network. If the weak point coordinates are successfully extracted, the coordinates with the highest confidence are selected as the strike point. If the weak point coordinates are not successfully extracted, the extraction is considered a failure and the target detection algorithm is re-run to obtain a new target slice.

[0043] S3: Initialize the target tracking network. Use the weak point coordinates of the target detection box and its corresponding target slice to initialize the target tracking network. The target tracking network is initialized.

[0044] S4: Dynamically adjust the focal length of the visual device to maintain the size consistency of the collected on-site images; measure the distance in real time through the rangefinder equipped with the visual device, and adjust the camera focal length of the visual device in real time according to the distance measurement results;

[0045] S5: The drone continuously tracks the target, and the target tracking network continuously updates the coordinates of the target bounding box and the coordinates of the weak points on the fuselage;

[0046] S6: Strike the weak points of the drone target. Upon receiving an external laser strike command, the servo turntable and quick-reflection mirror deflect in response to the drone target continuously tracked by S5. At the same time, the laser beam passes through the quick-reflection mirror and continuously burns the drone's weak points.

[0047] S7. Complete the target drone strike. When the intelligent processing unit detects that the characteristic map response value corresponding to the target center point position is less than 0.01, it is confirmed that the target is lost, the tracking and strike are stopped, and the target tracking network is re-initialized; when the stop light emission and cancel tracking instructions are received, the intelligent processing unit stops the target tracking network initialization operation, and then controls the servo turntable and the fast reflection mirror to stop deflection, completing the target drone strike; the high-precision locking and strike method of drones under complex backgrounds of the present invention realizes rapid and intelligent extraction of drone target weaknesses by constructing a deep convolution model algorithm; at the same time, a twin network model algorithm is constructed to achieve continuous locking and tracking of drone targets, and ensure that the laser strike point is always focused on the weak points of the drone target, thereby achieving rapid disposal of the drone target.

[0048] The high-precision locking and striking method for drones under complex backgrounds of the present invention requires tracking and processing of images through a GPU intelligent processing device when processing on-site images. In order to ensure processing efficiency, the tracking camera video stream data on the tracking turntable can be acquired with low latency through an image acquisition card, and the delay is controlled within 20ms. When the intelligent processing unit is operating, the image processing can be completed within 15ms, and the target's miss distance information can be converted into angle control information, and the miss distance information can be tracked and pushed to the servo control system and the quick reflex mirror system in real time. The servo control system drives the servo turntable to respond quickly; the servo turntable realizes tracking of the drone target, ensuring that the drone target is located in the center of the tracking video image; the quick reflex mirror system drives the quick reflex mirror to respond quickly; and the high-energy laser beam pointing is controlled in real time to ensure that the laser strike point is stably focused on the weak point of the drone target body.

[0049] The real-time adjustment of the camera focal length of the visual device according to the distance measurement result is as follows: The camera focal length value at the time of initialization of the known target tracking network and ranging values , then the focal length value and ranging values The conversion relationship is as follows:

[0050] Formula 1;

[0051] Since the camera focal length value at the initialization moment of the target tracking network and ranging values is a known condition, and the distance value It can be obtained in real time through the rangefinder. According to formula 1, the focal length value can be calculated ; and according to the focal length value Adjust the camera focus of the vision device in real time.

[0052] The target detection algorithm is the YOLO series algorithm; if yolov5 is used for detection, in the detection result, if the confidence level is greater than 0.5, it is assumed that the target is successfully detected.

[0053] The working process of the vulnerability extraction network is as follows:

[0054] The target slice obtained by S1 is scaled to a size of 64*64*3, and then input into the backbone network for feature extraction. In order to improve the video processing speed and achieve an effective frame rate, the backbone network selects lightweight convolutional neural networks such as MobileNetV3 or GhostNet. The output of the backbone convolutional neural network is an 8*8*3 feature map; the backbone network outputs an 8*8*3 feature map, which is used to extract the weak points of the fuselage. According to actual combat experience, the rotor of a drone can be easily penetrated by a high-energy laser. Striking the rotor can cause the drone to fall quickly. The rotor of the drone is selected as the weak point of the fuselage, such as Figure 2 The white dots shown are the weak points of the fuselage. Since weak point extraction is affected by factors such as camera imaging effects, background interference, and foreground occlusion, the weak point extraction network can output a set of recommended rankings of the fuselage weak point locations. When training the weak point extraction network, a loss function is used for training evaluation and model optimization. The loss function is defined as follows:

[0055] Formula 2;

[0056] As shown in Formula 2, during the training phase, the data annotation points are regressed by calculating the loss function. Specifically, during the training phase, the input image is divided into 8*8 grids. When a data annotation point falls into a certain grid, this grid is responsible for predicting this annotation point. The output feature map generates 8*8 groups of prediction values. Each group of prediction values contains x, y coordinates and confidence conf, where Represents grid coordinates, when the grid contains annotation points , otherwise 0, On the contrary, Represents the weight parameter when calculating coordinate loss, Represents the weight parameter when calculating the confidence loss if the grid does not contain the annotation point. represents the coordinate prediction value, represents the true value of the coordinate, represents the confidence prediction value, Represents the true confidence value when the grid contains the annotation point , otherwise it is 0; in the inference stage, the weak points of the input image are predicted. In the inference stage, the backbone network can predict the weak points of the input image and obtain 64 sets of prediction values. The loss function is used for evaluation. When the prediction confidence is When the prediction results are considered credible, the prediction results are mapped back to the initial image slice coordinate values and sorted from large to small according to the confidence level; finally, a set of recommended rankings of fuselage weak point locations is output.

[0057] The target tracking network can achieve continuous tracking of the drone target and continuous pointing to the key point area on the drone. The target tracking network is specifically as follows:

[0058] S51: Establish a twin network with the same structure as the weak point extraction network, and set the frame number of the acquired image to , and the image tracking box size ; Initialize the twin network, the initial frame is known , target bounding box and target key points, the initial frame is preprocessed based on the target bounding box and target key point coordinates, the initial frame preprocessing includes cropping and scaling, and a 32*32*3 key point frame and a 127*127*3 template frame are obtained; in order to avoid the violent jitter of the tracking target center point due to the continuous change of the target tracking frame size during target tracking, the focal length of the visual device is dynamically adjusted to maintain the size consistency of the collected scene image; that is, the prerequisite requirement for the use of the target tracking network, the scale of the tracked target image will not change significantly, and a fixed-size tracking frame size is used. To limit;

[0059] S52: Input the key point frame and template frame obtained in S51 into the backbone network of the twin network. The backbone network of the twin network uses a 5-layer convolutional network to output feature maps of 2*2*256 and 14*14*256 respectively. and ;

[0060] S53: Start target tracking, knowing the previous frame The target bounding box coordinates of the current frame are based on this coordinate position Perform preprocessing, including cropping and scaling, to obtain a 303*303*3 search frame;

[0061] S54: Input the search frame obtained in S53 into the backbone network of the twin network and output a 36*36*256 feature map ; When the weak point of the target is continuously affected by the laser focus, it will produce fire cover. At this time, if the same backbone network parameters are used for feature extraction of the key point frame, template frame and search frame, it is difficult for the backbone network to learn the similarity between the two features of whether the target is covered by fire or not. Therefore, during model training, the key point frame, template frame and search frame need to learn different backbone network parameters respectively; therefore, when extracting the features of the key point frame, template frame and search frame, they are completed respectively through the backbone network of their respective twin networks;

[0062] S55: and The input is sent to the KeyPoint branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 27*27*1 response map representing the similarity between different positions and the initial keypoint frame;

[0063] S56: respectively and Input to the Classification branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 17*17*1 response map, representing whether the target information is contained at different positions;

[0064] S57: and The input is sent to the Centerness branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 17*17*1 response map, which represents the probability of the target center at different positions in the image.

[0065] S58: First multiply the elements of the Classification branch response map and the Centerness branch response map, then take the location of the maximum response value, map it back to the original map coordinates, and get the final target center point coordinates. ;

[0066] S59: Map the KeyPoint branch response map back to the original image coordinates, knowing the previous frame The target key point coordinates and the previous frame The coordinate offset between the target center point coordinates ,by For the search center, The search radius is obtained, and the coordinates of the location with the largest response value in the search area are obtained to obtain the final key point prediction result. The key point prediction result includes the bounding box coordinates and the coordinate values of the weak point of the fuselage.

[0067] The deflection response of the servo turntable and the fast reflection mirror is as follows:

[0068] Calculate the servo miss distance based on the camera parameters of the visual device, the target center point coordinates, and the target weak point coordinates and the miss distance of the fast-reflecting mirror , servo miss distance and the miss distance of the fast-reflecting mirror This is the correction amount that the servo turntable and the quick-reflection mirror need to make; the servo turntable and the quick-reflection mirror respond with deflection, so that the servo turntable Continuously track the target center, the fast mirror Continue to accurately point to the drone's weak points.

[0069] The above embodiments are only preferred implementations of the present invention. Therefore, any equivalent changes or modifications made according to the structures, features and principles described in the scope of application of the present invention are included in the scope of application of the present invention.

Claims

1. A high-precision locking and striking method for UAVs in complex backgrounds, characterized by: The method is specifically as follows: S1: UAV target acquisition. After receiving the external lock and tracking command, the visual device captures the tracking scene image in real time and sends the scene image to the intelligent processing unit through the image acquisition card; The intelligent processing unit detects the drone target through the target detection algorithm. If the drone target is successfully detected, the detection result with the highest confidence is selected as the tracking target. If the drone target is not successfully detected, it is determined that the detection has failed and the target detection algorithm continues to detect the scene image sent to the image acquisition card; The drone target detected by the target detection algorithm is a target slice cropped by the detection frame; S2: Input the target slice obtained in S1 into the weak point extraction network. If the weak point coordinates are successfully extracted, the coordinates with the highest confidence are selected as the strike point. If the weak point coordinates are not successfully extracted, the extraction is considered a failure and the target detection algorithm is re-run to obtain a new target slice. S3: Initialize the target tracking network. Use the weak point coordinates of the target detection box and its corresponding target slice to initialize the target tracking network. The target tracking network is initialized. S4: Dynamically adjust the focal length of the visual device to maintain the size consistency of the collected on-site images; measure the distance in real time through the rangefinder equipped with the visual device, and adjust the camera focal length of the visual device in real time according to the distance measurement results; S5: The drone continuously tracks the target, and the target tracking network continuously updates the coordinates of the target bounding box and the coordinates of the weak points on the fuselage; S6: Strike the weak points of the drone target. Upon receiving an external laser strike command, the servo turntable and quick-reflection mirror deflect in response to the drone target continuously tracked by S5. At the same time, the laser beam passes through the quick-reflection mirror and continuously burns the drone's weak points. S7: The target drone strike is completed. When the intelligent processing unit detects that the characteristic map response value corresponding to the target center point position is less than 0.01, it is confirmed that the target is lost, the tracking strike is stopped, and the target tracking network is re-initialized. When the stop light emission and cancel tracking instructions are received, the intelligent processing unit stops the target tracking network initialization operation, and then controls the servo turntable and the fast reflection mirror to stop deflection, completing the target drone strike; The working process of the vulnerability extraction network is as follows: The target slice obtained by S1 is scaled to a size of 64*64*3 and then input into the backbone network for feature extraction. The backbone network outputs an 8*8*3 feature map, which is used to extract the weak points of the fuselage. When the weak point extraction network is trained, the loss function is used for training evaluation and model optimization. The loss function is defined as follows: Formula 2: As shown in Formula 2, during the training phase, the data annotation points are regressed by calculating the loss function. Specifically, during the training phase, the input image is divided into 8*8 grids. When a data annotation point falls within a certain grid, this grid is responsible for predicting this annotation point. The output feature map generates 8*8 groups of prediction values. Each group of prediction values contains x, y coordinates and confidence conf, where ij represents the grid coordinates. When the grid contains the annotation point , otherwise 0, On the contrary, Represents the weight parameter when calculating coordinate loss, Represents the weight parameter when calculating the confidence loss if the grid does not contain the annotation point. represents the coordinate prediction value, represents the true value of the coordinate, represents the confidence prediction value, Represents the true confidence value when the grid contains the annotation point , otherwise it is 0; in the inference stage, the weak points of the input image are predicted. In the inference stage, the backbone network can predict the weak points of the input image and obtain 64 sets of prediction values. The loss function is calculated for evaluation. When the prediction confidence When the prediction results are considered credible, the prediction results are mapped back to the initial image slice coordinate values and sorted from large to small according to the confidence level; finally, a set of recommended rankings of fuselage weak point locations is output.

2. The high-precision locking and striking method for drones in complex backgrounds according to claim 1 is characterized by: The real-time adjustment of the camera focal length of the visual device according to the distance measurement result is as follows: The camera focal length value at the time of initialization of the known target tracking network and ranging values , then the focal length value and ranging values The conversion relationship is as follows: Formula 1: Since the camera focal length value at the initialization moment of the target tracking network and ranging values is a known condition, and the distance value It can be obtained in real time through the rangefinder. According to formula 1, the focal length value can be calculated ; and according to the focal length value Adjust the camera focus of the vision device in real time.

3. The high-precision locking and striking method for drones in complex backgrounds according to claim 1 is characterized by: The target detection algorithm is the YOLO series algorithm; in the detection result, if the confidence level is greater than 0.5, it is assumed that the target is successfully detected.

4. The high-precision locking and striking method for drones in complex backgrounds according to claim 1 is characterized by: The backbone network is MobileNetV3 or GhostNet convolutional neural network.

5. The high-precision locking and striking method for drones in complex backgrounds according to claim 1 is characterized by: The target tracking network is as follows: S51: Establish a twin network with the same structure as the weak point extraction network, and set the frame number of the acquired image to , and the image tracking box size ; Initialize the twin network, the initial frame is known , target bounding box and target key points, preprocessing the initial frame based on the target bounding box and target key point coordinates, wherein the preprocessing of the initial frame includes cropping and scaling to obtain a 32*32*3 key point frame and a 127*127*3 template frame; S52: Input the key point frame and template frame obtained in S51 into the backbone network of the twin network. The backbone network of the twin network uses a 5-layer convolutional network to output feature maps of 2*2*256 and 14*14*256 respectively. and ; S53: Start target tracking, knowing the previous frame The target bounding box coordinates of the current frame are based on this coordinate position Perform preprocessing, including cropping and scaling, to obtain a 303*303*3 search frame; S54: Input the search frame obtained in S53 into the backbone network of the twin network and output a 36*36*256 feature map ; When extracting the features of the key point frame, template frame and search frame, they are completed through the backbone network of their respective twin networks; S55: and The input is sent to the KeyPoint branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 27*27*1 response map representing the similarity between different positions and the initial keypoint frame; S56: respectively and Input to the Classification branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 17*17*1 response map, representing whether the target information is contained at different positions; S57: and The input is sent to the Centerness branch, which includes a cross-correlation layer and a feature extraction layer, and outputs a 17*17*1 response map, which represents the probability of the target center at different positions in the image. S58: First multiply the elements of the Classification branch response map and the Centerness branch response map, then take the location of the maximum response value, map it back to the original map coordinates, and get the final target center point coordinates. ; S59: Map the KeyPoint branch response map back to the original image coordinates, knowing the previous frame The target key point coordinates and the previous frame The coordinate offset between the target center point coordinates ,by For the search center, The search radius is obtained, and the coordinates of the location with the largest response value in the search area are obtained to obtain the final key point prediction result. The key point prediction result includes the bounding box coordinates and the coordinate values of the weak point of the fuselage.

6. The high-precision locking and striking method for drones in complex backgrounds according to claim 1 is characterized by: The deflection response of the servo turntable and the fast reflection mirror is as follows: Calculate the servo miss distance based on the camera parameters of the visual device, the target center point coordinates, and the target weak point coordinates and the miss distance of the fast-reflecting mirror The servo turntable and the quick-reflection mirror respond to the deflection, so that the servo turntable can Continuously track the target center, the fast mirror Continue to accurately point to the drone's weak points.

Images