Unmanned aerial vehicle automatic intrusion inspection method and system and storage medium

Abstract

The invention provides an unmanned aerial vehicle automatic intrusion inspection method, which comprises the following steps: performing target detection on video frames acquired by an unmanned aerial vehicle to obtain image features of each frame of image, sequentially comparing the image features of each frame of image with preset target features, and when the image features in the images are successfully matched with the target features, determining that the unmanned aerial vehicle is intruded into the unmanned aerial vehicle. Taking the successfully matched image as the current frame; performing trajectory prediction on the successfully matched image features of the current frame to obtain a predicted image and a predicted trajectory; inputting the predicted image into a GhostNetV2 network for feature extraction to obtain predicted image features; inputting the next frame of image of the current frame into a GhostNetV2 network for feature extraction to obtain actual image features; calculating the similarity between the predicted image features and the actual image features, and completing the screening and correction of the predicted trajectory according to the similarity; and route planning is carried out according to the predicted trajectory, and a tracking route is generated, so that the calculation amount during target object identification in the feature extraction process is reduced, the calculation power demand is reduced, and the tracking real-time performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicles, and in particular to the field of an automatic unmanned aerial vehicle intrusion inspection method, system and storage medium. Background Art

[0002] With the advancement of science and technology, drone technology continues to develop. At present, remote inspections of designated areas by controlling drone gimbal cameras have improved the efficiency of inspections. However, when real-time inspections of target areas are required, drone operators are required to constantly operate the joystick to control the rotation of the gimbal camera and observe the image for any abnormalities. Although this is more efficient than traditional inspections, it is still relatively low, and long inspections will also reduce the operator's attention. Therefore, automatic tracking algorithms are needed to assist in completing inspection tasks.

[0003] Some existing technologies use the BoT-SORT multi-target tracking method to achieve automatic tracking of the tracking target, such as the invention with publication number "CN118072013.A", which uses the BoT-SORT multi-target tracking method based on region division, improves the BOT-SORT multi-target tracking algorithm, and realizes the function of target tracking in complex scenarios. The detection algorithm is used to obtain the position of the detection target, and then the improved tracking algorithm is used to associate the target and the trajectory to achieve automatic tracking of the specified target. However, these algorithms are insufficient in real-time performance, which often leads to the problem of untimely tracking. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a method and system for automatic intrusion inspection of drones, so that the target tracking algorithm can reduce the amount of calculation in the feature extraction process when identifying the target object, thereby reducing the computing power requirements.

[0005] Specifically, the present invention is achieved through the following technical solutions:

[0006] On the one hand, the present invention provides a method for automatic intrusion inspection of a drone, comprising:

[0007] Perform target detection on the video frames acquired by the drone to obtain the original image features of each frame;

[0008] The original image features of each frame of image are compared with the preset target features in sequence. When the original image features and the target features are successfully matched, the successfully matched image is used as the image to be predicted;

[0009] Perform trajectory prediction on the original image features of the image to be predicted to obtain a predicted image and a predicted trajectory;

[0010] Performing feature extraction on the predicted image through the GhostNetV2 network to obtain predicted image features;

[0011] The predicted image feature is compared with the original image feature of the next frame of the corresponding image to be predicted, and the predicted trajectory is screened and corrected according to the similarity;

[0012] Route planning is performed based on the predicted trajectory to generate a real-time tracking route.

[0013] The Re-ID module of its existing BoT-SORT-ReID algorithm is improved by replacing the Resnet50 network in ReID with the GhostNetV2 network, which effectively reduces the amount of calculation during Re-ID feature extraction and improves the real-time performance of target tracking.

[0014] Furthermore, the feature extraction of the predicted image through the GhostNetV2 network specifically includes:

[0015] The acquired predicted image data is convoluted and separated, and then batch normalized and ReLU activated to obtain the first intermediate feature;

[0016] The acquired predicted image data is processed by the decomposition fully connected attention mechanism to obtain the second intermediate feature;

[0017] The first intermediate feature and the second intermediate feature are multiplied to obtain a third intermediate feature.

[0018] The third intermediate feature is convolutionally separated by the second Ghost module and then batch normalized to obtain the fourth intermediate feature.

[0019] The fourth intermediate feature is added to the acquired original image data to obtain the image feature.

[0020] Furthermore, the GhostNetV2 network is trained by the following steps:

[0021] Input the acquired training image data into the Ghost Netv2 network to be trained to obtain the predicted image features;

[0022] Perform global average pooling on the predicted image features to obtain a feature map;

[0023] Calculate feature loss by comparing the feature map with its corresponding standard map; and calculate predicted probability distribution by processing the feature map through a fully connected layer, and calculate distribution loss based on the predicted probability distribution and the standard probability;

[0024] Modify the parameters of the Ghost Netv2 network according to the distribution loss and feature loss until the network converges.

[0025] Furthermore, the loss between the predicted image features and the standard image features is calculated by the following formula:

[0026]

[0027] in They represent the feature maps obtained after random samples, random positive samples, and random negative samples pass through the network. α represents the threshold.

[0028] Furthermore, at least one Ghost module of the GhostNetV2 network is subjected to structural reparameterization training to reduce the model structure.

[0029] Furthermore, after the model converges, the network model is fine-tuned using the following formula:

[0030]

[0031] Where ρ is the pruning operation, r is the pruning rate, and θ represents the fine-tuning operation. Represents the network after the t-th pruning.

[0032] On the other hand, the present invention also provides a drone automatic intrusion inspection system, which includes: a drone gimbal: used to track and obtain video frames of the inspection area;

[0033] Target detection module: used to perform target detection on the video frames acquired by the drone to obtain the original image features of each frame;

[0034] Target matching module: used to compare the original image features of each frame of image with the preset target features in sequence. When the original image features successfully match the target features, the successfully matched image is used as the current frame;

[0035] Trajectory prediction module: used to perform trajectory prediction on the original image features of the image to be predicted, and obtain the predicted image and predicted trajectory;

[0036] GhostNetV2 network: used to extract features of the predicted image to obtain predicted image features;

[0037] Feature matching module: used to compare the predicted image features with the original image features of the next frame of the corresponding image to be predicted, and complete the screening and correction of the predicted trajectory according to the similarity;

[0038] Route generation module: used to plan routes based on predicted trajectories and generate real-time tracking routes.

[0039] Furthermore, the GhostNetV2 network includes:

[0040] The first Ghost module layer is used to perform batch normalization and ReLU activation on the acquired predicted image data after convolution separation by the first Ghost module to obtain the first intermediate feature;

[0041] Decomposed fully connected attention layer: The acquired predicted image data is processed by the decomposed fully connected attention mechanism to obtain the second intermediate feature;

[0042] Feature aggregation layer: used for multiplying the first intermediate feature and the second intermediate feature to obtain a third intermediate feature;

[0043] Second Ghost module layer: the third intermediate feature is convolutionally separated by the second Ghost module and then batch normalized to obtain the fourth intermediate feature;

[0044] Feature concatenation layer: Add the fourth intermediate feature to the acquired predicted image data to obtain the predicted image feature.

[0045] Furthermore, the automatic drone intrusion inspection system also includes a network update module, which includes:

[0046] Feature map restoration submodule: used to perform global average pooling on the predicted image features to obtain the feature map;

[0047] Feature loss calculation submodule: used to calculate feature loss according to the feature map and its corresponding standard map;

[0048] Distribution loss calculation submodule: used to process the feature map through a fully connected layer to calculate the predicted probability distribution, and calculate the distribution loss according to the predicted probability distribution and the standard probability;

[0049] Network update submodule: used to correct the parameters of the Ghost Netv2 network according to the distribution loss and feature loss until the network converges.

[0050] On the other hand, the present invention also provides a storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the automatic intrusion inspection method of a drone as described in any one of the above items are implemented.

[0051] The method and system for automatic intrusion inspection of unmanned aerial vehicles described in the present invention first improve the Re-ID module of the existing BoT-SORT-ReID algorithm, replace the Resnet50 network in ReID with the GhostNetV2 network, effectively reduce the amount of calculation when extracting Re-ID features, improve the real-time performance of target tracking, and can correct the relevant parameters in the trajectory prediction module according to the prediction results and the actual offset results after each prediction is completed, so that it is more in line with the motion preference of the predicted target, and improve the accuracy of the prediction. Further, the feature reuse technology is usually used in the GhostNetV2 network to design the network, which increases the computational cost on the network hardware device, thereby affecting the execution efficiency. Therefore, the Ghostmodule is trained with structural reparameterization, so that the structure of the network deployment is reduced, and the process of feature fusion is carried out in the weight space, not in the feature space, which improves the efficiency of feature fusion; finally, the trained network model is pruned, which further simplifies the number of model parameters, reduces the computational and storage requirements, and improves the real-time performance of the system.

[0052] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A structural block diagram of an exemplary drone inspection system provided by the present invention;

[0054] Figure 2 A flowchart of a method for automatic intrusion inspection by a drone provided by the present invention;

[0055] Figure 3 A structural block diagram of a drone automatic intrusion inspection system provided by the present invention;

[0056] Figure 4 This is a flowchart of feature extraction of an exemplary GhostNetV2 network of the present invention;

[0057] Figure 5 A training flow chart of an exemplary GhostNetV2 network provided by the present invention;

[0058] Figure 6 A structural block diagram of an exemplary Ghost module provided by the present invention;

[0059] Figure 7 Schematic diagram of the derivation of the structural reparameterization described in the present invention. DETAILED DESCRIPTION

[0060] See also Figure 1, drone inspections require the deployment of hardware equipment in advance, and drone inspection tasks are usually completed by drones equipped with gimbal cameras. In the present invention, a three-axis gimbal camera is used as an example. The three-axis gimbal camera includes a three-axis electronically stabilized gimbal, a thermal imaging and visible light camera, and a laser ranging module; the above components are installed on the drone through a connecting device. The drone is equipped with a chip, which stores a target tracking algorithm and performs target recognition, detection and tracking on the image data collected by the gimbal camera according to preset target features. After the drone is assembled, the drone goes to the preset flight area to complete the inspection task, and collects image data in real time to detect whether there is a preset target in the flight area. If so, the target is located and the drone is controlled to track the target. If not, the inspection is performed according to the set program.

[0061] During the inspection process, it is necessary to collect image data from a large area and process and analyze the data in real time in order to identify, detect and track the target. In order to meet the real-time requirements of drones for automatic intrusion inspections, a large amount of computing power resources are required to assist in image data processing. However, the computing power of the drone itself is limited, and it is generally difficult to meet the requirements of the traditional BoT-SORT multi-target tracking method. If the image data processing is completed by a cloud server, the transmission delay will make the control instructions of the drone not issued quickly and timely.

[0062] Based on this, the present invention provides a method and system for automatic intrusion inspection of unmanned aerial vehicles, improves the Re-ID module of the existing BoT-SORT-ReID algorithm, and replaces the Resnet50 network in ReID with the GhostNetV2 network, so that the target tracking algorithm can reduce the amount of calculation in the feature extraction process when identifying the target object, thereby reducing the computing power requirement. The automatic intrusion inspection system of unmanned aerial vehicles includes a unmanned aerial vehicle gimbal, a target detection module 10, a trajectory prediction module 20, a GhostNetV2 network 30, a feature matching module 40, and a route generation module 50. The automatic intrusion inspection system of unmanned aerial vehicles is used to control the unmanned aerial vehicle gimbal to execute the automatic intrusion inspection method of unmanned aerial vehicles, please refer to Figure 2 and Figure 3 The automatic intrusion inspection method of the drone specifically includes:

[0063] The target detection module is used to execute step S10: perform target detection on the video frames acquired by the drone to obtain the original image features of each frame of the image.

[0064] The target matching module is used to execute step S20: sequentially compare the original image features of each frame of image with the preset target features, and when the original image features in the image successfully match the target features, the successfully matched image is used as the image to be predicted.

[0065] During the inspection process, the drone continuously takes pictures of the inspection area, and the pictures taken form a group of video frames. The target detection algorithm can identify the object of interest in each frame, and mark its position and category, providing key information for subsequent analysis, such as identifying people, cars, objects and other objects in the image. In the present invention, the captured picture is matched with the target features of the object to be tracked. When the matching features appear in the image, the tracking task is started. At this time, the object is calibrated in the image with the target frame to obtain the location information and status information of the object.

[0066] The trajectory prediction module 30 executes step S30: performing trajectory prediction on the image features of the current frame that have been successfully matched to obtain a predicted image and a predicted trajectory.

[0067] The present invention uses a Kalman filter to model the motion of the object and predict the position of the object in the next frame. The Kalman filter directly outputs the width and height of the bounding box instead of the scale factor, which helps to obtain a more accurate bounding box prediction. The camera motion compensation (CMC) technology is adopted to estimate the global motion of the camera and correct the parameters of the Kalman filter, so that the tracking algorithm can maintain the stability of tracking when the camera moves, solve the influence of the camera motion on the tracking accuracy, and generate a tracking trajectory and a predicted image according to the prediction results.

[0068] In order to ensure the accuracy of continuous tracking, the accuracy of the prediction results needs to be determined during the tracking process. When the prediction results are biased, the parameters of the Kalman filter or the parameters of the camera motion compensation need to be corrected. At this time, the accuracy of the prediction results is determined by IOU and ReID.

[0069] The GhostNetV2 network 40 executes step S40: inputs the predicted image into the GhostNetV2 network for feature extraction to obtain predicted image features.

[0070] In order to convert images into a form that is more suitable for analysis and understanding, it is necessary to extract their features. In this data conversion process, data transformation, filtering, dimensionality reduction, etc. are involved. In this application, the GhostNetV2 network is used to complete the feature extraction of image data to solve the problem of limited computing resources of drone loads and meet the needs of real-time tracking at the same time.

[0071] See also Figure 4 , the feature extraction process of the GhostNetV2 network includes:

[0072] The acquired predicted image data is convoluted and separated, and then batch normalized and ReLU activated to obtain the first intermediate feature;

[0073] The acquired predicted image data is processed by the decomposition fully connected attention mechanism to obtain the second intermediate feature;

[0074] The decomposed fully connected attention mechanism is different from the conventional fully connected layer. It is based on two fully connected layers, one of which aggregates features along the horizontal direction and the other aggregates features along the vertical direction. The feature aggregation in the two directions can be specifically characterized as follows:

[0075]

[0076] in, and are learnable weights and Z is the image data.

[0077] The two fully connected layers capture the long-range correlations along the two directions respectively. At this time, the fully connected attention mechanism is decomposed and used in conjunction with the Ghost module to capture the dependencies of long-distance spatial positions.

[0078] In order to improve the efficiency and accuracy of the UAV in performing automatic intrusion inspection tasks, the present invention also trains the existing Ghost NetV2 network so that it can adapt to each specific object or task. Figure 5 , the training process of the Ghost Netv2 network in the present invention is as follows:

[0079] Input the acquired training image data into the Ghost Netv2 network to be trained to obtain the predicted image features;

[0080] In one embodiment of the present invention, the Ghost module is trained using the open source dataset ImageNet. P identities and K images of each person are randomly selected as training samples, where P is 16 and K is 4. Considering that pedestrians under the perspective of the drone are small targets, the size of each image is adjusted to 64*32, and the adjusted image is padded with 10 pixels, and then randomly cropped to a 64*32 rectangular image. The rectangular image is input into the model, and the model outputs the ReID feature F.

[0081] Perform global average pooling on the predicted image features to obtain a feature map;

[0082] The predicted image feature, i.e., the ReID feature F, is input into a global average pooling layer and restored to a feature map.

[0083] Calculate feature loss based on the feature map and its corresponding standard map;

[0084] The prediction feature F uses the triplet loss as the feature loss, and its mathematical formula is as follows:

[0085]

[0086] in is the feature map obtained after the random sample passes through the network, is the feature map obtained after a random positive sample passes through the network, is the feature map obtained after the random negative sample passes through the network. α represents the threshold,

[0087] When the value of the formula in the brackets is greater than 0, the + sign is taken and it is itself. At this time, the loss is greater than 0 and the parameters are updated; when the value of the formula in the brackets is less than 0, the + sign is taken and it is 0. At this time, no gradient can be generated and the parameters are not updated. In this embodiment, α is 0.3.

[0088] The feature map is processed through a fully connected layer to calculate the predicted probability distribution, and the distribution loss is calculated based on the predicted probability distribution and the standard probability.

[0089] Modify the parameters of the Ghost Netv2 network according to the distribution loss and feature loss until the network converges.

[0090] The predicted probability distribution uses the cross entropy loss function, and its mathematical formula is as follows:

[0091]

[0092] Where y represents the actual probability distribution, Represents the predicted probability distribution, so the total training loss expression is as follows:

[0093] L total =L feature +L p

[0094] After the model converges, training is stopped and the latest model weights are saved for subsequent use. During the training process, the image feature loss of the training set is represented by triplet loss, so that the trained Ghost Netv2 network can better distinguish the differences between similar images and improve the drone's ability to recognize targets. The introduction of cross entropy loss speeds up the convergence of the model, reduces the training time required for switching between different targets, and enables faster and more effective responses when facing different tasks.

[0095] The feature matching module executes step S50: calculating the similarity between the predicted image features and the actual image features, and completing the screening and correction of the predicted trajectory according to the similarity.

[0096] There are many ways to compare similarity, such as comparing cosine similarity, Euclidean distance, grayscale image algorithm, etc. In the present invention, the cosine similarity between the image feature and the target feature is calculated, and finally it is determined whether the value of the cosine similarity exceeds a preset threshold. If it exceeds, it is determined that the predicted image feature and the actual image feature are the same object. At this time, it means that the confidence of the trajectory prediction is high. However, if the cosine similarity is lower than the preset threshold, it is determined that there is an error in the predicted trajectory of the Kalman filter and correction is required, and the parameters of the Kalman filter are corrected according to the predicted difference.

[0097] The route generation module executes step S60: performing route planning according to the predicted trajectory and generating a real-time tracking route.

[0098] Compare the real-time tracking route with the actual position of the tracked object, adjust the gimbal and aircraft posture of the drone so that the tracked object is in the center of the camera, then record the actual movement path and video of the tracked object, and generate the movement trajectory of the tracked object for subsequent analysis and investigation. It can also provide important evidence and convenience in the fields of security and monitoring.

[0099] In the GhostNetV2 network, feature reuse technology is usually used to design the network. At this time, the Ghost module will reuse the feature maps of other layers through splicing operations to ensure a large number of channels and network capacity, but this will increase the computational cost of the network on the hardware device, thus affecting the execution efficiency. Figure 6 As shown, in the existing Ghost module, the input feature map is subjected to a depth-separable convolution to obtain an intrinsic map, and then the intrinsic map is processed by a Relu activation function to obtain a Ghost map, and finally the input feature map and the Ghost map are feature spliced ​​to output the final result. This feature reuse technology will reduce the efficiency of the camera of the present invention in completing feature matching, affecting the real-time tracking. Based on the above shortcomings, the present invention refers to the idea of ​​re-parameterization and performs structural re-parameterization training on the convolution kernel used for dconv, so that the Ghost module finally deployed can reduce the processing time and meet the needs of real-time tracking.

[0100] See also Figure 7 , a represents the original Ghost module, b replaces the cat operation with the add operation based on a, c moves the ReLU operation after the add operation based on b, and d adds the BN operation in the jump connection based on c. According to the idea of ​​reparameterization, the operation in the box area of ​​d can be equivalent to the dconv operation, that is, the structure in e is obtained.

[0101] That is, according to Figure 7The structure shown in (d) performs structural re-parameterization training and continuously corrects the value of the convolution kernel of dconv, so that the final deployment of the Ghost module only includes the trained dconv layer and Relu layer. At this time, the feature fusion process is carried out in the weight space rather than in the feature space, which improves the efficiency of feature fusion.

[0102] In addition, after completing the training of the network model, in order to further streamline the model, the present invention also performs pruning on the trained model. By deleting redundant neurons or connections, pruning can reduce the number of model parameters and reduce computing and storage requirements without significantly affecting the model performance. The iterative pruning strategy adopted by the present invention has the following mathematical formula:

[0103]

[0104] where ρ is the pruning operation, r is the pruning rate, and θ represents the fine-tuning operation. Represents the network after the t-th pruning.

[0105] When the network cannot make valid predictions, the pruning process stops and the network obtained from the last valid pruning is used as the final network.

[0106] In summary, the automatic intrusion inspection method and system of unmanned aerial vehicles described in the present invention first improves the Re-ID module of the existing BoT-SORT-ReID algorithm, and replaces the Resnet50 network in ReID with the GhostNetV2 network, which effectively reduces the amount of calculation during Re-ID feature extraction and improves the real-time performance of target tracking. After each prediction is completed, the relevant parameters in the trajectory prediction module can be corrected according to the prediction results and the actual offset results, so that it is more in line with the motion preference of the predicted target and the prediction accuracy is improved. Furthermore, the feature reuse technology is usually used in the GhostNetV2 network to design the network, which increases the computing cost on the network hardware equipment, thereby affecting the execution efficiency. Therefore, the Ghost module is structurally re-parameterized and trained, so that the structure of the network deployment is reduced, and the feature fusion process is carried out in the weight space instead of the feature space, which improves the efficiency of feature fusion. Finally, the trained network model is pruned to further simplify the number of model parameters, reduce the computing and storage requirements, and improve the real-time performance of the system.

[0107] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the automatic intrusion inspection method for a drone described in any one of the above embodiments is implemented.

[0108] The present invention may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0109] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A method for automatic intrusion inspection by a drone, characterized in that: include: Perform target detection on the video frames acquired by the drone to obtain the original image features of each frame; The original image features of each frame of image are compared with the preset target features in sequence. When the original image features and the target features are successfully matched, the successfully matched image is used as the image to be predicted; Perform trajectory prediction on the original image features of the image to be predicted to obtain a predicted image and a predicted trajectory; Extracting features of the predicted image through a GhostNetV2 network to obtain predicted image features; The predicted image feature is compared with the original image feature of the next frame of the corresponding image to be predicted, and the predicted trajectory is screened and corrected according to the similarity; Route planning is performed based on the predicted trajectory to generate a real-time tracking route.

2. According to the method for automatic intrusion inspection of a drone according to claim 1, it is characterized in that: The feature extraction of the predicted image through the GhostNetV2 network specifically includes: The first Ghost module first performs convolution separation on the acquired predicted image data, and then performs batch normalization and ReLU activation to obtain the first intermediate feature; multiplying the first intermediate feature and the second intermediate feature to obtain a third intermediate feature; The third intermediate feature is convolved and separated by the second Ghost module, and then batch normalized to obtain a fourth intermediate feature; The fourth intermediate feature is added to the acquired predicted image data to obtain a predicted image feature.

3. The method for automatic intrusion inspection of a drone according to claim 2, characterized in that: The GhostNetV2 network is trained by the following steps: Input the acquired training image data into the Ghost Netv2 network to be trained to obtain the predicted image features; Perform global average pooling on the predicted image features to obtain a feature map; Calculate feature loss by comparing the feature map with its corresponding standard map; and calculate predicted probability distribution by processing the feature map through a fully connected layer, and calculate distribution loss based on the predicted probability distribution and the standard probability; Modify the parameters of the Ghost Netv2 network according to the distribution loss and feature loss until the network converges.

4. The method for automatic intrusion inspection of a drone according to claim 3 is characterized in that: The loss between the predicted image features and the standard image features is calculated by the following formula: in They represent the feature maps obtained after random samples, random positive samples, and random negative samples pass through the network. α represents the threshold.

5. A drone automatic intrusion inspection method according to any one of claims 2 to 4, characterized in that: Perform structural reparameterization training on at least one Ghost module of the GhostNetV2 network shown to reduce the model structure.

6. The method for automatic intrusion inspection of a drone according to claim 5, characterized in that: After the model converges, the network model is fine-tuned using the following formula: Where ρ is the pruning operation, r is the pruning rate, and θ represents the fine-tuning operation. Represents the network after the t-th pruning.

7. An automatic drone intrusion inspection system, characterized in that: include Drone gimbal: used to track and obtain video frames of the inspection area; Target detection module: used to perform target detection on the video frames acquired by the drone to obtain the original image features of each frame; Target matching module: used to compare the original image features of each frame of image with the preset target features in sequence. When the original image features successfully match the target features, the successfully matched image is used as the current frame; Trajectory prediction module: used to perform trajectory prediction on the original image features of the image to be predicted, and obtain the predicted image and predicted trajectory; GhostNetV2 network: used to extract features of the predicted image to obtain predicted image features; Feature matching module: used to compare the predicted image features with the original image features of the next frame of the corresponding image to be predicted, and complete the screening and correction of the predicted trajectory according to the similarity; Route generation module: used to plan routes based on predicted trajectories and generate real-time tracking routes.

8. The automatic intrusion inspection system of a drone according to claim 7 is characterized in that: The GhostNetV2 network includes: The first Ghost module layer is used to perform convolution separation on the acquired predicted image data, and then perform batch normalization and Relu activation to obtain the first intermediate feature; Decomposed fully connected attention layer: The acquired predicted image data is processed by the decomposed fully connected attention mechanism to obtain the second intermediate feature; Feature aggregation layer: used for multiplying the first intermediate feature and the second intermediate feature to obtain a third intermediate feature; Second Ghost module layer: the third intermediate feature is convolutionally separated by the second Ghost module and then batch normalized to obtain the fourth intermediate feature; Feature concatenation layer: Add the fourth intermediate feature to the acquired original image data to obtain the image feature.

9. The automatic intrusion inspection system of a drone according to claim 8, characterized in that: Also included is a network update module, which includes: Feature map restoration submodule: used to perform global average pooling on the predicted image features to obtain the feature map; Feature loss calculation submodule: used to calculate feature loss according to the feature map and its corresponding standard map; Distribution loss calculation submodule: used to process the feature map through the fully connected layer to calculate the predicted probability distribution, and calculate the distribution loss according to the predicted probability distribution and the standard probability; Network update submodule: used to correct the parameters of the Ghost Netv2 network according to the distribution loss and feature loss until the network converges.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, the steps of the automatic intrusion inspection method of a drone as described in any one of claims 1 to 6 are implemented.

Images