UAV detection and countermeasure method and device for power facilities

Abstract

The present invention discloses a method for detecting and countering drones for electric power facilities, which detects and counters external destructive drones, including step S1: installing and deploying a drone detection and countering device, so that the drone detection and countering device enters the detection stage of normal operation, and performs real-time detection of external destructive drones by combining visual detection algorithms and audio detection algorithms, so as to determine whether they are located in the invasion area. The present invention discloses a method for detecting and countering drones for electric power facilities, which can be deployed at power transmission and distribution facilities, and performs 360-degree real-time intelligent identification, detection and positioning, and video tracking of external destructive drones, and shouts and drives away drones and their operators, so as to counter the threat of external destructive drones within the safety range of the facilities.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle detection and countermeasures, and in particular relates to a unmanned aerial vehicle detection and countermeasures method for electric power facilities and a unmanned aerial vehicle detection and countermeasures device for electric power facilities. Background Art

[0002] With the rapid development and widespread application of drone technology, its potential threat to the safety of power grid operation has become increasingly prominent. Drones have been widely used in agricultural plant protection, three-dimensional mapping and other fields. At the same time, drone collision accidents caused by "illegal flying" and improper operation of flight operators are also increasing, causing physical damage to key power grid facilities such as transmission lines and substations, leading to power outages and operation interruptions, posing a threat to the stability of power supply. How to effectively complete drone prevention and control work for power facilities has become a key issue to ensure the stable operation of the power grid and the safety of personnel.

[0003] Problems and shortcomings of existing technologies:

[0004] Although various types of drones for industrial applications, including plant protection drones, are partially equipped with airborne obstacle avoidance systems such as millimeter-wave radars, it is difficult for drones to perform absolutely reliable detection and avoidance of power transmission and distribution lines during automatic flight operations because the reflection characteristics of power transmission and distribution lines are sparse and the drone's airborne obstacle avoidance system is easily affected by environmental factors such as weather and dust. At the same time, when drones fly near high-voltage power transmission and distribution lines, the flight control system may be interfered by strong magnetic fields, causing flight control system failures and flight accidents. In addition, illegal flight or loss of control of drones may also collide with power transmission and distribution lines, causing power outages and secondary disasters, and bringing adverse effects to society or the environment.

[0005] 1. The obstacle avoidance system of the drone itself is limited and cannot avoid sparsely-featured facilities such as power transmission and distribution lines;

[0006] 2. In scenes with low visibility such as cloudy and foggy days, it is difficult to accurately detect and locate the drone by vision alone;

[0007] 3. The power line protection zone is mainly strip-shaped and long-distance structure. Traditional prevention and control measures are mainly aimed at omnidirectional area protection. The installation and structural form of its prevention and control equipment are difficult to adapt to the power transmission and distribution line scenario, and the pole tower and deployment scenarios are not considered. Summary of the invention

[0008] The main purpose of the present invention is to provide a drone detection and countermeasure method and device for power facilities, which can be deployed at power transmission and distribution facilities, perform 360-degree real-time intelligent identification, detection and positioning, and video tracking of external destructive drones, and shout to drive away the drones and their operators, so as to counter the threat of external destructive drones within the safety range of the facilities.

[0009] To achieve the above objectives, the present invention provides a method for detecting and countering drones for power facilities, which detects and counters external sabotaging drones, and includes the following steps:

[0010] Step S1: Install and deploy the drone detection and countermeasure device so that the drone detection and countermeasure device enters the detection stage of normal operation, and detects external sabotage drones in real time by combining visual detection algorithms and audio detection algorithms to determine whether they are in the intrusion area;

[0011] Step S2: The drone detection and countermeasure device takes different actions according to the different intrusion areas where the external sabotage drone is located.

[0012] As a further preferred technical solution of the above technical solution, the visual detection algorithm of step S1 is specifically implemented as follows:

[0013] (First, the detection algorithm uses an improved deep learning algorithm YOLOv5s-360ID for detection. Compared with the traditional YOLOv5s algorithm, the target detection border regression loss function is improved. The improved loss function L EIoU As shown below:

[0014]

[0015] In the formula, IoU represents the overlap rate between the predicted border and the real border, ρ 2 (b,b gt ) represents the Euclidean distance between the center point of the anchor box and the target box, ρ 2 (w,w gt ) represents the width difference between the anchor box and the target box, ρ 2 (h,h gt ) represents the height difference between the anchor box and the target box, w c and h c represents the width and height of the minimum bounding rectangle containing the anchor box and the basic box, w and h represent the width and height of the target box output by the model prediction, b represents the target center point output by the model prediction, and w gt and h gt Indicates the width and height of the anchor box predicted by the model, b gt Represents the center point of the anchor box predicted by the model;

[0016] Through the trained model, the panoramic image is input in real time for detection. After the target in the image is detected, the model will output the coordinates of the center of the target box in the image (p x , p y), after detecting the external target of destroying the drone in the panoramic image, guide the horizontal turntable and the surveillance camera turntable to rotate to the target position; take the center of the panoramic image as the coordinate origin, the image to the right is the horizontal axis square, and the image upward is the vertical axis square;

[0017] Convert the detected target center pixel coordinates into the target angle of the horizontal turntable and the surveillance camera turntable, so as to control the rotation of the horizontal turntable and the surveillance camera turntable. It is necessary to calculate the azimuth angle from the plane coordinates to the spherical coordinates. and the zenith angle θ, as shown in the formula:

[0018]

[0019] In the formula, p x and p y They represent the horizontal and vertical coordinates of the center of the detected target in the image plane coordinates, respectively, and w and h represent the maximum effective pixel values ​​of the width and height of the panoramic image, respectively;

[0020] During installation, align the initial 0 angle of the horizontal turntable and the surveillance camera turntable to the center of the image. represents the absolute yaw angle target of the horizontal turntable rotation, and θ represents the absolute pitch angle target of the surveillance camera turntable rotation;

[0021] After the horizontal turntable and the surveillance camera turntable rotate to the target area, the surveillance camera starts detection and tracks the target by itself. After detecting the target, the two-dimensional coordinates of the target in the surveillance camera image are converted into the angles for controlling the horizontal turntable and the surveillance camera turntable;

[0022] Taking the coordinates of the panoramic camera as the reference coordinates, the monitoring camera image coordinates are converted into coordinates under the panoramic camera image coordinate axis reference system. After the coordinates are obtained, they are entered into the above formula to obtain the target absolute angle controlled by the monitoring camera.

[0023] As a further preferred technical solution of the above technical solution, the target absolute angle of controlling the horizontal turntable and the surveillance camera turntable is specifically implemented as follows:

[0024] First, the two-dimensional coordinate points of the panoramic camera image (p x , p y ) is converted to the coordinates (X p , Y p , Z p ), as shown in the formula:

[0025]

[0026] In the formula, K p is the internal parameter matrix of the panoramic camera;

[0027] The camera intrinsic parameter matrix is ​​calculated as shown in the formula:

[0028]

[0029] In the formula, f x and f y is the focal length, which represents the focal length in the horizontal and vertical directions of the image, c x and c y is the optical center coordinate of the image, indicating the position of the image center;

[0030] The parameter supremacy formula of the panoramic camera is introduced K can be calculated p ;

[0031] Use the rotation matrix R of the horizontal turntable and the surveillance camera turntable g The transformation relationship between the surveillance camera coordinate system and the panoramic camera coordinate system is obtained from the following formula:

[0032]

[0033] The rotation matrix R of the surveillance camera (i.e., PTZ camera) g It is obtained from the rotation matrices of the pitch axis (surveillance camera turntable axis) and the yaw axis (horizontal turntable axis), as shown in the following formula:

[0034]

[0035] In the formula, Represents the yaw angle of the horizontal turntable, θ c Represents the current pitch angle of the surveillance camera turntable;

[0036] The two-dimensional coordinate point of the surveillance camera image (g x ,g y ) and the coordinates in the surveillance camera coordinate system (X g ,Y g ,Z g ) The conversion relationship is shown as follows:

[0037]

[0038] In the formula, K g is the internal parameter matrix of the surveillance camera (which can be obtained by to obtain);

[0039] From the above formula The two-dimensional coordinates of the surveillance camera image are converted into the two-dimensional coordinates of the panoramic camera image and the relationship is:

[0040]

[0041] After the two-dimensional coordinates of the detection target are input, its coordinates in the panoramic camera image coordinate system are obtained, and then Obtain the rotation angle required for control.

[0042] As a further preferred technical solution of the above technical solution, the audio detection algorithm of step S1 is specifically implemented as follows:

[0043] To convert the acoustic signal received by the microphone into an MFCC feature map (a two-dimensional feature map of the Mel cepstrum), the signal must first be passed through a high-pass filter, as shown in the following formula:

[0044] y[n]=x[n]-0.97x[n-1];

[0045] In the formula, n represents the sampling point, y represents the output of the filter, x represents the sound signal obtained by microphone sampling, 0.97 is the pre-emphasis parameter, usually 0.9 to 1.0 (after experiments, 0.97 can better extract the high-frequency part of the sound signal in the current design);

[0046] The design uses a window function to frame the signal. The window function is shown in the following formula (Hamming window):

[0047]

[0048] Where N represents the length of each audio frame;

[0049] Multiply the filtered signal y[n] by the window function w(n) to get z[n], as shown in the following formula:

[0050] z[n]=y[n]*w[n];

[0051] Perform Fourier transform on the obtained z[n],

[0052] Where k represents the index of the point after Fourier transformation;

[0053] Then, a set of triangular filters is defined. The filters are set for the Mel scale. The number of filters is defined as M (usually set to 22-26. Through experiments, 23 is taken in this embodiment). The Mel frequency calculation method under the Mel scale is:

[0054]

[0055] Where f represents the frequency of the speech signal, m represents the Mel frequency corresponding to the frequency, and the maximum value of the sampling frequency f is substituted max The maximum Mel frequency m can be obtained max , change 0 to m maxDivide it into 23 equal parts to get the median of each triangular filter, which can then be expressed as The inverse operation is to obtain the frequency of each triangular filter corresponding to the Mel value. The inverse operation is:

[0056]

[0057] The frequency response of the triangular filter is obtained as follows:

[0058]

[0059] The logarithmic energy of each filter output is then calculated as shown below:

[0060]

[0061] Finally, the final feature map is obtained through discrete cosine transform, as shown in the following formula:

[0062]

[0063] Where K represents the number of triangular bandpass filters and n = 1, 2, ..., K, S k represents the energy of the kth filter, L represents the order of the required Mel-cepstrum coefficient, and the value of L is 13 (determined by experiment). Finally, the collected audio is sequentially brought into the 1st to 13th order Mel-cepstrum to obtain a 13-dimensional Mel-cepstrum feature map, that is, the final feature map size is N×L (after obtaining the feature map, the drone can be identified).

[0064] As a further preferred technical solution of the above technical solution, the drone detection algorithm based on the multi-source sensor combination of the visual detection algorithm and the audio detection algorithm is specifically implemented as follows:

[0065] Based on the CNN model, multi-source data is fused and detected. The logistic regression model is shown in the following formula (after the acoustic signal is extracted into a two-dimensional feature map, the extracted feature map data is input into the CNN model together with the light signal obtained by the gimbal camera for fusion detection):

[0066]

[0067] In the formula, is an N×1 vector, which is used to indicate the possibility of the existence of an external sabotage drone. p1 and p2 represent the optical image and acoustic feature map data of the drone, respectively. a0 is the bias term, a1 and a2 are weight coefficients, and g(x) is the S function shown in the following formula:

[0068]

[0069] The logistic regression model can be rewritten as:

[0070]

[0071] Where P = [1, p1, p2], a = [a0, a1, a2] T ;

[0072] Design the cost function as shown below to obtain the optimal coefficient combination {a0, a1, a2}:

[0073]

[0074] Among them, a o =[0,a1,a2], λ is the regularization parameter, y represents the human annotation result of the input data source, that is, the expected target, N represents the type of data source, and this formula includes two types of inputs: acoustic signal and optical signal, so N = 2, and the gradient function of J(a) is shown as follows:

[0075]

[0076] Where D(x)=diag([x1,x2,…,x N ]), is a diagonal matrix, x=[x1,x2,…,x N ] T , we can obtain the partial derivative of the logistic regression model by rewriting it:

[0077]

[0078] Substituting this formula into the gradient function of J(a), we have:

[0079]

[0080] Then use the gradient method to iterate a(j), as shown below:

[0081]

[0082] In the formula, j is the number of iterations, μ represents the learning rate;

[0083] After inputting the collected data set into the model and training the model, suitable a0, a1, and a2 are obtained, and the obtained parameters are brought into the logistic regression model to obtain the final model; after the model is trained by inputting the collected data set, the obtained model is imported into the program, and the image taken by the camera at the same time and the audio feature map collected by the microphone are synchronously input into the model for calculation each time, so as to obtain the fusion probability of whether the current external destructive drone exists in the invasion area.

[0084] As a further preferred technical solution of the above technical solution, step S2 is specifically implemented as the following steps:

[0085] Step S2.1: After detecting an external sabotage drone, determine whether the external sabotage drone is located in the intrusion warning zone. If yes, perform the first disposal, otherwise proceed to step S2.2:

[0086] Step S2.2: Determine whether the external sabotage drone is located in the intrusion restricted area, and if so, perform the second disposal;

[0087] Step S2.3: After the first or second treatment, determine whether the intrusion is eliminated. If so, continue to perform real-time detection of external destructive drones through the drone detection and countermeasure device; otherwise, perform the third treatment.

[0088] To achieve the above purpose, the present invention also provides a UAV detection and countermeasure device for power facilities, which is applied to the UAV detection and countermeasure method for power facilities, including a main body, a panoramic camera, a monitoring camera, a microphone array (one group each at the front and rear), a loudspeaker and a base, wherein:

[0089] The main body is provided with an edge computing terminal, a horizontal turntable, a surveillance camera turntable and a megaphone turntable, the panoramic camera is installed on the top of the main body, the surveillance camera is installed on the surveillance camera turntable, the microphone array is installed on the front and rear sides of the main body respectively, the megaphone is installed on the megaphone turntable, and the main body is installed on the base through the horizontal turntable;

[0090] The panoramic camera, the monitoring camera, the microphone array and the loudspeaker are electrically connected to the edge computing terminal respectively.

[0091] The beneficial effects of the present invention are:

[0092] 1. The drone detection and countermeasure device of the present invention combines a panoramic camera and a surveillance camera, and can use the panoramic image to detect drones over a large range. It is combined with the panoramic camera's visual detection and guidance algorithm to guide the surveillance camera to rotate to the location of the drone, and accurately detect and identify the drone, greatly improving the detection range and identification accuracy of a single drone detection device.

[0093] 2. The present invention also combines visual detection with acoustic wave detection methods, and uses both image recognition and audio recognition methods to detect drones. The drone detection method algorithm based on multi-source sensor fusion can improve the accuracy of drone detection.

[0094] 3. The drone detection and countermeasure device is also equipped with a loudspeaker, which can promptly warn drones and operators to prevent drone intrusion and damage.

[0095] 4. The present invention is applicable to the detection and countermeasure of external sabotaging drones in the operation environment of power transmission and distribution lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is a flow chart of the method and device for detecting and countering unmanned aerial vehicles for electric power facilities of the present invention.

[0097] Figure 2 It is a structural schematic diagram of the method and device for detecting and countering unmanned aerial vehicles for electric power facilities of the present invention.

[0098] Figure 3 It is a coordinate schematic diagram of the method and device for detecting and countering unmanned aerial vehicles for electric power facilities of the present invention.

[0099] Figure 4 It is a rotating schematic diagram of the method and device for detecting and countering unmanned aerial vehicles for electric power facilities of the present invention.

[0100] Figure 5 It is a schematic diagram of the existing YOLOv5s model. DETAILED DESCRIPTION

[0101] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0102] In the preferred embodiment of the present invention, those skilled in the art should note that the external sabotage drones and the like involved in the present invention may be regarded as prior art.

[0103] Preferred embodiments.

[0104] Since the power transmission and distribution facilities are mostly surrounded by empty roads, farmlands and other scenes with relatively simple background sounds, and it is difficult to effectively detect drones only by image features when the weather conditions are poor such as at night and in heavy fog, the present invention detects drones through both image features and audio features, and uses a drone detection algorithm based on multi-source sensor fusion to improve the reliability of drone detection and identification.

[0105] like Figure 1-5 As shown, the present invention discloses a method for detecting and countering drones for power facilities, which detects and counters external sabotaging drones, including the following steps:

[0106] Step S1: Install and deploy the drone detection and countermeasure device so that the drone detection and countermeasure device enters the detection stage of normal operation, and detects external sabotage drones in real time by combining visual detection algorithms and audio detection algorithms to determine whether they are in the intrusion area;

[0107] Step S2: The drone detection and countermeasure device takes different actions according to the different intrusion areas where the external sabotage drone is located.

[0108] Specifically, the visual detection algorithm of step S1 is implemented as follows (the detection of drone image features adopts a method combining panoramic images with ordinary images: the panoramic image can cover the situation around the panoramic camera at 360 degrees and can quickly locate suspected targets. However, due to its wide coverage, the proportion of drones in the picture is small, and the features are difficult to accurately identify. It is necessary to use the panoramic camera visual detection guidance algorithm to guide the monitoring camera toward the target direction, and combine it with the ordinary image it takes to further determine whether the drone exists):

[0109] (First, the detection algorithm uses an improved deep learning algorithm YOLOv5s-360ID for detection. Compared with the traditional YOLOv5s algorithm, the target detection border regression loss function is improved. The improved loss function L EIoU As shown below:

[0110]

[0111] In the formula, IoU represents the overlap rate between the predicted border and the real border, ρ 2 (b,b gt ) represents the Euclidean distance between the center point of the anchor box and the target box, ρ 2 (w,w gt ) represents the width difference between the anchor box and the target box, ρ 2 (h,h gt ) represents the height difference between the anchor box and the target box, w c and h c represents the width and height of the minimum bounding rectangle containing the anchor box and the basic box, w and h represent the width and height of the target box output by the model prediction, b represents the target center point output by the model prediction, and w gt and h gt Indicates the width and height of the anchor box predicted by the model, b gt Represents the center point of the anchor box predicted by the model;

[0112] It is worth mentioning that the structure of the YOLOv5s model is as follows Figure 5 As shown in the figure, after the panoramic image is input into the model for calculation, the detection result can be obtained. The model needs to be trained on the collected data set in advance, and the training of the model parameters needs to be adjusted according to the loss function. YOLOv5s-360ID improves its loss function compared to the traditional YOLOv5s. The loss function of the original model is the existing technology.

[0113] Through the trained model, the panoramic image is input in real time for detection. After the target in the image is detected, the model will output the coordinates of the center of the target box in the image (p x , p y ), after detecting the external target of destroying the drone in the panoramic image, guide the horizontal turntable and the surveillance camera turntable to rotate to the target position; take the center of the panoramic image as the coordinate origin, the image to the right is the horizontal axis square, and the image upward is the vertical axis square;

[0114] Convert the detected target center pixel coordinates into the target angle of the horizontal turntable and the surveillance camera turntable, so as to control the rotation of the horizontal turntable and the surveillance camera turntable. It is necessary to calculate the azimuth angle from the plane coordinates to the spherical coordinates. and the zenith angle θ, as shown in the formula:

[0115]

[0116] In the formula, p x and p y They represent the central horizontal and vertical coordinates of the detected target in the image plane coordinates, w and h represent the maximum effective pixel values ​​of the panoramic image width and height respectively (with the loss function L EIoU The w and h in are different);

[0117] During installation, align the initial 0 angle of the horizontal turntable and the surveillance camera turntable to the center of the image. represents the absolute yaw angle target of the horizontal turntable rotation, and θ represents the absolute pitch angle target of the surveillance camera turntable rotation;

[0118] After the horizontal turntable and the surveillance camera turntable rotate to the target area, the surveillance camera starts detection and tracks the target by itself. After detecting the target, the two-dimensional coordinates of the target in the surveillance camera image are converted into the angles for controlling the horizontal turntable and the surveillance camera turntable;

[0119] Taking the coordinates of the panoramic camera as the reference coordinates, the monitoring camera image coordinates are converted into coordinates under the panoramic camera image coordinate axis reference system. After the coordinates are obtained, they are entered into the above formula to obtain the target absolute angle controlled by the monitoring camera.

[0120] More specifically, the target absolute angles of the horizontal turntable and the surveillance camera turntable are controlled as follows:

[0121] First, the two-dimensional coordinate points of the panoramic camera image (p x ,p y ) is converted to the coordinates (X p ,Y p ,Z p ), as shown in the formula:

[0122]

[0123] In the formula, K p is the internal parameter matrix of the panoramic camera;

[0124] The camera intrinsic parameter matrix is ​​calculated as shown in the formula:

[0125]

[0126] In the formula, f x and f y is the focal length, which represents the focal length in the horizontal and vertical directions of the image, c x and c y is the optical center coordinate of the image, indicating the position of the image center;

[0127] The parameter supremacy formula of the panoramic camera is introduced K can be calculated p ;

[0128] Use the rotation matrix R of the horizontal turntable and the surveillance camera turntable g The transformation relationship between the surveillance camera coordinate system and the panoramic camera coordinate system is obtained from the following formula:

[0129]

[0130] The rotation matrix R of the surveillance camera (i.e., PTZ camera) g It is obtained from the rotation matrices of the pitch axis (surveillance camera turntable axis) and the yaw axis (horizontal turntable axis), as shown in the following formula:

[0131]

[0132] In the formula, Represents the yaw angle of the horizontal turntable, θ c Represents the current pitch angle of the surveillance camera turntable;

[0133] The two-dimensional coordinate point of the surveillance camera image (g x ,g y ) and the coordinates in the surveillance camera coordinate system (X g ,Y g ,Z g ) The conversion relationship is shown as follows:

[0134]

[0135] In the formula, K g is the internal parameter matrix of the surveillance camera (which can be obtained by to obtain);

[0136] From the above formula The two-dimensional coordinates of the surveillance camera image are converted into the two-dimensional coordinates of the panoramic camera image and the relationship is:

[0137]

[0138] After the two-dimensional coordinates of the detection target are input, its coordinates in the panoramic camera image coordinate system are obtained, and then Obtain the rotation angle required for control.

[0139] Furthermore, the audio detection algorithm of step S1 is specifically implemented as follows:

[0140] To convert the acoustic signal received by the microphone into an MFCC feature map (a two-dimensional feature map of the Mel cepstrum), the signal must first be passed through a high-pass filter, as shown in the following formula:

[0141] y[n]=x[n]-0.97x[n-1];

[0142] In the formula, n represents the sampling point, y represents the output of the filter, x represents the sound signal obtained by microphone sampling, 0.97 is the pre-emphasis parameter, usually 0.9 to 1.0 (after experiments, 0.97 can better extract the high-frequency part of the sound signal in the current design);

[0143] The design uses a window function to frame the signal. The window function is shown in the following formula (Hamming window):

[0144]

[0145] Where N represents the length of each audio frame;

[0146] Multiply the filtered signal y[n] by the window function w(n) to get z[n], as shown in the following formula:

[0147] z[n]=y[n]*w[n];

[0148] Perform Fourier transform on the obtained z[n],

[0149] Where k represents the index of the point after Fourier transformation;

[0150] Then, a set of triangular filters is defined. The filters are set for the Mel scale. The number of filters is defined as M (usually set to 22-26. Through experiments, 23 is taken in this embodiment). The Mel frequency calculation method under the Mel scale is:

[0151]

[0152] Where f represents the frequency of the speech signal, m represents the Mel frequency corresponding to the frequency, and the maximum value of the sampling frequency f is substituted max The maximum Mel frequency m can be obtained max , change 0 to m max Divide it into 23 equal parts to get the median of each triangular filter, which can then be expressed as The inverse operation is to obtain the frequency of each triangular filter corresponding to the Mel value. The inverse operation is:

[0153]

[0154] The frequency response of the triangular filter is obtained as follows:

[0155]

[0156] The logarithmic energy of each filter output is then calculated as shown below:

[0157]

[0158] Finally, the final feature map is obtained through discrete cosine transform, as shown in the following formula:

[0159]

[0160] Where K represents the number of triangular bandpass filters and n = 1, 2, ..., K, S k represents the energy of the kth filter, L represents the order of the required Mel-cepstrum coefficient, and the value of L is 13 (determined by experiment). Finally, the collected audio is sequentially brought into the 1st to 13th order Mel-cepstrum to obtain a 13-dimensional Mel-cepstrum feature map, that is, the final feature map size is N×L (after obtaining the feature map, the drone can be identified).

[0161] Furthermore, the drone detection algorithm based on the multi-source sensor combination of the visual detection algorithm and the audio detection algorithm is specifically implemented as follows:

[0162] Based on the CNN model, multi-source data is fused and detected. The logistic regression model is shown in the following formula (after the acoustic signal is extracted into a two-dimensional feature map, the extracted feature map data is input into the CNN model together with the light signal obtained by the gimbal camera for fusion detection):

[0163]

[0164] In the formula, is an N×1 vector, which is used to indicate the possibility of the existence of an external sabotage drone. p1 and p2 represent the optical image and acoustic feature map data of the drone, respectively. a0 is the bias term, a1 and a2 are weight coefficients, and g(x) is the S function shown in the following formula:

[0165]

[0166] The logistic regression model can be rewritten as:

[0167]

[0168] Where P = [1, p1, p2], a = [a0, a1, a2] T ;

[0169] Design the cost function as shown below to obtain the optimal coefficient combination {a0, a1, a2}:

[0170]

[0171] Among them, a o =[0,a1,a2], λ is the regularization parameter, y represents the human annotation result of the input data source, that is, the expected target, N represents the type of data source, and this formula includes two types of inputs: acoustic signal and optical signal, so N = 2, and the gradient function of J(a) is shown as follows:

[0172]

[0173] Where D(x)=diag([x1,x2,…,x N ]), is a diagonal matrix, x=[x1,x2,…,x N ] T , we can obtain the partial derivative of the logistic regression model by rewriting it:

[0174]

[0175] Substituting this formula into the gradient function of J(a), we have:

[0176]

[0177] Then use the gradient method to iterate a(j), as shown below:

[0178]

[0179] In the formula, j is the number of iterations, μ represents the learning rate;

[0180] After inputting the collected data set into the model and training the model, suitable a0, a1, and a2 are obtained, and the obtained parameters are brought into the logistic regression model to obtain the final model; after the model is trained by inputting the collected data set, the obtained model is imported into the program, and the image taken by the camera at the same time and the audio feature map collected by the microphone are synchronously input into the model for calculation each time, so as to obtain the fusion probability of whether the current external destructive drone exists in the invasion area.

[0181] Preferably, step S2 is specifically implemented as the following steps:

[0182] Step S2.1: When an external sabotage drone is detected, determine whether the external sabotage drone is in the intrusion warning zone. If it is, perform the first disposal (including (1) taking photos for evidence and reporting to the management platform; (2) using the loudspeaker to remind safe flight to prevent intrusion accidents). Otherwise, proceed to step S2.2:

[0183] Step S2.2: Determine whether the external sabotage drone is located in the intrusion restricted area. If so, perform the second disposal (including (1) taking photos for evidence and reporting to the management platform; (2) the platform notifies the operation and maintenance personnel to handle the situation on site; (3) using a high-power loudspeaker to deter the pilot from operating the drone to leave as soon as possible and change the route);

[0184] Step S2.3: After the first or second disposal, determine whether the intrusion has been eliminated. If so, continue to conduct real-time detection of external destructive drones through the drone detection and countermeasure device. Otherwise, perform the third disposal (upgrade the control measures, notify the relevant departments and operation and maintenance personnel to coordinate on-site disposal).

[0185] The present invention also discloses a UAV detection and countermeasure device for electric power facilities, which is applied to the UAV detection and countermeasure method for electric power facilities, and includes a main body, a panoramic camera, a monitoring camera, a microphone array (one group each at the front and the rear), a loudspeaker and a base, wherein:

[0186] The main body is provided with an edge computing terminal, a horizontal turntable, a surveillance camera turntable and a megaphone turntable, the panoramic camera is installed on the top of the main body, the surveillance camera is installed on the surveillance camera turntable, the microphone array is installed on the front and rear sides of the main body respectively, the megaphone is installed on the megaphone turntable, and the main body is installed on the base through the horizontal turntable;

[0187] The panoramic camera, the monitoring camera, the microphone array and the loudspeaker are electrically connected to the edge computing terminal respectively.

[0188] For the body:

[0189] The main part is the component that connects the other parts, and has built-in edge computing terminals, horizontal turntables, surveillance camera turntables, and megaphone turntables, and is powered by an external power supply. The edge computing terminal is connected to the panoramic camera, surveillance camera, and microphone array through cables, and can obtain video and audio signals in real time, and detect and identify drones and their location information after calculation and processing; the megaphone is connected through cables to play the expulsion audio; the horizontal turntable, surveillance camera turntable, and megaphone turntable are connected through cables to drive the surveillance camera and megaphone to rotate toward the specified direction, and provide real-time feedback on the azimuth angle;

[0190] For panoramic cameras:

[0191] The panoramic camera is composed of four lenses with 90-degree angles, and is a camera that can achieve 360-degree all-round monitoring. The panoramic camera is installed on the top of the main body through its bottom screws, and uses internal cables to connect to the edge computing terminal inside the main body to provide it with real-time 360-degree panoramic video signals;

[0192] For surveillance cameras:

[0193] The surveillance camera is equipped with a camera on the front for monitoring and recording video. The surveillance camera is connected to the surveillance camera turntable inside the main body through a rotating shaft, can rotate around the rotating shaft, and is connected to the edge computing terminal inside the main body through the cable inside the rotating shaft to provide it with real-time video signals;

[0194] For microphone arrays:

[0195] The microphone array is a device composed of multiple regularly arranged microphones and covered by a shell. It can obtain audio information through the circular hole array on the shell and transmit it to the edge computing terminal inside the main body through cables. There are two sets of microphone arrays in the present invention, which are located on the front and rear sides of the main body and fixed to the main body by internal screws;

[0196] For the Megaphone:

[0197] The megaphone is a sound amplifying device that is connected to the megaphone turntable inside the main body through a shaft and can rotate around the shaft. The megaphone is connected to the edge computing terminal inside the main body through a cable passing through the shaft, converting the sound signal into an audible high-volume sound;

[0198] For the base: The base is responsible for installing and supporting the main body. It is connected to the horizontal turntable inside the main body through a rotating shaft, which can achieve 360-degree horizontal rotation of the main body. The base can be fixed to the designated installation location, such as the top of the tower, the top of the building, etc., by bolts through the 4 through holes at the bottom.

[0199] Preferably, the workflow of the drone detection and countermeasure device for power facilities is as follows:

[0200] 1) Installation and deployment of drone detection and countermeasure devices

[0201] Deploy drone detection and countermeasure devices in power tower areas that require key prevention and control, and calibrate the installation location and coordinate information of drone detection and countermeasure devices to facilitate the identification of facilities where drones are located. Based on the video images of drone detection and countermeasure devices, calibrate drone intrusion warning areas and intrusion restricted areas, which serve as the basis for drone detection and countermeasure devices to identify dangerous behaviors of external drone sabotage.

[0202] 2) Detection of external sabotage drones

[0203] After the drone detection and countermeasure device is deployed, it enters the normal operation mode. The drone detection and countermeasure device starts visual detection of external sabotage drones as a priority detection method. At the same time, it runs sound wave (audio feature) detection of external sabotage drones, provides drone feature enhancement for visual features, reduces the false detection rate of pure visual detection, and improves the detection accuracy in low-light environments.

[0204] The drone detection and countermeasure device determines the status of external destructive drones based on the pre-calibrated drone intrusion warning area and intrusion restricted area and takes corresponding disposal measures.

[0205] 3) Countermeasures against external sabotage drones

[0206] When an external sabotage drone is detected in the intrusion warning area, the drone detection and countermeasure device will take photos for evidence and upload the information to remind staff to pay attention to the situation on the scene in time. At the same time, it will use a loudspeaker to remind the external sabotage drone to ensure that it flies outside the restricted area to prevent invasion accidents.

[0207] When an external sabotage drone is detected in an intrusion restricted area, the drone detection and countermeasure device will take photos for evidence and upload the information, notifying the operation and maintenance personnel to go to the scene for disposal. At the same time, it will use a loudspeaker to make high-power calls to deter the pilot from operating the drone to leave and change the route as soon as possible.

[0208] When it is detected that the intrusion has not been eliminated, the drone detection and countermeasure device will upgrade the control measures and notify the relevant departments and operation and maintenance personnel to jointly carry out on-site disposal.

[0209] It is worth mentioning that the technical features such as external destruction drones involved in the patent application of this invention should be regarded as prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected according to the conventional methods in the field, and should not be regarded as the inventive point of the patent of this invention. The patent of this invention will not be further elaborated.

[0210] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting and countering drones for power facilities, which detects and counters external sabotaging drones, characterized in that: The following steps are involved: Step S1: Install and deploy the drone detection and countermeasure device so that the drone detection and countermeasure device enters the detection stage of normal operation, and detects external sabotage drones in real time by combining visual detection algorithms and audio detection algorithms to determine whether they are in the intrusion area; Step S2: the drone detection and countermeasure device performs different actions according to the different intrusion areas where the external sabotage drone is located; The visual detection algorithm of step S1 is specifically implemented as follows: The target detection border regression loss function is improved. The improved loss function As shown below: ； In the formula, Indicates the overlap ratio between the predicted border and the true border, Represents the Euclidean distance between the center point of the predicted bounding box and the center point of the true bounding box, Represents the width difference between the predicted border and the true border, Represents the height difference between the predicted bounding box and the true bounding box, and Represents the width and height of the minimum bounding rectangle that contains the predicted border and the true border. and Represents the width and height of the predicted bounding box, Represents the center point of the predicted bounding box, and Indicates the width and height of the actual border, Indicates the center point of the real bounding box; Through the trained model, the panoramic image is input in real time for detection. After the target in the image is detected, the model will output the coordinates of the center of the real bounding box in the image. , after detecting the external target of destroying the drone in the panoramic image, guide the horizontal turntable and the surveillance camera turntable to rotate to the target position; take the center of the panoramic image as the coordinate origin, the image to the right is the horizontal axis square, and the image upward is the vertical axis square; Convert the detected target center pixel coordinates into the target angle of the horizontal turntable and the surveillance camera turntable, so as to control the rotation of the horizontal turntable and the surveillance camera turntable. It is necessary to calculate the azimuth angle from the plane coordinates to the spherical coordinates. and zenith angle , as shown in the formula: ； In the formula, and They represent the horizontal and vertical coordinates of the center of the detected target in the image plane coordinates, and Respectively represent the maximum effective pixel value of the panoramic image width and height; During installation, align the initial 0 angle of the horizontal turntable and the surveillance camera turntable to the center of the image. represents the absolute yaw angle target of the horizontal turntable rotation, Represents the absolute pitch angle target of the surveillance camera turntable rotation; After the horizontal turntable and the surveillance camera turntable rotate to the target area, the surveillance camera starts detection and tracks the target by itself. After detecting the target, the two-dimensional coordinates of the target in the surveillance camera image are converted into the angles for controlling the horizontal turntable and the surveillance camera turntable; Taking the coordinates of the panoramic camera as the reference coordinates, the monitoring camera image coordinates are converted into coordinates under the panoramic camera image coordinate axis reference system. After the coordinates are obtained, they are entered into the above formula to obtain the target absolute angle controlled by the monitoring camera.

2. A method for detecting and countering drones for electric power facilities according to claim 1, characterized in that: The target absolute angle of the horizontal turntable and the surveillance camera turntable is specifically implemented as follows: First, the two-dimensional coordinate points of the panoramic camera image Convert to the coordinates in the panoramic camera coordinate system , as shown in the formula: ； In the formula, is the internal parameter matrix of the panoramic camera; The camera intrinsic parameter matrix is ​​calculated as shown in the formula: ； In the formula, and is the focal length, which represents the focal length in the horizontal and vertical directions of the image, and is the optical center coordinate of the image, indicating the position of the image center; Substitute the parameters of the panoramic camera into the above formula and calculate ; Using the rotation matrix of the horizontal turntable and the surveillance camera turntable The transformation relationship between the surveillance camera coordinate system and the panoramic camera coordinate system is obtained from the following formula: ； The rotation matrix of the surveillance camera It is obtained from the rotation matrices of the pitch axis and the yaw axis, as shown below: ； In the formula, Represents the yaw angle of the horizontal turntable's current rotation. Represents the current pitch angle of the surveillance camera turntable; Two-dimensional coordinate points of surveillance camera images The coordinates in the surveillance camera coordinate system The conversion relationship is shown as follows: ； In the formula, is the internal parameter matrix of the surveillance camera; The two-dimensional coordinates of the surveillance camera image obtained by the above formula are converted into the two-dimensional coordinates of the panoramic camera image and the relationship is: ； After the two-dimensional coordinates of the detection target are input, its coordinates in the panoramic camera image coordinate system are obtained, and then Obtain the rotation angle required for control.

3. A method for detecting and countering drones for electric power facilities according to claim 2, characterized in that: The audio detection algorithm of step S1 is specifically implemented as follows: To convert the acoustic signal received by the microphone into an MFCC feature map, the signal must first be passed through a high-pass filter, as shown in the following formula: ； In the formula, Indicates the number of sampling points, represents the output of the filter, It represents the sound signal obtained by microphone sampling, and 0.97 is the pre-emphasis parameter; The design uses a window function to perform frame processing on the signal. The window function is shown in the following formula; ； In the formula, Indicates the length of each audio frame; The filtered signal With window functions Multiply to get , as shown below: ； To get Perform a Fourier transform, ; In the formula, Represents the index of the point after Fourier transformation; Then a set of triangular filters is defined, which is set up for the Mel scale, and the number of filters is defined as , the Mel frequency calculation method under Mel scale is: ； In the formula represents the frequency of the speech signal, Indicates the Mel frequency corresponding to this frequency, bringing in the maximum value of the sampling frequency The maximum Mel frequency can be obtained , change 0 to Divide it into 23 equal parts to get the median of each triangular filter, which can then be expressed by The inverse operation is to obtain the frequency of each triangular filter corresponding to the Mel value. The inverse operation is: ； The frequency response of the triangular filter is obtained as follows: ； The logarithmic energy of each filter output is then calculated as shown below: ； Finally, the final feature map is obtained through discrete cosine transform, as shown in the following formula: ； In the formula, represents the number of triangular bandpass filters and , Indicates The energy of the filter, represents the order of the required Mel cepstrum coefficient, L takes the value of 13, and finally the collected audio is sequentially brought into the 1st to 13th order Mel cepstrum to obtain a 13-dimensional Mel cepstrum feature map, that is, the final feature map size is .

4. A method for detecting and countering drones for electric power facilities according to claim 3, characterized in that: The drone detection algorithm based on the multi-source sensor combination of visual detection algorithm and audio detection algorithm is specifically implemented as follows: Based on the CNN model, multi-source data is fused and detected, and the logistic regression model is shown as follows: ； In the formula, yes A vector is used to represent the possibility of the existence of external sabotage drones. and Represent the optical image and acoustic feature map data of the drone respectively, is the bias term, and is the weight coefficient, It is the S function shown as follows: ； The logistic regression model can be rewritten as: ； In the formula, , ; Design the cost function as shown below to obtain the optimal coefficient combination : ； in, , is the regularization parameter, represents the human annotation result of the input data source, that is, the expected target, Indicates the type of data source. This formula includes two inputs: acoustic signal and optical signal. Therefore , The gradient function is as follows: ； In the formula, , is a diagonal matrix, , we can obtain the partial derivative of the logistic regression model by rewriting it: ； Substitute this formula into The gradient function is: ； Then, we use the gradient method to iterate , as shown below: ； In the formula, is the number of iterations, represents the learning rate; After inputting the collected data set into the model and training the model, a suitable , , , bring the obtained parameters into the logistic regression model to obtain the final model; after the model is trained by inputting the collected data set, the obtained model is imported into the program, and the image taken by the camera at the same time and the audio feature map collected by the microphone are synchronously input into the model for calculation each time, so as to obtain the fusion probability of whether the current external destructive drone exists in the invasion area.

5. The method for detecting and countering unmanned aerial vehicles for electric power facilities according to claim 3 is characterized in that: Step S2 is specifically implemented as follows: Step S2.1: After detecting an external sabotage drone, determine whether the external sabotage drone is located in the intrusion warning zone. If yes, perform the first disposal, otherwise proceed to step S2.2: Step S2.2: Determine whether the external sabotage drone is located in the intrusion restricted area, and if so, perform the second disposal; Step S2.3: After the first or second treatment, determine whether the intrusion is eliminated. If so, continue to perform real-time detection of external destructive drones through the drone detection and countermeasure device; otherwise, perform the third treatment.

6. A UAV detection and countermeasure device for electric power facilities, applied to a UAV detection and countermeasure method for electric power facilities as claimed in any one of claims 1 to 5, characterized in that: It includes a main body, a panoramic camera, a surveillance camera, a microphone array, a megaphone and a base, wherein: The main body is provided with an edge computing terminal, a horizontal turntable, a surveillance camera turntable and a megaphone turntable, the panoramic camera is installed on the top of the main body, the surveillance camera is installed on the surveillance camera turntable, the microphone array is installed on the front and rear sides of the main body respectively, the megaphone is installed on the megaphone turntable, and the main body is installed on the base through the horizontal turntable; The panoramic camera, the monitoring camera, the microphone array and the loudspeaker are electrically connected to the edge computing terminal respectively.

Images