A method for controlling small animals in substations based on dual-light detection

Abstract

This invention discloses a method for preventing small animals in substations based on dual-light detection. It simultaneously captures visible light and infrared video using a visible light camera and an infrared camera. An intrusion detection module marks any abnormal targets appearing in the dual-light monitoring video. Visible light AI models and infrared AI models intelligently analyze and identify small animals in the visible light and infrared video respectively. When the results analysis module determines that both the visible light and infrared AI models have detected an intrusion, an alarm module and a deportation device automatically trigger an alarm and drive the animal away, requiring no manual operation and making it convenient to use. The visible light and infrared AI models are compared and analyzed to improve detection accuracy and reduce false alarm rates. The monitoring device autonomously switches operating modes, improving accuracy while reducing energy consumption and operating costs.

Description

Technical Field

[0001] This invention relates to the technical field of substation protection methods, and in particular to a method for preventing and controlling small animals in substations based on dual-light detection. Background Technology

[0002] Substations, as intermediate stations in the power grid's transmission and distribution, play a crucial role in voltage transformation, energy transmission, and energy distribution, serving as a bridge between power plants and grid users. To maintain the stability of the power system and the reliability of equipment operation, and to prevent small animals from causing short circuits and damage to electrical appliances, small animal control measures are necessary.

[0003] Currently, the main methods for preventing small animals from entering substations include sealing holes, placing barriers, and administering medication. These methods all rely on manual labor, which is time-consuming, labor-intensive, inefficient, and has a short response time. This hinders the automated management and maintenance of substations and poses safety hazards to workers under high-voltage, ultra-high-voltage, and harsh conditions. Existing video recognition alarm technology suffers from low intrusion detection accuracy, frequently resulting in false alarms, missed alarms, and misreporting. It also has low accuracy in identifying intruders, failing to accurately identify the type of intruder. Furthermore, it operates 24 / 7, consuming significant energy and incurring high operating costs. Therefore, improvements are necessary. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method for controlling small animals in substations based on dual-light detection. This method simultaneously captures dual-light monitoring videos using visible light cameras and infrared cameras, and automatically identifies small animals using both visible light AI models and infrared AI models, thereby reducing false alarm interference and improving detection accuracy.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: a method for controlling small animals in substations based on dual-light detection, comprising the following steps:

[0006] Step 1: Install monitoring devices in each area of ​​the substation.

[0007] Each monitoring device is equipped with both a visible light camera and an infrared camera. The visible light camera and the infrared camera of the same monitoring device capture the same monitoring images. The monitoring device is equipped with an image overlay module, which combines the monitoring images captured by the visible light camera and the infrared camera of the same monitoring device into a dual-light monitoring video.

[0008] Step 2: The intrusion detection module marks any abnormal targets appearing in the dual-light surveillance video.

[0009] The monitoring device is equipped with a silent monitoring mode and a general monitoring mode. During continuous monitoring, the monitoring device switches to a power-saving mode. When the intrusion detection module detects an abnormal phenomenon, the monitoring device switches to a high-definition mode. At the same time, the intrusion detection module uses boundary-based video analysis technology to mark the target abnormal phenomenon in the dynamic scene of the dual-light monitoring video, and distinguishes the dual-light monitoring video into dual-light monitoring video with small animal intrusion and dual-light monitoring video without small animal intrusion.

[0010] Step 3: Identify the dual-light surveillance video using the dual-light AI model and output the identification information.

[0011] The dual-light AI model includes a visible light AI model and an infrared AI model. It collects visible light video and infrared video within the detection area in the dual-light monitoring video. The monitoring device extracts the monitoring images of the visible light video and infrared video at a set frame rate interval to obtain multiple visible light images and multiple infrared images. Then, it uses the visible light AI model to infer the visible light images and the infrared AI model to infer the infrared images to obtain target detection information, target type information, and target position coordinate information. The visible light bounding box B1 and the infrared prediction bounding box B2 are marked in the visible light video and infrared video, respectively.

[0012] The intrusion detection module is equipped with radar and infrared sensors. In silent monitoring mode, the monitoring device turns off the visible light camera and infrared camera and monitors the surrounding dynamic situation through the radar and infrared sensors. When the intrusion detection module detects an abnormality, the monitoring mode switches to normal monitoring mode. In normal monitoring mode, the visible light camera and infrared camera are turned on and the resolution of the visible light camera and infrared camera is adjusted to 720P~960P, and the frame rate interval is set to 3 seconds / frame.

[0013] The monitoring device is also equipped with an alarm monitoring mode and an enhanced monitoring mode. When any abnormality occurs in either the visible light video or the infrared video, the monitoring device switches to the alarm monitoring mode, and the resolution of the visible light camera and the infrared camera is adjusted to 1080P~4K, with the frame rate interval set to 1 second / frame. When abnormalities occur simultaneously in both the visible light video and the infrared video, the monitoring device switches to the enhanced monitoring mode, and the resolution of the visible light camera and the infrared camera is adjusted to 1080P~4K, with the frame rate interval set to 0.2 seconds / frame~0.5 seconds / frame.

[0014] The monitoring device is also equipped with a pan-tilt unit (PTZ). Visible light cameras and infrared cameras are mounted on the PTZ. After the visible light frame B1 and infrared prediction frame B2 are marked by the dual-light AI model, the PTZ drives the visible light camera and infrared camera to rotate so that the visible light frame B1 and infrared prediction frame B2 are respectively kept in the center of the visible light video and infrared video monitoring screen. The PTZ speed is set to high speed, medium speed and low speed. In general monitoring mode, the PTZ speed is set to low speed. In alert monitoring mode, the PTZ speed is set to medium speed. In enhanced monitoring mode, the PTZ speed is set to high speed.

[0015] The visible light camera and the infrared camera are each equipped with an optical zoom mechanism. The monitoring device is equipped with a synchronous zoom module. The synchronous zoom module synchronously controls the optical zoom mechanisms of the visible light camera and the infrared camera to zoom synchronously. When the area of ​​the visible light frame B1 or the infrared prediction frame B2 accounts for less than 20% of the visible light video or the infrared video, the synchronous zoom module is activated to magnify the dual-light monitoring video so that the area of ​​the visible light frame B1 or the infrared prediction frame B2 accounts for between 40% and 80% of the visible light video or the infrared video.

[0016] Step 4: Output intrusion information through the result analysis module.

[0017] The target detection information, target type information, and target position coordinate information inferred by the visible light AI model are compared with the target detection information, target type information, and target position coordinate information inferred by the infrared AI model, respectively. The overlap rate D of the visible light bounding box B1 and the infrared prediction bounding box B2 is calculated. When the target detection information and target type information inferred by the visible light AI model are the same as those inferred by the infrared AI model and the overlap rate D is greater than 0.5, an alarm message is output to the platform alarm module.

[0018] Step 5, platform alarm

[0019] After receiving the alarm information, the alarm module records the target type information and target location coordinate information inferred by the visible light AI model and the infrared AI model, uploads them to the host platform and sends an alarm. At the same time, it controls the visible light camera and the infrared camera to capture images, record the intrusion video images at the time of the intrusion and store them in the storage unit.

[0020] Step 6, Actively drive away

[0021] After receiving the alarm information, the alarm module activates the decoy device to drive away the small animals.

[0022] In a further technical solution, in step 4, the formula for calculating the overlap rate D is D=(B1∩B2) / (B1∪B2).

[0023] In a further technical solution, step 3 also includes the following sub-steps:

[0024] Step 31, Data Collection and Labeling Steps

[0025] Multiple visible light and infrared images of objects with small animals intruding were collected in various scenarios as positive samples, and multiple visible light and infrared images of objects without small animals intruding were collected in various scenarios as negative samples, with a positive-to-negative sample ratio of 1:1. At least 2000 images of each small animal target were collected, and the number of samples of different types was kept balanced. Visible light and infrared datasets were then established.

[0026] The visible light dataset is divided into a visible light training set, a visible light validation set, and a visible light test set in a 10:1:1 ratio.

[0027] The infrared dataset is divided into an infrared training set, an infrared validation set, and an infrared test set in a 10:1:1 ratio.

[0028] Positive samples in the visible light and infrared datasets were labeled using the labelImg local annotation tool. Visible light bounding boxes (B1) were labeled around small animals in the visible light images, and infrared predicted bounding boxes (B2) were labeled around small animals in the infrared images. The label types for the visible light bounding boxes B1 and the infrared predicted bounding boxes B2 were set, generating multiple XML-formatted annotation files. Each annotation file corresponds to the name of the visible light image and the infrared image, and each annotation file includes image information, target detection information, target type information, and target location coordinates for the corresponding visible light or infrared image.

[0029] Negative samples do not contain small animal targets and do not require annotation. The corresponding XML files are automatically generated in batches using Python code. The XML files only contain the corresponding image information and do not contain visible light bounding box B1 or infrared prediction bounding box B2.

[0030] Step 32, Data Augmentation and Extraction Steps

[0031] The visible light and infrared datasets are augmented synchronously with the annotation files using visible light and infrared images, enabling the annotation files to automatically match the augmented visible light and infrared images.

[0032] Step 33, Model building and training steps.

[0033] A network structure was built within the Darknet open-source framework to extract feature information from visible light and infrared images for model learning. The basic convolutional framework consists of 1x1 and 3x3 convolutional kernels. Low-level, intermediate-level, and high-level feature fusion routing layers were added to expand the receptive field of the dual-light AI model. Pooling layers reduced the spatial size of the data volume, decreased the number of parameters in the network, reduced computational resource consumption, and controlled overfitting.

[0034] Anchor boxes were generated using the kmenes++ clustering method. The aspect ratios of the visible light bounding box B1 and the infrared prediction bounding box B2 suitable for small animals were determined. The nine anchor boxes identified were (9,15), (24,18), (15,32), (44,29), (27,49), (52,59), (76,111), (137,66), and (168,150).

[0035] The training parameters were set as follows: network input was 320, batch size was 128, base learning rate was 0.002, maximum number of iterations was 500,000, learning rate decayed using step method with decay weight of 0.0005, momentum was 0.9, and multi-scale training was enabled on GPU.

[0036] Step 34, Model Testing and Deployment Steps

[0037] Observe the loss curve and mAP curve output during the training process. Stop training after the curves stabilize. Select the weight file of the dual-light AI model with the best performance based on mAP and perform inference tests on the visible light test set and the infrared test set. Output parameters such as mAP, precision, recall, and F1-score of the visible light test set and the infrared test set. Plot the confusion matrix and check the false positives and false negatives of the dual-light AI model. Finally, perform dual-light AI model conversion and deployment.

[0038] In a further technical solution, a verification step is included between steps 31 and 32.

[0039] Each visible light and infrared image of the positive sample is checked manually or through a verification procedure to ensure it corresponds to a specific annotation file.

[0040] Each visible light and infrared image of the negative samples is checked manually or through a testing procedure to ensure that each corresponds to an XML file.

[0041] If the annotation file or XML file does not correspond to the visible light image or infrared image, delete the annotation file or XML file, and repeat step 31 for the incorrect annotation file;

[0042] The image information in the annotation file is compared manually or through a verification procedure to see if it matches the image information in the corresponding visible light and infrared images.

[0043] The target detection information in the annotation file is compared manually or through a review procedure to determine if it matches the target detection information in the corresponding visible light and infrared images.

[0044] The target type information in the annotation file is compared manually or through a review procedure to determine if it matches the target type information in the corresponding visible light and infrared images.

[0045] The target location coordinates in the annotation file are compared manually or through a verification procedure to determine if they match the target location coordinates in the corresponding visible light and infrared images.

[0046] If any discrepancy is found, delete the erroneous annotation file and repeat step 31 for the erroneous annotation file.

[0047] In a further technical solution, in step 31, visible light images and infrared images containing erroneous information are manually added to the negative samples; or when no intrusion is found in the intrusion video image, multiple interfering images are obtained by extracting frames from the intrusion video image and the interfering images are added to the negative samples.

[0048] In a further technical solution, in step 32, portions of the visible light dataset and the infrared light dataset are extracted and normalized to a txt file by defining a coordinate transformation function. The new coordinate information in the txt file is the same as the target position coordinate information in the corresponding visible light image and infrared light image, respectively.

[0049] In a further technical solution, in step 34, after the dual-light AI model passes the test, it undergoes sparse training, channel and layer pruning, and quantization fine-tuning before conversion and deployment.

[0050] In a further technical solution, in step 2, the intrusion detection module is set with intrusion sensitivity, and different values ​​of intrusion sensitivity are set for different detection areas according to the security level of the substation protection area.

[0051] In a further technical solution, the monitoring device is also equipped with a pan-tilt unit, which includes an upper housing, a lower housing, a drive mechanism, and a circuit board. The upper housing is horizontally rotated and installed on the upper part of the lower housing. The drive mechanism and the circuit board are respectively installed in the inner cavity of the lower housing. The visible light camera and the infrared camera are respectively rotated and installed on the left and right sides of the upper housing. The drive mechanism is connected to the upper housing, the visible light camera, and the infrared camera respectively.

[0052] The circuit board is equipped with an AI chip, which stores a program for implementing a substation small animal control method based on dual-light detection according to any one of claims 1 to 7. Videos from the visible light camera and the infrared camera are sent to the AI ​​chip.

[0053] The AI ​​chip uses its built-in intrusion detection module to capture dual-light surveillance video of both instances where small animals have intruded and instances where small animals have not.

[0054] The AI ​​chip outputs intrusion information through its built-in result analysis module.

[0055] The AI ​​chip sends target type information, target location coordinates, and alarms to the host platform through its built-in alarm module.

[0056] In a further technical solution, the de-escalation device includes an ultrasonic de-escalation mechanism and an audible and visual alarm mechanism. The ultrasonic de-escalation mechanism and the audible and visual alarm mechanism are respectively installed on the monitoring device and are electrically connected to the AI ​​chip. When the AI ​​chip analyzes that a small animal has invaded the detection area, the AI ​​chip sends a de-escalation signal to the ultrasonic de-escalation mechanism and the audible and visual alarm mechanism respectively.

[0057] The advantages of this invention compared to existing technologies, based on the above structure, are as follows: Visible light and infrared video are captured simultaneously by a visible light camera and an infrared camera; the visible light AI model and infrared AI model respectively perform intelligent analysis and identification of small animals in the visible light and infrared video; when the result analysis module determines that both the visible light AI model and infrared AI model have intruded, the alarm module and repelling device automatically trigger an alarm and drive away the animal, eliminating the need for manual operation and making it convenient to use; the visible light AI model and infrared AI model are compared and analyzed to improve detection accuracy and reduce false alarm rate; the monitoring device autonomously switches working modes, improving accuracy while reducing energy consumption and operating costs. Attached Figure Description

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] Figure 1 This is a flowchart of the process of this invention;

[0060] Figure 2 This is a network structure diagram of the present invention for extracting features of small animals;

[0061] Figure 3 This is a graph showing the loss versus mAP curves during the training process in step 33 of this invention.

[0062] Figure 4 This is a schematic diagram of the digital module for the overlap rate D of the present invention;

[0063] Figure 5 These are comparative diagrams of the present invention and the prior art;

[0064] Figure 6 This is a flowchart of the working mode switching process of the present invention. Detailed Implementation

[0065] The following are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention.

[0066] A method for controlling small animals in substations based on dual-light detection, such as... Figures 1 to 4 As shown, it includes the following steps:

[0067] Step 1: Install monitoring devices in each area of ​​the substation.

[0068] Each monitoring device is equipped with both a visible light camera and an infrared camera. The visible light camera and the infrared camera of the same monitoring device capture the same monitoring images. The monitoring device is equipped with an image overlay module, which combines the monitoring images captured by the visible light camera and the infrared camera of the same monitoring device into a dual-light monitoring video.

[0069] Step 2: The intrusion detection module marks any abnormal targets appearing in the dual-light surveillance video.

[0070] The monitoring device is equipped with a silent monitoring mode and a general monitoring mode. During continuous monitoring, the device switches to a power-saving mode. When the intrusion detection module detects an anomaly, the device switches to a high-definition mode. Simultaneously, the intrusion detection module uses boundary-based video analysis technology to mark target anomalies in the dynamic scene of the dual-light monitoring video, distinguishing between dual-light monitoring videos with and without small animal intrusions. Specifically, the intrusion detection module is configured with intrusion sensitivity, setting different intrusion sensitivity values ​​for different detection areas based on the security level of the substation's protected area.

[0071] Step 3 also includes the following sub-steps,

[0072] Step 31, Data Acquisition and Labeling: Collect multiple visible light and infrared images of objects with small animal intrusions in various scenarios as positive samples. Collect multiple visible light and infrared images of objects without small animal intrusions in various scenarios as negative samples. The ratio of positive to negative samples is 1:1. The number of images collected for each small animal target should be no less than 2000, and the number of samples of different types should be balanced. Establish visible light and infrared datasets.

[0073] The visible light dataset is divided into a visible light training set, a visible light validation set, and a visible light test set in a 10:1:1 ratio.

[0074] The infrared dataset is divided into an infrared training set, an infrared validation set, and an infrared test set in a 10:1:1 ratio.

[0075] Positive samples in the visible light and infrared datasets were labeled using the labelImg local annotation tool. Visible light bounding boxes (B1) were labeled around small animals in the visible light images, and infrared predicted bounding boxes (B2) were labeled around small animals in the infrared images. The label types for the visible light bounding boxes B1 and the infrared predicted bounding boxes B2 were set, generating multiple XML-formatted annotation files. Each annotation file corresponds to the name of the visible light image and the infrared image, and each annotation file includes image information, target detection information, target type information, and target location coordinates for the corresponding visible light or infrared image.

[0076] Negative samples do not contain small animal targets and do not require annotation. The corresponding XML files are automatically generated in batches using Python code. The XML files only contain the corresponding image information and do not contain visible light bounding box B1 or infrared prediction bounding box B2.

[0077] Specifically, the intrusion detection module is equipped with radar and infrared sensors. In silent monitoring mode, the monitoring device shuts down the visible light and infrared cameras, relying solely on the radar and infrared sensors to monitor the surrounding dynamics. When the intrusion detection module detects an anomaly, the monitoring mode switches to normal monitoring mode. In normal monitoring mode, the visible light and infrared cameras are activated, and their resolution is adjusted to 720P-960P with a frame rate interval of 3 seconds per frame. In silent monitoring mode, the visible light and infrared cameras are shut down, relying solely on the radar and infrared sensors to monitor for dynamic situations or temperature anomalies. Once an anomaly is detected, the visible light and infrared cameras are activated again to capture images. The cameras are set to a lower resolution, with both the visible light and infrared cameras operating at low power. When any abnormality occurs in either the visible light or infrared video, the monitoring device switches to alert monitoring mode, adjusting the resolution of the visible light and infrared cameras to 1080P–4K and operating at high power to capture clearer images. The frame rate interval is set to 1 second / frame. When abnormalities occur simultaneously in both the visible light and infrared video, the monitoring device switches to enhanced monitoring mode, adjusting the resolution of the visible light and infrared cameras to 1080P–4K and setting the frame rate interval to 0.2–0.5 seconds / frame. In enhanced monitoring mode, the frame rate interval is shortened, increasing the number of visible light and infrared images captured and improving the accuracy of judgment. Figure 5As shown, by automatically switching working modes, the intrusion judgment accuracy is as high as 100%, the intrusion object recognition accuracy is as high as 100%, and the intrusion alarm accuracy is as high as 100%. By automatically tracking and zooming in on the image and autonomously changing the resolution, it provides the dual-light AI model with high-definition visible light video and infrared video, which facilitates the dual-light AI model to perform analysis and judgment, thereby improving the accuracy.

[0078] The monitoring device is also equipped with a pan-tilt unit (PTZ). Visible light and infrared cameras are mounted on the PTZ. After the dual-light AI model marks the visible light frame B1 and the infrared prediction frame B2, the PTZ rotates the visible light and infrared cameras to keep the visible light frame B1 and the infrared prediction frame B2 centered in the visible light and infrared video monitoring images, respectively. The PTZ rotation speed is set to high, medium, and low speeds. In general monitoring mode, the PTZ rotation speed is set to low; in alert monitoring mode, it is set to medium; and in enhanced monitoring mode, it is set to high. Each visible light and infrared camera is equipped with an optical zoom mechanism, and the monitoring device includes a synchronous zoom module. The synchronous zoom module synchronizes the optical zoom mechanisms of the visible light camera and the infrared camera. When the area of ​​the visible light frame B1 or the infrared prediction frame B2 accounts for less than 20% of the visible light video or infrared video, the synchronous zoom module is activated to magnify the dual-light monitoring video, ensuring that the area of ​​the visible light frame B1 or the infrared prediction frame B2 accounts for 40% to 80% of the visible light video or infrared video. The pan-tilt unit (PTZ) works in conjunction with the synchronous zoom module to track small animals. The PTG automatically adjusts its power according to the monitoring mode, switching between high, medium, and low speed settings to further reduce energy consumption. The synchronous zoom module simultaneously zooms the optical zoom mechanisms of both the visible light camera and the infrared camera, achieving optical zoom, magnifying the image, and capturing clear images of small animals. Figure 6 As shown, the monitoring device autonomously switches between silent monitoring mode, general monitoring mode, alert monitoring mode, and enhanced monitoring mode to automatically adjust the resolution of visible light and infrared cameras, the rotation speed of the pan-tilt unit, and the frequency of intrusion video images, thereby reducing energy consumption. Figure 5 As shown, the present invention consumes only 0.2 kWh in 24 hours, which improves the accuracy of intrusion judgment, identification and alarm while reducing energy consumption and reducing the cost of use.

[0079] Specifically, in step 31, visible light and infrared images containing erroneous information are manually added to the negative samples; or, when no intrusion is found in the intrusion video image, multiple interfering images are obtained by extracting frames from the intrusion video image, and these interfering images are added to the negative samples. Erroneous information refers to interference items that the dual-light AI model still identifies as intrusions even when no small animals have intruded, such as harmless moving objects like tree branches swaying in the wind or pedestrians passing by. By labeling the visible light and infrared images containing erroneous information, the dual-light AI model intelligently learns the interference items, gradually improving the detection accuracy during use. The detection accuracy is directly proportional to the usage time.

[0080] Specifically, a verification step is included after step 31.

[0081] Each visible light and infrared image of the positive sample is checked manually or through a verification procedure to ensure it corresponds to a specific annotation file.

[0082] Each visible light and infrared image of the negative samples is checked manually or through a testing procedure to ensure that each corresponds to an XML file.

[0083] If the annotation file or XML file does not correspond to the visible light image or infrared image, delete the annotation file or XML file, and repeat step 31 for the incorrect annotation file;

[0084] The image information in the annotation file is compared manually or through a verification procedure to see if it matches the image information in the corresponding visible light and infrared images.

[0085] The target detection information in the annotation file is compared manually or through a review procedure to determine if it matches the target detection information in the corresponding visible light and infrared images.

[0086] The target type information in the annotation file is compared manually or through a review procedure to determine if it matches the target type information in the corresponding visible light and infrared images.

[0087] The target location coordinates in the annotation file are compared manually or through a verification procedure to determine if they match the target location coordinates in the corresponding visible light and infrared images.

[0088] If any discrepancy is found, delete the erroneous annotation file and repeat step 31 for the erroneous annotation file.

[0089] Step 32, Data Augmentation and Extraction Steps

[0090] The visible light and infrared datasets are augmented simultaneously with the annotation files using visible light and infrared images. This allows the annotation files to automatically match the augmented visible light and infrared images, avoiding the time-consuming and laborious manual re-annotation and improving annotation efficiency and recognition speed. The simultaneous augmentation methods include simple methods such as changing the saturation, exposure, hue, direction, and angle of the visible light and infrared images, as well as operations such as cropping, flipping, and stitching to change the position and size of the small animal targets, thus significantly improving the diversity of the dataset.

[0091] Specifically, in step 32, portions of the visible light dataset and the infrared light dataset are extracted and normalized to new coordinate information in a txt file by defining a coordinate transformation function. The new coordinate information in the txt file is the same as the target position coordinate information in the corresponding visible light image and infrared light image, respectively.

[0092] Step 33, Model building and training steps.

[0093] A network structure is built within the Darknet open-source framework to extract feature information from visible light and infrared images for model learning. The basic convolutional framework of the network structure is composed of 1*1 and 3*3 convolutional kernels. Low-level, intermediate-level, and high-level feature fusion route layers are added to expand the receptive field of the dual-light AI model. Pool layers reduce the spatial size of the data volume, reduce the number of parameters in the network, reduce computational resource consumption, and control overfitting. The units of parameters in the convolutional framework, route layers, pool layers, anchor boxes, and model input settings in this invention are all default units.

[0094] Anchor boxes were generated using the kmenes++ clustering method. The aspect ratios of the visible light bounding box B1 and the infrared prediction bounding box B2 suitable for small animals were determined. The nine anchor boxes identified were (9,15), (24,18), (15,32), (44,29), (27,49), (52,59), (76,111), (137,66), and (168,150).

[0095] The training parameters were set as follows: network input was 320, batch size was 128, base learning rate was 0.002, maximum number of iterations was 500,000, learning rate decayed using step method with decay weight of 0.0005, momentum was 0.9, and multi-scale training was enabled on GPU.

[0096] The anchor boxes were generated using the kmeans++ clustering method, which optimized the initial cluster center selection method and achieved better clustering results.

[0097] Step 34, Model Testing and Deployment Steps

[0098] Observe the loss curve and mAP curve output during the training process. Stop training after the curves stabilize. Select the weight file of the dual-light AI model with the best performance based on mAP and perform inference tests on the visible light test set and the infrared test set. Output parameters such as mAP, precision, recall, and F1-score of the visible light test set and the infrared test set. Plot the confusion matrix and check the false positives and false negatives of the dual-light AI model. Finally, perform dual-light AI model conversion and deployment.

[0099] Specifically, in step 34, after the dual-light AI model passes the test, it undergoes sparse training, channel and layer pruning, and quantization fine-tuning before conversion and deployment. Channel and layer pruning and quantization fine-tuning support quantizing floating-point models into fixed-point models, reducing the memory footprint of the dual-light AI model and improving detection speed. The dual-light AI model uses an optimized high-performance neural network forward computation framework to achieve cross-platform model conversion and deployment, expanding its applicability and lowering the barrier to entry.

[0100] Step 4: Output intrusion information through the result analysis module.

[0101] The target detection information, target type information, and target position coordinate information inferred by the visible light AI model are compared with the target detection information, target type information, and target position coordinate information inferred by the infrared AI model, respectively. The overlap rate D of the visible light bounding box B1 and the infrared prediction bounding box B2 is calculated. When the target detection information and target type information inferred by the visible light AI model are the same as those inferred by the infrared AI model and the overlap rate D is greater than 0.5, an alarm message is output to the platform alarm module. Specifically, the formula for calculating the overlap rate D is D=(B1∩B2) / (B1∪B2).

[0102] Step 5, the platform issues an alarm.

[0103] After receiving the alarm information, the alarm module records the target type information and target location coordinate information inferred by the visible light AI model and the infrared AI model, uploads them to the host platform and sends an alarm. At the same time, it controls the visible light camera and the infrared camera to capture images, record the intrusion video images at the time of the intrusion and store them in the storage unit.

[0104] Step 6, actively drive them away.

[0105] After receiving the alarm information, the alarm module activates the decoy device to drive away the small animals.

[0106] Specifically, the monitoring device is also equipped with a pan-tilt unit, which includes an upper housing, a lower housing, a drive mechanism, and a circuit board. The upper housing is horizontally rotated and installed on the upper part of the lower housing. The drive mechanism and the circuit board are respectively installed in the inner cavity of the lower housing. The visible light camera and the infrared camera are respectively rotated and installed on the left and right sides of the upper housing. The drive mechanism is connected to the upper housing, the visible light camera, and the infrared camera respectively.

[0107] The circuit board is equipped with an AI chip, which stores a program for implementing a substation small animal control method based on dual-light detection according to any one of claims 1 to 7. Videos from the visible light camera and the infrared camera are sent to the AI ​​chip.

[0108] The AI ​​chip uses its built-in intrusion detection module to capture dual-light surveillance video of both instances where small animals have intruded and instances where small animals have not.

[0109] The AI ​​chip outputs intrusion information through its built-in result analysis module.

[0110] The AI ​​chip sends target type information, target location coordinates, and alarms to the host platform through its built-in alarm module.

[0111] Specifically, the repelling device includes an ultrasonic repelling mechanism and an audible and visual alarm mechanism. These mechanisms are installed on the monitoring device and electrically connected to an AI chip. When the AI ​​chip detects a small animal intruding into the detection area, it sends repelling signals to both the ultrasonic and audible / visual alarm mechanisms. The ultrasonic repelling mechanism emits ultrasonic waves that are sensitive to the nervous and physiological systems of small animals. Upon hearing these waves, the animals experience physiological disturbances and develop an aversion, thus avoiding the area covered by the ultrasonic waves and achieving the purpose of physical repelling. This process requires no human intervention, is completely free of chemical drugs, and is therefore non-toxic, odorless, and does not cause secondary pollution. The audible and visual alarm mechanism emits alarm sounds and warning lights, working in conjunction with the ultrasonic waves to repel the small animals. Since the frequency of the ultrasonic waves used to repel animals is generally above 20kHz, while the human hearing range is 16k-20kHz, security personnel stationed nearby cannot detect an intrusion using only the ultrasonic repelling mechanism. Therefore, the audible and visual alarm mechanism alerts nearby security personnel with its alarm sounds and warning lights.

[0112] 1. In this invention, a combination of positive and negative samples in the dataset is used, which reduces the false alarm rate and improves the model's anti-interference ability;

[0113] 2. In this invention, multiple synchronous data augmentation methods using images and XML are employed, which improves the richness and diversity of the dataset while reducing the burden of manual annotation.

[0114] 3. In this invention, Keams++ was used to generate the anchor box for the small animal, which has high adaptability and helps to improve the accuracy of the visible light bounding box B1 and the infrared prediction bounding box B2.

[0115] 4. In this invention, the model uses low-level, mid-level, and high-level feature fusion algorithms, which have strong network feature extraction capabilities and the ability to detect both small and large animals simultaneously.

[0116] 5. In this invention, a dual-light AI model consisting of a visible light AI model and an infrared AI model is used for detection. This model can automatically identify and drive away small animals, and it has high accuracy and good practicality.

[0117] The above description is only a preferred embodiment of the present invention. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the present invention. The content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for controlling small animals in substations based on dual-light detection, characterized in that: Includes the following steps, Step 1: Install monitoring devices in each area of ​​the substation. Each monitoring device is equipped with both a visible light camera and an infrared camera. The visible light camera and the infrared camera of the same monitoring device capture the same monitoring images. The monitoring device is equipped with an image overlay module, which combines the monitoring images captured by the visible light camera and the infrared camera of the same monitoring device into a dual-light monitoring video. Step 2: Mark any abnormal targets appearing in the dual-light surveillance video using the intrusion detection module. The monitoring device is equipped with a silent monitoring mode and a general monitoring mode. During continuous monitoring, the monitoring device switches to a power-saving mode. When the intrusion detection module detects an abnormal phenomenon, the monitoring device switches to a high-definition mode. At the same time, the intrusion detection module uses boundary-based video analysis technology to mark the target abnormal phenomenon in the dynamic scene of the dual-light monitoring video, and distinguishes the dual-light monitoring video into dual-light monitoring video with small animal intrusion and dual-light monitoring video without small animal intrusion. Step 3: Identify the dual-light surveillance video using the dual-light AI model and output the identification information. The dual-light AI model includes a visible light AI model and an infrared AI model. It collects visible light video and infrared video within the detection area in the dual-light monitoring video. The monitoring device extracts the monitoring images of the visible light video and infrared video at a set frame rate interval to obtain multiple visible light images and multiple infrared images. Then, it uses the visible light AI model to infer the visible light images and the infrared AI model to infer the infrared images to obtain target detection information, target type information, and target position coordinate information. The visible light bounding box B1 and the infrared prediction bounding box B2 are marked in the visible light video and infrared video, respectively. The intrusion detection module is equipped with radar and infrared sensors. In silent monitoring mode, the monitoring device turns off the visible light camera and infrared camera and monitors the surrounding dynamic situation through the radar and infrared sensors. When the intrusion detection module detects an abnormality, the monitoring mode switches to normal monitoring mode. In normal monitoring mode, the visible light camera and infrared camera are turned on and the resolution of the visible light camera and infrared camera is adjusted to 720P~960P, and the frame rate interval is set to 3 seconds / frame. The monitoring device is also equipped with an alarm monitoring mode and an enhanced monitoring mode. When any abnormality occurs in either the visible light video or the infrared video, the monitoring device switches to the alarm monitoring mode, and the resolution of the visible light camera and the infrared camera is adjusted to 1080P~4K, with the frame rate interval set to 1 second / frame. When abnormalities occur simultaneously in both the visible light video and the infrared video, the monitoring device switches to the enhanced monitoring mode, and the resolution of the visible light camera and the infrared camera is adjusted to 1080P~4K, with the frame rate interval set to 0.2 seconds / frame~0.5 seconds / frame. The monitoring device is also equipped with a pan-tilt unit (PTZ). Visible light cameras and infrared cameras are mounted on the PTZ. After the visible light frame B1 and infrared prediction frame B2 are marked by the dual-light AI model, the PTZ drives the visible light camera and infrared camera to rotate so that the visible light frame B1 and infrared prediction frame B2 are respectively kept in the center of the visible light video and infrared video monitoring screen. The PTZ speed is set to high speed, medium speed and low speed. In general monitoring mode, the PTZ speed is set to low speed. In alert monitoring mode, the PTZ speed is set to medium speed. In enhanced monitoring mode, the PTZ speed is set to high speed. The visible light camera and the infrared camera are each equipped with an optical zoom mechanism. The monitoring device is equipped with a synchronous zoom module. The synchronous zoom module synchronously controls the optical zoom mechanisms of the visible light camera and the infrared camera to zoom synchronously. When the area of ​​the visible light frame B1 or the infrared prediction frame B2 accounts for less than 20% of the visible light video or the infrared video, the synchronous zoom module is activated to magnify the dual-light monitoring video so that the area of ​​the visible light frame B1 or the infrared prediction frame B2 accounts for between 40% and 80% of the visible light video or the infrared video. Step 4: Output intrusion information through the result analysis module. The target detection information, target type information, and target position coordinate information inferred by the visible light AI model are compared with the target detection information, target type information, and target position coordinate information inferred by the infrared AI model, respectively. The overlap rate D of the visible light bounding box B1 and the infrared prediction bounding box B2 is calculated. When the target detection information and target type information inferred by the visible light AI model are the same as those inferred by the infrared AI model and the overlap rate D is greater than 0.5, an alarm message is output to the platform alarm module. Step 5, platform alarm After receiving the alarm information, the alarm module records the target type information and target location coordinate information inferred by the visible light AI model and the infrared AI model, uploads them to the host platform and sends an alarm. At the same time, it controls the visible light camera and the infrared camera to capture images, record the intrusion video images at the time of the intrusion and store them in the storage unit. Step 6, Actively drive away After receiving the alarm information, the alarm module activates the decoy device to drive away the small animals.

2. The method for controlling small animals in substations based on dual-light detection according to claim 1, characterized in that: In step 4, the overlap rate D is calculated using the formula D = (B1∩B2) / (B1∪B2).

3. The method for controlling small animals in substations based on dual-light detection according to claim 2, characterized in that: Step 3 also includes the following sub-steps. Step 31, Data Acquisition and Labeling Steps Multiple visible light images and multiple infrared images of objects with small animals intruding in various scenarios are collected as positive samples, and multiple visible light images and multiple infrared images of objects without small animals intruding in various scenarios are collected as negative samples. The ratio of positive to negative samples is 1:

1. The number of images of a single target, small animals, is no less than 2000, and the number of samples of different types is kept balanced. Visible light dataset and infrared dataset are established. The visible light dataset is divided into a visible light training set, a visible light validation set, and a visible light test set in a 10:1:1 ratio. The infrared dataset is divided into an infrared training set, an infrared validation set, and an infrared test set in a 10:1:1 ratio. Positive samples in the visible light and infrared datasets were labeled using the labelImg local annotation tool. Visible light bounding boxes (B1) were labeled around small animals in the visible light images, and infrared predicted bounding boxes (B2) were labeled around small animals in the infrared images. Label types were set for the visible light bounding boxes B1 and infrared predicted bounding boxes B2, generating multiple XML-formatted annotation files. Each annotation file corresponds to the name of the visible light and infrared images, and includes image information for the corresponding visible light or infrared image, target detection information, target type information, and target location coordinates. Negative samples do not contain small animal targets and do not require annotation. The corresponding XML files are automatically generated in batches using Python code. The XML files only contain the corresponding image information and do not contain visible light bounding box B1 or infrared prediction bounding box B2. Step 32, Data Augmentation and Extraction Steps The visible light and infrared datasets are augmented synchronously with the annotation files using visible light and infrared images, enabling the annotation files to automatically match the augmented visible light and infrared images. Step 33, Model building and training steps. A network structure was built within the Darknet open-source framework to extract feature information from visible light and infrared images for model learning. The basic convolutional framework consists of 1x1 and 3x3 convolutional kernels. Low-level, intermediate-level, and high-level feature fusion routing layers were added to expand the receptive field of the dual-light AI model. Pooling layers reduced the spatial size of the data volume, decreased the number of parameters in the network, reduced computational resource consumption, and controlled overfitting. Anchor boxes were generated using the kmenes++ clustering method. The aspect ratios of the visible light bounding box B1 and the infrared prediction bounding box B2 suitable for small animals were determined. The nine anchor boxes identified were (9,15), (24,18), (15,32), (44,29), (27,49), (52,59), (76,111), (137,66), and (168,150). The training parameters were set as follows: network input was 320, batch size was 128, base learning rate was 0.002, maximum number of iterations was 500,000, learning rate decayed using step method with decay weight of 0.0005, momentum was 0.9, and multi-scale training was enabled on GPU. Step 34, Model Testing and Deployment Steps Observe the loss curve and mAP curve output during the training process. Stop training after the curves stabilize. Select the weight file of the dual-light AI model with the best performance based on mAP and perform inference tests on the visible light test set and the infrared test set. Output parameters such as mAP, precision, recall, and F1-score of the visible light test set and the infrared test set. Plot the confusion matrix and check the false positives and false negatives of the dual-light AI model. Finally, perform dual-light AI model conversion and deployment.

4. The method for controlling small animals in substations based on dual-light detection according to claim 3, characterized in that: A verification step is also included between step 31 and step 32. The visible light image and the infrared image of the positive sample are checked manually or by a verification procedure to see if they correspond to a specific annotation file. The presence or infrared images of each negative sample are checked manually or by a testing procedure to determine if they correspond to a single XML file. If the annotation file or XML file does not correspond to the visible light image or infrared image, delete the annotation file or XML file, and repeat step 31 for the incorrect annotation file; The image information in the annotation file is compared manually or through a checking procedure to determine if it is identical to the image information in the corresponding visible light and infrared images. The target detection information in the annotation file is compared manually or through a review procedure to determine if it matches the target detection information in the corresponding visible light and infrared images. The target type information in the annotation file is compared manually or through an inspection procedure to determine if it matches the target type information in the corresponding visible light and infrared images. The target location coordinates in the annotation file are compared manually or through a checking procedure to see if they match the target location coordinates in the corresponding visible light and infrared images. If any discrepancy is found, delete the erroneous annotation file and repeat step 31 for the erroneous annotation file.

5. A method for controlling small animals in substations based on dual-light detection according to claim 4, characterized in that: In step 31, the visible light image and the infrared image containing the erroneous information are manually added to the negative sample; or when it is found manually that there is no intrusion in the intrusion video image, multiple interference images are obtained by extracting frames from the intrusion video image and the interference images are added to the negative sample.

6. A method for controlling small animals in substations based on dual-light detection according to claim 3, characterized in that: In step 32, portions of the visible light dataset and the infrared light dataset are extracted and normalized to new coordinate information in a txt file by defining a coordinate transformation function. The new coordinate information in the txt file is the same as the target position coordinate information in the corresponding visible light image and infrared light image.

7. A method for controlling small animals in substations based on dual-light detection according to claim 3, characterized in that: In step 34, after the dual-light AI model passes the test, it undergoes sparse training, channel and layer pruning, and quantization fine-tuning before conversion and deployment.

8. A method for controlling small animals in substations based on dual-light detection according to claim 1, characterized in that: In step 2, the intrusion detection module is set with intrusion sensitivity, and different values ​​of intrusion sensitivity are set for different detection areas according to the security level of the substation protection area.

9. A method for controlling small animals in substations based on dual-light detection according to any one of claims 1 to 8, characterized in that: The monitoring device is also equipped with a pan-tilt unit, which includes an upper housing, a lower housing, a drive mechanism, and a circuit board. The upper housing is horizontally rotated and mounted on the upper part of the lower housing. The drive mechanism and the circuit board are respectively installed in the inner cavity of the lower housing. The visible light camera and the infrared camera are respectively rotated and mounted on the left and right sides of the upper housing. The drive mechanism is connected to the upper housing, the visible light camera, and the infrared camera respectively. The circuit board is equipped with an AI chip, which stores a program for implementing the substation small animal control method based on dual-light detection as described in any one of claims 1 to 7. Videos from the visible light camera and the infrared camera are sent to the AI ​​chip. The AI ​​chip uses its built-in intrusion detection module to capture both dual-light surveillance video showing an intrusion by a small animal and dual-light surveillance video showing no intrusion by a small animal. The AI ​​chip outputs the intrusion information through its built-in result analysis module. The AI ​​chip sends the target type information, target location coordinates, and alarm to the host platform through its built-in alarm module.

10. A method for controlling small animals in substations based on dual-light detection according to claim 9, characterized in that: The expulsion device includes an ultrasonic expulsion mechanism and an audible and visual alarm mechanism, which are respectively installed on the monitoring device. The ultrasonic expulsion mechanism and the audible and visual alarm mechanism are electrically connected to the AI ​​chip. When the AI ​​chip detects that a small animal has invaded the detection area, the AI ​​chip sends expulsion signals to the ultrasonic expulsion mechanism and the audible and visual alarm mechanism respectively.

Images