Intelligent bird repelling method and system for extra-high voltage converter station

By improving the YOLOv8 model and multiple gimbal camera systems, the problem of difficulty in detecting birds in UHV converter stations is solved, and efficient bird identification and disengagement is achieved, which improves detection accuracy and bird damage prevention capabilities.

CN119073298APending Publication Date: 2024-12-06NANJING UNIV OF SCI & TECH +1

Patent Information

Application Number
CN202411197359.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

It is difficult to detect birds in ultra-high voltage converter stations, and existing bird damage prevention measures are invested and inefficient. It is difficult for traditional imaging equipment to achieve large-scale monitoring, and bird images lack surface texture characteristics and have low detection accuracy.

Method used

Improve the YOLOv8 bird recognition model, by adding SPD-Conv convolution to the Backbone network, adding CBAM attention module to the Neck network, and introducing an attention mechanism into the Head detection head, using the Shape-IoU loss function, combined with multiple gimbals The camera and embedded terminal realize the identification and expulsion of flying birds.

Benefits of technology

It significantly improves the detection accuracy and recognition ability of small-target birds, realizes effective monitoring and discharging of birds in ultra-high voltage converter stations, reduces the false detection rate, and improves the system's bird damage prevention ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119073298A_ABST
    Figure CN119073298A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent bird repelling method and system for an extra-high voltage converter station. The method comprises the steps of collecting a bird image in the converter station, making a bird data set in a converter station scene, and enhancing the bird data set by using an image synthesis method. Building and training a model framework: building a deep learning network based on YOLOv8, combining a novel convolution SPDConv and an attention module CBAM, enhancing the recognition capability of the model for a low, small and slow target, and training a target detection model; a two-degree-of-freedom heavy-load holder is used for scanning and collecting bird intrusion conditions in a monitoring area, a TensorRT is used for deploying a target detection model, and high-frame-rate and high-pixel video stream real-time reasoning is achieved; intrusion information collection: distributing a frame queue according to a video stream, generating an intrusion video and an intrusion picture, obtaining a flying bird intrusion direction by using a holder encoder, and storing flying bird intrusion information; an embedded platform is used as an intelligent terminal to realize multi-channel video stream deployment and control and information query; invasive birds in the converter station are repelled by using directional repelling sound wave equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of bird identification and bird driving away in ultra-high voltage converter stations, and in particular to a method and system for driving away birds in ultra-high voltage converter stations. Background Art

[0002] There are a large number of filters and other equipment installed outdoors in the UHV converter station. If there are birds flying through the installation area of ​​these equipment, it is easy to cause phase-to-phase short circuit or ground short circuit of the equipment and cause accidents. It is necessary to strictly prevent birds from entering. At present, the conventional bird damage prevention method for circulation stations is to spray RTV insulating paint. This passive protection measure not only has a large capital investment and poor cost-effectiveness, but also cannot eliminate the safety hazards caused by invading birds. As a typical "low, small and slow" target among flying objects, the detection difficulty of birds is much higher than that of traditional target detection due to its variety, large changes in its own shape and strong maneuverability. The converter station has the characteristics of complex environment and many obstructions, which further increases the difficulty of bird detection. In addition, affected by the imaging orientation and natural stray light such as illumination, the bird images collected by the pan-tilt camera lack surface texture features and only retain contour features; and when the imaging distance is far, the birds often occupy only very small pixels in the image. Therefore, it is necessary to study a recognition method for "low, small and slow" targets.

[0003] With the rapid development of deep learning, deep learning has broken through the bottleneck of traditional target detection algorithms and become a mainstream algorithm. As a typical first-order target detection algorithm, the YOLO algorithm has been iterated to the YOLOv9 version. Compared with other algorithms, it has the characteristics of fast detection speed and high detection accuracy, and shows strong recognition ability in small target detection tasks.

[0004] Existing imaging equipment is limited by the field of view angle and focal length, making it difficult to achieve large-scale monitoring. Summary of the invention

[0005] The object of the present invention is to provide a method and system for driving away birds in a UHV converter station, so as to identify birds in the converter station, monitor bird intrusion events in the converter station and drive them away.

[0006] The technical solution to achieve the purpose of the present invention is:

[0007] A method for driving away flying birds in a UHV converter station, comprising:

[0008] Collect flying bird images, create a data set, divide it into training set, validation set and test set, and annotate the flying bird targets in the data set; the collected flying bird images include birds in three postures: standing, gliding, and folding wings;

[0009] Establish an improved YOLOv8 flying bird recognition model: Based on the YOLOv8 network, add the five Conv convolutions of the Backbone network to the space-to-depth layer to replace them with SPD-Conv convolutions, add the last two C2f of the Neck network to the CBAM attention module to replace it with the C2f-CBAM module, and replace the Head detection head in YOLOv8 with the Dyhead detection head combined with the attention mechanism; Use the Shape-IoU loss function to replace the CIoU loss function in YOLOv8. Shape-IoU calculates the loss by focusing on the shape and scale of the border itself, making the border regression more accurate;

[0010] Train the model: Use the training set to train the improved YOLOv8 bird recognition model, use the validation set to evaluate the model performance, and adjust the model parameters and hyperparameters during training;

[0011] Get the YOLOv8 bird recognition model and use the test set to verify the actual recognition ability of the model;

[0012] Use TensorRT to deploy the YOLOv8 bird recognition model;

[0013] Intrusion information collection: Allocate frame queues according to video streams, generate intrusion videos and pictures, use pan / tilt encoders to obtain the intrusion position of flying birds, and store intrusion information of flying birds; use embedded terminals to realize multi-channel video stream control and information query;

[0014] It drives away identified birds by emitting directional sound waves.

[0015] An intelligent bird-repelling system for a UHV converter station, comprising:

[0016] The bird recognition module uses the trained improved YOLOv8 bird recognition model to process the collected images and recognize the birds in the converter station environment; Improved YOLOv8 bird recognition model: Based on the YOLOv8 network, the five Conv convolutions of the Backbone network are added to the space-to-depth layer to replace the SPD-Conv convolution, the last two C2f of the Neck network are added to the CBAM attention module to replace it with C2f-CBAM, and the Head detection head in YOLOv8 is replaced by the Dyhead detection head combined with the attention mechanism;

[0017] The video frame processing module takes out the video frame from the queue of image frames to be inspected. If a flying bird is detected in the image, it is added to the alarm frame queue and the next frame is detected. When a flying bird is detected in consecutive set frame images, it is considered that a flying bird intrusion event has occurred. If a flying bird is not detected in the set frame interval, it is considered that the flying bird target is lost. When a flying bird intrusion event occurs, a flying bird intrusion image queue is created, and all video frames before the bird is lost are added to the intrusion image queue. If the final cached video frame in the intrusion image queue is less than the set frame, it is considered to be a false detection. Otherwise, the alarm frame queue is input into the video encoding module to generate a flying bird intrusion video archive.

[0018] The video encoding and decoding module, when a bird intrusion event occurs, takes out the corresponding alarm video frame queue, reads the cached image frames in sequence, decompresses them, and generates a video file;

[0019] The data storage and upload module obtains the current azimuth and pitch angle of the gimbal, and calculates the current bird intrusion azimuth according to the set azimuth information. Once a bird intrusion event occurs, the bird intrusion event and intrusion azimuth information are stored in the database;

[0020] Converter station monitoring system; including multiple multi-degree-of-freedom pan / tilts to carry cameras and collect images of flying birds;

[0021] Bird repellent system: drives away flying birds by emitting directional sound waves.

[0022] Compared with the prior art, the present invention has the following significant advantages:

[0023] (1) In the Backbone feature extraction network, by replacing Conv convolution with SPD-Conv convolution and adding an attention mechanism in the convolution layer, the ability to extract small target features is improved; a CBAM attention module is added to the C2f module, and spatial attention and channel attention are introduced. Channel attention can enhance the feature representation of different channels, and spatial attention helps to extract key information at different positions in space.

[0024] (2) In the Neck multi-scale feature fusion network, the CBAM attention mechanism is added to the last two C2f modules; in the Head target detection network, the Head detection head is replaced with the dynamic detection head DyHead, and the attention mechanism is used to unify different target detection heads. The attention mechanism between feature levels is used for scale perception, the attention mechanism between spatial positions is used for spatial perception, and the attention mechanism within the output channel is used for task perception. This method can significantly improve the expressiveness of the model target detection head without increasing the amount of computation.

[0025] (3) The loss function CIoU in YOLOv8 is replaced by Shape-IoU. Shape-IoU calculates the loss by focusing on the shape and scale of the bounding box itself, making the bounding box regression more accurate.

[0026] (4) Establishing a bird data set for high-voltage circulation stations will help the bird recognition model to complete bird recognition in UHV converter stations.

[0027] (5) Establish a bird intrusion detection system: Through image frame processing, effectively eliminate model false detection and complete effective monitoring of bird intrusion in the converter station. Use multiple heavy-loaded gimbals to complete regional scanning and identification. Use high-pixel visible light cameras to collect available real-time images of the converter station. Combine video encoding and decoding with gimbal encoders to complete the collection of bird intrusion information such as bird intrusion direction and intrusion image video.

[0028] The present invention is based on an improved YOLOv8 network model. Aiming at the complex environment and background of the converter station, the large changes in the bird morphology, the small size and the limitations of existing imaging equipment, the converter station bird data set is collected, and the circulation station bird data set is produced through the data enhancement method. According to the characteristics of the working environment, the YOLOv8 network framework is improved, and the algorithm acceleration reasoning is realized through TensorRT to realize real-time reasoning and recognition of high-definition video streams. Multiple two-axis gimbals are used with cameras to realize all-round monitoring and identification of the target area. Combining streaming media servers, high-performance algorithm reasoning servers and embedded terminals, a monitoring system is designed to realize effective detection and data storage of bird intrusion events, and realize user control and access query of the bird intrusion monitoring system through embedded terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a typical posture diagram of a flying bird in a converter station in a specific embodiment of the present invention.

[0030] Figure 2 This is a partial image of the converter station bird data set in a specific embodiment of the present invention.

[0031] Figure 3 The figure is a flow chart of the method for identifying birds in a converter station in a specific embodiment of the present invention.

[0032] Figure 4 It is the network framework of the bird recognition algorithm improved based on YOLOv8 in the specific embodiment of the present invention.

[0033] Figure 5 Schematic diagram of the SPD-Conv module in a specific embodiment of the present invention.

[0034] Figure 6 Schematic diagram of a C2f-CBAM module in a specific embodiment of the present invention.

[0035] Figure 7 Schematic diagram of the DyHead module in a specific embodiment of the present invention.

[0036] Figure 8 It is a Shape-IoU block diagram in a specific embodiment of the present invention.

[0037] Fig. 9 This is a diagram of the training results of the improved YOLOv8 algorithm in a specific embodiment of the present invention.

[0038] Fig.10 This is a diagram of the bird recognition results of the improved YOLOv8 algorithm in a specific embodiment of the present invention.

[0039] Fig.11 The following is a flow chart of a converter station bird repelling system in a specific embodiment of the present invention.

[0040] Fig.12 It is a framework diagram of a bird intrusion detection system in a specific embodiment of the present invention.

[0041] Fig.13 It is a flow chart of a video frame processing module in a specific embodiment of the present invention.

[0042] Fig.14 It is a flow chart of a video generation module in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0044] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiment diagram of the present invention:

[0045] like Figure 3 As shown, a method for driving away birds from a UHV converter station of the present invention specifically comprises the following steps:

[0046] Step 1: Collect data and create a dataset

[0047] In open scenes, we use network technology to obtain public data sets (COCO, VOC bird detection data sets, etc.). The background of bird images under the sky and ocean background is relatively simple, and the birds occupy more image pixels. This is different from bird recognition in conventional scenes, such as Figure 2As shown in the images of some data sets, in the monitoring images of the converter station, the postures of flying birds are changeable, the pixels occupied are small, and the environment is complex. In order to further improve the actual recognition ability of the model, the public data sets are screened, and effective image sets with complex backgrounds and small pixels occupied by flying birds are collected; actual scene images in the converter station are collected, and the data set is expanded through data enhancement methods such as rotation, cropping, adding noise, and copy-paste; 3062 effective images are obtained, which are divided into training set, validation set and test set in a ratio of 7:2:1, and the bird detection model is trained using this data set.

[0048] The posture of birds in flight is changeable, and the environment of converter stations is relatively complex. When birds lack surface features, they can only be identified through contour information. Some of the postures are difficult to distinguish with the naked eye. Labeling all images reduces the actual recognition ability and greatly increases the possibility of false detection. By analyzing the characteristics of bird postures; Figure 1 As shown, in the collected bird images, the birds are mainly in three postures: standing, gliding, and folded wings. The bird images in these three postures are mainly annotated. The birds in the converter station are usually in these three postures. When other postures appear, identifying the birds in these three postures is sufficient to effectively identify and monitor the bird intrusion.

[0049] Label the flying bird targets in the dataset and export the annotation text file in YOLO format after labeling.

[0050] Step 2: Build an improved YOLOv8 bird recognition model

[0051] Combination Figure 4 The improved YOLOv8 bird recognition network is based on the YOLOv8 network, replaces the 5 Conv convolutions of the Backbone network with SPD-Conv convolutions, replaces the last two C2f of the Neck network with C2f-CBAM, and replaces the Head detection head in YOLOv8 with a Dyhead detection head combined with an attention mechanism.

[0052] The network structure of YOLOv8 is mainly composed of three parts: Backbone, Neck, and Head. The Backbone part is responsible for feature extraction, the Neck part is responsible for multi-scale feature fusion, and the Head part is responsible for the final target detection and classification tasks.

[0053] The improved YOLOv8 bird recognition model mainly includes the following steps:

[0054] 2.1. Establishing an improved Backbone feature extraction network

[0055] The improved Backbone feature extraction network is based on the original Backbone network. In Backbone, the five Conv convolutional layers are added with a space-to-depth (SPD) layer to become an SPD-Conv layer, and a CBAM attention module is added to the C2f module to become a C2f-CBAM module. The C2f-CBAM module combines the attention mechanisms of the feature channel and feature space in the C2f module. After the input passes through Backbone, the output of the third C2f-CBAM module is named out1, and the output of the last layer of SPPF is named out2 and out3.

[0056] Combination Figure 5 , the SPD-Conv consists of a space-to-depth (SPD) layer and a non-strided convolution layer. The space-to-depth (SPD) uses downsampling to convert the original length and width of S and the number of channels of C into intermediate features. Figure X Slice to get a series of sub-feature maps. As shown in the figure, when the scale is 2, 4 sub-feature maps f are obtained. 0,0 ,f 1,0 ,f 0,1 ,f 1,1 Then connect the four sub-feature maps along the channel dimension to output new features Figure X The non-strided convolution layer is a convolution with a stride of 1 and a convolution kernel size of 1*1.

[0057] The sub-feature graph f 0,0 ,f 1,0 ,f 0,1 ,f 1,1 It is obtained by slicing the intermediate feature map, where X is the original intermediate feature map, the size is S*S, the number of channels is C, and the slicing step size scale in the present invention is 2:

[0058] Where X[1:S:scale] means slicing X, where the starting index of the slice is 1, the ending index is S, and the slicing step is scale.

[0059] 2.2. Establishing an improved Neck multi-scale feature fusion network

[0060] The improved Neck feature fusion network adds the C2f module after the Concat layer and the convolution layer to the CBAM attention module to become a C2f-CBAM module, and names the outputs of the C2f module and the C2f-CBAM module out1_1, out2_1, and out3_1.

[0061] Combination Figure 6The C2f-CBAM module inputs the feature map Y, and sequentially performs ordinary two-dimensional convolution with a convolution kernel size of 1*1, Chunk layer, several CBAM_Bottleneck layers, Concat layer, and ordinary two-dimensional convolution with a convolution kernel size of 1*1. The Chunk layer divides the feature map into two blocks in dimension 1. In this embodiment, the number of CBAM_Bottleneck layers is 1. The CBAM_Bottleneck layer sequentially performs two two-dimensional convolutions with a convolution kernel size of 3, a channel attention module (CAM), and a spatial attention module (SAM). The CBAM module output is added to the original intermediate input and the result is output.

[0062] The channel attention module (CAM) generates two channel descriptors by passing the input through the average pooling layer and the maximum pooling layer, and then the two descriptors share the fully connected layer and the ReLU activation function, and then pass the Sigmoid function to obtain the channel attention weight, which is multiplied by the original feature map to achieve feature recalibration of the channel dimension. The spatial attention module (SAM) input passes through the average pooling layer, the maximum pooling layer, the ordinary two-dimensional convolution layer with a convolution kernel size of 7, and the Sigmoid activation function in sequence.

[0063] 2.3. Establishing an improved Head target detection layer

[0064] The outputs of the Neck network, out1_2, out2_1, and out3_1, are input into the DyHead detection head. Figure 7 , DyHead consists of three parts, named π L , π S , π C ;

[0065] The π L They are average pooling layer, ordinary two-dimensional convolution layer with convolution kernel size of 1*1, first activation function, and second activation function; π S The convolution layer is an ordinary two-dimensional convolution layer with index index and convolution kernel size of 3*3. The convolution layer is offset to obtain intermediate output 1, and the third activation function is used to obtain output 2. The two outputs are added to the original output to obtain the output; π C They are average pooling layer, fully connected layer, second activation function, fully connected layer, and average pooling layer.

[0066] The first activation function is ReLU(x)=max(x,0), and the second activation function is The third activation function is sigmoid(x)=1 / (1+exp(-x)). x is an input value.

[0067] The CIoU loss function bounding box regression method used by YOLOv8 considers the geometric relationship between the real box GT and the predicted box, and calculates the loss by using the relative position and relative shape of the bounding boxes. However, the inherent properties of the bounding box, such as its own shape and scale, will affect the bounding box regression. The Shape-IoU loss function can calculate the loss by focusing on the bounding box's own shape and scale, making the bounding box regression more accurate. Figure 8 ,The Shape-IoU principle is as follows,

[0068] The prototype of Shape-IoU loss function is: L shape-IoU =1-IoU+distance shape +0.5*Ω shape . IoU is the ratio of the intersection area and the union area of ​​the predicted box and the real box: Where B is the predicted box area, B gt is the real box area; distance shape is the weighted distance between the two bounding boxes: where β w and β h are the weight coefficients in the horizontal and vertical directions, c is the length of the hypotenuse of the detection frame, is the coordinate of the center point of the real frame, (x c ,y c ) is the coordinate of the center point of the prediction box; Ω shape Consider the shape cost: θ is a constant θ = 4, which is used to describe the rotation state of the bounding box, where t = w Describes the shape cost in the w direction, when t = h Describes the shape cost in the h direction, w and h are the width and height of the prediction box (Anchor Box).

[0069] The vertical weight coefficient is: Horizontal weight coefficient: Where scale is the scale factor, which is related to the target scale in the data set. The smaller the target, the larger the scale should be. gt With w gt is the height and width of the real box (GT Box), h and w are the height and width of the predicted box.

[0070] Step 3: Train the model

[0071] Training equipment configuration: Work in Ubuntu environment, use PyTorch as the framework, main parameters: epoch is 300, imgSize = 640*640, batch = 8, use NVIDIA 3070Ti graphics card acceleration. Use the training set to train the above improved YOLOv8 bird detection model, and use the validation set to evaluate the model performance, adjust the model parameters and hyperparameters.

[0072] Step 4: Evaluate the model

[0073] After the improved YOLOv8 model is trained using the dataset, the model is evaluated using precision (Precision, P), recall (Recall, R), and meanAveragePrecision (mAP). Precision indicates the precision, which indicates the proportion of targets that actually exist in the algorithm recognition results. The higher the precision, the higher the detection accuracy of the model; Recall indicates the recall, which represents the proportion of correct targets in the recognition results to the total targets of the class. The higher the recall, the higher the probability that the target is correctly detected; the average precision reflects the area surrounded by the precision, recall curves and coordinate axes. The larger the average precision (AP) value, the better the algorithm detection effect and the higher the accuracy.

[0074]

[0075] Among them, TP is the number of correctly identified positive samples, TN is the number of negative samples correctly identified as negative samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples: p(r) represents the precision-recall (PR) curve, r represents the recall rate on the horizontal axis, and the average precision is the integral value of the PR curve in the range of 0 to 1.

[0076] After loading the trained model parameters, the model effect is verified using the flying bird test set, and evaluated using indicators such as AP and Precision. It is compared with the original YOLOv8, proving that the improved YOLOv8 has improved recognition capabilities and can more effectively identify flying birds in the complex environment of UHV converter stations.

[0077] The training results of the bird recognition model are as follows: Fig. 9 As shown in the figure, the validation set is used to evaluate the model performance. According to the above indicators, the average accuracy of the model is 84.9%, the precision is 92.8%, and the recall is 78.1%, which are 5.6%, 5.8%, and 4.1% higher than the original YOLOv8 network. The recognition ability of the model is tested using the test set. Some flying bird recognition results are shown in the figure. Fig.10 , which can effectively detect small target flying birds in the converter station.

[0078] Step 5: Model deployment

[0079] Birds are small in size, and due to the limitations of imaging equipment, distant birds occupy very few pixels in the image. Figure 2 As shown, the flying bird target only occupies 50*50 pixels in the 1090*1080 image. In order to realize target recognition within a range of 500m, an imaging device with high pixels and strong zoom capability is required. The present invention uses a Hikvision PTZ camera with a maximum pixel of 2688*1520 and a 1-55 times zoom capability. When the image pixels are large, the processing time of the image frame is greatly increased. In order to realize real-time recognition of high-pixel video frames, the present invention uses an Nvidia 3070Ti graphics card to accelerate the reasoning of flying bird recognition. As shown in the figure, the reasoning time of the 1920*1080 pixel image after acceleration is about 26ms.

[0080] After model training, we get the bird recognition model. We convert the weight file into an onnx general model file, and use the Nvidia official TensorRT tool to export the .trt model deployment file, deserialize the model, and allocate GPU memory for model input and output. We use the Hikvision gimbal SDK to obtain real-time video frames, input the image frames into the bird recognition model, and copy the output data to the CPU after executing the model inference. We also mark the bird targets on the image based on the output data.

[0081] The present invention provides an intelligent bird-repelling system for a UHV converter station. The process of the intelligent bird-repelling system is as follows: Fig.11 As shown, a pan-tilt camera is used to scan and collect high-definition video streams of flying birds in the UHV converter station. The high-definition video stream is forwarded by the video streaming server. After the inference server pulls the high-definition video stream, the bird recognition model is used to identify bird intrusions. After video frame processing, effective bird intrusion events are extracted. If a bird intrusion event occurs, a directional sound wave bird repellent device is used to repel the birds. At the same time, the database stores information such as bird intrusion videos and intrusion directions. At the same time, the intrusion event information is synchronized to the cloud service platform, which can centrally count and browse the bird intrusion situation in the UHV converter station. The intelligent bird repellent hardware system framework is shown in the figure. Fig.12 As shown in the figure, multiple PTZ cameras collect real-time video, and the video stream is forwarded to the streaming media server; the PTZ encoder communicates with the embedded terminal through the serial port. The embedded terminal queries and deploys the video stream through the HTTP protocol, and the inference server uses the RTSP protocol to pull the video stream. The inference results are saved in the database. The cloud service platform can view the inference result video stream, count and browse the bird intrusion information in the database.

[0082] The bird-repellent system specifically includes: bird recognition module, video frame processing module, video encoding and decoding module, data storage and upload module, converter station monitoring system, and bird repellent system; the converter station monitoring system consists of four heavy-loaded two-degree-of-freedom gimbals and an embedded control terminal. The gimbal is equipped with a visible light camera with a zoom capability of 1-55 and a maximum support of 2688*1520 pixels, and a built-in two-degree-of-freedom encoder.

[0083] This system uses multi-spectral PTZ equipment to collect real-time images of the surrounding areas of the UHV converter station, and uses a streaming media server to centrally manage high-definition video streams. The streaming media server distributes the video streams to the inference server, which deploys a bird recognition model, infers the coordinates of bird intrusion, and uses directional sound waves to drive away the birds. The time and direction of bird intrusion are stored on the server and synchronized to the cloud server platform for centralized browsing and statistics.

[0084] The hardware of the bird-repellent system adopts directional sound wave emission technology and has two operating modes: normal prevention and control and directional expulsion. Normal prevention and control is a space denial prevention and control based on a sound pressure of 100dBA combined with the audio of natural enemies; directional expulsion is guided by the target detection direction, and multiple devices are linked to carry out highly directional sound wave attack and expulsion operations with a maximum outlet sound pressure of 140dBA.

[0085] Bird recognition module: The bird recognition module uses the improved YOLOv8 algorithm after the above training to achieve real-time reasoning for 2688*1520 pixel images. The improved YOLOv8 algorithm adds a DyHead detection head to the YOLOv8 infrastructure, uses SPDConv to replace part of Conv, and uses the ShapeIou loss function to achieve the recognition of birds in the converter station environment. The bird recognition module is deployed on the inference server, which is configured with: Windows 8 operating system, Inter CORE i7 CPU, Nvidia 3070Ti graphics card, and 2TB solid-state drive.

[0086] Image frame processing module: The PTZ is connected to the inference server via an Ethernet cable, obtains video frames frame by frame, takes out video frames from the queue of image frames to be inspected, performs inference on the frame image, and obtains the inference result. Fig.13If a bird is detected in the image, it will be added to the alarm frame queue and the next frame will be detected. When there are 5 consecutive frames of images with a bird detected, it is considered that a bird intrusion event has occurred. If no bird is detected after 5 frames, it is considered that the bird target is lost. When a bird intrusion event occurs, a bird intrusion image queue is created, and all video frames before the bird is lost are added to the intrusion image queue. If the final cached video frames in the intrusion image queue are less than 10 frames, it is considered a false detection; otherwise, the alarm frame queue is input into the video encoding module to generate a bird intrusion video archive. The 2688*1520 pixel RGB format image frame occupies a large amount of memory. It is compressed into JPG format using the Opencv image library and added to the cache queue. The JPG format image is decompressed into RGB format before encoding the video.

[0087] Data storage and processing module: The data that needs to be collected include the time of bird intrusion, intrusion direction (PTZ pitch angle and azimuth), intrusion video, number of intrusions, etc. When a bird intrusion event occurs, the current azimuth and pitch angle of the PTZ are obtained through the PTZ encoder, and the current bird intrusion direction is calculated based on the set direction information. Once a bird intrusion event occurs, the bird intrusion event, intrusion direction and other information are stored in the database and uploaded to the web server.

[0088] Video encoding module: The video encoding module uses the Ffmpeg video library and runs on the inference server to generate intrusion videos. The video encoding module runs separately in another working thread and combines Fig.14 ,When a bird intrusion event occurs, the corresponding alarm video frame queue is taken out, the cached image frames are read in sequence, and decompressed into RGB format video frames, and encoded through H264 to generate a video file.

[0089] Bird monitoring system: combined Fig.13 The bird monitoring system includes four heavy-duty two-degree-of-freedom PTZs equipped with visible light cameras that support up to 2688*1520 pixels and 1-55 magnification. The embedded control terminal is equipped with a touch screen. The embedded control terminal uses an Ethernet cable to communicate with the four PTZ cameras. The embedded terminal can realize centralized preview and control of the PTZ cameras. The embedded terminal is also connected to the inference server to realize centralized algorithm access to the video streams of the four PTZ cameras.

[0090] Due to the limitation of the existing imaging system, it is difficult for one camera to complete all-round monitoring of a large area. Multiple PTZ cameras are used to achieve all-round recognition in the area. The working area of ​​a PTZ is set by the PTZ encoder, and the PTZ scans cyclically within the set interval. Four PTZs are installed at the four corners of the monitoring area, and the recognition areas of the four PTZs intersect, completely covering the target detection area.

[0091] When the monitoring area is large, install multiple PTZ cameras to analyze, process and store multiple video streams; push multiple video streams to the streaming media server, and perform bird detection on multiple video streams. Configure a streaming media server on a server, connect it to the streaming media server via an Ethernet cable, and push each PTZ video stream to the streaming media server via the RTSP streaming media transmission protocol;

[0092] The streaming media server uses the same hardware configuration as the inference server. The streaming media server provides HTTP services. An embedded terminal with a touch screen is used to connect to the streaming media service. The client program is designed to access the HTTP service of the streaming media server to obtain the address of each pan-tilt video stream, monitor the status of each pan-tilt video stream, and control the disconnection, connection, and preview of each pan-tilt video stream. The pan-tilt communicates with the embedded platform through USART, and the pan-tilt posture is controlled by the embedded platform.

[0093] The video stream analysis service is deployed in an inference server equipped with a high-performance graphics card. The inference server is connected to the control terminal device and the streaming media server at the same time. The video stream address of each pan-tilt camera is pushed to the inference server through the embedded client. The inference server pulls the target video stream and uses the trained deep learning algorithm for bird recognition.

[0094] The inference server is mainly composed of video stream processing module, image inference module, information processing module and multimedia file processing module. According to the video interface of different PTZ, the PTZ official API, RTSP video transmission protocol and other methods can be used to extract RGB format image frames from the video stream frame by frame, and the output image frames are processed by the image inference module.

[0095] Each inference server is equipped with a file proxy service and connected to the WEB server. When a bird intrusion is detected, the intrusion video and screenshots are stored in the SSD. The web server can obtain the corresponding video and screenshots through the file proxy. When a bird is detected in the video frame, the system time and other information are obtained, the video file is named according to the intrusion time, and the HTTP protocol is used to synchronize to the WEB server.

Claims

1. A method for driving away birds from a UHV converter station, characterized in that: include: Collect flying bird images, create a data set, divide it into training set, validation set and test set, and annotate the flying bird targets in the data set; the collected flying bird images include birds in three postures: standing, gliding, and folding wings; Establish an improved YOLOv8 flying bird recognition model: Based on the YOLOv8 network, add the five Conv convolutions of the Backbone network to the space-to-depth layer to replace them with SPD-Conv convolutions, add the last two C2f of the Neck network to the CBAM attention module to replace it with the C2f-CBAM module, and replace the Head detection head in YOLOv8 with the Dyhead detection head combined with the attention mechanism; Use the Shape-IoU loss function to replace the CIoU loss function in YOLOv8. Shape-IoU calculates the loss by focusing on the shape and scale of the border itself, making the border regression more accurate; Train the model: Use the training set to train the improved YOLOv8 bird recognition model, use the validation set to evaluate the model performance, and adjust the model parameters and hyperparameters during training; Get the YOLOv8 bird recognition model and use the test set to verify the actual recognition ability of the model; Use TensorRT to deploy the YOLOv8 bird recognition model; Intrusion information collection: Allocate frame queues according to video streams, generate intrusion videos and pictures, use PTZ encoders to obtain the intrusion position of flying birds, and store intrusion information of flying birds; Use embedded terminals to achieve multi-channel video stream control and information query; It drives away identified birds by emitting directional sound waves.

2. The method for driving away birds from a UHV converter station according to claim 1, characterized in that: The SPD-Conv convolution consists of a space-to-depth layer and a non-strided convolution layer; the space-to-depth uses downsampling to slice the intermediate feature map X with an original length and width of S and a number of channels of C to obtain multiple sub-feature maps; then connects multiple sub-feature maps along the channel dimension to output a new feature map X′; the non-strided convolution layer is a convolution with a stride of 1 and a convolution kernel size of 1*1.

3. The method for driving away birds from a UHV converter station according to claim 1, characterized in that: The C2f-CBAM module sequentially performs a two-dimensional convolutional layer with a convolution kernel size of 1*1, a Chunk layer, several CBAM_Bottleneck layers, a Concat layer, and an ordinary two-dimensional convolutional layer with a convolution kernel size of 1*1 on the input feature map Y; the Chunk layer divides the feature map into two blocks in dimension 1, and the CBAM_Bottleneck layer sequentially performs two two-dimensional convolutional layers with a convolution kernel size of 3, a channel attention module CAM, and a spatial attention module SAM.

4. The method for driving away birds from a UHV converter station according to claim 3, characterized in that: The channel attention module CAM generates two channel descriptors by inputting the average pooling layer and the maximum pooling layer respectively, and then the two descriptors share the fully connected layer and the ReLU activation function, and then pass the Sigmoid function to reach the channel attention weight, which is multiplied with the original feature map to achieve feature recalibration of the channel dimension; the intermediate feature map of the spatial attention module SAM sequentially undergoes average pooling and maximum pooling, ordinary two-dimensional convolution with a convolution kernel size of 7, and Sigmoid activation function.

5. The method for driving away birds from a UHV converter station according to claim 1, characterized in that: DyHead consists of three parts, named π L , π S , π C ; π L Sequentially, average pooling layer, ordinary two-dimensional convolution layer with convolution kernel size of 1*1, first activation function, and second activation function; π S Sequentially pass through the common two-dimensional convolutional layers with index index and convolution kernel size of 3*3. The output of the convolutional layer is passed through the offset layer to obtain the intermediate output 1, and then through the third activation function to obtain the intermediate output 2. The two intermediate outputs are summed with the input to obtain the output; π C Some of them are sequentially average pooling layer, fully connected layer, second activation function, fully connected layer, and average pooling layer; The first activation function is the ReLU function; the second activation function is the hard_sigmode function; the third activation function is the sigmoid function.

6. The method for driving away birds from a UHV converter station according to claim 1, characterized in that: The Shape-IoU loss function is: L shape-IoU =1-IoU+distance shape +0.5*Ω shape IoU is the ratio of the intersection area and union area of ​​the predicted box and the true box: Where B is the predicted box area, B gt is the real box area; distance shape is the weighted distance between the two bounding boxes: where β w and β h are the weight coefficients in the horizontal and vertical directions, c is the length of the hypotenuse of the detection frame, is the coordinate of the center point of the real frame, (x c ,y c ) is the coordinate of the center point of the prediction box; Ω shape Consider the shape cost: θ is a constant; Describes the shape cost in the w direction; Describes the shape cost in the h direction; w and h are the width and height of the prediction box, h gt With w gt is the height and width of the real frame.

7. The method for driving away birds from a UHV converter station according to claim 1, characterized in that: The YOLOv8 bird recognition model will be improved to improve the Precision, Recall, and Average Precision (AP) evaluation model accuracy of the YOLOv8 bird recognition model.

8. The method for driving away birds from a UHV converter station according to claim 1, characterized in that: After model training, the converter station bird recognition model is obtained. The original model file is converted into an onnx general model file, and the Nvidia official TensorRT tool is used to export the trt model deployment file, deserialize the model, and allocate GPU memory for model input and output.

9. An intelligent bird-repellent system for a UHV converter station, characterized in that: include: The bird recognition module uses the trained improved YOLOv8 bird recognition model to process the collected images and recognize the birds in the converter station environment; Improved YOLOv8 bird recognition model: Based on the YOLOv8 network, the five Conv convolutions of the Backbone network are added to the space-to-depth layer to replace the SPD-Conv convolution, the last two C2f of the Neck network are added to the CBAM attention module to replace it with C2f-CBAM, and the Head detection head in YOLOv8 is replaced by the Dyhead detection head combined with the attention mechanism; The video frame processing module takes out the video frame from the queue of image frames to be inspected. If a flying bird is detected in the image, it is added to the alarm frame queue and the next frame is detected. When a flying bird is detected in consecutive set frame images, it is considered that a flying bird intrusion event has occurred. If a flying bird is not detected in the set frame interval, it is considered that the flying bird target is lost. When a flying bird intrusion event occurs, a flying bird intrusion image queue is created, and all video frames before the bird is lost are added to the intrusion image queue. If the final cached video frame in the intrusion image queue is less than the set frame, it is considered to be a false detection. Otherwise, the alarm frame queue is input into the video encoding module to generate a flying bird intrusion video archive. The video encoding and decoding module, when a bird intrusion event occurs, takes out the corresponding alarm video frame queue, reads the cached image frames in sequence, decompresses them, and generates a video file; The data storage and upload module obtains the current azimuth and pitch angle of the gimbal, and calculates the current bird intrusion azimuth according to the set azimuth information. Once a bird intrusion event occurs, the bird intrusion event and intrusion azimuth information are stored in the database; Converter station monitoring system; It includes multiple multi-DOF gimbals to carry cameras and collect images of flying birds; Bird repellent system: drives away flying birds by emitting directional sound waves.

Citation Information

Patent Citations

  • Power transmission line active bird repelling device based on video study and judgment

    CN113287597A

  • Monitoring management method and system based on three-dimensional panoramic video fusion

    CN115409751A

  • Forest weed detection method based on improved YOLOv5 model

    CN115546639A

  • Insulator defect detection method based on improved YOLOv7

    CN115731164A

  • Method for identifying bird species related to bird-related fault of transformer substation

    CN116229380A

Cited By

  • Target detection method and device based on infrared image, equipment and storage medium

    CN120953593A