A method and system for identifying bird nests in substations based on deep learning

Abstract

The present invention relates to a method for identifying bird nests in substations based on deep learning, comprising: building and training a recognition model: acquiring a sample image, processing the sample image by using a first feature extraction algorithm, and obtaining a sample feature map; inputting the sample feature map into a recognition model, wherein the recognition model performs several convolution operations on the sample feature map, and obtains several feature maps of different scales, and generates several anchor frames in the several feature maps; using a quantum particle swarm algorithm, determining the ratio value and confidence threshold of positive samples to negative samples; dividing the several anchor frames into positive samples and negative samples according to the confidence threshold; updating the recognition model according to the several positive samples and negative samples; acquiring an image to be inspected; processing the image to be inspected by using a first feature extraction algorithm, and obtaining a first feature map; inputting the first feature map into the recognition model, and obtaining a first recognition result and the confidence of the first recognition result.

Description

Technical Field

[0001] The present invention relates to a substation bird nest recognition method and system based on deep learning, belonging to the field of substation monitoring. Background Art

[0002] At present, substations have basically achieved the goal of less people and less people, and video inspections have become the main means of monitoring substations. Substations are mostly located on the outskirts of cities, and due to the special structure of the electrical equipment in the station, it is very common for birds to build nests in them, which poses a great hidden danger to the safe operation of the substation. Bird nests are mostly made of straw and branches, etc., which are very easy to lean against the lines and cause short circuit faults. When encountering rainy and snowy weather, wet bird nests will cause the insulation strength of the equipment to decrease, which is easy to cause breakdown and other faults. Therefore, it is necessary to use image recognition algorithms to realize automatic monitoring and alarm of bird nests. The monitoring device obtains bird nest pictures under different care, weather, and background environments, and the bird nests are of different sizes. The recognition effect of existing image recognition algorithms is not good. Therefore, a bird nest recognition method with better recognition effect is needed.

[0003] For example, the patent "Transmission Line Bird Nest Detection Method Based on Improved SSD Algorithm" with publication number CN111241905A discloses the following steps: (1) Obtain images of transmission towers through drones, pre-process them, and manually label them to create a data set; (2) Divide the data set obtained in step (1) into mutually exclusive training sets, validation sets, and test sets, and use the improved SSD algorithm to train them to obtain a model for identifying targets; (3) Identify the target according to the model obtained in step (2); (4) Statistic the logs obtained from the training to obtain the accuracy of the model. This invention realizes intelligent real-time monitoring of bird nests on transmission lines. However, this method lacks the processing of the training set, and the distribution results of positive and negative samples will directly affect the final recognition effect. It is difficult to achieve the optimal effect of the model without processing; at the same time, the parameters of the model lack reasonable selection. For specific objects, the parameters in the model should be reasonably set to maximize the potential of the network model and achieve the optimal result. Summary of the invention

[0004] In order to overcome the problems existing in the prior art, the present invention designs a substation bird nest recognition method and system based on deep learning, uses the quantum particle swarm algorithm to determine the optimal positive and negative sample ratio and confidence threshold, maximizes the potential of the recognition model, and adjusts the recognition model parameters to the most suitable recognition of the characteristics of the bird's nest type, so as to improve the final recognition effect.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] Technical solution 1:

[0007] A method for identifying bird nests in a substation based on deep learning, comprising the following steps:

[0008] Constructing and training a recognition model: obtaining a sample image, processing the sample image using a first feature extraction algorithm to obtain a sample feature map; inputting the sample feature map into the recognition model, and the recognition model performing several convolution operations on the sample feature map to obtain several feature maps of different scales, and generating several anchor frames in the several feature maps; using a quantum particle swarm algorithm to determine the ratio of positive samples to negative samples and a confidence threshold; dividing the several anchor frames into positive samples and negative samples according to the confidence threshold; and updating the recognition model according to the several positive samples and negative samples;

[0009] Acquire an image to be inspected; process the image to be inspected using a first feature extraction algorithm to obtain a first feature map; input the first feature map into a recognition model to obtain a first recognition result and a confidence level of the first recognition result.

[0010] Furthermore, it also includes:

[0011] The image to be inspected is processed using a second feature extraction algorithm to obtain a second feature map; the second feature map is input into a recognition model to obtain a second recognition result and a confidence level of the second recognition result; and the one with a higher confidence level between the first recognition result and the second recognition result is taken as the final recognition result.

[0012] Furthermore, the first feature extraction algorithm is a HOG algorithm, and the second feature extraction algorithm is a SIFT algorithm.

[0013] Furthermore, the quantum particle swarm algorithm is used to determine the ratio of positive samples to negative samples and the confidence threshold, specifically:

[0014] The ratio value and confidence threshold are used as the position information of the particle, and the recognition effect is used as the output information, and each particle is updated cyclically until the cycle stop condition is reached. When the cycle stops, the position information of the particle is the ratio value and the confidence threshold.

[0015] Furthermore, the method also includes performing a preprocessing operation on the image to be inspected before obtaining the first feature map, wherein the preprocessing operation includes removing image noise and enhancing the image.

[0016] Furthermore, it also includes performing data conversion on the first feature map, specifically:

[0017] The first feature map is decomposed into a three-channel image, and the image is converted into binary data according to each channel, and the binary data is input into the recognition model.

[0018] Furthermore, in the recognition model, the loss values ​​of the positive samples and the negative samples are calculated by using the loss function CIoU.

[0019] Technical solution 2:

[0020] A substation bird nest recognition system based on deep learning, comprising:

[0021] A plurality of image collectors, wherein the image collectors are used to obtain and send images to be inspected to a server;

[0022] A server, wherein a recognition model is provided in the server, wherein the recognition model processes the image to be inspected using a first feature extraction algorithm to obtain a first feature map; the first feature map is input into the recognition model to obtain a first recognition result; and the server sends the recognition result to the monitoring system;

[0023] A monitoring system, wherein the monitoring system is used to issue an early warning according to the identification result;

[0024] Among them, the recognition model uses the quantum particle swarm algorithm during the training process to determine the ratio of positive samples to negative samples and the confidence threshold.

[0025] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0026] 1. The present invention utilizes the HOG algorithm and the SIFT algorithm to obtain the first feature map and the second feature map respectively, thereby obtaining the first recognition result and the second recognition result, and the one with higher confidence is taken as the final recognition result. Since HOG maintains good invariance to image geometry and optical deformation, and SIFT has a certain degree of stability to perspective changes and noise, and has a small feature dimension and is resistant to occlusion, the combination of the two can adapt to the recognition needs in different image environments and improve the recognition accuracy.

[0027] 2. Considering that the hyperparameters in SSD will be continuously updated during the iterative training process, and the positive and negative sample ratio is generally determined by the obtained image, there is a great deal of uncertainty and a great impact on the recognition effect; in addition, the confidence threshold and the positive and negative sample ratio in the prior art generally adopt default values ​​or are determined based on experience, which has great limitations. Therefore, the present invention uses the quantum particle swarm algorithm to determine the optimal positive and negative sample ratio and confidence threshold, maximizes the potential of the recognition model, and adjusts the recognition model parameters to the most suitable recognition of the characteristics of the bird's nest, so as to improve the final recognition effect.

[0028] 3. In some images, multiple bird nests may overlap. In general, only the bird nest closest to the monitoring device can be identified. The present invention uses CIoU as a loss function to improve the anti-occlusion capability and avoid missing objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the present invention;

[0030] Figure 2 It is a schematic diagram of the recognition model of the present invention;

[0031] Figure 3 This is a diagram of the aiming frame. DETAILED DESCRIPTION

[0032] The present invention is described in more detail below with reference to the embodiments.

[0033] Embodiment 1

[0034] A method for identifying bird nests in a substation based on deep learning, comprising the following steps:

[0035] S1. Construct a recognition model. In this embodiment, the recognition model adopts the SSD model. Figure 2 As shown, the SSD model includes a VGG16 network, several convolutional layers, pooling layers, and classification layers.

[0036] S2. Training recognition model:

[0037] S21, obtaining a sample image set, wherein the sample image is a substation inspection image marked with a bird's nest location;

[0038] S22, performing image enhancement on each sample image, such as removing noise, grayscale inversion, contrast stretching, grayscale slicing, etc.; then rotating and folding each sample image to expand the database; finally scaling each sample image to 300×300;

[0039] S23, respectively extracting features from each sample image using the HOG algorithm and the SIFT algorithm to obtain multiple sample feature maps;

[0040] S24. Decompose each sample feature map into an RGB three-channel image, and convert the sample feature map into binary data by channel. The data format includes but is not limited to TFRecord, hdf5 and other file formats, which are suitable for large-scale data storage, reading, copying and moving under different frameworks.

[0041] S25. Input the sample feature map into the recognition model. The recognition model performs several convolution operations on the sample feature map to obtain several feature maps of different scales, and generates several anchor frames in the several feature maps; calculate the confidence (i.e., IOU value) of each aiming frame; use the quantum particle swarm algorithm to determine the ratio of positive samples to negative samples and the confidence threshold; divide the several anchor frames into positive samples and negative samples according to the confidence threshold; calculate the loss value of each positive sample and negative sample through the loss function CIoU; update the recognition model according to the direction of the loss value;

[0042] S26, repeat step S25 until the recognition accuracy reaches the requirement;

[0043] S3. Identify bird nests:

[0044] The image to be inspected is acquired by using the image collector of the substation; the image to be inspected is processed by using the HOG algorithm and the SIFT algorithm respectively to obtain the first feature map and the second feature map; the first feature map and the second feature map are input into the recognition model respectively to obtain the first recognition result, the confidence of the first recognition result, the second recognition result and the confidence of the second recognition result; the one with the higher confidence is taken as the final recognition result.

[0045] The beneficial effects of this embodiment are:

[0046] 1. It is generally believed in the prior art that the fusion of multiple features combines multi-angle and multi-level features of the image (such as image color, shape, gradient), spatial features and temporal features, global features and local features, and has better classification performance than a single feature.

[0047] However, the technicians of the present invention considered that the process and mathematical expression of image features extracted by the HOG algorithm and the SIFT algorithm are different, and the advantages of each other are weakened while complementing each other's shortcomings after fusion. Therefore, for the images to be inspected with large fluctuations in image features / characteristics obtained by each image collector of the substation at different positions, angles and lighting conditions, fusion of multiple features is not applicable, and its recognition effect is not as good as the single image feature extraction method.

[0048] In summary, this embodiment overcomes technical bias, and uses the HOG algorithm and the SIFT algorithm to obtain the first feature map and the second feature map respectively, thereby obtaining the first recognition result and the second recognition result, and the one with higher confidence is taken as the final recognition result. Since HOG maintains good invariance to image geometry and optical deformation, and SIFT has a certain degree of stability to perspective changes and noise, and has a small feature dimension and is resistant to occlusion, the combination of the two can adapt to the recognition needs in different image environments and improve the recognition accuracy.

[0049] 2. Considering that the hyperparameters in the SSD model will be continuously updated during the iterative training process, and the positive and negative sample ratio values ​​are generally determined by the obtained images, there is a great deal of uncertainty, and it has a great impact on the recognition effect; in addition, the confidence threshold and the positive and negative sample ratio values ​​in the prior art generally adopt default values ​​or are determined based on experience, which has great limitations. Therefore, this embodiment uses the quantum particle swarm algorithm to determine the optimal positive and negative sample ratio value and confidence threshold, maximize the potential of the recognition model, and adjust the recognition model parameters to the most suitable recognition of the characteristics of the bird's nest type, so as to improve the final recognition effect.

[0050] 3. In some images, multiple bird nests may overlap. In general, only the bird nest closest to the monitoring device can be identified. This embodiment uses CIoU as the loss function to improve the anti-occlusion capability and avoid missing the identified object.

[0051] Embodiment 2

[0052] Furthermore, in this embodiment, the positive and negative sample ratio and the confidence threshold are used as the position information of the particles, and the recognition effect, such as the mean average precision (MAP), is used as the output information. The quantum particle swarm algorithm is used to determine the positive and negative sample ratio and the confidence threshold, and the specific steps are as follows:

[0053] A1. Initialize each particle;

[0054] A2. Input the sample feature map into the recognition model, perform recognition based on the position information of the current optimal particle, and obtain the recognition result of each cycle;

[0055] A3, determine whether the recognition result meets the loop stop condition, if not, update the particle position information and repeat step A2; otherwise, the particle position information is what is required at this time, and it is used as the positive and negative sample ratio value and confidence threshold in the final recognition model. In this embodiment, the loop stop condition is: the MAP value exceeds 0.9.

[0056] Embodiment 3

[0057] A substation bird nest recognition system based on deep learning, comprising:

[0058] A plurality of image collectors, wherein the image collectors are used to obtain and send images to be inspected to a server;

[0059] A server, wherein a recognition model is provided in the server, wherein the recognition model processes the image to be inspected by using the HOG algorithm and the SIFT algorithm respectively to obtain a first feature map and a second feature map; the first feature map and the second feature map are respectively input into the recognition model to obtain a first recognition result, a confidence level of the first recognition result, a second recognition result and a confidence level of the second recognition result; the one with a higher confidence level is taken as the final recognition result; the server sends the recognition result to the monitoring system;

[0060] The monitoring system issues an early warning when the identification result is a bird's nest.

[0061] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Claims

1. A method for identifying bird nests in substations based on deep learning, characterized in that: The following steps are involved: Construct and train a recognition model: obtain a sample image, process the sample image using a first feature extraction algorithm to obtain a sample feature map, wherein the first feature extraction algorithm is a HOG algorithm; input the sample feature map into the recognition model, and the recognition model performs several convolution operations on the sample feature map to obtain several feature maps of different scales, and generates several anchor frames in the several feature maps; use a quantum particle swarm algorithm to determine the ratio of positive samples to negative samples and a confidence threshold; divide the several anchor frames into positive samples and negative samples according to the confidence threshold; calculate the loss values ​​of the positive samples and negative samples in the recognition model through a loss function CIoU, and update the recognition model according to the several positive samples and negative samples; Acquire an image to be inspected; process the image to be inspected using a first feature extraction algorithm to obtain a first feature map; input the first feature map into a recognition model to obtain a first recognition result and a confidence level of the first recognition result; Processing the image to be inspected by using a second feature extraction algorithm to obtain a second feature map, wherein the second feature extraction algorithm is a SIFT algorithm; inputting the second feature map into a recognition model to obtain a second recognition result and a confidence level of the second recognition result; taking the one with a higher confidence level between the first recognition result and the second recognition result as the final recognition result; The quantum particle swarm algorithm is used to determine the ratio of positive samples to negative samples and the confidence threshold, specifically: The ratio value and confidence threshold are used as the position information of the particle, and the recognition effect is used as the output information, and each particle is updated cyclically until the loop stop condition is reached; when the loop stops, the position information of the particle is the ratio value and the confidence threshold.

2. According to a deep learning-based method for identifying bird nests in substations according to claim 1, it is characterized in that: The method also includes performing a preprocessing operation on the image to be inspected before obtaining the first feature map, wherein the preprocessing operation includes removing image noise and enhancing the image.

3. According to a method for identifying bird nests in substations based on deep learning according to claim 1, it is characterized in that: The method also includes performing data conversion on the first feature map, specifically: decomposing the first feature map into a three-channel image, converting the image into binary data according to each channel, and inputting the binary data into the recognition model.

4. A substation bird nest recognition system based on deep learning, characterized in that: include: A plurality of image collectors, wherein the image collectors are used to obtain and send images to be inspected to a server; A server, wherein a recognition model is provided in the server, and the recognition model processes the image to be inspected using a feature extraction algorithm to obtain a feature map, specifically: processing the image to be inspected using a first feature extraction algorithm to obtain a first feature map, wherein the first feature extraction algorithm is a HOG algorithm; processing the image to be inspected using a second feature extraction algorithm to obtain a second feature map, wherein the second feature extraction algorithm is a SIFT algorithm; Input the feature map into the recognition model to obtain a recognition result, specifically: input the first feature map into the recognition model to obtain a first recognition result and a confidence level of the first recognition result; input the second feature map into the recognition model to obtain a second recognition result and a confidence level of the second recognition result; and take the one with a higher confidence level between the first recognition result and the second recognition result as the final recognition result; The server sends the final recognition result to the monitoring system; A monitoring system, wherein the monitoring system is used to issue an early warning according to the identification result; The construction and training process of the recognition model is as follows: Acquire a sample image, process the sample image using a first feature extraction algorithm to obtain a sample feature map, input the sample feature map into a recognition model, the recognition model performs a number of convolution operations on the sample feature map to obtain a number of feature maps of different scales, and generate a number of anchor frames in the feature maps; The quantum particle swarm algorithm is used to determine the ratio value and confidence threshold of positive samples and negative samples. Specifically, the ratio value and confidence threshold are used as the position information of the particles, and the recognition effect is used as the output information. Each particle is cyclically updated until the cycle stop condition is reached. When the cycle stops, the position information of the particle is the ratio value and the confidence threshold. According to the confidence threshold, the plurality of anchor frames are divided into positive samples and negative samples; the loss values ​​of the positive samples and negative samples are calculated in the recognition model through the loss function CIoU, and the recognition model is updated according to the plurality of positive samples and negative samples.

Images