Catenary bird nest identification system based on low-power internet of things edge computing device

Abstract

The application provides a catenary bird nest identification system based on a low-power Internet of Things edge computing device, comprising: an input module, a feature extraction module, a model construction module and an output module; the input module is used for inputting an original image to the feature extraction module, wherein the original image comprises an image of high-voltage power transmission equipment and an image of a catenary; the feature extraction module is used for performing feature extraction on the original image to obtain feature information; the model construction module is used for constructing a bird nest identification model based on the feature information; and the output module is used for outputting a bird nest identification result of the original image according to the bird nest identification model. The application can reduce the actual information input quantity, has high detection precision, and realizes real-time online detection.

Description

Technical Field

[0001] This invention belongs to the field of bird nest detection technology, and particularly relates to a contact wire bird nest identification system based on low-power IoT edge computing devices. Background Technology

[0002] Birds frequently nest in high-voltage power transmission equipment and overhead contact lines, causing adverse effects on these devices. This can easily lead to line tripping, component damage, and other safety accidents, even severely impacting the safe and efficient operation of high-voltage transmission lines. Specific causes include bird droppings flashover, short circuits in nest materials, and short circuits in the birds themselves. In my country, bird damage has become the third leading cause of power outages, after lightning strikes and external force damage. Therefore, identifying and removing bird nests from power transmission equipment is a crucial measure to reduce bird damage. Current bird nest identification systems in power systems mostly utilize deep learning computing units from NVIDIA, which can achieve high-throughput parallel computing. While these units offer superior performance, they are costly and consume significant power. Therefore, there is an urgent need for a novel algorithm-based system to identify bird nests in power transmission equipment and maintain the safe operation of the national energy system. Summary of the Invention

[0003] To address the aforementioned technical issues, this invention proposes a contact wire bird nest identification system based on low-power IoT edge computing devices, enabling real-time bird nest detection.

[0004] To achieve the above objectives, the present invention provides a contact wire bird nest identification system based on a low-power IoT edge computing device, comprising: an input module, a feature extraction module, a model building module, and an output module;

[0005] The input module is used to receive the original image and transmit it to the feature extraction module, wherein the original image includes: an image of high-voltage transmission equipment and an image of the overhead contact line;

[0006] The feature extraction module is used to extract features from the original image and obtain feature information;

[0007] The model building module is used to build a bird nest recognition model based on the feature information;

[0008] The output module is used to output the bird nest recognition result of the original image according to the bird nest recognition model.

[0009] Optionally, the feature extraction module includes: a first extraction unit and a second extraction unit;

[0010] The first extraction unit is used to extract macroscopic feature information of the bird's nest in the original image;

[0011] The second extraction unit is used to extract the microscopic texture information of the bird's nest in the original image;

[0012] The first extraction unit and the second extraction unit are respectively connected to the model building module.

[0013] Optionally, the macroscopic feature information includes: color feature information and outline feature information of the bird's nest.

[0014] Optionally, the first extraction unit extracts the color feature information by means of one-dimensional discrete wavelet transform;

[0015] The one-dimensional discrete wavelet transform is:

[0016] A j+1 =H j A j

[0017] D j+1 =G j A j

[0018] Among them, A j+1 H is an approximation of the (j+1)th order signal. j A is the high-pass component of the j-th order signal. j D is an approximation of the j-th order signal. j+1 G represents the detail values ​​of the (j+1)th order signal. j A is the low-pass component of the j-th order signal. j It is an approximation of the j-th order signal.

[0019] Optionally, the first extraction unit extracts the contour feature information by means of two-dimensional discrete wavelet transform;

[0020] The two-dimensional discrete wavelet transform is:

[0021] A i,j+1 =H j A i,j

[0022] D j,j+1 =G j A i,j

[0023] A i+1,j+1 =H i A i,j+1

[0024] D i+1,j+1 =G j A i,j+1

[0025] Among them, A i,j+1 A is an approximation of the horizontal order i and the vertical order j+1.i,j D is an approximation of the horizontal order i and the vertical order j signal. i,j+1 For the detail values ​​of the horizontal order i and the vertical order j+1, A i,j A is an approximation of the horizontal order i and the vertical order j signal. i+1,j+1 D is an approximation of the horizontal (i+1)th order and the vertical (j+1)th order signal. i+1,j+1 G represents the detail values ​​of the horizontal (i+1)th order and the vertical (j+1)th order signal. j Let be the low-pass component of the j-th order signal.

[0026] Optionally, the microtexture information includes: texture information of the details of the branches inside the bird's nest.

[0027] Optionally, the second extraction unit extracts the texture information of the tree branch details inside the bird's nest by using db0 wavelet processing.

[0028] The db0 wavelet processing is as follows:

[0029]

[0030]

[0031] Where h[n] is a high-pass filter, g[n] is a low-pass filter, and n is the length.

[0032] Optionally, the method for the model building module to build the bird nest recognition model includes: training and validating a support vector machine or a Gaussian Bayes classifier using the feature information to obtain the bird nest recognition model.

[0033] Optionally, the process of constructing a model using the Gaussian Bayes classifier includes: obtaining the Gaussian distribution coefficients of the image when there are bird nests and when there are no bird nests through the Gaussian Bayes classifier;

[0034] Obtaining the Gaussian distribution coefficients when there are bird nests is equivalent to obtaining the standard deviation of each pixel.

[0035] The standard deviation σ i,c for:

[0036]

[0037] Where m is the number of sampling points, x i,c This represents the weight value of the sampled pixel at a distance i from the current point. It is the weighted average of the surrounding pixels;

[0038] Obtaining the Gaussian distribution coefficients when there are no bird nests is equivalent to obtaining the expected value of each pixel.

[0039] The expected value μ i,c for:

[0040]

[0041] Where, x m,c This is the weight value of the sampled pixel at a distance of m from the current point.

[0042] Optionally, the bird's nest recognition model is:

[0043]

[0044] In this context, a value greater than 0 indicates the presence of a bird's nest, while a value less than 0 indicates the absence of a bird's nest. It has a bird's nest type. For the type without bird nests, m is the number of sampling points, x i This represents the i-th sampling point, where i is the number and order of the sampling points, and T is the judgment model symbol.

[0045] Compared with the prior art, the present invention has the following advantages and technical effects:

[0046] This invention is the first to employ a low-cost, low-power IoT edge microcontroller to achieve real-time bird nest detection based on machine learning and edge computing. Most existing solutions rely on drone image transmission technology for cloud processing, which makes real-time detection difficult. This invention uses an IoT microcontroller unit, which is low-cost and low-power, and its functional modules can be directly installed on drones, enabling real-time online detection.

[0047] Discrete wavelet transform is used to perform three transformations on the captured image to extract macroscopic and texture features. Macroscopic color and contour information, as well as texture information of the tree branches inside the bird's nest, are extracted separately. Then, machine learning classifiers are used to identify the macroscopic and microscopic information respectively. This method can reduce the actual amount of information input and has high detection accuracy. Attached Figure Description

[0048] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0049] Figure 1 This is a schematic diagram of the overhead contact line bird nest identification system based on a low-power IoT edge computing device, according to an embodiment of the present invention. Detailed Implementation

[0050] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0051] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0052] Example

[0053] like Figure 1 As shown, this embodiment provides a contact wire bird nest identification system based on a low-power IoT edge computing device, including: an input module, a feature extraction module, a model building module, and an output module;

[0054] The input module is used to input the original image to the feature extraction module, wherein the original image includes: an image of high-voltage transmission equipment and an image of the overhead contact line;

[0055] The feature extraction module is used to extract features from the original image and obtain feature information;

[0056] The model building module is used to build a bird nest recognition model based on the feature information;

[0057] The output module is used to output the bird nest recognition result of the original image according to the bird nest recognition model.

[0058] Furthermore, the feature extraction module includes: a first extraction unit and a second extraction unit;

[0059] The first extraction unit is used to extract macroscopic feature information of the bird's nest in the original image;

[0060] The second extraction unit is used to extract the microscopic texture information of the bird's nest in the original image;

[0061] The first extraction unit and the second extraction unit are respectively connected to the model building module.

[0062] Furthermore, the macroscopic feature information includes: the color feature information and the outline feature information of the bird's nest.

[0063] Furthermore, the first extraction unit extracts the color feature information using a one-dimensional discrete wavelet transform.

[0064] The first extraction unit extracts the contour feature information using a two-dimensional discrete wavelet transform.

[0065] Furthermore, the micro-texture information includes: texture information of the details of the branches inside the bird's nest.

[0066] Furthermore, the method for constructing the bird nest recognition model by the model building module includes: training and validating a support vector machine or a Gaussian Bayes classifier using the feature information to obtain the bird nest recognition model.

[0067] Furthermore, the process of constructing a model using the Gaussian Bayes classifier includes: obtaining the Gaussian distribution coefficients of images with and without bird nests using the Gaussian Bayes classifier;

[0068] Obtaining the Gaussian distribution coefficients when there are bird nests is equivalent to obtaining the standard deviation of each pixel.

[0069] Obtaining the Gaussian distribution coefficients when there are no bird nests is equivalent to obtaining the expected value of each pixel.

[0070] In specific embodiments of the present invention, the following system functions are implemented:

[0071] 1. Image Input Module

[0072] The image input module uses the OV2640 integrated into the ESP32S3, or inputs image data streams into the ESP32-S3 via SPI or UART interfaces. In this embodiment, the ESP-S3-eye comes with an OV2640 camera, which, when used with the ESP-WHO, allows for the retrieval of camera data.

[0073] 2. Multi-level detail feature extraction module based on discrete wavelet transform

[0074] The discrete wavelet transform module performs three transformations on the original image to obtain macroscopic information LL3 and detail information HH1, HH2, and HH3, respectively. According to the principle of discrete wavelet transform, the one-dimensional discrete wavelet transform formula is:

[0075]

[0076]

[0077] Written in matrix form:

[0078] A j+1 =H j A j

[0079] D j+1 =G j A j

[0080] Among them, A j+1 H is an approximation of the (j+1)th order signal. j A is the high-pass component of the j-th order signal. j D is an approximation of the j-th order signal. j+1G represents the detail values ​​of the (j+1)th order signal. j A is the low-pass component of the j-th order signal. j It is an approximation of the j-th order signal.

[0081] The two-dimensional wavelet transform can be written as:

[0082] A i,j+1 =H j A i,j

[0083] D i,j+1 =G j A i,j

[0084] A i+1,j+1 =H i A i,j+1

[0085] D i+1,j+1 =G j A i,j+1

[0086] Among them, A i,j+1 A is an approximation of the horizontal order i and the vertical order j+1. i,j D is an approximation of the horizontal order i and the vertical order j signal. i,j+1 For the detail values ​​of the horizontal order i and the vertical order j+1, A i,j A is an approximation of the horizontal order i and the vertical order j signal. i+1,j+1 D is an approximation of the horizontal (i+1)th order and the vertical (j+1)th order signal. i+1,j+1 G represents the detail values ​​of the horizontal (i+1)th order and the vertical (j+1)th order signal. j Let be the low-pass component of the j-th order signal.

[0087] For the simplest db0 wavelet, i.e., the haar wavelet, h[n] and g[n] are defined as follows:

[0088]

[0089]

[0090] Where h[n] is a high-pass filter, g[n] is a low-pass filter, and n is the length.

[0091] The pywt module of Python was used to perform db0 wavelet processing on the bird's nest on the iron tower, and three wavelet images were obtained in succession to obtain the texture details of the bird's nest.

[0092] In this embodiment, wavelet transform is implemented using haar.c and haar.h, employing the simplest db0 wavelet, i.e., the haar wavelet. During the transform process, only the low-frequency image LL and the high-frequency diagonal detail image HH are calculated. Then, wavelet transform is performed on LL to obtain LL2 and HH2. For images with a resolution greater than 800*600, a third discrete wavelet transform is used; for images with a resolution less than or equal to 640*480, a second wavelet transform is used.

[0093] 3. Machine learning classifier training and judgment module (i.e., model building module)

[0094] The machine learning training module is used to train model parameters on a PC. Alternatively, it can be migrated to ESPS3 for local training using images captured by the OV2640 camera, directly obtaining model parameters.

[0095] This embodiment uses a Gaussian Bayes classifier, the principle of which is as follows:

[0096] Suppose an image belongs to a certain type (with or without a bird's nest), and the distribution of the values ​​of each pixel xi in the image follows a Gaussian distribution, that is:

[0097]

[0098] Where simga_i and miu_i are the expected value and standard deviation of the i-th pixel, and the subscript c represents the type, which includes two types: bird's nest and no bird's nest. The judgment formula (i.e., the bird's nest recognition model) is as follows:

[0099]

[0100] If T is greater than 1, it is determined that there is a bird's nest; if T is less than 1, it is determined that there is no bird's nest. Since calculating the product in the MCU consumes a lot of memory, both sides are logarithmized, resulting in:

[0101]

[0102] Since i >> 1, and the ratio of the number of samples in the two classes is usually no greater than 10, i.e., logT = logp(c Trie )-logp(c_False) is much smaller than the third term, therefore it simplifies to:

[0103]

[0104] In this context, a value greater than 0 indicates the presence of a bird's nest, while a value less than 0 indicates the absence of a bird's nest. It has a bird's nest type. For the type without bird nests, m is the number of sampling points, x i This represents the i-th sampling point, where i is the number and order of the sampling points, and T is the judgment model symbol.

[0105] The Gaussian Bayes classifier only needs to calculate logT to classify images and determine whether it is greater than 0. If it is greater than 0, it is considered that there is a bird's nest; if it is less than 0, it is considered that there is no bird's nest.

[0106] The training process for Gaussian Bayes is also quite simple: it involves calculating the Gaussian distribution coefficients for two types of scenarios, with and without bird nests, which is the standard deviation σ of each pixel's value. i,c_True σ i,c_False And mathematical expectation value μ i,c_True μ i,c_False ,Right now

[0107]

[0108]

[0109] Where m is the number of sampling points, x i,c This represents the weight value of the sampled pixel at a distance i from the current point. x is the weighted average of the surrounding pixels. m,c This is the weight value of the sampled pixel at a distance of m from the current point.

[0110] The implementation process of the machine learning training module in this embodiment can be implemented on a personal computer using the pandas library in Python, and after training on the PC to obtain model parameters, it can be transferred to the ESP32-s3. Alternatively, it can be trained locally on the ESP32-s3 using the built-in OV2640 camera.

[0111] 4. Model Parameter Import Module

[0112] There are two methods for importing the model: importing the trained σ... i,c_True ,σ i,c_False μ i,c_True ,μ i,c_R The data is stored as a CSV file on an SD card and read multiple times during program execution. This method can reduce memory usage.

[0113] The second method is to directly generate a .hpp file and define a static global variable in the file (since esp32s3 may update model parameters through local training, it is not defined as a constant) and include the file during program compilation. However, this method consumes too much memory when processing high-resolution images, which can cause the program to fail to compile.

[0114] This embodiment first implements a Gaussian Bayes classifier in Python, then rewrites it in C, and debugs it on a computer using the Linux subsystem make and the g++ compiler.

[0115] This embodiment is the first to employ a low-cost, low-power IoT edge microcontroller to achieve real-time bird nest detection based on machine learning and edge computing. Most existing solutions use drone image transmission technology for cloud processing, which makes real-time detection difficult. This solution uses an IoT microcontroller unit, which is low-cost and low-power, and the functional modules can be directly installed on the drone, enabling real-time online detection.

[0116] This embodiment employs discrete wavelet transform to perform three transformations on the captured image, extracting macroscopic and texture features. Macroscopic color and contour information, as well as texture information detailing the branches within the bird's nest, are extracted separately. Then, machine learning classifiers are used to identify the macroscopic and microscopic information respectively. This method reduces the actual amount of input information and achieves high detection accuracy.

[0117] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A catenary bird nest identification system based on a low-power Internet of Things edge computing device, characterized in that, The system comprises an input module, a feature extraction module, a model construction module and an output module. The input module is configured to receive an original image and transmit the original image to the feature extraction module, wherein the original image comprises an image of high-voltage power transmission equipment and an image of a catenary. The feature extraction module is configured to perform feature extraction on the original image to obtain feature information. The model construction module is configured to construct a bird nest recognition model based on the feature information. The output module is configured to output a bird nest recognition result of the original image according to the bird nest recognition model. The feature extraction module comprises a first extraction unit and a second extraction unit. The first extraction unit is configured to extract macro feature information of a bird nest in the original image. The second extraction unit is configured to extract micro texture information of the bird nest in the original image. The first extraction unit and the second extraction unit are connected to the model construction module. The macro feature information comprises color feature information and contour feature information of the bird nest. The color feature information is extracted by one-dimensional discrete wavelet transform in the first extraction unit. The contour feature information is extracted by two-dimensional discrete wavelet transform in the first extraction unit. The texture information of branch details in the bird nest is extracted by db0 wavelet processing in the second extraction unit. The model construction module constructs the bird nest recognition model by training and verifying a Gaussian Bayesian classifier based on the feature information. The Gaussian Bayesian classifier is used to construct a model by obtaining Gaussian distribution coefficients when there is a bird nest and when there is no bird nest in an image. The Gaussian distribution coefficient when there is a bird nest is obtained by obtaining a standard deviation of each pixel point. The Gaussian distribution coefficient when there is no bird nest is obtained by obtaining a mathematical expectation value of each pixel point. The micro texture information comprises texture information of branch details in the bird nest. The bird nest recognition model is as follows: wherein, greater than 0 is considered to have a bird nest, less than 0 is considered to have no bird nest, is a bird nest type, is a no bird nest type, is the number of sampling points, represents the i-th sampling point, i is the number order of the sampling point, is a judgment model symbol.

2. The catenary bird nest recognition system based on a low-power Internet of Things edge computing device according to claim 1, wherein the one-dimensional discrete wavelet transform is as follows:

3. The catenary bird nest recognition system based on a low-power Internet of Things edge computing device according to claim 2, wherein the two-dimensional discrete wavelet transform is as follows: wherein is an approximation of the j+1 order signal, is a high-pass component of the j order signal, is an approximation of the j order signal, is a detail value of the j+1 order signal, is a low-pass component of the j order signal.

4. The catenary bird nest recognition system based on a low-power Internet of Things edge computing device according to claim 1, wherein the db0 wavelet processing is as follows:

5. The catenary bird nest recognition system based on a low-power Internet of Things edge computing device according to claim 1, wherein wherein is an approximation of the transversal i-th longitudinal j+1 -th signal, is an approximation of the transversal i-th longitudinal j-th signal, is a detail of the transversal i-th longitudinal j+1 -th signal, is an approximation of the transversal i+1 -th longitudinal j+1 -th signal, is a detail of the transversal i+1 -th longitudinal j+1 -th signal, is a low-pass component of the j-th signal.

6. The catenary bird nest recognition system based on a low-power Internet of Things edge computing device according to claim 1, wherein ​ wherein is a high pass filter, is a low pass filter, is a length. ​ The standard deviation is: wherein, is the number of sampling points, is the weight value of the sampling pixel point with the distance i from the current point, is the weighted average value of the surrounding pixels. ​ The mathematical expectation value is: wherein, is the weight value of the sampling pixel point and the current point distance m.

Images