A method and apparatus for image recognition of small components of power transmission line towers

Abstract

This invention provides a method and apparatus for image recognition of small components of transmission line towers. The method includes acquiring pre-annotated image information of transmission line tower components and constructing an image dataset; performing data augmentation and data balancing on the image dataset to obtain a preprocessed image dataset; performing clustering operations on the preprocessed image dataset to obtain predicted anchor frame sizes; constructing a convolutional neural network and training the convolutional neural network using the preprocessed image dataset to obtain a network recognition model; identifying components in the currently acquired transmission line tower images and transmitting image data of identified defective components back to the operation and maintenance center. This invention addresses the problems of existing technologies, such as wasted human resources, low efficiency of manual inspections, difficulty in deploying algorithm models, inability to meet the needs of the power industry, and various other issues with existing image recognition algorithms.

Description

Technical Field

[0001] This invention relates to the field of intelligent air conditioning control technology, specifically to an image recognition method for small components of power transmission line towers and an apparatus applied to the method. Background Technology

[0002] In recent years, the contradictions of insufficient personnel for power transmission line inspection, low efficiency of manual inspection, large scale of power transmission lines and large number of small components have become increasingly prominent. Traditional inspection methods can no longer meet the needs of the power industry's development, and intelligent inspection of small components of power transmission line towers is receiving increasing attention from the power industry.

[0003] Currently, most power companies only use monitoring devices to perform image capture tasks, while the determination of defects in small components is still done manually, which is far from meeting the current inspection needs. There is a need for an image recognition algorithm that can be deployed to monitoring devices to identify and process small components of transmission line towers in real time. However, in the process of researching and applying existing technologies, it has been found that various image recognition algorithms proposed by existing technologies have problems such as large model parameters and difficulty in deployment, and cannot be applied in actual inspections. Summary of the Invention

[0004] This invention provides a method and apparatus for image recognition of small components of power transmission line towers, which solves the problems existing in the current technology, such as waste of human resources, low efficiency of manual inspection, difficulty in deploying algorithm models, inability to meet the needs of the power industry, and various problems existing in the current image recognition algorithm.

[0005] In a first aspect, the present invention provides an image recognition method for small components of transmission line towers, the method comprising:

[0006] Acquire pre-annotated image information of transmission line tower components and construct an image dataset;

[0007] The image dataset is augmented and balanced to obtain a preprocessed image dataset.

[0008] Clustering is performed on the preprocessed image dataset, and the number of anchor boxes is determined by the preset number of clusters to obtain the predicted anchor box size.

[0009] A convolutional neural network is constructed, and the convolutional neural network is trained using a preprocessed image dataset to obtain a network recognition model for identifying components of power transmission line towers.

[0010] Based on the network recognition model, component identification is performed on the currently acquired images of transmission line towers, and the image data of the identified defective components is transmitted back to the operation and maintenance center.

[0011] According to the present invention, a method for image recognition of small components of transmission line towers includes obtaining image information of pre-annotated transmission line tower components, which includes:

[0012] The panoramic image of the transmission line collected during the inspection is cropped into a partial image that includes small components of the transmission line tower. The small components of the transmission line tower include normal small components and defective small components.

[0013] In local images, annotation boxes are generated to mark the category and location of the small components of the transmission line tower, and the annotation box information of each local image is saved in an annotation file.

[0014] According to the present invention, a method for image recognition of small components of transmission line towers includes data augmentation and data balancing of the image dataset, comprising:

[0015] Data augmentation of the image dataset: Based on the actual shooting situation of the monitoring device, a data augmentation algorithm is used to generate corresponding annotation files;

[0016] Data balancing of the image dataset: The number of bounding boxes for each type of component in the augmented image dataset is statistically analyzed. Images of components with fewer bounding boxes are further augmented, while images of components with more bounding boxes are appropriately removed, so that the number of bounding boxes for each type of component remains consistent, resulting in the preprocessed image dataset.

[0017] According to the present invention, a method for image recognition of small components of transmission line towers is provided, which uses the K-means clustering algorithm to perform clustering operations on the preprocessed image dataset, including:

[0018] The number of cluster categories is set according to the number of anchor boxes defined in the output network of the convolutional neural network, and the cluster centers are randomly initialized. ;

[0019] For each sample data point of the preprocessed image dataset The distance between each cluster center and the sample data points is calculated according to the Euclidean distance formula, and the sample data that are closer are assigned to the class where the cluster center is located.

[0020] For each category, determine whether the iteration termination condition is met. If it is met, output the cluster label; otherwise, update the cluster center of that category, as shown in formula (1):

[0021] (1)

[0022] in, Represents clusters Cluster center in This indicates the total number of sample data points in this category;

[0023] Then, the values ​​of the cluster centers are continuously updated until the positions of the cluster centers no longer change or the set number of iterations is reached, thereby obtaining the predicted anchor frame size.

[0024] According to the present invention, a method for image recognition of small components of transmission line towers includes, wherein the construction of a convolutional neural network comprises:

[0025] The input network uses PCNet to process the input image through a feature channel with a value of 1. The convolution kernel reduces the number of parameters in subsequent convolution operations, and then features with 1 feature channel are used in parallel. Convolution kernel with 2 feature channels Convolution kernel with 3 feature channels Convolution kernel with 4 feature channels Convolution kernel with 5 feature channels Convolutional kernels extract contrast information from widget images and fuse multiple receptive fields;

[0026] The backbone network employs a lightweight inverted residual structure, replacing traditional convolutional kernels with a bottleneck structure of pointwise convolution-depthmial-pointwise convolution. It also performs dimensionality upscaling followed by dimensionality reduction on feature channels, enabling feature extraction to occur in a high-dimensional space. The feature fusion stage utilizes SPP-tiny, which, based on the SPP module, retains only skip connections and... The max pooling layer is then used, and a channel attention mechanism is employed to assign appropriate weights to the feature channels, thereby achieving the fusion of local and global features.

[0027] The output network uses the concept of FPN feature pyramid structure for multi-scale prediction, setting two scales of information, and setting three anchor boxes for each scale for regression analysis.

[0028] According to the present invention, a method for image recognition of small components of transmission line towers includes training a convolutional neural network using a preprocessed image dataset, comprising:

[0029] The preprocessed image dataset is divided into a training set and a validation set according to a certain ratio. The training set is used to train the constructed convolutional neural network to obtain a convolutional neural network model. The validation set is used to select the convolutional neural network model with the best parameters from the trained convolutional neural network models.

[0030] The background information of the object to be identified is enriched by using the mosaic data augmentation method. Several images are randomly selected from the divided training set, randomly scaled, cropped and then stitched to form a new image. The new image and the training set images are input into the constructed convolutional neural network in batches to perform convolutional pooling operations to obtain the preliminary output prediction results.

[0031] The output network of the convolutional neural network is used to calculate the loss of the initial output prediction results and construct the loss function. The convolution kernel parameters of the convolutional neural network model are iteratively updated using the backpropagation algorithm. Training stops when the set number of iterations is reached. The convolutional neural network model with the optimal parameters is obtained by screening based on the validation set.

[0032] According to the present invention, an image recognition method for small components of transmission line towers is provided. After selecting the convolutional neural network model with optimized parameters, a pruning method is used to compress the model, and the scaling factor of the batch normalized BN layer is utilized. Multiplying by the output channel determines the importance of the convolutional layer's output channel; a scaling factor is added to the loss function. The regularization term is sparsely trained as shown in formula (2):

[0033]

[0034] in, The loss function of the model. As a sparsity factor, For sparse regularization, for A set; the learning rate during sparse training. and sparsity factor Adjustments are made to achieve better sparsity by adjusting the scaling factor in the sparsely trained network model. Output channels approaching zero are pruned, and after fine-tuning the pruning, a network identification model for recognizing small components of power transmission line towers is obtained.

[0035] Secondly, the present invention also provides an image recognition device for small components of transmission line towers, comprising:

[0036] The storage module is used to deploy a network identification model for identifying components of transmission line towers, and to store images of transmission line towers collected by the monitoring device.

[0037] The recognition module is used to input the collected images of transmission line towers into the network recognition model to obtain the recognition results of the corresponding transmission line tower components;

[0038] The positioning module is used to locate the monitoring device and acquire the location information of the transmission line tower components corresponding to the collected images of the transmission line tower.

[0039] The transmission module is used to compress and transmit image data and location information of identified defective parts in real time.

[0040] Compared with the prior art, the beneficial effects of the present invention include the following:

[0041] 1. The convolutional neural network constructed in this invention has a significantly smaller model size compared to existing image recognition algorithms, solving the problems of large model parameter quantity and difficulty in deployment on monitoring devices.

[0042] 2. This invention uses a pruning method to compress the convolutional neural network model with optimized parameters, further reducing the number of model parameters, improving the real-time performance of recognition, and reducing the power consumption of the monitoring device.

[0043] 3. This invention is the first to use PCNet, which overcomes the problem of difficulty in identifying small targets in complex backgrounds in existing technologies.

[0044] 4. This invention is the first to adopt the lightweight inverted residual structure Ires, which can appropriately deepen the network to better extract deep features, while overcoming the drawback of the original inverted residual structure that cannot perform skip connections when the feature map size is changed, thus improving the reusability of features.

[0045] Thirdly, the present invention also provides an electronic device, comprising:

[0046] Memory, which stores computer-executable instructions;

[0047] The processor is configured to run computer-executable instructions.

[0048] The computer-executable instructions are executed by the processor to implement the steps of the above-described method for image recognition of small components of transmission line towers.

[0049] Fourthly, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described image recognition method for small components of transmission line towers.

[0050] Therefore, the present invention provides an electronic device and storage medium for image recognition of transmission line tower components, comprising: one or more memories and one or more processors. The memories are used to store program code and intermediate data generated during program execution, storage of model output results, and storage of the model and model parameters; the processors are used for the processor resources occupied by the code execution and multiple processor resources occupied during model training.

[0051] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0052] Figure 1This is a flowchart of an embodiment of the image recognition method for small components of power transmission line towers according to the present invention.

[0053] Figure 2 This is a schematic diagram of the structure of the convolutional neural network built in an embodiment of the image recognition method for small components of power transmission line towers according to the present invention.

[0054] Figure 3 This is a schematic diagram of the structure of the input network PCNet in an embodiment of the image recognition method for small components of power transmission line towers according to the present invention.

[0055] Figure 4 This is a schematic diagram of the lightweight inverted residual structure Ires in an embodiment of the image recognition method for small components of power transmission line towers according to the present invention.

[0056] Figure 5 This is a comparison diagram of the SPP module and the SPP-tiny module in an embodiment of the image recognition method for small components of a power transmission line tower according to the present invention.

[0057] Figure 6 This is a schematic diagram of an embodiment of an image recognition device for small components of power transmission line towers according to the present invention. Detailed Implementation

[0058] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0060] First, some of the terms and related technologies used in this application will be explained in conjunction with the accompanying drawings to facilitate understanding by those skilled in the art.

[0061] K-means clustering algorithm: Also known as k-means clustering, K-means clustering is a simple and classic distance-based clustering algorithm. It uses distance as the evaluation metric, assuming that the closer two objects are, the greater their similarity. This algorithm considers clusters to be composed of objects that are close to each other, thus aiming to obtain compact and independent clusters as its ultimate goal.

[0062] Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that include convolutional computations and have a deep structure. CNNs have representation learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure.

[0063] Receptive field: The receptive field is one of the important concepts in convolutional neural networks. It is defined as the area of ​​the input image that the features of the convolutional neural network can see. In other words, the feature output is affected by the pixels within the receptive field.

[0064] Transmission line tower components: including normal small components and defective small components.

[0065] An embodiment of an image recognition method for small components of power transmission line towers:

[0066] See Figure 1 A method for image recognition of small components of power transmission line towers, comprising the following steps:

[0067] Step S1: Obtain image information of pre-annotated transmission line tower components and construct an image dataset.

[0068] Step S2 involves data augmentation and data balancing of the image dataset to obtain a preprocessed image dataset.

[0069] Step S3: Perform clustering operation on the preprocessed image dataset, output the preset number of anchor boxes to determine the number of cluster categories, and obtain the predicted anchor box size.

[0070] Step S4: Convolutional neural network is built, and the convolutional neural network is trained using the preprocessed image dataset to obtain a network recognition model for identifying components of power transmission line towers.

[0071] Step S5: Based on the network recognition model, component identification is performed on the currently acquired images of transmission line towers, and the image data of identified defective components is transmitted back to the operation and maintenance center. Specifically, the network recognition model for identifying small components of transmission line towers is deployed to the monitoring device installed on the tower. The monitoring device's camera captures images of the transmission line towers, and small component identification is performed. Image data of identified defective small components is then transmitted back to the operation and maintenance center.

[0072] In step S1 above, obtaining the image information of the transmission line tower components that have been pre-annotated includes:

[0073] The panoramic image of the transmission line collected during the inspection is cropped into a partial image that includes small components of the transmission line tower. These small components include normal components and defective components.

[0074] In local images, annotation boxes are generated to mark the category and location of the small components of the transmission line tower, and the annotation box information of each local image is saved in an annotation file.

[0075] In step S2 above, the data augmentation and data balancing of the image dataset includes:

[0076] Image dataset augmentation: Based on the actual shooting conditions of the monitoring device, data augmentation algorithms are used to generate corresponding annotation files. Specifically, image dataset augmentation employs methods such as small-angle rotation, brightness adjustment, blurring, and sliding window slicing algorithms based on the actual shooting conditions of the monitoring device, and then generates corresponding annotation files.

[0077] Data balancing of the image dataset: The number of bounding boxes for each type of component in the augmented image dataset is statistically analyzed. Images of components with fewer bounding boxes are further augmented, while images of components with more bounding boxes are appropriately removed, so that the number of bounding boxes for each type of component remains consistent, resulting in the preprocessed image dataset.

[0078] In step S3 above, the K-means clustering algorithm is used to cluster the preprocessed image dataset. This requires determining the number of clusters based on the number of anchor boxes set by the convolutional neural network model output, and then predicting the anchor box size. This includes:

[0079] The number of cluster categories is set according to the number of anchor boxes defined in the output network of the convolutional neural network, and the cluster centers are randomly initialized. ;

[0080] For each sample data point of the preprocessed image dataset The distance between each cluster center and the sample data points is calculated according to the Euclidean distance formula, and the sample data that are closer are assigned to the class where the cluster center is located.

[0081] For each category, determine whether the iteration termination condition is met. If it is met, output the cluster label; otherwise, update the cluster center of that category, as shown in formula (1):

[0082] (1)

[0083] in, Represents clusters Cluster center in This indicates the total number of sample data points in this category;

[0084] Then, the values ​​of the cluster centers are continuously updated until the positions of the cluster centers no longer change or the set number of iterations is reached, thereby obtaining the predicted anchor frame size.

[0085] In step S4 above, the construction of the convolutional neural network is divided into three parts: the input network, the backbone network, and the output network, including:

[0086] The input network uses PCNet to process the input image through a feature channel with a value of 1. The convolution kernel reduces the number of parameters in subsequent convolution operations, and then features with 1 feature channel are used in parallel. Convolution kernel with 2 feature channels Convolution kernel with 3 feature channels Convolution kernel with 4 feature channels Convolution kernel with 5 feature channels Convolutional kernels extract contrast information from widget images and fuse multiple receptive fields. The input network employs a self-built parallel convolutional structure, PCNet, to fuse multiple receptive fields.

[0087] The backbone network employs a lightweight inverted residual structure, replacing traditional convolutional kernels with a bottleneck structure of pointwise convolution-depthmial-pointwise convolution. It also performs dimensionality upscaling followed by dimensionality reduction on feature channels, enabling feature extraction to occur in a high-dimensional space. The feature fusion stage utilizes SPP-tiny, which, based on the SPP module, retains only skip connections and... The network employs a max-pooling layer, followed by a channel attention mechanism to assign appropriate weights to the feature channels, thereby fusing local and global features. The backbone network primarily utilizes a lightweight inverted residual structure and an improved spatial pyramid pooling module (SPP-tiny) for feature extraction.

[0088] The output network employs the feature pyramid structure of the Feature Pyramid Network (FPN) for multi-scale prediction, setting two scales and three anchor boxes for each scale for regression analysis. Specifically, the output network uses multi-scale output to set multiple anchor boxes for prediction. The convolutional neural network is trained using a preprocessed image dataset to obtain a network recognition model for identifying small components of power transmission line towers.

[0089] In step S4 above, training the convolutional neural network using the preprocessed image dataset includes:

[0090] The preprocessed image dataset is divided into a training set and a validation set in a 9:1 ratio. The training set is used to train the constructed convolutional neural network to obtain a convolutional neural network model. The validation set is used to select the convolutional neural network model with the best parameters from the trained convolutional neural network models.

[0091] Mosaic data augmentation is used to enrich the background information of objects being identified. Several images are randomly selected from the training set, randomly scaled, cropped, and then stitched together to form new images. These new images and training set images are then input into a constructed convolutional neural network in batches for convolutional pooling operations to obtain preliminary output prediction results. Specifically, a batch of images is taken from the training set, and several images are randomly selected (in this case, four images are randomly selected) for random scaling and cropping. The processed images are then stitched together according to the top left, top right, bottom left, and bottom right directions to form new images. This process is repeated, and the sizes of the new images and training set images are adjusted accordingly. The size of the input is determined by batching the inputs into the constructed convolutional neural network, which then undergoes convolutional pooling operations to obtain preliminary output prediction results.

[0092] The output network of the convolutional neural network is used to calculate the loss of the initial output prediction results and construct the loss function. The convolution kernel parameters of the convolutional neural network model are iteratively updated using the backpropagation algorithm. Training stops when the set number of iterations is reached. The convolutional neural network model with the optimal parameters is obtained by screening based on the validation set.

[0093] In this embodiment, after selecting the convolutional neural network model with optimized parameters, a pruning method is used to compress the model, and the scaling factor of the batch normalized BN layer is utilized. Multiplying by the output channel determines the importance of the convolutional layer's output channel; a scaling factor is added to the loss function. The regularization term is sparsely trained as shown in formula (2):

[0094] (2)

[0095] in, The loss function of the model. As a sparsity factor, For sparse regularization, for A set; the learning rate during sparse training. and sparsity factor Adjustments are made to achieve better sparsity by adjusting the scaling factor in the sparsely trained network model. The output channels close to 0 are pruned, and after fine-tuning the pruning, a network identification model for identifying small components of transmission line towers is obtained. The selected regularization term is L1 regularization, and the above formula can be rewritten as the following formula (3):

[0096] (3)

[0097] In this embodiment, the transmission line tower components are small transmission line tower components, which include normal small components and defective small components. Normal small components include ordinary bolts, overhead bolts, vibration dampers, etc., while defective small components include overhead bolt detachment, vibration damper collision, and missing vibration dampers, etc. Obtaining images of small transmission line tower components involves cropping the panoramic image of the transmission line collected during inspection to create a local image containing the small transmission line tower components. In the local image, Labelimg software is used to annotate the category and location of the tower components to generate annotation boxes, and small component type labels are added. The type labels should be expressed in English. A local image may contain one or more small components that need to be annotated. The annotation box information of each local image is saved in an annotation file, which is automatically generated when a local image is saved after being annotated using Labelimg software.

[0098] In this embodiment, data augmentation of the image dataset is performed using augmentation methods such as small-angle rotation, brightness change, blurring, and sliding window slicing algorithms based on the actual shooting situation of the monitoring device, and corresponding annotation files are generated. Data balancing of the image dataset is performed by statistically analyzing the number of bounding boxes for each type of small part in the augmented image dataset based on the data from all annotation files. Images of small parts with fewer bounding boxes are further augmented, while images of small parts with more bounding boxes are appropriately reduced, so that the number of bounding boxes for each type of small part is approximately the same, resulting in a preprocessed image dataset.

[0099] For example, this embodiment uses a sliding window slicing algorithm for data augmentation, specifically including:

[0100] The sliding window size is determined based on the statistical distribution of width and height and the quantity histogram of the categories in the annotation file;

[0101] For the image dataset, slide slices sequentially from left to right and from top to bottom using a sliding window with specified width, height, and step size. The overlap rate is generally controlled between 0.1 and 0.5.

[0102] During the slicing process, some labeled target boxes will be segmented by a sliding window. The intersection ratio between the target box and the sliding window is calculated, and target boxes with an intersection ratio greater than 0.6 are retained. The size of the target box is then limited to the sliding window area.

[0103] In this embodiment, the anchor frame size is predicted using the K-means clustering algorithm, specifically including:

[0104] S31. Based on the number of anchor boxes set by the output network of the convolutional neural network model, set the number of clusters to 6, and randomly initialize 6 cluster centers. ;

[0105] S32. For each sample data point in the dataset The distance between each cluster center and the sample data points is calculated according to the Euclidean distance formula, and the sample data that are closer are assigned to the class where the cluster center is located.

[0106] S33. For each category, determine whether the iteration termination condition is met. If it is met, output the cluster label. If it is not met, update the cluster center of the category using the following formula, expressed as formula (1).

[0107] S34. Repeat steps S32 and S33 to update the values ​​of the cluster centers until the positions of the cluster centers no longer change or the set number of iterations is reached, and obtain the predicted anchor frame size.

[0108] In this embodiment, a convolutional neural network is constructed, as follows: Figure 2 As shown, it specifically includes:

[0109] The input network uses PCNet, such as Figure 3 As shown, the input image is first processed through a feature channel with a value of 1. The convolution kernel (Conv) reduces the number of parameters in subsequent convolution operations, better extracting detailed information, and then uses features with 1 feature channel in parallel. Convolution kernel with 2 feature channels Convolution kernel with 3 feature channels Convolution kernel with 4 feature channels Convolution kernel with 5 feature channels Convolutional kernels extract contrast information from small components and fuse multiple receptive fields.

[0110] The backbone network can be divided into two parts: the feature extraction stage and the feature fusion stage.

[0111] The feature extraction stage employs a lightweight inverted residual block (IRS) structure, such as... Figure 4As shown, replacing the traditional convolution kernel with a bottleneck structure of pointwise convolution (PWConv)-depth convolution (DWConv)-pointwise convolution can significantly reduce the time and space complexity of convolution operations. Furthermore, the feature channels are first increased in dimensionality and then decreased in dimensionality, allowing feature extraction to be performed in a high-dimensional space. Using the ReLU6 activation function after the first pointwise and depth convolution effectively increases non-linearity in the high-dimensional space. The use of a linear operation after the second pointwise convolution is because performing non-linear operations in a low-dimensional space would damage the features. Adding a convolution kernel before the bottleneck structure appropriately deepens the network to better extract deeper features, while overcoming the limitation of the original inverted residual structure in not being able to perform skip connections when the feature map size changes, thus improving feature reusability.

[0112] The SPP-tiny used in the feature fusion stage, such as Figure 5 As shown, the left side represents the structural diagram of the SPP module, and the right side represents the structural diagram of the SPP-tiny module. Only jump connections and... are retained based on the SPP module. Max pooling has a significant impact on the receptive field, mainly depending on the size of the pooling window. Therefore, using a large pooling window can greatly increase the receptive field. Experiments show that using the SPP-tiny module and the SPP module has almost the same effect, while reducing the computational cost. Then, a channel attention mechanism (SE block) is used to assign appropriate weights to the feature channels to achieve the fusion of local and global features.

[0113] The output network employs the Feature Pyramid Network (FPN) structure for multi-scale prediction. Considering the number of model parameters and the target size, two scales are set, with three anchor boxes for each scale for regression analysis, resulting in a total of six anchor box sizes. The anchor box sizes predicted by K-means are shown in Table 1.

[0114] Taking Table 1 as an example, when the input image size is set to... When the size is small, the scale information obtained is as follows: and Scale information is The feature map has a large receptive field, which can be used to identify small parts that are relatively large, such as vibration dampers. The scale information is... The feature map has a small receptive field, which can be used to identify small parts such as bolts. In practical applications, the scale information should be adjusted accordingly to meet the actual needs.

[0115] Table 1 Anchor Frame Size Correspondence Table

[0116]

[0117] An embodiment of an image recognition device for small components of power transmission line towers:

[0118] In this embodiment, as Figure 6 As shown, the present invention also provides an image recognition device for small components of transmission line towers, comprising:

[0119] The storage module is used to deploy a network identification model for identifying components of transmission line towers, and to store images of transmission line towers collected by the monitoring device.

[0120] Specifically, the storage module is used to deploy a network model of small components of transmission line towers on the GPU platform of the monitoring device, and temporarily store the images of transmission line towers captured by the camera of the monitoring device in the storage module.

[0121] The recognition module is used to input the collected images of transmission line towers into the network recognition model to obtain the recognition results of the corresponding transmission line tower components.

[0122] Specifically, the recognition module is used to preprocess the acquired images to a uniform size and input them into the network model of small components of the transmission line tower for recognition. For each small component in the image, multiple predicted anchor boxes are obtained. Non-maximum suppression is performed on the multiple predicted anchor boxes obtained for each small component to obtain the anchor box with the highest confidence. The anchor box with the highest confidence for each small component is converted into the corresponding coordinates of the original image, and the category, position and corresponding confidence of each small component on the original image are determined.

[0123] The positioning module is used to locate the monitoring device and acquire the location information of the transmission line tower components corresponding to the collected images of the transmission line tower.

[0124] Specifically, for the positioning module, when the camera of the monitoring device captures images of the transmission line tower, the monitoring device is positioned to obtain the location information corresponding to the captured images of the transmission line tower and the small components of the transmission line tower.

[0125] The transmission module is used to compress and transmit image data and location information of identified defective parts in real time.

[0126] Specifically, for the transmission module, the images and location information of the defective components are selected for real-time compression and transmission based on the component category. After the operation is completed, the corresponding files and images are cleared to release memory space.

[0127] In one embodiment, an electronic device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the electronic device provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the electronic device stores data. The network interface of the electronic device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for image recognition of small components of power transmission line towers.

[0128] Those skilled in the art will understand that the electronic device structure shown in this embodiment is only a partial structure related to the solution of this application and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in this embodiment, or combine certain components, or have different component arrangements.

[0129] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0131] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] Therefore, the present invention provides an electronic device and storage medium for image recognition of transmission line tower components, comprising: one or more memories and one or more processors. The memories are used to store program code and intermediate data generated during program execution, storage of model output results, and storage of the model and model parameters; the processors are used for the processor resources occupied by the code execution and multiple processor resources occupied during model training.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for image recognition of small components of transmission line towers, characterized in that, include: Acquire pre-annotated image information of transmission line tower components and construct an image dataset; The image dataset is augmented and balanced to obtain a preprocessed image dataset. Clustering is performed on the preprocessed image dataset, and the number of anchor boxes is determined by the preset number of clusters to obtain the predicted anchor box size. A convolutional neural network is constructed, and the convolutional neural network is trained using a preprocessed image dataset to obtain a network recognition model for identifying components of power transmission line towers. Based on the network recognition model, the system performs component recognition on the currently acquired images of transmission line towers and transmits the image data of the identified defective components back to the operation and maintenance center. The construction of the convolutional neural network includes: The input network processes the input image through a feature channel with a value of 1. The convolution kernel reduces the number of parameters in subsequent convolution operations, and then features with 1 feature channel are used in parallel. Convolution kernel, feature channels of 2 Convolution kernel, 3 feature channels Convolution kernel, 4 feature channels Convolution kernel, 5 feature channels Convolutional kernels extract contrast information from widget images and fuse multiple receptive fields; The backbone network employs a lightweight inverted residual structure, replacing traditional convolutional kernels with a bottleneck structure of pointwise convolution-depthmial-pointwise convolution. It also performs dimensionality upscaling followed by dimensionality reduction on feature channels, enabling feature extraction to occur in a high-dimensional space. The feature fusion stage utilizes SPP-tiny, which, based on the SPP module, retains only skip connections and... The max pooling layer is followed by a channel attention mechanism to assign appropriate weights to the feature channels, thereby achieving the fusion of local and global features. Among them, the SPP module is a spatial pyramid pooling module, and SPP-tiny is a lightweight spatial pyramid pooling module, which is an improved version of the SPP module. The output network uses the concept of FPN feature pyramid structure for multi-scale prediction, setting two scales of information, and setting three anchor boxes for each scale for regression analysis.

2. The method according to claim 1, characterized in that, The acquisition of image information pre-annotated for transmission line tower components includes: The panoramic image of the transmission line collected during the inspection is cropped into a partial image that includes small components of the transmission line tower. The small components of the transmission line tower include normal small components and defective small components. In local images, annotation boxes are generated to mark the category and location of the small components of the transmission line tower, and the annotation box information of each local image is saved in an annotation file.

3. The method according to claim 2, characterized in that, The data augmentation and data balancing of the image dataset includes: Data augmentation of the image dataset: Based on the actual shooting situation of the monitoring device, a data augmentation algorithm is used to generate corresponding annotation files; Data balancing of the image dataset: The number of bounding boxes for each type of component in the augmented image dataset is statistically analyzed. Images of components with fewer bounding boxes are further augmented, while images of components with more bounding boxes are appropriately removed, so that the number of bounding boxes for each type of component remains consistent, resulting in the preprocessed image dataset.

4. The method according to claim 1, characterized in that: The K-means clustering algorithm is used to cluster the preprocessed image dataset, including: The number of cluster categories is set according to the number of anchor boxes defined in the output network of the convolutional neural network, and the cluster centers are randomly initialized. ; For each sample data point of the preprocessed image dataset The distance between each cluster center and the sample data points is calculated according to the Euclidean distance formula, and the sample data that are closer are assigned to the class where the cluster center is located. For each category, determine whether the iteration termination condition is met. If it is met, output the cluster label; otherwise, update the cluster center of that category, as shown in formula (1): （1） in, Represents clusters Cluster center in This indicates the total number of sample data points in this category; Then, the values ​​of the cluster centers are continuously updated until the positions of the cluster centers no longer change or the set maximum number of iterations is reached, thereby obtaining the predicted anchor box size.

5. The method according to claim 1, characterized in that, The step of training the convolutional neural network using the preprocessed image dataset includes: The preprocessed image dataset is divided into a training set and a validation set according to a certain ratio. The training set is used to train the constructed convolutional neural network to obtain a convolutional neural network model; the validation set is used to select a convolutional neural network model from the trained convolutional neural network models. The background information of the object to be identified is enriched by using the mosaic data augmentation method. Several images are randomly selected from the divided training set, randomly scaled, cropped and then stitched to form a new image. The new image and the training set images are input into the constructed convolutional neural network in batches to perform convolutional pooling operations to obtain the preliminary output prediction results. The output network of the convolutional neural network is used to calculate the loss of the initial output prediction results and construct the loss function. The convolution kernel parameters of the convolutional neural network model are iteratively updated using the backpropagation algorithm. Training stops when the set number of iterations is reached, and the convolutional neural network model is obtained by filtering based on the validation set.

6. The method according to claim 5, characterized in that: After selecting the convolutional neural network (CNN) models, a pruning method was used to compress the models, and the scaling factor of the batch normalized (BN) layer was utilized. Multiplying by the output channel determines the importance of the convolutional layer's output channel; a scaling factor is added to the loss function. The regularization term is sparsely trained as shown in formula (2): （2） in, The loss function of the model. As a sparsity factor, For sparse regularization, for A set; the learning rate during sparse training. and sparsity factor Adjustments were made to the scaling factor in the sparsely trained network model. Output channels approaching zero are pruned, and after fine-tuning the pruning, a network identification model for recognizing small components of power transmission line towers is obtained.

7. An image recognition device for small components of transmission line towers, characterized in that, The device is applied to image recognition of small components of a transmission line tower as described in any one of claims 1 to 6, including: The storage module is used to deploy a network identification model for identifying components of transmission line towers, and to store images of transmission line towers collected by the monitoring device. The recognition module is used to input the collected images of transmission line towers into the network recognition model to obtain the recognition results of the corresponding transmission line tower components; The positioning module is used to locate the monitoring device and acquire the location information of the transmission line tower components corresponding to the collected images of the transmission line tower. The transmission module is used to compress and transmit image data and location information of identified defective parts in real time.

Images