A method for identifying power equipment based on a lightweight network and a related device

Abstract

The application provides a kind of identification method and related device of power equipment based on lightweight network, applied to computer vision technical field.The method comprises: determining sample data set, sample data set includes multiple sample images, and power equipment is included in sample image;Lightweight network model is modified to obtain the to-be-trained model;The to-be-trained model is trained using the sample data set to obtain the power equipment identification model;Collect to-be-identified image, and to-be-identified image is infrared image including power equipment;The to-be-identified image is identified using the power equipment identification model, and the identification result image is obtained, and the identification result image is the image that power equipment in to-be-identified image is marked out.Through the above method, the lightweight network model is modified, so that the model can detect power equipment in complex background, and based on lightweight network model, the operation amount can be effectively reduced, and the identification speed is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and related apparatus for identifying power equipment based on a lightweight network. Background Technology

[0002] With the rapid development of power systems, people are becoming increasingly dependent on electricity, and the demand for electricity is also growing. Today, to meet people's growing demand for sustainable energy, power systems face enormous challenges, thus requiring larger and more complex power systems.

[0003] In power systems, post porcelain insulators are a special type of electrical equipment widely used in overhead transmission lines. Due to long-term exposure to complex outdoor environments, post porcelain insulator strings are affected by various complex factors such as mechanical loads, dust, hail, acid rain, and vibration. This causes a gradual decline in the mechanical and insulation properties of the post porcelain insulators, leading to defects. Defective post porcelain insulators cannot withstand normal electromechanical loads, resulting in breakage, localized overheating, and other phenomena, seriously threatening the safe and stable operation of the power system. Therefore, rapid and accurate identification and location of electrical equipment is crucial for the maintenance of power systems. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method and apparatus for identifying power devices based on a lightweight network, which aims to quickly and accurately identify power devices.

[0005] In a first aspect, embodiments of this application provide a method for identifying power devices based on a lightweight network, the method comprising:

[0006] A sample dataset is determined, comprising multiple sample images, the sample images including electrical equipment;

[0007] The lightweight network model is modified to obtain the model to be trained;

[0008] The model to be trained is trained using the sample dataset to obtain a power equipment identification model;

[0009] Acquire an image to be identified, wherein the image to be identified is an infrared image including power equipment;

[0010] The power equipment identification model is used to identify the image to be identified, and an identification result image is obtained. The identification result image is an image in which the power equipment in the image to be identified is marked.

[0011] Optionally, the lightweight network model is the MobileNetV2 model;

[0012] The modification of the lightweight network model to obtain the model to be trained includes:

[0013] The inverted residual structure in the MobileNetV2 model is replaced by a multi-branch stacking module and a branch stacking downsampling module to obtain the model to be trained.

[0014] Optionally, the model to be trained includes at least one of the multi-branch stacking modules and one of the branch stacking downsampling modules.

[0015] Optionally, in the model to be trained, when the sum of the number of the multi-branch stacking modules and the branch stacking downsampling modules is greater than or equal to 3, the multi-branch stacking modules and the branch stacking downsampling modules are arranged alternately.

[0016] Optionally, determining the sample dataset includes:

[0017] Acquire multiple infrared images of electrical equipment;

[0018] The electrical equipment in the multiple infrared images of electrical equipment is labeled and saved to obtain the sample dataset.

[0019] Optionally, the method further includes:

[0020] Each sample image in the sample dataset is rotated, scaled, and / or sheared.

[0021] Optionally, the method further includes:

[0022] The recognition result image is then subjected to score filtering and nonmaximum suppression.

[0023] Secondly, embodiments of this application provide an identification device for power equipment based on a lightweight network, the device comprising:

[0024] A determination module is used to determine a sample dataset, which includes multiple sample images, and the sample images include power equipment;

[0025] The modification module is used to modify the lightweight network model to obtain the model to be trained.

[0026] The training module is used to train the model to be trained using the sample dataset to obtain the power equipment identification model;

[0027] The acquisition module is used to acquire the image to be identified, which is an infrared image including power equipment;

[0028] The recognition module is used to recognize the image to be recognized using the power equipment recognition model to obtain a recognition result image, wherein the recognition result image is an image in which the power equipment in the image to be recognized is marked.

[0029] Optionally, the lightweight network model is the MobileNetV2 model;

[0030] The modification module is specifically used to replace the inverted residual structure in the MobileNetV2 model with a multi-branch stacking module and a branch stacking downsampling module to obtain the model to be trained.

[0031] Optionally, the model to be trained includes at least one of the multi-branch stacking modules and one of the branch stacking downsampling modules.

[0032] Optionally, in the model to be trained, when the sum of the number of the multi-branch stacking modules and the branch stacking downsampling modules is greater than or equal to 3, the multi-branch stacking modules and the branch stacking downsampling modules are arranged alternately.

[0033] Optionally, the determining module is specifically used to acquire multiple infrared images of electrical equipment;

[0034] The electrical equipment in the multiple infrared images of electrical equipment is labeled and saved to obtain the sample dataset.

[0035] Optionally, the device further includes an image processing module for rotating, scaling, and / or shearing each sample image in the sample dataset.

[0036] Optionally, the device further includes a filtering module for performing score filtering and nonmaximum suppression on the recognition result image.

[0037] Thirdly, embodiments of this application provide a device including a memory and a processor. The memory is used to store instructions or code, and the processor is used to execute the instructions or code to cause the device to perform the identification method for power devices based on a lightweight network as described in any of the first aspects above.

[0038] Fourthly, embodiments of this application provide a computer storage medium storing code, wherein when the code is executed, a device running the code implements the identification method for power devices based on a lightweight network as described in any of the first aspects above.

[0039] This application provides a method and related apparatus for identifying power equipment based on a lightweight network. When executing the method, a sample dataset is first determined, and a lightweight network model is modified to obtain a model to be trained. Then, the model to be trained is trained using the sample dataset to obtain a power equipment identification model. Finally, the power equipment identification model is used to identify the acquired image to be identified, obtaining the identification result image, thus completing the identification of the power equipment. In this way, by modifying the lightweight network model, the model can detect power equipment in complex backgrounds, and the identification based on the lightweight network model effectively reduces the computational load and improves the identification speed. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a method for identifying power devices based on a lightweight network, provided in an embodiment of this application;

[0042] Figure 2 A schematic diagram of a multi-branch stacked module provided in an embodiment of this application;

[0043] Figure 3 A schematic diagram of a branch stacking downsampling module provided in an embodiment of this application;

[0044] Figure 4 A schematic diagram of a model to be trained provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram of a feature pyramid provided for an embodiment of this application;

[0046] Figure 6 A schematic diagram of another feature pyramid provided for an embodiment of this application;

[0047] Figure 7 A schematic diagram of a spatial pyramid pooling module provided in an embodiment of this application;

[0048] Figure 8 This is a schematic diagram of a device for identifying power equipment based on a lightweight network, provided in an embodiment of this application. Detailed Implementation

[0049] Currently, infrared thermography is widely used to identify power equipment. This involves capturing infrared images using on-site equipment, extracting the temperature from the images, and then identifying the power equipment based on the temperature. However, this method of identification based on temperature is limited by the background quality of the infrared images. In complex background environments, the accuracy of identifying power equipment is poor or even impossible. Furthermore, the recognition speed of intelligent algorithms based on complex neural networks is limited by computing power and power consumption, making it difficult to promote and apply.

[0050] To address the problems existing in the prior art, this application provides a method and related apparatus for identifying power equipment based on a lightweight network. The lightweight network model is modified and trained, enabling the model to detect power equipment in complex backgrounds. Furthermore, by using a lightweight network instead of a complex neural network, the impact of computing power and power consumption is reduced, making it highly applicable.

[0051] Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0052] See Figure 1 , Figure 1 A flowchart illustrating a method for identifying power devices based on a lightweight network, provided in this application embodiment, includes:

[0053] S101: Determine the sample dataset.

[0054] The sample dataset includes multiple sample images, which include power equipment. In this embodiment, the sample images are infrared images. The sample images in the dataset include images from various complex environments, as well as images of power equipment in normal operation and power equipment that has malfunctioned.

[0055] This application does not limit the method for determining the sample dataset in any way; it can be obtained from open-source databases or from historical power system operation data. Preferably, in one feasible implementation provided in this application, determining the sample dataset includes:

[0056] Acquire multiple infrared images of electrical equipment;

[0057] The electrical equipment in the multiple infrared images of electrical equipment is labeled and saved to obtain the sample dataset.

[0058] First, acquire multiple infrared images of electrical equipment. These images include those from various complex environments, as well as images of normally operating and malfunctioning electrical equipment. These infrared images can be captured by a camera or downloaded from the internet. Use annotation tools or manually to annotate the electrical equipment in each infrared image. Specifically, enclose all electrical equipment in the infrared image with rectangles. Then, save the original infrared images, the annotated infrared images, and the coordinates of the corresponding rectangles to obtain the sample dataset. The same method can also be used to determine the dataset for model validation.

[0059] In some special cases, insufficient data may occur, potentially affecting model training and training results. To address this issue, this application embodiment can also perform rotation, scaling, and / or shearing on each sample image in the sample dataset to increase the image data. These three processing methods can be used individually or in combination. Other methods that can increase the number of images, such as photometric distortion methods that change image brightness, contrast, or saturation, and geometric distortion methods such as translation and random flipping, are also applicable to this application embodiment. Of course, while processing the sample images, the corresponding annotation information and other relevant information are also modified accordingly.

[0060] S102: Modify the lightweight network model to obtain the model to be trained.

[0061] The lightweight network model (MobileNet) aims to reduce the number of model parameters and complexity while maintaining model accuracy. Compared with traditional deep neural network models, lightweight network models have better general applicability. Lightweight network models include MobileNetV1, MobileNetV2, and MobileNetV3. In this embodiment, the lightweight network model used is MobileNetV2. In the MobileNetV2 model, a reciprocal residual structure is used to first perform projective convolution for dimensionality increase, then depthwise convolution, and finally projective convolution for dimensionality reduction, performing this process only on one channel of the feature map.

[0062] In one feasible implementation provided in this application, the modification of the lightweight network model to obtain the model to be trained includes:

[0063] The inverted residual structure in the MobileNetV2 model is replaced by a multi-branch stacking module and a branch stacking downsampling module to obtain the model to be trained.

[0064] The multi-branch stacking module includes multiple branches that extract features separately. The extraction results from each branch are then stacked. Since the number of times the convolutional normalization activation function is applied differs between branches, resulting in feature map sizes, a further convolutional normalization activation function is applied after stacking to ensure successful stacking. The multi-branch stacking module effectively reduces the size of the feature map while increasing its depth.

[0065] It should be noted that the number of branches and the number of convolutions performed on each branch in the multi-branch stacking module are not limited and can be set by technical personnel according to actual needs. This application provides a preferred multi-branch stacking module, see [link to preferred module]. Figure 2 This multi-branch stacking module has four branches. From left to right, the first branch has one convolutional normalized activation function, the second branch has one convolutional normalized activation function, the third branch has three convolutional normalized activation functions, and the fourth branch has five convolutional normalized activation functions. The four branches produce four feature maps. The four feature maps are stacked, and after stacking, another convolutional normalized activation function is applied to integrate the features, resulting in a single feature map.

[0066] The branch-stacking downsampling module includes at least two branches for feature extraction, one of which performs max pooling on the feature map to increase the receptive field. See also... Figure 3 , Figure 3 This is a schematic diagram of a branch stacking downsampling module provided in an embodiment of this application.

[0067] In this branch stacking downsampling module, the left branch is a max pooling with a stride of 2x2 plus a 1x1 convolution, and the right branch is a 1x1 convolution plus a convolution with a kernel size of 3x3 and a stride of 2x2. The extraction results of the two branches are stacked at output. In this branch stacking downsampling module, there is no need to perform convolution normalization activation function processing on the stacked extraction results of the two branches.

[0068] Specifically, the convolution formula is as follows:

[0069] rawpicture=Rlayer+Glayer+Blayer;

[0070] purplelayer=α×rawpicture;

[0071] yellowlayer=β×rawpicture;

[0072] newpicture=purplelayer+yellowlayer;

[0073] newpicture=(α+β)(R+G+B);

[0074] Here, α and β are weight parameters during the training process. The RGB three channels are converted into new channels according to the weight parameters. Increasing or decreasing the number of channels increases the effective feature layer.

[0075] The standardized formula is as follows:

[0076]

[0077]

[0078]

[0079]

[0080] μ B Let be the mean of the data x, m be the number of data points x, and σ be the mean of the data x. B Let x be the variance of the data. The data is normalized, ε is a randomly added minimum value, and y i γ is the input data to the next layer, θ is the data scaling, and θ is the translation parameter.

[0081] The activation function formula is as follows:

[0082]

[0083] The maximum pooling formula is as follows:

[0084]

[0085] n represents the feature map, c is the channel, h is the row number, w is the column number, and k is the column number. h ∈[1,K],k w ∈[1,K], k is the length and width of the pooling window, hs is the row corresponding to h with a step size of s; ic is the column corresponding to h with a step size of s.

[0086] The inverse residual structure in the MobileNetV2 model is replaced by a multi-branch stacking module and a branch stacking downsampling module to increase the number of image channels. Stacking corresponds to a denser residual structure, which is easier to optimize, and increasing depth improves accuracy. The model to be trained includes at least one multi-branch stacking module and one branch stacking downsampling module. This application provides a preferred embodiment in which, when the sum of the number of multi-branch stacking modules and branch stacking downsampling modules in the model to be trained is greater than or equal to 3, the multi-branch stacking modules and branch stacking downsampling modules are arranged alternately. Skip connections are used in the internal residual blocks to alleviate the gradient vanishing problem caused by increasing the depth in the model. See also... Figure 4 , Figure 4 The model to be trained shown includes four multi-branch stacked modules and three branch stacked downsampling modules, with the multi-branch stacked modules and branch stacked downsampling modules arranged alternately.

[0087] In one feasible implementation provided in this application, a feature pyramid is constructed in the model to be trained, and the feature pyramid is used to enhance feature extraction. See [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a pyramid.

[0088] In another feasible implementation provided in this application, the output of the feature extraction part can also be input into the spatial pyramid pooling module to increase the receptive field, see [link to relevant documentation]. Figure 6 Specifically, the structure of the spatial pyramid pooling module is as follows: Figure 7 As shown in the figure, 5, 9, and 13 represent the number of max-pooling kernels. Different numbers of max-pooling kernels represent the ability to process images of different resolutions. In other words, these three types of max-pooling can increase the receptive field in three ways. Max-pooling is performed on the feature map using the spatial pyramid pooling module to increase the receptive field, enabling the algorithm to adapt to images of different resolutions.

[0089] S103: Train the model to be trained using the sample dataset to obtain the power equipment identification model.

[0090] Input the sample dataset into the model to be trained, extract features from the sample images, and set parameters such as image input size, category division, and mosaic data enhancement. Output the weight file corresponding to each category. Use the weight file to modify the initial weight file and categories in the model to be trained to obtain the power equipment recognition model.

[0091] S104: Acquire the image to be identified.

[0092] The image to be identified is an infrared image including electrical equipment. Specifically, the image to be identified can be captured by a field camera, and it includes images from various complex environments.

[0093] S105: The image to be identified is identified using the power equipment identification model to obtain the identification result image.

[0094] The recognition result image is an image in which the electrical equipment in the image to be recognized is labeled. The image to be recognized is used as input to the electrical equipment recognition model, which then identifies and labels the electrical equipment in the image and outputs the recognition result image. Each electrical device in the recognition result image is marked with a rectangle as a label box, thus completing the electrical equipment recognition. The recognition result image can then be used as a reference for the maintenance of the power system.

[0095] To improve the accuracy of recognition, in one embodiment of this application, after obtaining the recognition result image using the power equipment recognition model, the recognition result image is further subjected to score filtering and non-maximum suppression.

[0096] Score filtering selects bounding boxes whose scores satisfy a threshold function, while non-maximum suppression (NMS) selects the highest-scoring bounding boxes within a given region that belong to the same category. Specifically, each bounding box in the recognition result image corresponds to a score. Bounding boxes with scores greater than the threshold function are selected to reduce the number of bounding boxes. Then, the categories are iterated over. NMS selects the highest-scoring bounding boxes within a given region that belong to the same category. By iterating through the categories, NMS is applied to each category individually. Categories are sorted according to their scores, and each time the highest-scoring bounding box is selected, its overlap with all other bounding boxes is calculated, and bounding boxes with an overlap greater than a preset threshold are removed. The smaller the NMS value, the stricter the selection criteria for the bounding boxes.

[0097] The above are some specific implementations of a method for identifying power devices based on a lightweight network provided in this application. Based on this, this application also provides a corresponding device. The device provided in this application will be described below from the perspective of functional modularity.

[0098] See Figure 8 , Figure 8 This is a schematic diagram of a lightweight network-based identification device 800 for power equipment provided in an embodiment of this application. The device 800 includes a determination module 801, a modification module 802, a training module 803, a data acquisition module 804, and an identification module 805.

[0099] The determination module 801 is used to determine a sample dataset, which includes multiple sample images, and the sample images include power equipment;

[0100] Modify module 802 to modify the lightweight network model to obtain the model to be trained;

[0101] Training module 803 is used to train the model to be trained using the sample dataset to obtain a power equipment identification model;

[0102] Acquisition module 804 is used to acquire an image to be identified, wherein the image to be identified is an infrared image including power equipment;

[0103] The recognition module 805 is used to recognize the image to be recognized using the power equipment recognition model to obtain a recognition result image, wherein the recognition result image is an image in which the power equipment in the image to be recognized is marked.

[0104] In one feasible implementation provided in this application, the lightweight network model is the MobileNetV2 model;

[0105] The modification module 802 is specifically used to replace the inverted residual structure in the MobileNetV2 model with a multi-branch stacking module and a branch stacking downsampling module to obtain the model to be trained.

[0106] In one feasible implementation provided in this application, the model to be trained includes at least one multi-branch stacking module and one branch stacking downsampling module.

[0107] In one feasible implementation provided in this application, when the sum of the number of the multi-branch stacking modules and the branch stacking downsampling modules in the model to be trained is greater than or equal to 3, the multi-branch stacking modules and the branch stacking downsampling modules are arranged alternately.

[0108] In one feasible implementation provided in this application, the determining module 801 is specifically used to acquire multiple infrared images of electrical equipment;

[0109] The electrical equipment in the multiple infrared images of electrical equipment is labeled and saved to obtain the sample dataset.

[0110] In one feasible implementation provided in this application, the apparatus further includes an image processing module for rotating, scaling, and / or shearing each sample image in the sample dataset.

[0111] In one feasible implementation provided in this application, the device further includes a filtering module for performing score filtering and nonmaximum suppression on the recognition result image.

[0112] This application provides a device for identifying power equipment based on a lightweight network. First, a sample dataset is determined, and the lightweight network model is modified to obtain a model to be trained. Then, the model to be trained is used with the sample dataset to obtain a power equipment identification model. Finally, the power equipment identification model is used to identify the acquired image to be identified, obtaining the identification result image, thus completing the identification of the power equipment. In this way, by modifying the lightweight network model using the above method, the model can detect power equipment in complex backgrounds, and the identification based on the lightweight network model can effectively reduce the computational load and improve the recognition speed.

[0113] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0114] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to enable the device to perform the identification method for power devices based on a lightweight network as described in any embodiment of this application.

[0115] The computer storage medium stores code. When the code is executed, the device running the code implements the power equipment identification method based on a lightweight network as described in any embodiment of this application.

[0116] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0117] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0118] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0119] The above description is merely an exemplary implementation of this application and is not intended to limit the scope of protection of this application.

Claims

1. A method for identifying power devices based on lightweight networks, characterized in that, The method includes: Acquire multiple infrared images of electrical equipment; label and save the electrical equipment in the multiple infrared images of electrical equipment to obtain a sample dataset; the sample dataset includes multiple sample images, and the sample images include electrical equipment; Each sample image in the sample dataset is rotated, scaled, and / or sheared. The inverted residual structure in the lightweight network model is replaced by a multi-branch stacking module and a branch stacking downsampling module to obtain a model to be trained; the lightweight network model is the MobileNetV2 model; a feature pyramid is constructed in the model to be trained to enhance feature extraction; the output of the feature extraction part in the model to be trained is input into a spatial pyramid pooling module to increase the receptive field; in the model to be trained, when the sum of the number of the multi-branch stacking module and the branch stacking downsampling module is greater than or equal to 3, the multi-branch stacking module and the branch stacking downsampling module are arranged alternately. The model to be trained is trained using the sample dataset to obtain a power equipment identification model; Acquire an image to be identified, wherein the image to be identified is an infrared image including power equipment; The power equipment recognition model is used to identify the image to be identified, and a recognition result image is obtained; the recognition result image is subjected to score filtering and non-maximum suppression; the recognition result image is an image in which the power equipment in the image to be identified is labeled.

2. The method according to claim 1, characterized in that, The model to be trained includes at least one multi-branch stacking module and one branch stacking downsampling module.

3. A device for identifying power equipment based on a lightweight network, characterized in that, The device includes: A determination module is used to acquire multiple infrared images of electrical equipment; to label and save the electrical equipment in the multiple infrared images of electrical equipment to obtain a sample dataset; the sample dataset includes multiple sample images, and the sample images include electrical equipment; The modification module is used to rotate, scale, and / or shear each sample image in the sample dataset; replace the inverted residual structure in the lightweight network model with a multi-branch stacking module and a branch stacking downsampling module to obtain the model to be trained; the lightweight network model is a MobileNetV2 model; construct a feature pyramid in the model to be trained, the feature pyramid is used to enhance feature extraction; input the output of the feature extraction part in the model to be trained into a spatial pyramid pooling module to increase the receptive field; in the model to be trained, when the sum of the number of the multi-branch stacking module and the branch stacking downsampling module is greater than or equal to 3, the multi-branch stacking module and the branch stacking downsampling module are arranged alternately; The training module is used to train the model to be trained using the sample dataset to obtain the power equipment identification model; The acquisition module is used to acquire the image to be identified, which is an infrared image including power equipment; The recognition module is used to recognize the image to be recognized using the power equipment recognition model to obtain a recognition result image; to perform score filtering and non-maximum suppression on the recognition result image; the recognition result image is an image in which the power equipment in the image to be recognized is labeled.

4. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method for identifying power devices based on a lightweight network as described in claim 1 or 2.

5. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the power device identification method based on a lightweight network as described in claim 1 or 2.

Images