Variable-depth power equipment lightweight identification method

Abstract

The invention relates to a variable depth power equipment lightweight identification method, and belongs to the field of power equipment defect identification and computer vision. Comprising the following steps: constructing an image complexity characterization method for multi-dimensional joint coding, and obtaining a feature map of electric power image data; the variable depth power equipment lightweight identification method is constructed, and self-adaptive identification of the power equipment is realized based on the obtained feature map of the power image data. According to the method, the model depth can be adaptively adjusted according to the image complexity, and the calculation reasoning time is effectively shortened.

Description

Technical Field

[0001] The present invention belongs to the field of power equipment defect recognition and computer vision, and in particular relates to a variable-depth power equipment lightweight recognition method. Background Art

[0002] With the widespread application of deep learning models in image recognition, the computational cost and resource consumption of deep neural networks have become increasingly prominent. Traditional deep learning models usually use a fixed network depth in the inference stage, and perform the same number of layer operations regardless of the complexity of the input image. This approach wastes computing resources when processing simple tasks, and in complex tasks, the fixed depth may not be enough to process difficult images. Therefore, it is of great practical significance to propose a lightweight recognition method that can adaptively adjust the network depth according to the complexity of the input image.

[0003] In the inspection scene of power equipment, with the large-scale use of cameras, drones, inspection robots and other collection equipment, the amount of image and video data has increased significantly, which has brought tremendous pressure to network transmission, data storage, cloud computing, manual review and other tasks. It is urgent to combine edge computing technology to realize the rapid initial screening and pre-identification analysis of inspection images on the edge side to reduce the pressure of cloud data computing. How to maximize the efficiency of resource utilization and achieve a balance between computing speed and recognition accuracy under the condition of limited edge computing power has important practical application value for the practical promotion of edge recognition technology and the construction of cloud-edge collaborative system.

[0004] Current deep learning models generally have the following technical problems and limitations in image recognition tasks:

[0005] 1. Waste of computing resources caused by fixed depth: Existing deep learning networks are usually designed with fixed depth, and the network will perform the same number of layer operations regardless of the complexity of the input image. This design may cause a waste of computing resources when processing simple images, because these images may only require shallow feature extraction to complete the recognition task.

[0006] 2. Lack of adaptive capabilities: Traditional deep learning models lack the ability to adaptively adjust the complexity of input data. When processing complex images, a fixed network depth may not be sufficient to extract enough high-level features, thus affecting recognition accuracy. At the same time, for simple images, an overly deep network structure will increase the computational burden and reduce reasoning efficiency.

[0007] 3. Computational efficiency issues in the inference phase: On resource-constrained edge devices or mobile devices, the computational efficiency of fixed-depth deep learning models in the inference phase is difficult to guarantee. Due to the limited computing resources of these devices, fixed-depth structures are prone to increased inference latency and excessive energy consumption.

[0008] 4. Trade-off between model complexity and performance: Existing methods have difficulty finding the best balance between model complexity and performance. Although simple network structures have low computational overhead, they are insufficient in performance when processing complex tasks; deep and complex networks have high resource requirements and are difficult to adapt to diverse application scenarios. Summary of the invention

[0009] The purpose of the present invention is to provide a variable-depth lightweight recognition method for power equipment in view of the limitations of existing methods, adaptively adjust the model depth according to image complexity, and effectively reduce the calculation and reasoning time.

[0010] To achieve the above object, the technical solution of the present invention is: a variable depth power equipment lightweight identification method, comprising:

[0011] Construct a multi-dimensional joint coding method for representing image complexity and obtain the feature map of power image data;

[0012] A variable-depth lightweight recognition method for power equipment is constructed to achieve adaptive recognition of power equipment based on the feature map of the obtained power image data.

[0013] In one embodiment of the present invention, the image complexity characterization method of multi-dimensional joint coding includes a feature coding method based on image features and a feature coding method based on convolutional neural network.

[0014] In one embodiment of the present invention, the specific calculation formula of the feature encoding method based on image features is as follows:

[0015]

[0016] Among them, V Tr represents feature encoding based on image features, w and h represent the resolution of the image, and H img is the entropy value of the image, C g , H g 、E g They represent the contrast, homogeneity, and energy texture features extracted by the gray-level co-occurrence matrix GLCM respectively;

[0017]

[0018] C g =∑ i,j (ij) 2 P(i,j)

[0019]

[0020] E g =∑ i,jP(i,j) 2

[0021] Among them, p(i) is the probability distribution of gray value i in the image, L is the gray level, and P(i,j) is the probability that a pixel with gray value i and a pixel with gray value j appear at the same time in a given direction and distance.

[0022] In one embodiment of the present invention, the specific calculation formula of the feature encoding method based on convolutional neural network is as follows:

[0023] V conv =N f (N e (img))

[0024] Among them, N e is the Unet-based encoder, N f is a dimensionality reduction network; after the image in the power image data is downsampled to img∈[64,64,3], it passes through the encoder N e Perform feature extraction to obtain feature layer F e ∈[4,4,128], and then through the dimension reduction network N f Dimensionality reduction to F f ∈[1,1,16], and obtain the feature encoding V based on the convolutional neural network conv .

[0025] In one embodiment of the present invention, the convolutional neural network adopts a Unet structure, which is composed of a four-layer convolutional block structure N e = {N e1 ,N e2 ,N e3 ,N e4} and 4 layers of upsampling blocks N d = {N d1 ,N d2 ,N d3 ,N d4} composition; the convolutional neural network uses power image data for training sample recovery capability. During training, the codec training is based on the power image data to generate the restoration prediction results of the image in the power image data. When applying the encoding, only N e +N f .

[0026] In one embodiment of the present invention, the characteristic graph V∈[1,1,22] of the power image data is specifically expressed as follows:

[0027] V = concat[V Tr ,V conv ]

[0028] Among them, V Trrepresents the feature encoding based on image features, V conv It represents feature encoding based on convolutional neural network, and concat represents concatenation operation.

[0029] In one embodiment of the present invention, the variable-depth lightweight recognition method for electric power equipment includes a variable-depth-based multi-branch lightweight recognition method for electric power equipment and a model hierarchical training method based on image distribution differences.

[0030] In one embodiment of the present invention, the variable depth power equipment lightweight identification method is specifically implemented as follows:

[0031] Construct a feature extraction network and a multi-branch detection head that can separate convolutional neural networks across layers;

[0032] The feature extraction network includes an N-layer network structure, and the feature graphs between each layer are directly connected through a jump structure; the feature graph of the power image data is extracted according to the feature graph V train =[V train-L1 , V train-L2 , V train-L3 ] is divided into three types of routing complexity, corresponding to the three model structures of L1, L2, and L3. If the i-th layer in a model structure chooses to retain the calculation model, then the input feature map f of its i+1th layer i+1 for

[0033] f i+1 =f i +F i (f i )

[0034] If the computational model of the i-th layer is not retained, the input feature map f of the i+1th layer i+1 for

[0035] f i+1 =f i

[0036] The feature maps generated by the feature extraction network correspond to different predictors and parameter settings according to the three model structures of the feature extraction network, namely, the multi-branch detection head; among them, the L1 predictor corresponding to L1 and the L2 predictor corresponding to L2 both output prediction results by 1 3×3 convolution and 1 1×1 convolution cascade, and the L3 predictor corresponding to L3 outputs prediction results by 2 3×3 convolutions and 1 1×1 convolution cascade.

[0037] In one embodiment of the present invention, the model hierarchical training method based on image distribution difference is specifically implemented as follows:

[0038] Divide three data sets Dataset = {D1, D2, D3} according to the difficulty level, and then loop the following steps to train the feature extraction network and multi-branch detection head:

[0039] S1, use the Dataset dataset to train L3 and 3 predictors. This process takes the average of the loss functions of each predictor as the final loss function, optimizes the model parameters, and sets the learning rate to 1e -3 ;

[0040] S2, train L3 and L3 predictor using D3 dataset, with learning rate set to 5e -5 ;

[0041] S3, use the D2 dataset to train L2 and L2 predictors, and the learning rate is set to 5e -5 ;

[0042] S4, use the D1 dataset to train L1 and L1 predictors, and the learning rate is set to 5e -5 ;

[0043] Repeat steps S2-S4 until the accuracy requirement is met.

[0044] The present invention also provides a variable-depth power equipment lightweight identification system, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. Improve computing efficiency: By adaptively adjusting the network depth, the waste of computing resources for simple images is reduced and computing efficiency is improved. In an environment where edge computing resources are limited, the computing overhead can be significantly reduced while ensuring recognition accuracy.

[0047] 2. Enhance model adaptability: The model can adaptively adjust the depth according to the complexity of the image, thereby improving the recognition ability of complex images. Using network structures of different depths for images of different complexities improves the recognition performance and accuracy of the overall model.

[0048] 3. Balance accuracy and efficiency: Through multi-stage training and complexity adaptation methods, a good balance is achieved between model complexity and performance. A shallow network is used for simple images, and a deep network is used for complex images, effectively taking into account both computational efficiency and recognition accuracy.

[0049] 4. Adapt to different data sources: The model can process data from multiple cameras, select the appropriate network structure through image complexity analysis, and realize adaptive calculation of multi-channel camera data by a single model, with strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a schematic diagram of a network structure adopted by the method. DETAILED DESCRIPTION

[0051] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0052] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0053] The present invention provides a method for lightweight identification of variable-depth power equipment, comprising:

[0054] Construct a multi-dimensional joint coding method for representing image complexity and obtain the feature map of power image data;

[0055] A variable-depth lightweight recognition method for power equipment is constructed to achieve adaptive recognition of power equipment based on the feature map of the obtained power image data.

[0056] The present invention also provides a variable-depth power equipment lightweight identification system, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0057] like Figure 1 As shown, this embodiment provides a variable-depth lightweight identification method for power equipment based on image complexity coding, which mainly includes an image complexity characterization method of multi-dimensional joint coding and a variable-depth lightweight identification method for power equipment.

[0058] The multi-dimensional joint coding method for representing image complexity specifically includes: a feature coding method based on image features and a feature coding method based on convolutional neural networks.

[0059] The specific calculation formula of the feature encoding method based on image features is as follows:

[0060]

[0061] Among them, w and h represent the resolution of the image, H img is the entropy value of the image, C g, H g 、E g They respectively represent the contrast, homogeneity, and energy texture features extracted by the gray-level co-occurrence matrix (GLCM).

[0062]

[0063] C g =∑ i,j (ij) 2 P(i,j)

[0064]

[0065] E g =∑ i,j P(i,j) 2

[0066] Among them, p(i) is the probability distribution of gray value i in the image, L is the gray level, and P(i,j) is the probability that a pixel with gray value i and a pixel with gray value j appear at the same time in a given direction and distance.

[0067] The above indicators measure the randomness and information content of the image, the texture repeatability of the image, the uniformity of the grayscale in the image, and the difference of the grayscale in the image respectively.

[0068] The specific calculation formula of the feature encoding method based on convolutional neural network is as follows:

[0069] V conv =N f (N e (img))

[0070] The convolutional neural network in this encoding method adopts the Unet structure, which consists of a four-layer convolutional block structure N e = {N e1 ,N e2 ,N e3 ,N e4} and 4 layers of upsampling blocks N d = {N d1 ,N d2 ,N d3 ,N d4}, and train the sample recovery ability on the pre-prepared power data.

[0071] N e Each convolutional block consists of 2 convolutional layers and 1 max pooling layer. The convolutional size is 3×3, 16 output channels, stride 1, and ReLU activation. The number of channels of the convolutional layer is 16, 32, 64, and 128 respectively. The pooling layer is 2×2 with a stride of 2.

[0072] Nd Each upsampling block consists of 1 upsampling layer and 2 convolutional layers. The convolutional layer has a size of 3×3, 16 output channels, a stride of 1, and ReLU activation, and a pooling layer of 2×2 and stride of 2. The number of channels in the convolutional layer is 64, 32, 16, and 3 respectively. The upsampling layers are all 2×2 and have a stride of 2.

[0073] In the application encoding process, for the input image, it is first downsampled to img∈[64,64,3] and then passed through the encoder N e Perform feature extraction to obtain feature layer F e ∈[4,4,128], and then through the dimension reduction network N f Dimensionality reduction to F f ∈[1,1,16]. First, add the 4×4 feature maps of each layer, and then reduce the dimension to 16 through PCA to obtain the feature encoding V based on the convolutional neural network conv .

[0074] During training, the encoder N based on Unet e and N d The final result of the codec training is the restoration prediction result of the 64x64x3 input image. However, when applying it, N d , using N e +N f , and the final output is a 1x16 feature matrix.

[0075] Through this method, the feature map V∈[1,1,22] of the power image data is obtained.

[0076] V = concat[V Tr ,V conv ]

[0077] The variable depth power equipment lightweight recognition method specifically includes: a multi-branch power equipment lightweight recognition method based on variable depth, and a model hierarchical training method based on image distribution differences.

[0078] The lightweight identification method of multi-branch power equipment based on variable depth is specifically implemented as follows:

[0079] Construct a feature extraction network and a multi-branch detection head that can separate convolutional neural networks across layers;

[0080] The feature extraction network contains 18 layers of network structure, and the feature maps between each layer are directly connected through a jump structure. train =[V train-L1 , V train-L2 , V train-L3] is divided into three types of routing complexity, corresponding to the three model structures of L1, L2, and L3. The calculation graphs retained by each network structure are shown in Table 1. For the input feature map f of the i-th layer i and the computational model F i , its feature map size and calculation model are shown in Table 1. If the layer chooses to retain the calculation model, the input feature map f of the i+1th layer i+1 for

[0081] f i+1 =f i +F i (f i )

[0082] If you choose not to keep this layer, the input feature map f of the i+1th layer i+1 for

[0083] f i+1 =f i

[0084] Table 1

[0085]

[0086]

[0087] For the feature map f generated by the feature extraction network 19 ∈[1, 1, 960], according to the L1, L2, and L3 structures of the feature extraction network, they correspond to different predictors and parameter settings, namely, multi-branch detection heads. Among them, the L1 predictor and the L2 predictor both output prediction results by 1 3×3 convolution and 1 1×1 convolution cascade, and the L3 predictor outputs prediction results by 2 3×3 convolutions and 1 1×1 convolution cascade.

[0088] The model hierarchical training method based on image distribution difference is specifically implemented as follows:

[0089] First, three data sets Dataset = {D1, D2, D3} are divided according to the difficulty level, and then the following steps are looped to train the feature extraction network and the multi-branch detection head.

[0090] S1. Use the Dataset dataset to train the L3 model structure and three predictors. The process takes the average of the loss functions of each predictor as the final loss function, optimizes the model parameters, and sets the learning rate to 1e -3 .

[0091] S2, use the D3 dataset to train the L3 model structure and L3 predictor, and the learning rate is set to 5e -5 .

[0092] S3, use the D2 dataset to train the L2 model structure and L2 predictor, and the learning rate is set to 5e -5 .

[0093] S4, use the D1 dataset to train the L1 model structure and L1 predictor, and the learning rate is set to 5e -5 .

[0094] Repeat steps S2-S4 until the model reaches the required accuracy.

[0095] Application of the method of the present invention:

[0096] like Figure 1 As shown in the figure, the model consisting of the convolutional neural network, feature extraction network and multi-branch detection head involved in the variable depth power equipment lightweight recognition method based on image complexity coding of the present invention is deployed on the edge computing terminal, and the multi-channel camera data is connected to monitor the operation of the power equipment in real time. First, the data of each camera is collected, 100 samples are taken from each camera, and the feature map V of the power image data is calculated. local-100 Then, use the Euclidean distance to calculate V local-100 and V train The similarities of the three categories are calculated respectively to determine the L1, L2, and L3 models corresponding to the camera data, thereby realizing the adaptive calculation of a single model for multi-channel camera data and reducing the calculation efficiency.

[0097] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0098] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0101] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for lightweight identification of variable depth power equipment, characterized in that: include: Construct a multi-dimensional joint coding method for representing image complexity and obtain the feature map of power image data; A variable-depth lightweight recognition method for power equipment is constructed to achieve adaptive recognition of power equipment based on the feature map of the obtained power image data.

2. A variable depth power equipment lightweight identification method according to claim 1, characterized in that: The multi-dimensional joint coding method for representing image complexity includes a feature coding method based on image features and a feature coding method based on convolutional neural networks.

3. A variable depth power equipment lightweight identification method according to claim 2, characterized in that: The specific calculation formula of the feature encoding method based on image features is as follows: Among them, V Tr represents feature encoding based on image features, w and h represent the resolution of the image, and H img is the entropy value of the image, C g , H g 、E g They represent the contrast, homogeneity, and energy texture features extracted by the gray-level co-occurrence matrix GLCM respectively; C g =∑ i，j (i-j) 2 ·P(i,j) HAVE BEEN g =∑ i，j P(i,j) 2 Among them, p(i) is the probability distribution of gray value i in the image, L is the gray level, and P(i, j) is the probability that a pixel with gray value i and a pixel with gray value j appear at the same time in a given direction and distance.

4. A variable depth power equipment lightweight identification method according to claim 3, characterized in that: The specific calculation formula of the feature encoding method based on convolutional neural network is as follows: V conv =N f (N e (img)) Among them, N e is the Unet-based encoder, N f is a dimensionality reduction network; after the image in the power image data is downsampled to img∈[64,64,3], it passes through the encoder N e Perform feature extraction to obtain feature layer F e ∈[4, 4, 128], and then through the dimension reduction network N f Dimensionality reduction to F f ∈[1, 1, 16], and obtain the feature encoding V based on the convolutional neural network conv .

5. A variable depth power equipment lightweight identification method according to claim 4, characterized in that: The convolutional neural network adopts the Unet structure, which consists of N e = {N e1 , N e2 , N e3 , N e4 } and 4 layers of upsampling blocks N d = {N d1 , N d2 , N d3 , N d4 } composition; the convolutional neural network uses power image data for training sample recovery capability. During training, the codec training is based on the power image data to generate the restoration prediction results of the image in the power image data. When applying the encoding, only N e +N f .

6. A variable depth power equipment lightweight identification method according to claim 1, characterized in that: The characteristic graph V∈[1, 1, 22] of the power image data is specifically expressed as follows: V=concat[V Tr ,In conv ] Among them, V Tr represents the feature encoding based on image features, V conv It represents feature encoding based on convolutional neural network, and concat represents concatenation operation.

7. A variable depth power equipment lightweight identification method according to claim 1, characterized in that: The variable-depth lightweight recognition method for electric power equipment includes a variable-depth-based multi-branch lightweight recognition method for electric power equipment and a model hierarchical training method based on image distribution differences.

8. A variable depth power equipment lightweight identification method according to claim 7, characterized in that: The variable depth power equipment lightweight identification method is specifically implemented as follows: Construct a feature extraction network and a multi-branch detection head that can separate convolutional neural networks across layers; The feature extraction network includes an N-layer network structure, and the feature graphs between each layer are directly connected through a jump structure; the feature graph of the power image data is extracted according to the feature graph V train =[V train-L1 , V train-L2 , V train-L3 ] is divided into three types of routing complexity, corresponding to the three model structures of L1, L2, and L3. If the i-th layer in a model structure chooses to retain the calculation model, then the input feature map f of its i+1th layer i+1 for f i+1 =f i +F i (f i ) If the computational model of the i-th layer is not retained, the input feature map f of the i+1th layer i+1 for f i+1 =f i The feature maps generated by the feature extraction network correspond to different predictors and parameter settings according to the three model structures of the feature extraction network, namely, the multi-branch detection head; among them, the L1 predictor corresponding to L1 and the L2 predictor corresponding to L2 both output prediction results by 1 3×3 convolution and 1 1×1 convolution cascade, and the L3 predictor corresponding to L3 outputs prediction results by 2 3×3 convolutions and 1 1×1 convolution cascade.

9. A variable depth power equipment lightweight identification method according to claim 8, characterized in that: The model hierarchical training method based on image distribution difference is specifically implemented as follows: Divide three data sets Dataset = {D1, D2, D3} according to the difficulty level, and then loop the following steps to train the feature extraction network and multi-branch detection head: S1, use the Dataset dataset to train L3 and 3 predictors. This process takes the average of the loss functions of each predictor as the final loss function, optimizes the model parameters, and sets the learning rate to 1e -3 ; S2, train L3 and L3 predictor using D3 dataset, with learning rate set to 5e -5 ; S3, use the D2 dataset to train L2 and L2 predictors, and the learning rate is set to 5e -5 ; S4, use the D1 dataset to train L1 and L1 predictors, and the learning rate is set to 5e -5 ; Repeat steps S2-S4 until the accuracy requirement is met.

10. A variable depth power equipment lightweight identification system, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 9 can be implemented.

Images