Variable-depth power equipment lightweight identification method

Abstract

The application relates to a variable-depth power equipment lightweight identification method, and belongs to the fields of power equipment defect identification and computer vision. The method comprises the following steps: constructing a multi-dimensional joint coding image complexity representation method to obtain a feature map of power image data; and constructing a variable-depth power equipment lightweight identification method to realize self-adaptive identification of power equipment based on the obtained feature map of the power image data. The application can adaptively adjust the model depth according to the image complexity, and effectively reduce the calculation and reasoning time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of power equipment defect recognition and computer vision, and particularly relates to a variable-depth power equipment lightweight recognition method. BACKGROUND

[0002] With the wide application of deep learning models in image recognition, the computing cost and resource consumption problem of deep neural networks is increasingly prominent. Traditional deep learning models usually use fixed network depth in the inference stage, regardless of the complexity of the input image, the same number of layer operations will be performed. This way wastes computing resources when dealing with simple tasks, and in complex tasks, the fixed depth may not be enough to handle high-difficulty images. Therefore, it is of great practical significance to propose a lightweight recognition method that can adaptively adjust the network depth according to the complexity of the input image.

[0003] In the power equipment inspection scene, with the large-scale use of cameras, drones, inspection robots and other collection devices, the number of image and video data has increased significantly, bringing great pressure to network transmission, data storage, cloud computing, manual review and other work, and it is urgent to combine edge computing technology to realize fast preliminary screening and pre-recognition analysis of inspection images on the edge side, and reduce the pressure of cloud data calculation. How to improve resource utilization efficiency as much as possible under the condition of limited edge computing power, and achieve the balance of calculation speed and recognition accuracy, has important practical application value for the practical popularization of edge side recognition technology and the construction of cloud-edge collaborative system.

[0004] The current deep learning model has the following technical problems and limitations in image recognition tasks:

[0005] 1. Wasted computing resources due to fixed depth: Existing deep learning networks are usually designed with fixed depth, regardless of the complexity of the input image, the network will perform the same number of layer operations. This design may cause waste of computing resources when dealing with simple images, because these images may only need shallow feature extraction to complete the recognition task.

[0006] 2. Lack of adaptive ability: Traditional deep learning models lack the ability to adaptively adjust to the complexity of the input data. When dealing with complex images, the fixed network depth may not be enough to extract sufficient high-level features, affecting the recognition accuracy. At the same time, for simple images, the deep network structure increases the computational burden and reduces the inference efficiency.

[0007] 3. Inference stage computing efficiency problem: In resource-limited edge devices or mobile devices, the fixed depth of the deep learning model in the inference stage is difficult to guarantee the computing efficiency. Due to the limited computing resources of these devices, the fixed depth structure is easy to cause the problem of increased inference delay and high energy consumption.

[0008] 4. Trade-off between model complexity and performance: Existing methods have difficulty in finding the best balance point between model complexity and performance. Simple network structure, although with small computational overhead, is insufficient in performance when dealing with complex tasks; while deep and complex network requires high resources and is difficult to adapt to diverse application scenarios. SUMMARY

[0009] The present application aims to provide a variable-depth power equipment lightweight identification method that can adaptively adjust the model depth according to the image complexity, effectively reducing the computational inference time, in view of the limitations of existing methods.

[0010] To achieve the above-mentioned purpose, the technical scheme of the present application is: a variable-depth power equipment lightweight identification method, comprising:

[0011] A multi-dimensional joint coding image complexity representation method is constructed to obtain the feature map of power image data;

[0012] A variable-depth power equipment lightweight identification method is constructed, which realizes adaptive identification of power equipment based on the obtained feature map of power image data.

[0013] In an embodiment of the present application, the multi-dimensional joint coding image complexity representation method includes an image feature-based feature encoding method and a convolutional neural network-based feature encoding method.

[0014] In an embodiment of the present application, the image feature-based feature encoding method has the following specific calculation formula:

[0015]

[0016] wherein V Tr represents the feature encoding based on image features, w and h represent the resolution of the image, H img is the entropy value of the image, C g , H g , E g respectively represent the contrast, homogeneity and energy texture features formed by the gray level co-occurrence matrix GLCM extraction;

[0017]

[0018] C g =∑ i,j (i-j) 2 ·P(i,j)

[0019]

[0020] E g =∑ i,jP(i,j) 2

[0021] wherein, p(i) is the probability distribution of the gray value i in the image, L is the number of gray levels, P(i,j) is the probability of the pixel with the gray value i and the pixel with the gray value j appearing simultaneously at a given direction and distance.

[0022] In an embodiment of the present application, the specific calculation formula of the feature encoding method based on the convolutional neural network is as follows:

[0023] V conv =N f (N e (img))

[0024] wherein, N e is an encoder based on Unet, N f is a dimension reduction network; after the image in the power image data is down-sampled to img∈[64,64,3], the feature extraction is performed through the encoder N e , the feature layer F e ∈[4,4,128] is obtained, then the dimension reduction network N f is used for dimension reduction to F f ∈[1,1,16], and the feature encoding V conv based on the convolutional neural network is obtained.

[0025] In an embodiment of the present application, the convolutional neural network adopts the Unet structure, which is composed of four layers of convolution block structure N e ={N e1 ,N e2 ,N e3 ,N e4} and four layers of up-sampling block N d ={N d1 ,N d2 ,N d3 ,N d4}; the convolutional neural network adopts the power image data to restore the training sample, and during the training, the training based on the encoding and decoding of the power image data is performed to generate the restoration prediction result of the image in the power image data; and during the application of the encoding, only N e +N f is used.

[0026] In an embodiment of the present application, the feature map V∈[1,1,22] of the power image data is specifically represented as follows:

[0027] V=concat[V Tr ,V conv ]

[0028] wherein, V Trdenotes a feature encoding based on image features, V conv denotes a feature encoding based on a convolutional neural network, and concat denotes a concatenation operation.

[0029] In an embodiment of the present application, the variable-depth power equipment lightweight identification method includes a variable-depth-based multi-branch power equipment lightweight identification method and a model hierarchical training method based on image distribution differences.

[0030] In an embodiment of the present application, the variable-depth power equipment lightweight identification method is implemented as follows:

[0031] A feature extraction network and a multi-branch detection head of a cross-layer separable convolutional neural network are constructed.

[0032] The feature extraction network includes N-layer network structures, and the feature maps between each layer are directly connected through a jump structure; the feature maps of the power image data are divided into three types of distribution complexities according to the feature map V train = [V train-L1 , V train-L2 , V train-L3 ], which correspond to three model structures L1, L2, and L3; if the calculation model of the i-th layer is selected to be retained in a model structure, the input feature map f i+1 of the i+1-th layer is

[0033] f i+1 = f i + F i (f i )

[0034] If the calculation model of the i-th layer is not retained, the input feature map f i+1 of the i+1-th layer is

[0035] f i+1 = f i

[0036] The feature maps generated by the feature extraction network correspond to different predictors and parameter settings, i.e., the multi-branch detection head, according to the three model structures L1, L2, and L3 of the feature extraction network; the L1 predictor corresponding to L1 and the L2 predictor corresponding to L2 are both output prediction results by cascading one 3x3 convolution and one 1x1 convolution, and the L3 predictor corresponding to L3 is output prediction results by cascading two 3x3 convolutions and one 1x1 convolution.

[0037] In an embodiment of the present application, the model hierarchical training method based on image distribution differences is implemented as follows:

[0038] Three data sets Dataset={D1, D2, D3} are divided according to difficulty levels, and the following steps are cycled to train the feature extraction network and the multi-branch detection head:

[0039] S1, training L3 and 3 predictors using Dataset dataset, the process takes the average of each predictor loss function as the final loss function, optimizing model parameters, learning rate is set to 1e -3 ;

[0040] S2, training L3 and L3 predictor using D3 dataset, learning rate is set to 5e -5 ;

[0041] S3, training L2 and L2 predictor using D2 dataset, learning rate is set to 5e -5 ;

[0042] S4, training L1 and L1 predictor using D1 dataset, learning rate is set to 5e -5 ;

[0043] Repeat steps S2-S4 until the accuracy requirement is reached.

[0044] The application also provides a variable depth power equipment lightweight identification system, comprising a memory, a processor and computer program instructions stored on the memory and capable of being executed by the processor, when the processor executes the computer program instructions, the method steps as described above can be implemented.

[0045] Compared with the prior art, the application has the following beneficial effects:

[0046] 1. Improve computing efficiency: by adaptively adjusting the network depth, reduce the waste of computing resources for simple images, improve the computing efficiency. In the environment of limited edge computing resources, it can significantly reduce the computing cost under the premise of ensuring the recognition accuracy.

[0047] 2. Enhance model adaptability: the model can adaptively adjust the depth according to the complexity of the image, thereby improving the recognition ability of complex images. Different network structures are used for images of different complexity, which improves the recognition performance and accuracy of the overall model.

[0048] 3. Balance accuracy and efficiency: through multi-stage training and complexity adaptive method, a good balance between model complexity and performance is achieved. For simple images, use shallow network, for complex images, use deep network, effectively balance computing efficiency and recognition accuracy.

[0049] 4. Adapt to different data sources: the model can process data from multiple cameras, select the appropriate network structure through image complexity analysis, realize adaptive calculation of single model for multi-camera data, and has strong adaptability. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A network structure schematic diagram is used for the method of the application. DETAILED DESCRIPTION

[0051] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0052] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit the exemplary embodiments according to the application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a reference to the presence of a feature, step, operation, device, component and / or combinations thereof.

[0053] The application provides a variable-depth power equipment lightweight identification method, comprising:

[0054] A multi-dimensional joint coding image complexity representation method is constructed to obtain a feature map of power image data;

[0055] A variable-depth power equipment lightweight identification method is constructed, and based on the obtained feature map of power image data, power equipment adaptive identification is realized.

[0056] The application also provides a variable-depth power equipment lightweight identification system, comprising a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, when the processor executes the computer program instructions, the method steps as described above can be realized.

[0057] As shown in Figure 1 The embodiment provides a variable-depth power equipment lightweight identification method based on image complexity coding, mainly comprising a multi-dimensional joint coding image complexity representation method and a variable-depth power equipment lightweight identification method.

[0058] The multi-dimensional joint coding image complexity representation method specifically comprises: an image feature-based feature coding method and a convolutional neural network-based feature coding method.

[0059] The image feature-based feature coding method specifically calculates the formula as follows:

[0060]

[0061] Wherein, w and h represent the resolution of the image, H img is the entropy value of the image, C g, H g , E g respectively represent the contrast, homogeneity, energy texture features formed by the gray level co-occurrence matrix (GLCM) extraction.

[0062]

[0063] C g =∑ i,j (i-j) 2 ·P(i,j)

[0064]

[0065] E g =∑ i,j P(i,j) 2

[0066] wherein p(i) is the probability distribution of the gray value i in the image, L is the gray level number, and P(i,j) is the probability of the pixel with the gray value i and the pixel with the gray value j appearing simultaneously at a given direction and distance.

[0067] The above indexes respectively measure the randomness and information amount of the image, the texture repetition of the image, the uniformity of the gray level in the image, and the difference of the gray level in the image.

[0068] The feature encoding method based on the convolutional neural network specifically calculates the formula as follows:

[0069] V conv =N f (N e (img))

[0070] In the encoding method, the convolutional neural network adopts the Unet structure, which is composed of four convolutional block structures N e ={N e1 ,N e2 ,N e3 ,N e4} and four up-sampling blocks N d ={N d1 ,N d2 ,N d3 ,N d4}, and the sample recovery capability is trained on the pre-prepared power data.

[0071] N e Each layer of the convolutional block is composed of two convolutional layers and one maximum pooling layer. The convolutional layer has a convolution size of 3x3, 16 output channels, a step of 1, and a ReLU activation, and the number of channels of the convolutional layer is 16, 32, 64, and 128, respectively. The pooling layer is 2x2 with a step of 2.

[0072] Nd Each layer of the up-sampling block is composed of 1 up-sampling layer and 2 convolutional layers. The convolutional layer is composed of a convolutional layer with a convolution size of 3x3, 16 output channels, a step size of 1, ReLU activation, and a pooling layer with a size of 2x2 and a step size of 2, and the number of channels of the convolutional layer is 64, 32, 16, and 3, respectively. The up-sampling layer is 2x2 with a step size of 2.

[0073] In the application encoding process, for the input image, it is first down-sampled to img∈[64,64,3], and then the encoder N e performs feature extraction to obtain the feature layer F e ∈[4,4,128], and then the dimension reduction network N f is used for dimension reduction to F f ∈[1,1,16]. First, the 4x4 feature maps of each layer are added, and then PCA is used for dimension reduction to 16 to obtain the feature encoding V conv based on the convolutional neural network.

[0074] During training, the encoder N e and the decoder N d are trained, and the final generated is the restoration prediction result of the input image of 64x64x3. However, in the application, N d is directly discarded, and N e +N f is used, and finally a 1x16 feature matrix is output.

[0075] Through this method, the feature map V∈[1,1,22] of the power image data is obtained.

[0076] V=concat[V Tr ,V conv ]

[0077] The variable-depth power equipment lightweight identification method specifically includes: a variable-depth-based multi-branch power equipment lightweight identification method and a model hierarchical training method based on image distribution differences.

[0078] The variable-depth-based multi-branch power equipment lightweight identification method is specifically implemented as follows:

[0079] A feature extraction network and a multi-branch detection head of a cross-layer separable convolutional neural network are constructed.

[0080] The feature extraction network includes 18 network structures, and the feature maps between each layer are directly connected through a jump structure. Secondly, the image is divided into V train =[V train-L1 , V train-L2 , V train-L3The network is divided into three routing complexities, corresponding to L1, L2, and L3 model structures, and the retained computation graphs of each network structure are shown in Table 1. For the input feature map f i and the computation model F i , the feature map size and computation model are shown in Table 1. If the layer selects to retain the computation model, the input feature map f i+1 of the i+1 layer is

[0081] f i+1 = f i +F i (f i )

[0082] If the layer is not selected to be retained, the input feature map f i+1 of the i+1 layer is

[0083] f i+1 = f i

[0084] Table 1

[0085]

[0086]

[0087] For the feature map f 19 generated by the feature extraction network, ∈[1, 1, 960], according to the L1, L2, and L3 structures of the feature extraction network, different predictors and parameter settings are respectively selected, that is, multiple branch detection heads. The L1 predictor and the L2 predictor are both output prediction results by cascading 1 3x3 convolution and 1 1x1 convolution, and the L3 predictor is output prediction results by cascading 2 3x3 convolutions and 1 1x1 convolution.

[0088] The model hierarchical training method based on image distribution difference is implemented as follows:

[0089] First, three data sets Dataset={D1, D2, D3} are divided according to the difficulty level, and then the following steps are cycled to train the feature extraction network and the multiple branch detection head.

[0090] S1, the L3 model structure and the three predictors are trained using the Dataset data set. The process takes the average of the loss functions of each predictor as the final loss function, optimizes the model parameters, and the learning rate is set to 1e -3 .

[0091] S2, the L3 model structure and the L3 predictor are trained using the D3 data set, and the learning rate is set to 5e -5 .

[0092] S3, training L2 model structure and L2 predictor using D2 dataset, learning rate set to 5e -5 .

[0093] S4, training L1 model structure and L1 predictor using D1 dataset, learning rate set to 5e -5 .

[0094] Repeat S2-S4 steps until the model reaches the accuracy requirement.

[0095] Application of the method of the present application:

[0096] As Figure 1 shown, the model composed of the convolutional neural network, feature extraction network and multi-branch detection head obtained by training involved in the variable depth power equipment lightweight identification method based on image complexity coding is deployed on an edge computing terminal, and is connected to multiple camera data for real-time monitoring of power equipment operation. First, each camera data is collected, 100 images are sampled from each camera, and the feature map V local-100 of the power image data is calculated. Then, the Euclidean distance is used to calculate the similarity of three categories in V local-100 and V train , so as to determine the L1, L2 and L3 models corresponding to the camera data, so as to realize adaptive calculation of a single model for multiple camera data and reduce the calculation efficiency.

[0097] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0098] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows or one or more blocks. Figure 1 The device that implements the functions specified in one or more flows or one or more blocks.

[0099] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0100] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A variable-depth power equipment lightweight identification method, characterized in that, The application relates to a multi-dimensional joint coding image complexity representation method, and a variable-depth power equipment lightweight identification method. The multi-dimensional joint coding image complexity representation method comprises an image feature-based feature coding method and a convolutional neural network-based feature coding method. The specific calculation formula of the image feature-based feature coding method is as follows: Wherein, p(i) is the probability distribution of the gray value i in the image, L is the gray level, and P(i,j) is the probability of the pixels with the gray value i and the pixels with the gray value j appearing simultaneously in a given direction and distance. The variable-depth power equipment lightweight identification method comprises a variable-depth multi-branch power equipment lightweight identification method and a model hierarchical training method based on image distribution difference. where V Tr represents the feature encoding based on image features, w and h represent the resolution of the image, H img is the entropy value of the image, C g , H g , E g respectively represent the contrast, homogeneity, and energy texture features formed by the gray level co-occurrence matrix GLCM extraction; C g =∑ i,j (i-j) 2 ·P (i,k) E g =∑ i,j P(i,k) 2 The variable-depth power equipment lightweight identification method is specifically implemented as follows: A feature extraction network and a multi-branch detection head of a cross-layer separable convolutional neural network are constructed. The feature extraction network generates a feature map, which is respectively corresponding to different predictors and parameter settings, i.e. a multi-branch detection head, according to the L1, L2 and L3 model structures of the feature extraction network; wherein the L1 predictor corresponding to L1 and the L2 predictor corresponding to L2 are both output prediction results by cascading one 3*3 convolution and one 1*1 convolution, and the L3 predictor corresponding to L3 is output prediction results by cascading two 3*3 convolutions and one 1*1 convolution. The specific calculation formula of the convolutional neural network-based feature coding method is as follows: The feature extraction network comprises an N-layer network structure, and feature maps between each layer are directly connected through a jump structure; the feature maps of the power image data are divided into three kinds of feature maps V train , V train-L1 , and V train-L2 according to the feature map V train-L3 , and the feature maps are divided into three kinds of network complexity, corresponding to L1, L2 and L3 three model structures, if the i-th layer in a model structure selects to reserve the calculation model, then the input feature map f i+1 of the i+1-th layer is f i+1 = f i +F i (f i ) If the computation model of the i-th layer is not preserved, the input feature map f i+1 is f i+1 = f i The feature map V of the power image data is specifically represented as follows:

2. The variable-depth power equipment lightweight identification method of claim 1, wherein The model hierarchical training method based on image distribution difference is specifically implemented as follows: V conv = N f (N e (img)) Wherein, N e is an encoder based on Unet, N f is a dimension reduction network; after the image in the power image data is down-sampled to img∈[64,64,3], the feature extraction is carried out through the encoder N e , and the feature layer F e ∈[4,4,128] is obtained, then the dimension reduction network N f is used for dimension reduction to F f ∈[1,1,16], and the feature encoding V conv based on the convolutional neural network is obtained.

3. The variable-depth power equipment lightweight identification method of claim 2, wherein The convolutional neural network adopts Unet structure, which is composed of N e ={N e1 ,N e2 ,N e3 ,N e4} of four layers of convolutional block structure and N d =N d1 ,N d2 ,N d3 ,N d4} of four layers of up-sampling block; the convolutional neural network adopts power image data to train sample recovery capability; during training, based on the encoding and decoding of the power image data, the recovery prediction result of the image in the power image data is generated; and during application of the encoding, only N e +N f is used.

4. The variable-depth power equipment lightweight identification method of claim 1, wherein Three data sets Dataset={D1, D2, D3} are divided according to the difficulty, and then the following steps are repeated for feature extraction network and multi-branch detection head training: V = concat[V Tr ,V conv ] wherein V Tr denotes a feature encoding based on image features, V conv denotes a feature encoding based on a convolutional neural network, and concat denotes a concatenation operation.

5. The variable-depth power equipment lightweight identification method of claim 1, wherein The steps S2-S4 are repeated until the accuracy requirement is reached. The application further provides a computer readable storage medium, which comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, and when the processor executes the computer program instructions, the method steps of any one of claims 1-5 can be realized. S1, train L3 and 3 predictors using Dataset dataset, the process takes the mean of each predictor loss function as the final loss function, optimize model parameters, learning rate is set to 1e -3 ; S2, train L3 and L3 predictor using D3 dataset, learning rate set to 5e -5 ; S3, train L2 and L2 predictor using D2 dataset, learning rate set to 5e -5 ; S4, train L1 and L1 predictor using D1 dataset, learning rate set to 5e -5 ; ​ 6. A variable-depth power equipment lightweight identification system, characterized by, ​

Images