Lightweight electric tower identification algorithm based on improved MobileNetV3

Abstract

The invention provides an improved MobileNetV3 lightweight electric tower identification algorithm, and belongs to the technical field of electric tower identification algorithms, and the algorithm comprises the steps: firstly, introducing an expansion convolution method into MobileNetV3 to expand the receptive field range of a convolution kernel, so as to reduce the loss of electric tower characteristics; then, a channel attention mechanism SENet in the network is replaced with a two-dimensional attention mechanism SimAM introduced without parameters, and the parameters of the network are reduced; and finally, in normalization operation, batch data normalization of the backbone network is replaced with group data normalization of which the accuracy is stable under various batch sizes, so that errors caused by small-batch data processing are reduced. Experiments on a self-made electric tower database prove that compared with a baseline algorithm, the network keeps a high recognition rate of 95.1%, the parameter quantity of the network is reduced, and rapid convergence and accurate recognition of the network on an electric tower image are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of electric tower recognition algorithms, and in particular to a lightweight electric tower recognition algorithm based on an improved MobileNetV3. Background Art

[0002] At present, the inspection of power towers is first carried out by manual visual inspection, and then the drone is operated to fly nearby to compare the position of the aerial photography return image to the specific position of the power tower, or the power tower is detected by image processing methods. However, image recognition and target detection rely on good lighting conditions and clear image imaging. At the same time, it is difficult to effectively distinguish places with a large number and types of power towers. Weather changes and foreign objects will also greatly affect the inspection accuracy, resulting in poor inspection results. In order to effectively solve this problem, the deep neural network method has become a feasible solution for combining power tower detection with AI technology. However, the existing deep learning network has problems such as complex calculations, large amount of calculations, and long detection time, making it difficult to apply under conditions of limited equipment resources. Summary of the invention

[0003] In view of this, the present invention aims at the deficiencies in the prior art and provides a lightweight power tower recognition algorithm based on improved MobileNetV3.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A lightweight electric tower recognition algorithm based on improved MobileNetV3 includes the following steps:

[0006] S1. Use the improved MobileNetV3 algorithm to extract the tower feature information;

[0007] S2. Use the attention mechanism to enhance the extracted tower feature information and build a neural network model;

[0008] S3, using the group normalization algorithm to train the neural network model;

[0009] S4. Based on the trained neural network model, the power towers are inspected and identified.

[0010] Furthermore, the improved MobileNetV3 algorithm includes:

[0011] In the MobileNetV3 algorithm, a dilated convolution operation is added to the first convolution operation, and a dilated convolution operation and a maximum pooling operation are added to the shortcut connection of each inverted residual structure;

[0012] Replace the original SENet attention mechanism in the MobileNetV3 algorithm with the SimAM attention mechanism.

[0013] Furthermore, the method for extracting the characteristic information of the electric tower is:

[0014] Infer the three-dimensional weights from the current neurons, and then optimize these neurons in turn. The energy function formula is:

[0015]

[0016] In the formula, is t and x i The linear transformation of t Represents the energy function value, which measures the linear separability between neuron t and other neurons. The lower the energy value, the higher the importance of neuron t; ω t ,b t are the weight and bias of the linear transformation, which are used to linearly transform the input features; y is a label or mark used to indicate the target neuron (usually set to 1) and other neurons (usually set to -1). This label plays a key role in simplifying calculations; x i represents the other neurons in the feature map except the target neuron t, i is the index in the spatial dimension, M is the number of neurons in the channel; t represents the target neuron, that is, the neuron currently being considered;

[0017] Using binary labels and adding regularization terms, the final energy function is defined as:

[0018]

[0019] Each channel has M = H × W energy functions;

[0020]

[0021] In the formula, The minimum energy is obtained by the formula:

[0022]

[0023] The neuron with the minimum energy is the characteristic information of the electric tower.

[0024] Furthermore, the SimAM attention mechanism is used to enhance the extracted tower feature information and construct a neural network model:

[0025]

[0026] Where E is the energy function obtained above t .

[0027] Furthermore, the method of training the neural network model using the group normalization algorithm is:

[0028] The channels in the neural network are divided into several groups, and then the normalized mean and variance are calculated in each group.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention uses the lightweight convolutional neural network MobileNetV3 as the basic network framework. First, the expansion convolution operation is added to the shortcut branch of the first convolution and the bottleneck layer to expand the receptive field. In the process of convolution, more characteristic information of the power tower is obtained, and non-characteristic information is suppressed to improve the convergence speed of the network, thereby reducing the number of network training times; secondly, the SimAM attention mechanism without parameter introduction is used to replace the attention mechanism of the original network; finally, the GN, which is very stable under various batch sizes, replaces the BN of the original network, reducing the error caused by small batch data processing. The experimental results on the power tower database show that the improved method improves the convergence speed of network training, reduces the parameters of the network, and demonstrates that the network in this chapter has good advancement and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of receptive fields with different expansion rates in an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of a flow chart of an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of an improved network structure in an embodiment of the present invention;

[0034] Figure 4 It is a schematic diagram of some pictures of the self-made electric tower data set in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0036] MobileNet is a lightweight convolutional neural network proposed by the Google team in 2017, which focuses on applications on mobile terminals or embedded devices. Compared with traditional convolutional neural networks, this method can greatly reduce model parameters and computational complexity, and the accuracy rate is also improved. Based on MobileNetV3, this paper proposes a lightweight tower recognition algorithm that improves MobileNetV3.

[0037] Embodiment: A lightweight electric tower recognition algorithm based on improved MobileNetV3 includes the following steps:

[0038] S1. Use the improved MobileNetV3 algorithm to extract the tower feature information;

[0039] S2. Use the attention mechanism to enhance the extracted tower feature information and build a neural network model;

[0040] S3, using the group normalization algorithm to train the neural network model;

[0041] S4. Based on the trained neural network model, the power towers are inspected and identified.

[0042] In S1, the improved MobileNetV3 algorithm includes:

[0043] (1) In the MobileNetV3 algorithm, a dilated convolution operation is added to the first convolution operation, and a dilated convolution operation and a maximum pooling operation are added to the shortcut connection of each inverted residual structure;

[0044] In order to extract as much characteristic information of the power tower as possible, the improved solution uses dilated convolution for convolution operation; the specific improvement is to add dilated convolution operation to the first convolution operation, and add dilated convolution operation and maximum pooling operation to the shortcut connection of each inverted residual structure to increase the receptive field of the convolutional neural network. In this way, more characteristic information can be extracted and the network convergence speed can be accelerated;

[0045] The dilated convolution operation is added according to the first convolution operation. The dilated convolution avoids the negative impact of the pooling operation. The dilated convolution is a special convolution behavior. This method increases the receptive field by injecting holes into the convolution area of ​​the standard convolution, and ensures that the size of the input and output feature maps is the same as that of the ordinary convolution feature map, thereby increasing the receptive field of the convolution kernel and extracting more detail information.

[0046] First, the introduction of the dilation rate can expand the receptive field without losing resolution, and the relative spatial position of the pixels remains unchanged. Second, the dilated convolution can obtain multi-scale context information. When multiple dilated convolutions with different dilation coefficients are superimposed, different receptive fields can provide multi-scale information. Finally, the dilated convolution can reduce the amount of computation because no additional parameters need to be introduced;

[0047]

[0048] Among them, x represents the horizontal coordinate in the feature map, y represents the vertical coordinate in the feature map; i represents the horizontal coordinate in the whole set, j represents the vertical coordinate in the convolution kernel; f represents the parameter value of the feature map, and g represents the parameter value of the convolution kernel;

[0049] The expansion rate represents the multiple by which an original element is expanded, such as Figure 1 The receptive field of the 3×3 convolution kernel at different dilation rates is shown. The change in the central area can clearly show the size of the expansion. Figure 1 (a) shows the receptive field of a normal convolution kernel with a dilation rate of d = 1 and a receptive field size of 3 × 3. Figure 1 (b) shows the receptive field of the convolution kernel when the dilation rate d = 2, and the size of the receptive field is expanded to 5×5. Figure 1 (c) shows the receptive field of the convolution kernel with a dilation rate of d = 3, and the size of the receptive field is expanded to 7 × 7. It can be seen that the dilated convolution expands the receptive field of the feature without the need for additional parameters, and the larger receptive field obtains more feature information.

[0050] (2) Replace the original SENet attention mechanism in the MobileNetV3 algorithm with the SimAM attention mechanism;

[0051] In order to reduce network parameters and improve network operation efficiency, the original SENet attention mechanism of MobileNetV3 is replaced with the SimAM attention mechanism. Compared with the original single-dimension (channel) attention mechanism, the replaced attention mechanism not only realizes parameter-free introduction, but also infers from two independent dimensions (space and channel);

[0052] Replace the original SENet attention mechanism of MobileNetV3 with the SimAM attention mechanism. Channel attention and spatial attention are the two main attention mechanisms at present. In neural networks, different channels in different feature maps usually represent different objects. Channel attention can adaptively redistribute the weights of each channel and determine the target of attention by selecting the object. The SimAM attention mechanism can make the channel and spatial domain attention work together well, avoiding the need to manually fine-tune the network structure.

[0053] The proposed SimAM attention mechanism can make the channel and spatial domain attention work together well, avoiding the need to manually fine-tune the network structure.

[0054] In S1, the method for extracting the characteristic information of the electric tower is:

[0055] Infer the three-dimensional weights from the current neurons, and then optimize these neurons in turn. The energy function formula is:

[0056]

[0057] In the formula, is t and x i Linear transformation of t Represents the energy function value, which measures the linear separability between neuron t and other neurons. The lower the energy value, the higher the importance of neuron t; ω t ,b t are the weight and bias of the linear transformation, which are used to linearly transform the input features; y is a label or mark used to indicate the target neuron (usually set to 1) and other neurons (usually set to -1). This label plays a key role in simplifying calculations; x i represents the other neurons in the feature map except the target neuron t, i is the index in the spatial dimension, M is the number of neurons in the channel; t represents the target neuron, that is, the neuron currently being considered;

[0058] Using binary labels and adding regularization terms, the final energy function is defined as:

[0059]

[0060] Each channel has M = H × W energy functions;

[0061]

[0062] In the formula, The minimum energy is obtained by the formula:

[0063]

[0064] The neuron with the minimum energy is the characteristic information of the electric tower;

[0065] In order to better implement the attention mechanism, the SimAM mechanism needs to evaluate the importance of each neuron and measure the linear separability between neurons to find important neurons. The lower the energy, the greater the difference between the neuron and the surrounding neurons, and the higher the importance level. Therefore, the importance of a neuron can be calculated by 1 / e * Compared with CBAM’s two-step method which requires too much calculation, SimAM’s calculation method using three-dimensional weights is relatively simple, which keeps the attention module lightweight.

[0066] For the existing attention machine modules, this paper conducts comparative analysis through Table 1. CAP stands for channel average pooling, GMP stands for spatial maximum pooling, CMP stands for channel maximum pooling, GSP stands for spatial dimension standard deviation, C2D stands for 2D convolution, FC stands for fully connected network, and BN stands for batch normalization. k stands for convolution layer parameters, r stands for restoration ratio, and C stands for current feature channel.

[0067] Table 1 Parameters of different attention modules

[0068]

[0069] As can be seen from the table above, the SimAM attention module has a clear advantage in terms of the number of parameters. Compared with the CBAM attention mechanism, this mechanism is a simple parameter-free attention mechanism, so it does not bring additional parameters to the network.

[0070] In S2, the SimAM attention mechanism is used to enhance the extracted tower feature information and construct a neural network model:

[0071]

[0072] Where E is the energy function obtained above t .

[0073] In S3, the method of training the neural network model using the group normalization algorithm is:

[0074] The channels in the neural network are divided into several groups, and then the normalized mean and variance are calculated in each group.

[0075] Batch Normalization (BN) is one of the important techniques in deep learning. It can enable various networks to be trained in parallel and plays an important role in the field of computer vision. However, the use of batch normalization also brings some problems. Batch normalization requires the batch size to be large enough to work properly. For small batches of data processing, the estimation of statistical data becomes inaccurate; in addition, if the batch size is reduced, the error of the model will increase rapidly. However, in large network training tasks, memory limitations force researchers to use batch normalization with smaller batches. The group normalization (GN) method divides the channels in the neural network into several groups, and then calculates the normalized mean and variance in each group. Unlike the traditional batch normalization method, the calculation of group normalization is independent of the batch size and can maintain very stable accuracy under various batch sizes. Since the group normalization method does not need to consider the batch size, the neural network can be trained at a smaller batch size, which is very useful for devices and applications with limited memory and computing resources.

[0076] like Figure 2 As shown, the process of the present invention is:

[0077] ① Image input, first increase the number of channels through 1×1 convolution;

[0078] ②Use deep convolution in high-dimensional space;

[0079] ③ Optimize feature map data through SimAM attention mechanism;

[0080] ④The number of channels is reduced after 1x1 convolution;

[0081] ⑤When the stride is equal to 1 and the size of the input and output feature maps are the same, the residual connection is used to connect the input and output.

[0082] The improved lightweight network structure is as follows Figure 3 As shown in Table 2, the module structures of the improved MobileNetV3 network are shown in Table 2.

[0083] Table 2 Module structures of the improved MobileNetV3 network

[0084] enter operate Expanded size Number of output channels SimAM NL Step Length <![CDATA[ 224 2 ×3 ]]> conv2d,dilation=2,GN 16 no HS 2 <![CDATA[112 2 × 16]]> bneck,dilation=2,,3*3,GN 16 16 no RE 1 <![CDATA[112 2 × 16]]> bneck,3*3,GN 64 24 no RE 2 <![CDATA[56 2 × 24]]> bneck,dilation=2,3*3,GN 72 24 no RE 1 <![CDATA[ 56 2 ×24 ]]> bneck,5*5,GN 72 40 yes RE 2 <![CDATA[ 28 2 ×40 ]]> bneck,dilation=2,5*5,GN 120 40 yes RE 1 <![CDATA[ 28 2 ×40 ]]> bneck,dilation=2,5*5,GN 120 40 yes RE 1 <![CDATA[ 28 2 ×40 ]]> bneck,3*3,GN 240 80 no HS 2 <![CDATA[14 2 × 80]]> bneck,dilation=2,3*3,GN 200 80 no HS 1 <![CDATA[14 2 × 80]]> bneck,dilation=2,3*3,GN 184 80 no HS 1 <![CDATA[14 2 × 80]]> bneck,dilation=2,3*3,GN 184 80 no HS 1 <![CDATA[ 14 2 ×80 ]]> bneck,dilation=2,3*3,GN 480 112 yes HS 1 <![CDATA[ 14 2 ×112 ]]> bneck,dilation=2,3*3,GN 672 112 yes HS 1 <![CDATA[ 14 2 ×112 ]]> bneck,dilation=2,5*5,GN 672 160 yes HS 1 <![CDATA[7 2 ×160 ]]> bneck,5*5,GN 960 160 yes HS 2 <![CDATA[7 2 ×160 ]]> bneck,dilation=2,5*5,GN 960 160 yes HS 1 <![CDATA[7 2 × 160]]> conv2d,1*1,GN - 960 - HS 1 <![CDATA[7 2 × 960]]> pool,7*7,GN - - - - 1 <![CDATA[1 2 × 960]]> conv2d,1*1 - 1280 - HS 1 <![CDATA[1 2 × 1280]]> conv2d,1*1 - 8 - - 1

[0085] In order to test the performance of the method proposed in the present invention, an experiment was conducted on the electric tower database. The hardware configuration is that the CPU is Intel (R) Core (TM) i5-11400, the GPU is NVIDIA GeForce RTX 3060Ti, and the RAM is 16GB. The experiment uses the stochastic gradient descent method to update the parameters, sets the initial learning rate to 0.0001, and sets the total training batches to 16. The purpose of parameter setting is to avoid the problem of gradient disappearance or gradient explosion during model training.

[0086] The steps to improve the network are as follows:

[0087] First, MobileNetV3 Large is used as the basic framework. On this basis, the dilated convolution operation is used for the first convolution of the network, and then the dilated convolution operation is used for the shortcut connection. Finally, the maxpool operation is added at the shortcut connection to perform power tower recognition on the self-made power tower dataset.

[0088] In order to further compare the performance of the present invention, the experiment selected the residual neural network ResNet18 as the baseline algorithm. This model not only won the championship in the 2015 ILSVRC (ImageNet Large Scale Visual Recognition Challenge), but is also a representative deep learning method in power detection and recognition. At the same time, the experiment also selected the MobileNetV2 model as the comparison algorithm, and conducted precision detection experiments on the lightweight model.

[0089] The self-made electricity tower dataset comes from electricity tower pictures in various shooting environments selected from the Internet, including 21 cat-shaped electricity towers, 19 wine glass-shaped electricity towers, and 18 sheep-head-shaped electricity towers. Figure 4 As shown in the figure, the types of electric towers displayed from left to right are sheep head type electric tower, cat type electric tower and wine glass type electric tower. The pictures are expanded using data enhancement methods including picture rotation, flipping, blurring, adding noise and changing brightness. The three types of pictures after expansion are 378, 342 and 324, respectively, for a total of 1044. The experiment was carried out in a ratio of 9:1 between the training and test sets, with 20 epochs, 16 batch_size and 224 input image size. The experimental results are shown in Table 3. The present invention can automatically complete the recognition of electric towers. Compared with ResNet18 with fewer layers, the recognition rate of the present invention is higher and the required parameters are smaller; compared with the lightweight network, the recognition rate of the method of the present invention is slightly lower than that of MobileNetV3. The reason is that the homemade electric tower database is small, which objectively cannot make the attention mechanism and dilated convolution operation of the algorithm fully play its role, but the number of parameters is much lower than that of the original method; compared with MobileNetV2, the convergence speed of the algorithm network is comparable, and the recognition rate is much higher than that of the MobileNetV2 algorithm.

[0090] Table 3 Comparison of the proposed method and the baseline algorithm on the power tower database

[0091]

[0092] In summary, the experimental results of the present invention on the homemade power tower database show that the improved method improves the convergence speed of network training, reduces network parameters, and has higher detection accuracy, which proves that the algorithm of the present invention has good advancement and robustness.

[0093] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A lightweight power tower recognition algorithm based on improved MobileNetV3, characterized in that: The following steps are involved: S1. Use the improved MobileNetV3 algorithm to extract the tower feature information; S2. Use the attention mechanism to enhance the extracted tower feature information and build a neural network model; S3, using the group normalization algorithm to train the neural network model; S4. Based on the trained neural network model, the power towers are inspected and identified.

2. According to the lightweight power tower recognition algorithm based on improved MobileNetV3 according to claim 1, it is characterized in that: The improved MobileNetV3 algorithm includes: In the MobileNetV3 algorithm, a dilated convolution operation is added to the first convolution operation, and a dilated convolution operation and a maximum pooling operation are added to the shortcut connection of each inverted residual structure; Replace the original SENet attention mechanism in the MobileNetV3 algorithm with the SimAM attention mechanism.

3. According to claim 2, the lightweight power tower recognition algorithm based on improved MobileNetV3 is characterized in that: The method for extracting the characteristic information of the electric tower is: Infer the three-dimensional weights from the current neurons, and then optimize these neurons in turn. The energy function formula is: In the formula, is t and x i Linear transformation of t Represents the energy function value, which measures the linear separability between neuron t and other neurons. The lower the energy value, the higher the importance of neuron t; ω t ,b t They are the weight and bias of the linear transformation, which are used to linearly transform the input features; y is a label or tag used to indicate the target neuron and other neurons; x i represents the other neurons in the feature map except the target neuron t, i is the index in the spatial dimension, M is the number of neurons in the channel; t represents the target neuron, that is, the neuron currently being considered; Using binary labels and adding regularization terms, the final energy function is defined as: Where λ is a regularization parameter, which is used to prevent overfitting and help optimize the solution. It is a summation operation used to calculate the contribution of all neurons except the target neuron t to the energy function. Each channel has M = H × W energy functions; In the formula, The minimum energy is obtained by the formula: The neuron with the minimum energy is the characteristic information of the electric tower.

4. According to claim 3, the lightweight power tower recognition algorithm based on improved MobileNetV3 is characterized in that: The SimAM attention mechanism is used to enhance the extracted tower feature information and construct a neural network model: Where E is the energy function obtained above t .

5. According to claim 4, the lightweight power tower recognition algorithm based on improved MobileNetV3 is characterized in that: The method of training the neural network model using the group normalization algorithm is: The channels in the neural network are divided into several groups, and then the normalized mean and variance are calculated in each group.

Images