Power intelligent inspection tracking method based on target perception depth feature compression

Abstract

The present invention discloses a method for intelligent power inspection and tracking based on target-aware deep feature compression. The method involves inputting an image of the power equipment to be tracked into a pre-trained convolutional neural network to extract deep features containing the power equipment to be tracked and its background area. These features, along with given Gaussian soft labels, are then input into a single-layer convolutional network and learned using negative balance loss until the network converges. Global average pooling is used to quantize the back-propagated gradient of each channel to obtain an initial importance weight, which is then binarized to obtain the final importance weight. During tracking, deep features of the target template are extracted only in the first frame, and deep features are extracted and compressed from the search area image in the next frame. Correlation operations are performed on the compressed target template features and the search area features to obtain the target's position. The present invention uses importance weights obtained by back-propagating network gradients to guide the deep feature compression process, obtaining concise deep features that reduce computational costs and improve tracking robustness.

Description

TECHNICAL FIELD

[0001] The application relates to a target tracking method based on deep feature compression, in particular to a power intelligent inspection tracking method based on target perception deep feature compression. BACKGROUND

[0002] A stable and reliable power system is the key to guarantee people's livelihood and economic development. In recent years, visual-based detection technology has attracted widespread attention in power detection. Both robot-based detection technology and unmanned aerial vehicle-based detection technology need to track a certain target, so a robust tracking method is very important. The development of deep learning has improved the tracking performance, but since the deep neural network model provides a large number of deep feature parameters, the calculation and storage costs are high, which is difficult to meet the real-time requirements of intelligent detection and is difficult to deploy on mobile intelligent detection equipment.

[0003] In recent years, many model lightweight methods have been used to compress deep models in order to migrate to mobile devices. The main five methods are: 1 parameter pruning, 2 parameter sharing, 3 low-rank decomposition, 4 design of compact convolution filters, and 5 knowledge distillation. Parameter pruning mainly removes redundant parameters by designing a criterion to judge whether a parameter is important. Parameter sharing mainly explores the redundancy of model parameters, and uses Hash or quantization technology to compress the weight. Low-rank decomposition uses matrix or tensor decomposition technology to estimate and decompose the original convolution kernel in the deep model. Designing compact convolution filters mainly reduces the storage and calculation complexity of the model by designing special structured convolution kernels or compact convolution calculation units. Knowledge distillation mainly uses the knowledge of a large network to migrate its knowledge to a distillation model. These methods tend to compress from the perspective of the entire network, while only the feature output of a certain layer is used in tracking.

[0004] Therefore, how to compress deep features is very important for improving the timeliness of tracking. SUMMARY

[0005] To solve the problems in the prior art, the application provides a power intelligent inspection tracking method based on target perception deep feature compression, which uses the importance weight obtained by back-propagating network gradient to guide the deep feature compression process; simple and effective deep features can be obtained to reduce the operation cost, and the robustness of tracking is improved.

[0006] The application adopts the following technical solutions.

[0007] A power intelligent inspection tracking method based on target perception deep feature compression, the method comprising the steps of:

[0008] Step 1, determine the context region image according to the power equipment to be tracked and the corresponding coordinates, input the context region image into the pre-trained convolutional neural network, and extract the deep features of the power equipment to be tracked and the background region thereof;

[0009] Step 2, construct a single-layer convolutional network, input the deep features of the power equipment to be tracked and the background region thereof and the given Gaussian soft label Y into the network, learn by using the negative balance loss until the network converges, and obtain the gradient information stored in the converged network;

[0010] Step 3, quantize the gradient of each channel by using global average pooling to obtain the initial importance weight Δ of each channel deep feature i , binary the initial importance weight to obtain the final importance weight Δf i ;

[0011] Step 4, during tracking, only extract the target template deep feature χ1 in the first frame, input the next frame search region image into the pre-trained convolutional neural network to extract the deep feature χ t , then use Δf i to compress χ t and χ1 to obtain χ′ t and χ′1.

[0012] Step 5, input the compressed target template feature and the search region feature into the matching network to perform cross-correlation operation to obtain the final response map, and the position with the maximum response value is the target position.

[0013] Further, in step 1, the corresponding coordinates include the upper left corner longitudinal and transverse axis positions and the length and width, forming a rectangular frame; the context region image with a specified length and width centered on the power equipment is calculated according to the coordinate information.

[0014] The context region image includes the power equipment to be tracked and the background region.

[0015] Further, in step 1, the deep feature extraction includes: inputting the image into the pre-trained convolutional neural network for one forward propagation, extracting the Conv4_3 and Conv4_1 layer features of the network, and then splicing the two layer features to obtain the final deep feature.

[0016] Further, in step 2, the negative balance loss is:

[0017]

[0018] Wherein, * is a correlation operation, lambda is a regularization parameter, X is a sample obtained from the depth feature of the power equipment to be tracked and its background area by taking a sliding window operation, W is a network weight value, Y is a Gaussian soft label, T(·) is a function used to balance the balance between difficult negative samples and simple negative samples.

[0019] Further, the function T(·) used to balance the balance between difficult negative samples and simple negative samples is:

[0020]

[0021] Wherein, m and n are two hyperparameters, and r is the loss W*X-Y.

[0022] Further, in step 3, the method for calculating the initial importance weight of each channel depth feature is:

[0023]

[0024] Wherein, i is the index of each channel feature, G AP (·) is a global average pooling function, L is a negative balance loss function, z i is the depth feature of the i-th channel.

[0025] Further, in step 3, the method for calculating the final importance weight of each channel depth feature is:

[0026]

[0027] When the initial importance weight is greater than 0, the final importance weight is 1, that is, the channel feature is retained; when the initial importance weight is less than or equal to 0, the final importance weight is 0, that is, the channel feature is deleted.

[0028] Further, in step 4, the depth feature compression method is:

[0029]

[0030] Wherein, chi is the depth feature before compression, chi' is the depth feature obtained after compression, is a multiplication operation.

[0031] Further, in step 5, the method for obtaining the position of the target is:

[0032]

[0033] Wherein, t is the index of the current frame, chi'1 is the compressed template feature, chi' t is the compressed search area feature of the t-th frame.

[0034] The application has the beneficial effect that, compared with the prior art, the application calculates the importance parameter of each channel feature through the gradient of the network back propagation, deletes the channel feature with low importance, and retains the important channel feature for representing the target; the simple and effective deep feature is obtained, and the operation speed is greatly improved.

[0035] The application can effectively compress the deep feature, solves the problem that the deep feature cannot meet the real-time requirement when used for power inspection tracking, indirectly highlights the important feature, and makes the tracking more robust. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The application is a power intelligent inspection tracking method based on target perception deep feature compression;

[0037] Figure 2 The application is a tracking effect comparison chart of the tracking method on a general data set TC-128;

[0038] Figure 3 The application is a tracking effect comparison chart of the tracking method on a power inspection scene. DETAILED DESCRIPTION

[0039] The application will be further described below with reference to the drawings. The following examples are only used to more clearly illustrate the technical solutions of the application, and cannot be used to limit the protection scope of the application.

[0040] As shown in Figure 1 The implementation process of the power intelligent inspection tracking method based on target perception deep feature compression is as follows:

[0041] Step 1, after determining the power equipment to be tracked and giving the corresponding coordinates, inputting the power equipment context area picture shot by the power intelligent inspection into a pre-trained convolutional neural network, extracting the deep feature containing the power equipment to be tracked and the background area thereof, and performing offline feature compression coefficient calculation;

[0042] The coordinates of the power equipment to be tracked are represented by a rectangular frame, and the corresponding coordinates have four position information, which are the vertical and horizontal axis positions of the upper left corner and the length and width. According to the coordinate information, an image containing the power equipment context area with the center position of the power equipment to be tracked as the center and the length and width of 180*180 is calculated.

[0043] The power equipment picture shot by the power intelligent inspection contains the power equipment to be tracked and the background area.

[0044] The image is input into a pre-trained convolutional neural network VGG-16 for one forward propagation, and the Conv4_3 and Conv4_1 layer features of the network are extracted, with sizes of 45x45x512 and 45x45x512 respectively. Among them, 45x45 is the feature length and width, and 512 is the feature channel number. Then the two layer features are spliced to obtain the final deep feature 45x45x1024.

[0045] Step 2, construct a single-layer convolutional network, input the deep features of the power equipment to be tracked and its background area and the given Gaussian soft label Y into the network, and learn using the negative balance loss until the network converges;

[0046] The negative balance loss is:

[0047]

[0048] Where * is the correlation operation, λ is the regularization parameter, X is the sample obtained by taking a sliding window operation on the deep features of the power equipment to be tracked and its background area, W is the network weight, and Y is the Gaussian soft label. T(·) is a function used to balance the difficult negative samples and simple negative samples:

[0049]

[0050] Where m and n are two hyperparameters, and r is the loss W*X-Y.

[0051] The deep features are input into a single-layer network for forward propagation, and the given label Y and the network output W*X are input into the negative balance loss function to calculate the loss r. When the loss r is not 0, the network is backpropagated for learning until the loss r is 0, and the network training is completed, i.e. the network is trained to convergence.

[0052] The gradient information stored in the network after the network training converges is used to calculate the importance weight of each channel.

[0053] Step 3, use global average pooling to quantize the backpropagation gradient of each channel to obtain the initial importance weight of each channel deep feature i , and the initial importance weight is binarized to obtain the final importance weight i .

[0054] The function of global average pooling is to add the gradient information stored in the network after the training to obtain a value, which can be used to represent the initial importance weight of each channel deep feature i .

[0055] The method for calculating the initial importance weight of each channel deep feature is:

[0056]

[0057] where i is the index of each channel feature, G AP is a global average pooling function, L is a negative balanced loss function, z i is the deep feature of the i-th channel.

[0058] The final importance weight of each channel deep feature is calculated by the following method:

[0059]

[0060] When the initial importance weight is greater than 0, the final importance weight is 1, that is, the channel feature is retained; when the initial importance weight is less than or equal to 0, the final importance weight is 0, that is, the channel feature is deleted.

[0061] The context region deep feature m x n x d is obtained, and the channel importance weight 1 x d is obtained. In the embodiment of the application, the context region deep feature is 45 x 45 x 1024, and the channel importance weight 1 x 1024 is obtained.

[0062] The importance weight parameter of each channel deep feature is calculated by the gradient of the network back propagation, the channel feature with low importance weight is deleted, the important channel feature is retained for representing the target, the simple and effective deep feature is obtained, and the operation speed is greatly improved.

[0063] Step 4, online tracking is performed, in the tracking, only the target template deep feature χ1 in the first frame is extracted, the next frame search region image is input into the pre-trained convolutional neural network to extract the deep feature χ t Then, Δf i is used to compress χ t and χ1 to obtain χ' t and χ'1.

[0064] The deep feature compression method is:

[0065]

[0066] where χ is the deep feature before compression, χ' is the deep feature obtained after compression, is a multiplication operation.

[0067] The target template deep feature w x h x d is compressed to obtain the deep feature w x h x c, the search region image deep feature m x n x d is compressed to obtain the deep feature m x n x c.

[0068] The target template depth feature is w x h x 1024, and the length and width w and h of the specific target template are calculated according to given target coordinate information, and the target template depth feature w x h x c is obtained after compression; the search area image depth feature is 45 x 45 x 1024, and the depth feature 45 x 45 x c is obtained after compression; in order to ensure the tracking speed, 100 and 30 channel features ranked first are selected from the Conv4_3 and Conv4_1 layer features respectively, so c <= 130.

[0069] In step 5, finally, the compressed target template feature and the search area feature are sent into the matching network for cross-correlation operation to obtain the final response map, and the position of the maximum response value is the target position.

[0070] The method for obtaining the position of the target is:

[0071]

[0072] Wherein, t is the index of the current frame, χ'1 is the compressed template feature, χ' is the compressed search area feature of the tth frame. t

[0073] The examples of the present application are tested on an Intel i7-9700 CPU and NVDIA GeForce RTX 2060 computer using MATLAB R2019a.

[0074] As shown in Figure 2 , we compared our method with classic tracking methods on the general dataset TC-128, including depth-based tracking methods TADT and ECO, and hand-crafted feature-based tracking methods KCF and CSK. Compared with the hand-crafted feature-based method, our method has better effect, and compared with the depth feature-based method, our method also has better performance. Figure 3 is the comparison of tracking effect in the power inspection scene with other methods, and our method also has excellent performance.

[0075] The present application has the beneficial effect that, compared with the prior art, the present application calculates the importance parameters of each channel feature through the gradient of network backpropagation, deletes the channel features with low importance, and retains the important channel features for representing the target; a simple and effective depth feature is obtained, and the operation speed is greatly improved.

[0076] The present application can effectively compress the depth feature, solve the problem that the depth feature cannot meet the real-time requirement when used for power inspection tracking, indirectly highlight the important features, and make the tracking more robust.

[0077] ​The applicant of the present application has made detailed description and illustration on the embodiments of the present application in combination with the drawings of the present application. However, those skilled in the art should understand that the above embodiments are only the preferred embodiments of the present application, and the detailed description is only for helping the readers to better understand the spirit of the present application, and is not intended to limit the protection scope of the present application. On the contrary, any improvement or modification based on the spirit of the present application should fall within the protection scope of the present application.

Claims

1. A power intelligent inspection and tracking method based on target perception deep feature compression, characterized in that: The method comprises the steps of: Step 1: Determine a context area image based on the power equipment to be tracked and its corresponding coordinates, input the context area image into a pre-trained convolutional neural network, and extract deep features of the power equipment to be tracked and its background area; Step 2: Construct a single-layer convolutional network. Input the deep features of the power equipment to be tracked and its background area together with the given Gaussian soft label Y into the network. Use negative balance loss to learn until the network converges, and obtain the gradient information stored inside the converged network. The negative balance loss is expressed as follows: Where * is the correlation operation, λ is the regularization parameter, X is the sample obtained by sliding window operation on the deep features of the power equipment to be tracked and its background area, W is the network weight, Y is the Gaussian soft label, and T(·) is the function used to balance the hard negative samples with the easy negative samples. The function T(·) used to balance the hard negative samples with the easy negative samples is expressed as follows: Among them, m and n are two hyperparameters, and r is the loss W*XY; Step 3: Use global average pooling to quantize the back-propagation gradient of each channel to obtain the initial importance weight Δ of each channel depth feature i , binarize the initial importance weight to get the final importance weight Δf i ; Step 4: During tracking, the target template depth feature χ1 is extracted only in the first frame, and the next frame search area image is input into the pre-trained convolutional neural network to extract the depth feature χ t , and then use Δf i χ t Compress with χ1 to get χ t ′ and χ1′; In step 5, the compressed target template features and the search area features are sent to the matching network for cross-correlation operation to obtain the final response map. The location with the maximum response value is the target position.

2. The electric power intelligent inspection and tracking method based on target perception deep feature compression according to claim 1 is characterized in that: In step 1, the corresponding coordinates include the vertical and horizontal axis positions and length and width of the upper left corner to form a rectangular frame; a context area image of a specified length and width centered on the power equipment is calculated based on the coordinate information; The context area image includes the electric power equipment to be tracked and the background area.

3. The electric power intelligent inspection and tracking method based on target perception deep feature compression according to claim 1 is characterized in that: In step 1, extracting deep features includes: inputting the image into a pre-trained convolutional neural network for forward propagation, extracting the network's Conv4_3 and Conv4_1 layer features, and then splicing the two layers of features to obtain the final deep features.

4. The electric power intelligent inspection and tracking method based on target perception deep feature compression according to claim 1 is characterized in that: In step 3, the method for calculating the initial importance weight of each channel depth feature is: Among them, i is the index of each channel feature, G AP (·) is the global average pooling function, L is the negative balance loss function, z i is the depth feature of the i-th channel.

5. The electric power intelligent inspection and tracking method based on target perception deep feature compression according to claim 4 is characterized in that: In step 3, the final importance weight of each channel depth feature is calculated as follows: When the initial importance weight is greater than 0, the final importance weight is 1, which means that the channel feature is retained; When the initial importance weight is less than or equal to 0, the final importance weight is 0, that is, the channel feature is deleted.

6. The electric power intelligent inspection and tracking method based on target perception deep feature compression according to claim 1 is characterized in that: In step 4, the deep feature compression method is: Among them, χ is the depth feature before compression, χ′ is the depth feature obtained after compression, It is a multiplication operation.

7. The electric power intelligent inspection and tracking method based on target perception deep feature compression according to claim 6 is characterized in that: In step 5, the method for obtaining the target position is: Among them, t is the index of the current frame, χ1′ is the compressed template feature, and χ t ′ is the search area feature after compression of the tth frame.

Images