Real-time detection method and system based on key points in distribution network overhead line inspection

Abstract

The application discloses a real-time detection method and system based on key points in network distribution overhead line inspection, and belongs to the technical field of unmanned aerial vehicle automatic inspection real-time detection. Mobilenetv 3In the network structure, the first feature map is output Mobilenetv 3The network structure outputs the first feature map, and the first feature map is output Head The convolution layer outputs the second feature map, and the second feature map is output Soft - Argmax The regressor obtains coordinate expectation and variance through a full connection layer; a self-probability distribution constraint loss model is created, the coordinate expectation and the variance are taken as inputs of the self-probability distribution constraint loss model, a real probability distribution is simulated, and the real probability distribution is taken as supervision information to constrain Head The second feature map of the convolution layer. The detection performance of the key points is improved through the self-probability distribution constraint loss model, wherein the self-probability distribution constraint loss model gives the ability to perceive the surrounding distribution in the training process, and an adaptive Gaussian heat map is constructed to make up for the lack of distribution supervision information, so that more accurate probability distribution is provided for the predicted expected coordinate value.

Description

BACKGROUND

[0001] The present application belongs to the technical field of unmanned aerial vehicle automatic inspection real-time detection, in particular to a real-time detection method and system based on key points in overhead line inspection of distribution network. TECHNICAL FIELD

[0002] The existing unmanned aerial vehicle automatic inspection technology of overhead line of distribution network integrates target detection, key point detection, image classification, obstacle avoidance and other algorithms. The front-end computing power of the unmanned aerial vehicle is limited, and the inference speed of each algorithm is required to be high. At present, the key point detection algorithm deployed in the front-end of the unmanned aerial vehicle mainly uses the Heatmap-based method to ensure the detection accuracy.

[0003] The Heatmap-based method supervises the model learning of the Gaussian probability distribution graph. Since the Gaussian heat map needs to be rendered, and since the maximum value point in the heat map directly corresponds to the result, it is inevitable to maintain a relatively high-resolution heat map (commonly 64x64), which causes the front-end to consume a large amount of calculation and memory overhead, increases the algorithm inference time, and reduces the real-time performance of the unmanned aerial vehicle automatic inspection process of overhead line of distribution network. SUMMARY

[0004] The present application provides a real-time detection method and system based on key points in overhead line inspection of distribution network to solve the technical problems in the background art.

[0005] The present application adopts the following technical solution: a real-time detection method based on key points in overhead line inspection of distribution network, at least comprising the following steps:

[0006] A sample picture of a specified size is input into a Mobilenetv3 network structure, a first feature map is output from the Mobilenetv3 network structure, a second feature map is output from the first feature map through a Head convolution layer, a coordinate expectation is obtained from the second feature map through a Soft-Argmax regressor, and a variance is obtained through a fully connected layer; wherein the first feature map has an output dimension feature, and the second feature map has a number of key points on the sample image feature;

[0007] A self-probability distribution constraint loss model is created, the coordinate expectation and variance are taken as inputs of the self-probability distribution constraint loss model, a real probability distribution is simulated, and the real probability distribution is taken as supervision information to constrain the second feature map of the Head convolution layer.

[0008] In a further embodiment, the specified size of the sample picture is Nx256x256x3, wherein N represents batch-size;

[0009] The size and feature of the first feature map are represented as Nx Cx 8x 8, wherein C is the output dimension.

[0010] The size and feature of the second feature map are represented as N x J x 8 x 8, wherein J represents the number of key points;

[0011] The size of the coordinate expectation is N x J x 2; and the size of the variance σ is N x J x 2.

[0012] In a further embodiment, the self-probability distribution constraint loss model comprises: a flow generation model and a probability distribution constraint model.

[0013] The flow generation model is configured to pass through a loss function L mle The probability distribution constraint model is configured to pass through a constraint loss function L js Supervision information constraint is generated on the output of the Head convolutional layer; then, the self-probability distribution constraint loss function L sgs is represented as: L sgs = L mle + αL js , wherein α is an adjustment coefficient.

[0014] In a further embodiment, the flow generation model is created in the following process:

[0015] L mle represents a maximum likelihood loss function, and is represented as follows:

[0016] In the formula, represents the probability that the labeled coordinate GT of the key point appears at the position coordinate x axis of the second feature map, θ represents a detection parameter, represents the parameter of the flow generation model, X represents the position of the second feature map, I represents the input sample picture, μ g represents the label of the key point g, represents the learnable parameter predicted by maximum likelihood estimation.

[0017] In a further embodiment, the creation of the flow generation model further comprises the following process:

[0018] μ g is reparameterized using a reparameterization trick. In the formula, is the coordinate expectation; the second feature map output by the Head convolutional layer is generated by average pooling to obtain a feature of N x J x 1 x 1, and then passes through a fully connected layer to output a feature of N x 2J, and then the error In the formula, is obtained by reshaping the feature of N x 2J, The size of the error is N x J x 2; when the key point is clear and unoccluded, the error tends to 0; otherwise, when the key point is occluded, the predicted differs from the label μ g .

[0019] In further embodiments, the creation of the flow generation model further includes the following process:

[0020] P θ (X|I) is decomposed into wherein represents a (0, 1) normal distribution, represents the maximum likelihood loss of the flow generation model; then the formula is updated as follows:

[0021]

[0022] wherein,

[0023] then, the corresponding:

[0024]

[0025] As L mle loss gradient descends, the predicted will be closer and closer to the label μ g . represents the Jacobian matrix of the inverse of G.

[0026] In further embodiments, the creation process of the probability distribution constraint model is as follows:

[0027] Take as the variance, and μ g as the mean, to construct an adaptive Gaussian heat map, the size of which is the same as the size of the second feature map output by the Head convolution layer;

[0028] The adaptive Gaussian heat map is obtained by the following formula: After normalization, the value of x is [0.0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875];

[0029] L js represents the JS constraint loss function, then wherein D(*||*) represents the divergence measure, the distance between two distributions is measured using the JS divergence, f represents the second feature map output by the Head layer, represents the Softmax operation on the second feature map, represents a normal distribution with mean μ g and variance .

[0030] A real-time detection system based on key points in overhead power line inspection includes:

[0031] The Mobilenetv3 network structure includes a Head convolutional layer connected to the Mobilenetv3 network structure, a Soft-Argmax regressor connected to the Head layer, and a fully connected layer connected to the Soft-Argmax regressor.

[0032] It also includes: a self-probability distribution-constrained loss model embedded in the Soft-Argmax regressor and the fully connected layer.

[0033] In a further embodiment, the self-probability distribution constrained loss model includes: a flow generation model and a probability distribution constrained model.

[0034] The beneficial effects of this invention: This invention improves the detection performance of key points through a self-probability distribution constrained loss model, wherein the self-probability distribution constrained loss model assigns... Perception The ability to distribute around, and with An adaptive Gaussian heatmap is constructed to compensate for the lack of distribution supervision information and provide a more accurate probability distribution for predicting expected coordinate values. Attached Figure Description

[0035] Figure 1 This is a flowchart of a real-time detection method for key points in the inspection of overhead distribution lines. Detailed Implementation

[0036] Traditional heatmap-based methods explicitly render Gaussian heatmaps, allowing the model to learn the target distribution of the output. The input image is filtered to obtain the desired Gaussian heatmap. To ensure model performance and reduce quantization errors, it inevitably requires deconvolution to maintain a relatively high-resolution heatmap (commonly 64×64), resulting in significant computational and memory overhead on the front end and increased algorithm inference time. The invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] Example 1

[0038] like Figure 1 As shown, in order to reduce the algorithm inference time, this embodiment adopts the Mobilenetv3 network structure as the backbone network for the real-time detection method of key points in the inspection of overhead distribution lines.

[0039] The specific process includes the following steps: A sample image of a specified size is input into the MobileNetv3 network structure. The MobileNetv3 network structure outputs a first feature map. The first feature map is then passed through a Head convolutional layer to output a second feature map. The second feature map is passed through a Soft-Argmax regressor to obtain the coordinate expectation, and through a fully connected layer to obtain the variance σ. The first feature map has features representing the output dimension, and the second feature map has features representing the number of keypoints on the sample image. In a further embodiment, the specified size of the sample image is N×256×256×3, where N represents the batch size; the size and feature representation of the first feature map are N×C×8×8, where C is the output dimension, set to 96 in this embodiment; the size and feature representation of the second feature map are N×J×8×8, where J represents the number of keypoints, and in this embodiment, J is set to 2.

[0040] In other words, the sample image of size N×256×256×3 is passed through the backbone network to output a first feature map of size N×C×8×8. Then, it passes through a 1×1 head convolutional layer to output a second feature map of size N×J×8×8. The second feature map undergoes a soft-Argmax operation to obtain the coordinate expectation (size N×J×2) and then passes through a fully connected layer to obtain σ (size N×J×2). Because there is no deconvolution upsampling module in the above structure, compared with the head convolutional layer of the heatmap-based method, this method has almost no time consumption and memory consumption in the head convolutional layer.

[0041] To improve the detection performance of key points on insulators, this embodiment further includes the following process: creating a self-probability distribution-constrained loss model, using the coordinate expectation and variance σ as inputs to the self-probability distribution-constrained loss model to simulate the true probability distribution, and using this as supervisory information to constrain the second feature map of the Head convolutional layer. In a further embodiment, the size of the coordinate expectation is N×J×2; the size of the variance σ is N×J×2. Compared with regression-based methods, the self-probability distribution-constrained loss model generates different probability distributions based on the input samples, thereby constraining the second feature map of the Head convolutional layer, allowing the model to adaptively perceive the variance of the corresponding samples, thus improving the key point detection performance.

[0042] To achieve the above functionality, in a further embodiment, the self-probability distribution-constrained loss model includes: a flow generation model and a probability distribution-constrained model; the flow generation model is set to pass through a loss function L. mle The probability distribution constrained model is set to simulate a real probability distribution through a constrained loss function L. js The output of the Head convolutional layer generates supervisory information constraints; then, the loss function L is constrained by the probability distribution.sgs Represented as: L sgs =L mle +αL js Where α is an adjustment coefficient, and in this embodiment, α is 5.

[0043] To simulate the generation error probability distribution, the flow generation model creation process in this embodiment is as follows:

[0044] In the formula, The keypoint's labeled coordinates GT represent the probability that it appears on the x-axis of the second feature map, where θ represents the detection parameters. Let X represent the parameters of the flow generation model, X represent the position of the second feature map, I represent the input sample image, and μ represent the parameters of the flow generation model. g The label representing the key point g, This indicates that maximum likelihood estimation is used to predict the learnable parameters of θ.

[0045] Because the label μ of key point g in the sample image g The distribution of keypoints is inconsistent, and the training samples are too scattered. Directly learning the distribution of the true coordinates of keypoints will make it difficult for the model to fit the data. Therefore, to address this technical problem, the creation of a flow generation model also includes the following steps: using reparameterization techniques to adjust μ... g Reparameterization error in, The coordinate expectation is obtained by performing a softmax operation on the second feature map of size N×J×8×8 output from the Head convolutional layer and then integrating the result. The feature size is N×J×2. The second feature map output by the Head convolutional layer is used to generate N×J×1×1 features through average pooling, and then passed through a fully connected layer to output features of size N×2J. The error is then... In It is obtained from a feature of size N×2J through a reshape operation. The dimensions are N×J×2; when the key points are clear and unobstructed, the error is... The value approaches 0; conversely, when key points are occluded, the predicted value approaches 0. With label μ g There is a difference between the two. In other words, when the keypoints of the input sample image are semantically clear and unobstructed, the error tends to approach 0; conversely, when the input keypoints are occluded, the predicted error is lower. Errors can easily occur with labels, making... It is not easy to approach 0.

[0046] Since the error must satisfy the central limit theorem, then P... θ (X|I) decomposes into in Represents a (0, 1) normal distribution. Let L represent the maximum likelihood loss of the flow generation model. mle The trained generator G is adaptively adjusted based on the Q-normal distribution. The updated formula is as follows:

[0047] In the formula,

[0048] Then, the corresponding:

[0049]

[0050] With L mle Loss gradient descent, prediction It will get closer and closer to the label μ g , Let represent the Jacobian matrix of the inverse of G. In the above process, σ is initially a random value predicted by the model, and as L... mle Loss gradient descent, prediction It will get closer and closer to the label μ g , and L mle exist This will make It tends to 0. Therefore, as the gradient descent... It has the ability to predict the uncertainty of key points in input images, that is... Can represent coordinates The variance of the surrounding distribution.

[0051] As described above, this embodiment performs a softmax operation on the output features of the Head convolutional layer with a size of N×J×8×8, and calculates the expectation of the softmax features as the coordinate values. When the keypoints are clear and unobstructed, the distribution of the second feature map output by the Head convolutional layer conforms to μ. g The point is sharp, meaning the variance at point μ is smaller, and the distribution tends to be a normal distribution approaching 0 around the edges. However, the coordinates of the predicted keypoints obtained by expectation cannot constrain the probability distribution. To provide adaptive distribution supervision information to the second feature map output by the Head convolutional layer, this embodiment proposes to constrain the loss function L... js Supervisory information constraints are generated for the output of the Head convolutional layer.

[0052] Because of L mle Maximum likelihood loss can give This embodiment will demonstrate the ability to predict the uncertainty of key points in an input image. As variance, μ gAn adaptive Gaussian heatmap is constructed as the mean, the size of which is the same as the size of the second feature map output by the Head layer.

[0053] The adaptive Gaussian heatmap is obtained using the following formula: After normalization, x is [0.0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875];

[0054] L js To represent the JavaScript constraint loss function, then In the formula, D(*||*) represents the divergence measure, using JS divergence to measure the distance between two distributions, and f represents the second feature map output by the Head layer. This indicates that a Softmax operation is performed on the second feature map. The mean is μ g variance is The normal distribution.

[0055] In summary, this embodiment utilizes L sgs Self-probabilistic constraint loss improves keypoint detection performance, where L sgs During training Perception The ability to distribute around, and with An adaptive Gaussian heatmap is constructed to compensate for the lack of distribution supervision information, providing a more accurate probability distribution for predicting expected coordinate values. Because L... sgs This method does not require deconvolution to improve feature resolution. The model requires relatively fewer parameters: compared to existing technologies with the same backbone network, heatmap-based methods require 30M parameters, while this method only requires 3M. ​​It also significantly reduces model inference time: compared to existing technologies with the same backbone network, heatmap-based methods require 60ms, while this method only requires 3ms. Furthermore, this method maintains high keypoint detection performance with a low parameter count: in self-tested data, the heatmap-based method achieved a Map score of 0.675; in this embodiment, the Map score is 0.71.

[0056] Example 2

[0057] This embodiment provides a real-time detection system based on key points in overhead distribution line inspection to implement the real-time detection method described in Embodiment 1, including:

[0058] A Mobilenetv3 network structure, a Head convolution layer connected to the Mobilenetv3 network structure, a Soft-Argmax regressor connected to the Head layer, and a fully connected layer connected to the Soft-Argmax regressor;

[0059] Further comprising: a self-probability distribution constraint loss model embedded in the Soft-Argmax regressor and the fully connected layer.

[0060] The self-probability distribution constraint loss model comprises: a flow generation model and a probability distribution constraint model. The flow generation model is configured to pass through a loss function L mle simulate a real probability distribution, and the probability distribution constraint model is configured to pass through a constraint loss function L js generate supervised information constraints on the output of the Head convolution layer; then, the self-probability distribution constraint loss function L sgs is expressed as: L sgs = L mle + αL js , wherein α is an adjustment coefficient.

Claims

1. A real-time detection method based on key points in the inspection of distribution overhead lines, characterized in that, At least comprising the following steps: The sample picture of a specified size is input into a Mobilenetv3 network structure, a first feature map is output from the Mobilenetv3 network structure, the first feature map is output from a Head convolution layer to obtain a second feature map, the second feature map is subjected to a Soft-Argmax regressor to obtain a coordinate expectation, and a variance is obtained through a fully connected layer ; wherein the first feature map has features of an output dimension, and the second feature map has a number of key points on a sample image creating a self-probability distribution constraint loss model, the coordinate expectation and variance as an input of the self-probability distribution constraint loss model, simulating a real probability distribution and taking it as supervision information to constrain the second feature map of the Head convolution layer; The self-probability distribution constraint loss model comprises a flow generation model and a probability distribution constraint model; The flow generation model is configured to pass through a loss function Analog real probability distribution, the probability distribution constraint model is configured to pass through a constraint loss function Generate supervision information constraint to the output of the Head convolutional layer; then, the self-probability distribution constraint loss function Indicated as: , wherein, Is the adjustment coefficient; The flow process of creating the probability distribution constraint model is as follows: will be described below. As the variance, As the mean, an adaptive Gaussian heat map is constructed, which has the same size as the second feature map output by the Head convolutional layer.

2. The method of claim 1, wherein the method comprises: The specified size of the sample picture is N*256*256*3, where N represents batch-size; The size and feature of the first feature map are represented as N*C*8*8, where C is the output dimension; The size and feature of the second feature map are represented as N*J*8*8, where J represents the number of key points; The size of the coordinate expectation is N x J x 2; the size of the variance is N x J x 2.

3. The method of claim 1, wherein, The flow process of creating the flow generation model is as follows: Definitions denotes the maximum likelihood loss function, which is given by the formula: wherein, denotes a key point the probability that the annotated coordinates GT of a key point denotes a learned parameter, denotes a given parameter of the flow generating model, X denotes the position of the second feature map, I denotes the input sample picture, denotes a label of a key point denotes the prediction of the annotated coordinates GT of a key point the learned parameters.​ 4. The method of claim 3, wherein, The flow process of creating the flow generation model further comprises the following: Using the reparameterization trick reparam error where, is the coordinate expectation; the second feature map output by the Head convolution layer is generated by average pooling to generate a feature of size N x J x 1 x 1, and then output by a fully connected layer to generate a feature of size N x 2J, then the error in is obtained by reshaping the feature of size N x 2J, of size N x J x 2; when the key points are clear and unoccluded, the error tends to 0; otherwise, when the key points are occluded, the predicted coordinate expectation differs from the label .

5. The method of claim 3, wherein the method further comprises: The flow process of creating the flow generation model further comprises the following: will be described below with reference to the drawings. decomposed into wherein denotes a (0, 1) normal distribution, denotes the maximum likelihood loss of the stream generation model; then the formula is updated as follows: ； In the formulae, , ; Then, the corresponding: ； As Loss gradient descent, predicted Will be closer and closer to the label , The Jacobian matrix represents the inverse of the matrix G.

6. The method of claim 4, wherein the method further comprises: The adaptive Gaussian heat map is obtained by the following formula: ; After normalization, the value of x is [0.0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875]; denotes the JS constraint loss function, then ; where, denotes the divergence measure, the JS divergence is used to measure the distance between two distributions, f denotes the second feature map of the Head layer output, denotes the Softmax operation on the second feature map, denotes the normal distribution with mean and variance .

7. A real-time detection system based on key points in the inspection of distribution overhead lines, characterized in that, Comprise: The Mobilenetv3 network structure, the Head convolution layer connected to the Mobilenetv3 network structure, the Soft-Argmax regressor connected to the Head layer, and the full connection layer connected to the Soft-Argmax regressor; Further comprising: the self-probability distribution constraint loss model embedded in the Soft-Argmax regressor and the full connection layer; The self-probability distribution constraint loss model comprises a flow generation model and a probability distribution constraint model; The flow generation model is configured to pass through a loss function Analog real probability distribution, the probability distribution constraint model is configured to pass through a constraint loss function Supervision information constraint is generated to the output of Head convolution layer; then, the probability distribution constraint loss function It is expressed as: , wherein, The adjustment coefficient is: The flow process of creating the probability distribution constraint model is as follows: The As variance, As mean, construct an adaptive Gaussian heat map, which has the same size as the second feature map output by the Head convolutional layer. 8.The real-time detection system based on key points in the inspection of the distribution network overhead line according to claim 7, characterized in that, The self-probability distribution constraint loss model comprises a flow generation model and a probability distribution constraint model.

Images