A target detection method based on dynamic super-resolution, a transmission line component detection method and system

Abstract

The present invention discloses a target detection method based on dynamic super-resolution, a power transmission line component detection method and a system. The method sends an input image to be detected to a dynamic super-resolution module; the dynamic super-resolution module consists of a dynamic path selection network and two different types of super-resolution networks, and the dynamic path selection network adaptively selects the super-resolution network according to the characteristics of the input image; the super-resolution network performs image super-resolution processing on the input image and then sends it to the target detection module to obtain the target detection result. The method of the present invention uses a dynamic super-resolution method to improve the detection performance of low-resolution input images and small targets, and has a fast detection speed, which can meet the requirements of real-time detection. In particular, when the method is applied to the inspection process of power transmission lines by unmanned aerial vehicles, it can promote the replacement of manual inspection of power transmission lines by unmanned aerial vehicle inspection, and has high practical value.

Description

Technical Field

[0001] The present invention belongs to the field of image target detection, and in particular relates to a target detection method based on dynamic super-resolution, and a power transmission line component detection method and system. Background Art

[0002] Transmission lines are an indispensable part of the power system. In my country, a large number of transmission lines are built in places with harsh conditions such as mountains and wilderness. They are easily affected by various external conditions. Once the transmission line components are damaged by external influences, it is bound to cause power outages for a large area of ​​users, which will have a great impact on people's lives and industrial production. Therefore, daily inspections of transmission line components are crucial to the stable operation of the entire power grid. Traditional manual inspections consume a lot of manpower, material resources, and financial resources, and the accuracy is prone to errors due to differences in staff experience and ability. With the rapid development of the power grid, the workload of manual inspections will become larger and larger. However, under some extremely harsh conditions, manual inspections not only cannot guarantee the timeliness of work, but also pose great safety risks. Automatic detection of transmission line components can not only save a lot of human resources, but also greatly improve the speed and accuracy of inspections. Therefore, it is imperative to replace traditional manual inspections with robot inspections and drone inspections with automatic detection and recognition capabilities. Exploring suitable automatic detection algorithms for transmission line components is of great significance to ensuring the smooth operation of the entire power grid.

[0003] With the increasing development of deep learning and machine vision technology, more and more scholars have applied target detection technology based on image processing to the inspection process of power transmission lines. Most methods can achieve good performance on high-resolution input images. However, during robot inspection and drone inspection, since the equipment must maintain a certain safe distance from the transmission line and some transmission line components are small in themselves, the image resolution is low and the target is small, making it difficult to detect. The image super-resolution network (super-resolution network) reconstructs low-resolution images to generate corresponding high-resolution images, but the features of each image are different. For images with different features, the same super-resolution network cannot achieve good detection results.

[0004] Similarly, for other application fields, target detection in low-resolution images or small target images also faces the above problems. Therefore, the present invention proposes a target detection method based on dynamic super-resolution to overcome the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a target detection method based on dynamic super-resolution, a transmission line component detection method and system to address the problem of poor target detection effect caused by low image resolution and small detection targets, such as in the application scenario of power transmission line component detection, the target in the image captured during the transmission line inspection by drone is small, the target resolution is low, and it is difficult to accurately complete component detection. The method introduces image super-resolution into the model, and uses the dynamic super-resolution method to adaptively select a suitable super-resolution network, improve the quality of the input image, and improve the detection performance of low-resolution input images and small targets.

[0006] On the one hand, the present invention provides a target detection method based on dynamic super-resolution, comprising the following steps:

[0007] Step 1: The image to be detected is sent to the dynamic super-resolution module to adaptively select the super-resolution network and perform image super-resolution processing;

[0008] The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks, and the dynamic path selection network adaptively selects the super-resolution network according to the characteristics of the input image;

[0009] Step 2: Input the image output by the super-resolution network into the trained target detection module to obtain the target detection result;

[0010] The input data of the target detection module is an image, and the output data is the target detection result on the image.

[0011] The target detection method of the present invention performs outstandingly in the target detection accuracy of low-resolution images and in the detection of small targets in images. Therefore, the method of the present invention can be applied to low-resolution images, high-resolution small target images, or any image with low target detection accuracy due to low image resolution and small targets. Among them, the classification criteria of low resolution and small targets are determined by the industry standards and experimental results in this field, and the present invention does not make specific limitations on this. As long as it is used to solve the problem of low target detection accuracy due to low image resolution and small targets, the technical solutions that do not deviate from the concept of the present invention belong to the protection scope of the present invention.

[0012] In a second aspect, the present invention provides a method for detecting power transmission line components based on dynamic super-resolution, comprising the following steps:

[0013] Step 1: The transmission line image is transmitted to the dynamic super-resolution module to adaptively select the super-resolution network and perform image super-resolution processing;

[0014] The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks, and the dynamic path selection network adaptively selects the super-resolution network according to the characteristics of the input image;

[0015] Step 2: Input the image output by the super-resolution network into the trained target detection module to obtain the component detection result on the transmission line image;

[0016] The input data of the target detection module is an image, and the output data is the target detection result on the image.

[0017] Optionally, two super-resolution networks are provided in the dynamic super-resolution module, one super-resolution network is a RFDN network (Residual Feature Distillation Network for Lightweight Image Super-Resolution), and the other super-resolution network is a SRCNN network (Super-Resolution Convolutional Neural Network).

[0018] Among them, the RFDN network uses shallow residual blocks to learn more representative features through multiple feature distillation connections, which can take into account contextual information at the same time. The use of shallow residual blocks does not introduce additional parameters. Therefore, the network can still achieve very good super-resolution effects while being lightweight enough.

[0019] The SRCNN network is the pioneering work of deep learning image super-resolution. Although the SRCNN structure is simple and only uses three convolutional layers, its effect is better than traditional image super-resolution methods. SRCNN first uses bicubic interpolation to enlarge the low-resolution image to the target size, then fits the nonlinear mapping through a three-layer convolutional network, and finally outputs the high-resolution image result. SRCNN uses 9×9, 1×1, and 5×5 convolution kernels in the feature extraction, nonlinear mapping, and image reconstruction stages. The two larger convolution kernels can obtain a larger receptive field. Although the SRCNN structure is very simple and only uses three convolutional layers, its super-resolution effect is still greatly improved compared to traditional methods.

[0020] Optionally, the dynamic super-resolution module and the target detection module constitute a target detection network, and the training process of the target detection network is divided into pre-training and fine-tuning;

[0021] The pre-training process is:

[0022] (1) collecting original images and marking detection targets on the original images, thereby constructing a target detection data set, and then using the target detection data set to pre-train a network of a target detection module;

[0023] Wherein, the original image of the target detection data set is downsampled to obtain a downsampled image;

[0024] (2) inputting the downsampled image in step (1) into a pre-trained target detection module for detection, wherein the image is classified according to the confidence of the detection result;

[0025] (3) using the classified image in step (2) to pre-train the dynamic path selection network in the dynamic super-resolution module;

[0026] Wherein, the correspondence between the classification image category and the super-resolution network category is set, the input data of the dynamic path selection network is the image, and the output data is the selection probability or selection result of each super-resolution network;

[0027] (4) using the super-resolution data set to pre-train each super-resolution network in the dynamic super-resolution module, wherein, based on a preset super-resolution multiple, the super-resolution network is pre-trained using the super-resolution data set to obtain a super-resolution weight of the super-resolution network;

[0028] The fine-tuning process is:

[0029] Based on the trained target detection module and dynamic super-resolution module, and using the downsampled image dataset in step (1) or the training image dataset of the same category as the image to be detected, and the loss function, the target detection network is fine-tuned to obtain a target detection model based on dynamic super-resolution.

[0030] In some application scenarios, a high-resolution original image is obtained for the current application object, and pre-training is performed before fine-tuning using the image downsampled from the original image. After the original image is downsampled, the resolution is reduced to obtain a low-resolution image or a small target image with reduced resolution.

[0031] In some application scenarios, for the current application object, the target detection model of other application objects is directly used for fine-tuning to obtain a target detection model suitable for the current application object; the fine-tuning process is performed based on the training image dataset of the current application object.

[0032] Optionally, the fine-tuning process uses a composite loss function for training, and the composite loss function is as follows:

[0033]

[0034] In the formula, Loss represents the composite loss value, Loss SR Represents the dynamic super-resolution module loss, Loss Det Represents the target detection module loss, α and β are weight coefficients;

[0035] The dynamic super-resolution module loss adopts the MSE (Mean Square Error) loss function, and the target detection module loss consists of anchor box regression loss, classification loss and confidence loss.

[0036] The MSE loss calculation formula is as follows, X(i,j,m), Y(i,j,m) represent the reference image and the network output image respectively, where (i,j) (i=1,2,...,H; j=1,2,...,W) represents the pixel coordinates, W is the width of the image, H is the height of the image, m (m=1,2,...,B) represents the current channel, and B is the number of channels;

[0037]

[0038] The target detection module loss is as follows:

[0039] Loss Det =Loss box +Loss cl +Loss cf

[0040] In the formula, Loss box The anchor box regression loss GIoU not only considers the overlap between the predicted box and the true box, but also the distance between them. During the training process, the predicted box can be closer and closer to the true box. Therefore, in YOLOv5, the GIoU loss function is used to optimize the anchor box regression process. Its definition is as follows:

[0041]

[0042] Among them, A c It is the smallest rectangular area containing the real box and the predicted box, U is the union of the real box and the predicted box; IoU is the intersection over union of the real box and the predicted box, and its definition is as follows:

[0043]

[0044] Among them, A and B represent the real box and the predicted box respectively.

[0045] Among them, Loss cl and Loss cf They represent target classification loss and confidence loss respectively. Both losses use the logistic cross entropy loss function. The logistic cross entropy loss function combines the Sigmoid function with the cross entropy loss function to synthesize the two steps into one step.

[0046] Optionally, the dynamic path selection network consists of a convolutional layer, an average pooling layer, a fully connected layer and a Softmax layer, and the dynamic path selection network outputs the selection probability of each super-resolution network.

[0047] For example, the dynamic path selection network consists of five convolutional layers responsible for extracting features, an average pooling layer, a fully connected layer, and a Softmax layer responsible for outputting prediction probabilities. The first convolutional layer has a 4×4 convolution kernel with a step size of 4, and the next four have 1×1 convolution kernels with a step size of 1. The dynamic path selection network determines the probability p of the best detection effect after the two super-resolution networks based on the features of the input image. 1 and p 2 , if p 1 Greater than p 2 , the image is sent to super-resolution network 1, otherwise the input image is sent to super-resolution network 2. The dynamic path selection network can adaptively select a more suitable super-resolution network according to the characteristics of the input image.

[0048] Optionally, the target detection module is a target detection model constructed based on YOLOv5, and the YOLOv5 at least includes a backbone network, a neck network and a prediction network.

[0049] For example, the input image is randomly rotated, randomly translated, and randomly arranged using the Mosaic data enhancement method at the input end, and four images are spliced ​​into one image. At the same time, the adaptive anchor frame operation is used to automatically calculate the most appropriate anchor frame size. After the image is preprocessed, it will be sent to the backbone network for feature extraction.

[0050] The backbone network mainly consists of Focus slicing operation, CBL module, CSP module and SPP module. The Focus operation first divides the input image into 4 feature layers of the same size, then splices these four feature layers and passes them through a CBL module, and the output size is half of the original input image. CBL is a module composed of convolution, batch normalization and Leaky Relu. There are two types of CSP structures. The two CSP structures are similar. The second one just replaces the residual module in the first one with a CBL module. The features extracted by the backbone feature network will be sent to the neck network. The neck network draws on the FPN and PANet structures to strengthen information propagation and retain more useful spatial information. In the prediction stage, the three feature layers of different sizes output by the neck network are convolved by 1×1, and three feature maps are output, which are 8 times, 16 times and 32 times the input image of the detection network. These three feature maps of different sizes are responsible for predicting small, medium and large targets respectively.

[0051] In a third aspect, the present invention provides a system based on the above-mentioned target detection method, which at least comprises: an image acquisition module, a dynamic super-resolution module and a target detection module;

[0052] Wherein, the image acquisition module is used to acquire the image to be detected;

[0053] The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks, and the dynamic path selection network adaptively selects a super-resolution network to perform image super-resolution processing according to the features of the input image;

[0054] The target detection module is used to detect the image output by the super-resolution network to obtain the target detection result.

[0055] In a fourth aspect, the present invention provides a terminal, comprising: one or more processors, and a memory storing one or more computer programs;

[0056] The processor calls the computer program to execute:

[0057] The invention provides steps of a target detection method based on dynamic super-resolution or steps of a power transmission line component detection method based on dynamic super-resolution.

[0058] In a fifth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program is called by a processor to execute:

[0059] The invention provides steps of a target detection method based on dynamic super-resolution or steps of a power transmission line component detection method based on dynamic super-resolution.

[0060] Beneficial Effects

[0061] 1. The present invention provides a target detection method and a power transmission line component detection method based on dynamic super-resolution, which introduces a super-resolution network and uses a dynamic path selection network to adaptively select a more suitable super-resolution network, thereby improving the detection performance of low-resolution input images and small targets, and the detection speed is fast, which can meet the requirements of real-time detection, and effectively solves the problem of poor target detection effect caused by low image resolution and small detection targets. In particular, it is applied to the detection of power transmission line components, suitable for images taken during the inspection process of unmanned aerial vehicle power transmission lines, and promotes unmanned aerial vehicle inspections to replace manual inspections of power transmission lines, which has high practical value.

[0062] 2. The target detection algorithm based on dynamic super-resolution provided by the present invention sets two or more super-resolution networks. As in the preferred embodiment of the present invention, two super-resolution networks, RFDN and SRCNN, are set. Although SRCNN has a simple structure, SRCNN uses a larger convolution kernel, has a larger receptive field, and can obtain better edge features. RFDN uses feature distillation links and shallow residual blocks, which can take into account contextual information at the same time, and compared with the pre-upsampling mode used by SRCNN, the post-upsampling mode used by RFDN can greatly reduce the amount of network calculations. In order to combine the advantages of the two super-resolution networks, the present invention proposes a dynamic path selection network, which learns the features of the input image through the dynamic path selection network, and can adaptively select the most suitable super-resolution network, thereby achieving the best super-resolution effect and improving the performance of subsequent detection modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of the training process of the target detection network in the present invention;

[0064] Figure 2 A network structure diagram of the method described in the example of the present invention;

[0065] Figure 3 This is a structural diagram of the target detection module in the method described in the example of the present invention. DETAILED DESCRIPTION

[0066] The target detection method based on dynamic super-resolution provided by the present invention is applicable to solving the problem of poor target detection effect in other fields due to low image resolution and small detection target in addition to the application field of power transmission line component detection. The power transmission line component detection method based on dynamic super-resolution provided by the present invention is aimed at the application field of transmission lines, and is particularly applicable to images taken during the inspection process of unmanned aerial vehicles. The low-resolution images and small targets referred to in the present invention are divided according to industry standards and determined by test results, and the present invention does not make specific restrictions. The core idea of ​​the present invention is to adaptively select a suitable super-resolution network through a dynamic super-resolution method to improve image quality and detection accuracy for the problem of low image resolution and small target detection accuracy. Therefore, without departing from the concept of the present invention, the target detection technology of any resolution image and the target detection technology of any size target that solves the problem of poor target detection effect due to low image resolution and small detection target fall within the protection scope of the present invention. The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0067] Embodiment 1:

[0068] This embodiment provides a power transmission line component detection method based on dynamic super-resolution, which is applied to power transmission line component detection. In the actual application process, it includes the following steps:

[0069] Step 1: The transmission line image is transmitted to the dynamic super-resolution module to adaptively select the super-resolution network and perform image super-resolution processing;

[0070] Step 2: Input the image output by the super-resolution network into the trained target detection module to obtain the component detection result on the transmission line image.

[0071] The target detection network of the present invention is composed of two parts: a dynamic super-resolution module and a target detection module. The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks. In this embodiment, two super-resolution networks are set.

[0072] The dynamic path selection network in this embodiment consists of five convolutional layers responsible for extracting features, an average pooling layer, a fully connected layer, and a Softmax layer responsible for outputting predicted probabilities. The first convolutional layer has a 4×4 convolution kernel with a step size of 4, and the next four have a 1×1 convolution kernel with a step size of 1. The dynamic path selection network determines the probability p of the best detection effect after the two super-resolution networks based on the features of the input image. 1 and p 2 , if p 1 Greater than p 2 , the image is sent to super-resolution network 1, otherwise the input image is sent to super-resolution network 2. The dynamic path selection network can adaptively select a more suitable super-resolution network according to the characteristics of the input image.

[0073] It should be understood that in other feasible embodiments, if the number of super-resolution networks exceeds 2, the corresponding probability and the division criteria of the super-resolution network are adjusted, and the training rules set in the training phase shall prevail. In addition, the number, size, and size of the convolutional layers in the dynamic path selection network can be set according to the actual application environment and effect, and are not limited to the above examples.

[0074] The purpose of the super-resolution network is to multiply the size of the input image by the super-resolution factor to obtain a high-resolution image result, which is then sent to the target detection module.

[0075] The purpose of the target detection module is to obtain target detection results from the input image. In this embodiment, the YOLOv5 algorithm is selected. The YOLOv5 network structure is as follows: Figure 3As described above. Mosaic data enhancement is used at the input end to perform random rotation, random translation, random arrangement and other operations on the input image, and 4 images are spliced ​​into 1 image. At the same time, adaptive anchor frame operation is used to automatically calculate the most appropriate anchor frame size. After the image is preprocessed, it will be sent to the backbone network for feature extraction.

[0076] The backbone network mainly consists of Focus slicing operation, CBL module, CSP module and SPP module. The Focus operation first divides the input image into 4 feature layers of the same size, then splices these four feature layers and passes them through a CBL module, and the output size is half of the original input image. CBL is a module composed of convolution, batch normalization and Leaky Relu. There are two types of CSP structures. The two CSP structures are similar. The second one just replaces the residual module in the first one with a CBL module. The features extracted by the backbone feature network will be sent to the neck network. The neck network draws on the FPN and PANet structures to strengthen information propagation and retain more useful spatial information. In the prediction stage, the three feature layers of different sizes output by the neck network are convolved by 1×1, and three feature maps are output, which are 8 times, 16 times and 32 times the input image of the detection network. These three feature maps of different sizes are responsible for predicting small, medium and large targets respectively.

[0077] It should be understood that in other feasible embodiments, adaptive adjustment of the network structure also falls within the protection scope of the present invention, and the present invention is not limited to the YOLOv5 network structure to achieve target detection.

[0078] Based on the above logic, in practical applications, it is necessary to first train the dynamic super-resolution module and the target detection module, that is, train the target detection network to obtain a target detection model based on dynamic super-resolution. The training process is as follows:

[0079] (1.0) Create a data set for power transmission line component detection.

[0080] Transmission line inspection images are collected and detection targets on the transmission line inspection images are annotated, thereby constructing a target detection data set. In this embodiment, the target detection data set is divided into a training set and a test set.

[0081] Among them, the inspection images of power lines collected by drones or obtained by other means are collected, and the resolution of the collected inspection images is higher than the low-resolution inspection images conventionally described in the art. The collected images are then screened, and images that do not contain transmission line components are discarded. If the collected data is small, the number of data sets can be expanded by using data augmentation methods such as image rotation, translation, and horizontal flipping. For images containing transmission line components, use Labelme software to annotate the transmission line components. After the annotation is completed, the training set and the test set are allocated to complete the production of the transmission line component detection data set.

[0082] (2.0) Use the dataset to pre-train the network of the target detection module (YOLOv5).

[0083] This embodiment uses the data set produced in step (1.0) to train YOLOv5. The input data of the training process is the power transmission line component detection data set, which includes the image and the annotation information of the detection target. The input data of the trained target detection module (YOLOv5) is the power transmission line inspection image, and the output result is the target detection result in the image.

[0084] (3.0) Create a dataset for training dynamic path selection networks.

[0085] Detecting the low-resolution image using the pre-trained target detection module, and classifying the low-resolution image based on confidence;

[0086] The low-resolution image is obtained by downsampling the power transmission line inspection image in step (1). In this embodiment, for two super-resolution networks, images with a confidence level lower than 0.75 in the detection results are classified into one category A, and the remaining images are classified into another category B.

[0087] (4.0) Use the data set of step (3.0) to pre-train the dynamic path selection network in the dynamic super-resolution module to obtain pre-trained weights.

[0088] During the training process, a class of images A with a confidence prediction score lower than 0.75 are set to RFDN processing; a class of images B with a confidence prediction score greater than or equal to 0.75 are set to SRCNN processing, that is, the dynamic path selection network is pre-trained using the data set composed of these two types of data.

[0089] (5.0) Each super-resolution network is pre-trained using the DIV-2K dataset.

[0090] Wherein, based on the preset super-resolution multiple, the super-resolution network is pre-trained using the DIV-2K data set to obtain the super-resolution weights of the super-resolution network. The DIV-2K data set is a commonly used data set for super-resolution in this field, so it is not described in detail.

[0091] At this point, the pre-training of the dynamic path selection network and the super-resolution network in the dynamic super-resolution module, as well as the pre-training of the target detection module, are completed, and a pre-trained target detection network is obtained.

[0092] (6.0) Fine-tune the overall network model.

[0093] Based on the pre-trained target detection network, the target detection network is fine-tuned using the low-resolution image and the annotation information of the detected target and the composite loss function to obtain a target detection model based on dynamic super-resolution. In other feasible embodiments, the loss function can be adjusted.

[0094] (7.0) Use the test set to test the dynamic super-resolution based object detection model in step (6.0).

[0095] It should be understood that for different application objects, pre-training and fine-tuning can be performed in the above manner; or the target detection model that has been trained for other application objects can be introduced for fine-tuning to obtain a target detection model suitable for the current application object. The present invention does not make specific limitations on this.

[0096] Embodiment 2:

[0097] Similar to Example 1, the difference of this embodiment is that it is not limited to the application of power transmission lines, and the target detection application field facing the same problem is also applicable to the following method, that is, in order to solve the problem that the target detection effect is poor in other fields due to the low image resolution and the small detection target, this embodiment provides a target detection method based on dynamic super-resolution, including the following steps:

[0098] Step 1: The image to be detected is sent to the dynamic super-resolution module to adaptively select the super-resolution network and perform image super-resolution processing;

[0099] The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks, and the dynamic path selection network adaptively selects the super-resolution network according to the characteristics of the input image;

[0100] Step 2: Input the image output by the super-resolution network into the trained target detection module to obtain the target detection result;

[0101] The input data of the target detection module is an image, and the output data is the target detection result on the image.

[0102] The training process of the target detection network may refer to the description in the aforementioned embodiment 1, and the two are the same.

[0103] Embodiment 3:

[0104] This embodiment provides a system based on the above target detection method, which at least includes: an image acquisition module, a dynamic super-resolution module and a target detection module.

[0105] The image acquisition module is used to acquire low-resolution images. In hardware implementation, the image acquisition module can be regarded as a camera, a webcam or other device.

[0106] The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks, and the dynamic path selection network adaptively selects a super-resolution network to perform image processing according to the features of the input image;

[0107] The target detection module is used to detect the image output by the super-resolution network to obtain the target detection result.

[0108] In some feasible embodiments, the dynamic super-resolution module and the target detection module imported into the system are already trained. In other feasible embodiments, the dynamic super-resolution module and the target detection module are trained in the system, so that the system also includes: a training module, wherein the training module is used to train the target detection network to obtain a target detection model based on dynamic super-resolution.

[0109] Among them, the training module includes: a data set acquisition unit, a pre-training unit of the target detection module, a pre-training unit of the dynamic path selection network, a pre-training unit of the super-resolution network and a fine-tuning unit; or the training module includes: a data set and target detection model acquisition unit and a fine-tuning unit.

[0110] In a feasible manner, a data set acquisition unit is used to construct a data set, and the data set consists of an image and annotation information of a detection target on the image. The pre-training unit of the target detection module uses the data set to pre-train the network of the target detection module. Among them, the pre-trained target detection module detects a low-resolution image to obtain a detection result, and the low-resolution image is obtained by downsampling the image in the data set. The pre-training unit of the dynamic path selection network is used to classify the low-resolution image according to the confidence in the detection result; and use the classified data to train the dynamic path selection network in the dynamic super-resolution module. The pre-training unit of the super-resolution network is used to pre-train each super-resolution network using a super-resolution data set, such as using the DIV-2K data set. The fine-tuning unit is used to train the target detection network based on the pre-trained target detection module and the dynamic super-resolution module using the down-sampled image and the loss function to obtain a target detection model based on dynamic super-resolution.

[0111] In another feasible embodiment, the dataset acquisition unit is used to construct a dataset, which consists of an image and annotation information of the detection target on the image; it is also used to obtain a trained target detection model; the fine-tuning unit is used to train the target detection network based on the trained target detection model using the dataset data and a loss function to obtain a target detection model based on dynamic super-resolution.

[0112] It should be understood that the specific implementation process of the above-mentioned unit modules refers to the method content, and the present invention will not be specifically repeated here, and the division of the above-mentioned functional module units is only a division of logical functions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. At the same time, the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0113] Embodiment 4:

[0114] This embodiment provides a terminal, which includes: one or more processors, and a memory storing one or more computer programs;

[0115] The processor calls the computer program to execute:

[0116] The steps of a target detection method based on dynamic super-resolution or the steps of a power transmission line component detection method based on dynamic super-resolution. The specific implementation process can refer to Example 1 and Example 2.

[0117] It should also be noted that the training process of the target detection network can be implemented in the terminal, or the trained target detection network can be imported into the terminal.

[0118] The terminal also includes: a communication interface for communicating with external equipment and performing data interactive transmission.

[0119] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0120] If the memory, processor and communication interface are implemented independently, the memory, processor and communication interface can be connected to each other through a bus and communicate with each other. The bus can be an industrial standard architecture bus, a peripheral device interconnection bus or an extended industrial standard architecture bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0121] Optionally, in a specific implementation, if the memory, processor and communication interface are integrated on a chip, the memory, processor and communication interface can communicate with each other through an internal interface.

[0122] For the specific implementation process of each step, please refer to the description of the above method.

[0123] It should be understood that in the embodiments of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), graphics processing units (GPU), etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0124] Embodiment 5:

[0125] This embodiment provides a readable storage medium storing a computer program, wherein the computer program is called by a processor to execute:

[0126] The steps of a target detection method based on dynamic super-resolution or the steps of a power transmission line component detection method based on dynamic super-resolution. The specific implementation process can refer to Example 1 and Example 2.

[0127] It should also be noted that the computer program of the training process of the target detection network can be loaded into the readable storage medium, or the trained target detection network can be imported into the readable storage medium.

[0128] For the specific implementation process of each step, please refer to the description of the above method.

[0129] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the controller described in any of the foregoing embodiments, such as a hard disk or memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Further, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0130] Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0131] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive, and therefore the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A method for detecting power transmission line components based on dynamic super-resolution, characterized in that: The following steps are involved: Step 1: The transmission line image is transmitted to the dynamic super-resolution module to adaptively select the super-resolution network and perform image super-resolution processing; The dynamic super-resolution module includes a dynamic path selection network and at least two different types of super-resolution networks, and the dynamic path selection network adaptively selects the super-resolution network according to the characteristics of the input image; Step 2: Input the image output by the super-resolution network into the trained target detection module to obtain the component detection result on the transmission line image; The input data of the target detection module is an image, and the output data is the target detection result on the image; The dynamic super-resolution module and the target detection module constitute a target detection network, and the training process of the target detection network is divided into pre-training and fine-tuning; The pre-training process is: (1) collecting original images and marking detection targets on the original images, thereby constructing a target detection data set, and then using the target detection data set to pre-train a network of a target detection module; Wherein, the original image of the target detection data set is downsampled to obtain a downsampled image; (2) inputting the downsampled image in step (1) into a pre-trained target detection module for detection, wherein the image is classified according to the confidence of the detection result; (3) using the classified image in step (2) to pre-train the dynamic path selection network in the dynamic super-resolution module; Wherein, the correspondence between the classification image category and the super-resolution network category is set, the input data of the dynamic path selection network is the image, and the output data is the selection probability or selection result of each super-resolution network; (4) using the super-resolution data set to pre-train each super-resolution network in the dynamic super-resolution module, wherein, based on a preset super-resolution multiple, the super-resolution network is pre-trained using the super-resolution data set to obtain a super-resolution weight of the super-resolution network; The fine-tuning process is: Based on the trained target detection module and dynamic super-resolution module, the target detection network is fine-tuned using the down-sampled image dataset in step (1) or a training image dataset of the same category as the image to be detected, and a loss function to obtain a target detection model based on dynamic super-resolution; The fine-tuning process uses a composite loss function for training, which is as follows: In the formula, Loss represents the composite loss value, Loss SR Represents the dynamic super-resolution module loss, Loss Det Represents the target detection module loss, α and β are weight coefficients; The dynamic super-resolution module loss adopts a mean square error loss function, and the target detection module loss is composed of anchor box regression loss, classification loss, and confidence loss. The dynamic path selection network consists of a convolutional layer, an average pooling layer, a fully connected layer and a Softmax layer, and the dynamic path selection network outputs the selection probability of each super-resolution network.

2. The method according to claim 1, characterized in that: The dynamic super-resolution module is provided with two super-resolution networks, one of which is an RFDN network and the other is an SRCNN network.

3. The method according to claim 1, characterized in that: The target detection module is a target detection model built based on YOLOv5, and the YOLOv5 at least includes a backbone network, a neck network and a prediction network.

4. A terminal, characterized in that: include: one or more processors; a memory storing one or more computer programs; The processor invokes the computer program to execute: The steps of the power transmission line component detection method as claimed in claim 1.

5. A readable storage medium, characterized in that: A computer program is stored, which is called by a processor to execute: The steps of the power transmission line component detection method as claimed in claim 1.

Images