Method and device for identifying tiny foreign objects in power transmission lines

Abstract

The present application relates to a method and device for identifying tiny foreign objects on power transmission lines. The method comprises: acquiring a power transmission line image and performing convolution processing to obtain tiny foreign object features; processing the tiny foreign object features using a preset extraction rule to obtain first data to be input; processing the first data to be input using a multi-layer ladder processing model to obtain low-level feature data of tiny foreign objects output by a direct processing layer and high-level feature data to be processed by a high-level processing layer; processing the first data to be input using a tiny foreign object position enhancement model to obtain tiny foreign object position enhancement data; obtaining high-level feature data of tiny foreign objects output by a high-level processing layer based on the high-level feature data to be processed and the tiny foreign object position enhancement data; merging the low-level feature data of tiny foreign objects and the high-level feature data of tiny foreign objects to obtain pooled data; and obtaining the location and type of tiny foreign objects based on the pooled data. The present application can effectively identify tiny foreign objects on power transmission lines.

Description

Technical Field

[0001] The present application relates to the field of electric power and power transmission technology, and in particular to a method and device for identifying tiny foreign objects in power transmission lines. Background Art

[0002] As an important component of the power system, transmission lines are directly exposed to the air for a long time. They are not only subject to the pressure from the transmission line's own charge load, but also to the harm of the uncertain external environment. This causes the components in the transmission line to continue to age. If these problems are not discovered and handled in a timely manner, circuit failures may occur if the transmission line operates in this condition for a long time, posing a certain threat to the stable operation of the power system.

[0003] With the development of transmission line detection technology, technologies based on AI (Artificial Intelligence) edge computing have emerged to identify and monitor hidden dangers in transmission channels. These technologies are mainly used for remote inspections of ultra-high voltage transmission line channels and to prevent damage to power transmission caused by external factors.

[0004] During the implementation process, the inventors discovered that the conventional technology has at least the following problems:

[0005] The current AI-based edge detection and identification methods for monitoring hidden dangers in power transmission channels or traditional methods have problems such as difficulty in effectively identifying tiny foreign objects on the transmission lines. Summary of the Invention

[0006] Based on this, it is necessary to provide a method and device that can effectively identify tiny foreign objects in transmission lines to address the above technical problems.

[0007] To achieve the above objectives, on the one hand, an embodiment of the present application provides a method for identifying small foreign objects in a power transmission line, comprising:

[0008] Acquire the transmission line image, perform convolution processing on the transmission line image, and obtain the characteristics of tiny foreign objects;

[0009] Using preset extraction rules to process the characteristics of tiny foreign bodies to obtain first data to be input;

[0010] A multi-layer ladder processing model is used to process the first input data to obtain low-level feature data of tiny foreign objects output by the direct processing layer and high-level feature data to be processed by the advanced processing layer; the multi-layer ladder processing model is a plurality of processing layers each including a corresponding number of convolution kernels; the direct processing layer is the first processing layer in the multi-layer ladder processing model; and the advanced processing layer is the processing layer in the multi-layer ladder processing model other than the direct processing layer;

[0011] The first input data is processed using a micro foreign body position enhancement model to obtain micro foreign body position enhancement data; and micro foreign body high-level feature data output by the advanced processing layer is obtained based on the high-level feature data to be processed and the micro foreign body position enhancement data. The micro foreign body position enhancement model is a model for extracting position features of micro foreign bodies and includes a plurality of cross-concatenated convolution operations and data rectification operations.

[0012] The low-level feature data of tiny foreign objects and the high-level feature data of tiny foreign objects are merged to obtain the data to be pooled; and the location and type of the tiny foreign objects are obtained based on the data to be pooled.

[0013] In one embodiment, the preset extraction rules include: selecting two groups of feature convolution layers from the tiny foreign body features, convolving the two groups of feature convolution layers separately and then merging them to obtain the first data to be input; the feature convolution layer is composed of adjacent convolution layers cascaded together.

[0014] In one embodiment, the step of using the micro-foreign matter position enhancement model to process the first data to be input, and obtaining the micro-foreign matter position enhancement data also includes: selecting the micro-foreign matter features to obtain the second data to be input; using the micro-foreign matter position enhancement model to process the first data to be input and the second data to be input respectively to obtain the first processing result and the second processing result; and merging the first processing result and the second processing result to obtain the micro-foreign matter position enhancement data.

[0015] In one embodiment, the tiny foreign body feature includes a plurality of convolution modules connected in series;

[0016] The step of selecting the features of the tiny foreign matter to obtain the second data to be input includes: selecting a group of adjacent convolution layers in cascade from the convolution module as the second data to be input.

[0017] In one embodiment, the micro foreign body location enhancement model further includes an output extension operation; the output extension operation is located after a plurality of cross-series convolution operations and data rectification operations; and the previous operation of the output extension operation is a convolution operation.

[0018] In one embodiment, the high-level feature data to be processed and the enhanced data of the position of the tiny foreign matter are processed by dot product to obtain the high-level feature data of the tiny foreign matter output by the advanced processing layer.

[0019] In one embodiment, the number of groups of convolution kernels included in the direct processing layer is 0 or 1; the number of groups of convolution kernels included in each advanced processing layer is greater than 1; the data to be pooled are the low-level feature data of tiny foreign objects and the high-level feature data of tiny foreign objects obtained by merging in a cascade manner; the position and type of the tiny foreign objects are the data to be pooled obtained by pooling, full connection and feature matching in sequence.

[0020] A device for identifying tiny foreign objects in a power transmission line, comprising:

[0021] An image acquisition module is used to acquire images of power transmission lines and perform convolution processing on the images of power transmission lines to obtain features of tiny foreign objects;

[0022] A feature extraction module, configured to process features of tiny foreign bodies using preset extraction rules to obtain first data to be input;

[0023] A multi-layer ladder processing module is used to process the first input data using a multi-layer ladder processing model to obtain low-level feature data of tiny foreign objects output by the direct processing layer and high-level feature data to be processed by the advanced processing layer; the multi-layer ladder processing model is a plurality of processing layers each including a corresponding number of convolution kernels; the direct processing layer is the first processing layer in the multi-layer ladder processing model; and the advanced processing layer is the processing layer in the multi-layer ladder processing model other than the direct processing layer;

[0024] a position enhancement module for processing the first input data using a micro-foreign matter position enhancement model to obtain micro-foreign matter position enhancement data; and obtaining micro-foreign matter high-level feature data output by the high-level processing layer based on the high-level feature data to be processed and the micro-foreign matter position enhancement data; the micro-foreign matter position enhancement model is a model for extracting micro-foreign matter position features that includes a plurality of cross-convolution operations and data rectification operations;

[0025] The result output module is used to merge the low-level feature data of tiny foreign objects and the high-level feature data of tiny foreign objects to obtain the data to be pooled; and obtain the location and type of the tiny foreign objects based on the data to be pooled.

[0026] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of the above method when executing the computer program.

[0027] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0028] One of the above technical solutions has the following advantages and beneficial effects:

[0029] The multi-layer ladder processing model is used to process image data, which can not only process the semantic information of the high-level image data, but also include the detailed information of the low-level image data, so as to achieve the purpose of detecting tiny foreign objects; the tiny foreign object position enhancement model is parallel to the multi-layer ladder processing model and can provide evaluation data for position identification; the multi-layer ladder processing model and the tiny foreign object position enhancement model are combined into a feature enhancement module, which can enhance the features of tiny foreign objects, facilitate target positioning, and effectively identify tiny foreign objects on the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A schematic diagram of the convolutional architecture of FASTER RCNN in an example;

[0032] Figure 2 FIG1 is a first schematic flow chart of a method for identifying tiny foreign objects in a power transmission line according to an embodiment;

[0033] Figure 3 A schematic diagram of the basic architecture of a method for identifying tiny foreign objects in a power transmission line according to one embodiment;

[0034] Figure 4 A schematic diagram of the basic architecture of a method for identifying tiny foreign objects in a power transmission line according to another embodiment;

[0035] Figure 5 A structural block diagram of a method and apparatus for identifying tiny foreign objects in a power transmission line according to one embodiment;

[0036] Figure 6 is a diagram of the internal structure of a computer device in one embodiment;

[0037] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0038] To facilitate understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The accompanying drawings provide embodiments of the present application. However, the present application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to make the disclosure of the present application more thorough and comprehensive.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0040] It will be understood that the terms "first," "second," etc. used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element.

[0041] Spatially relative terms such as "under," "beneath," "beneath," "under," "above," "above," etc., may be used herein to describe the relationship of an element or feature shown in the figures to other elements or features. It should be understood that in addition to the orientations shown in the figures, spatially relative terms also include different orientations of the device in use and operation. For example, if the device in the drawings is turned over, the element or feature described as "under" or "beneath" or "beneath" the other elements will be oriented as "above" the other elements or features. Thus, the exemplary terms "under" and "under" can include both upper and lower orientations. In addition, the device can also include alternative orientations (e.g., rotated 90 degrees or other orientations), and the spatial descriptors used herein are interpreted accordingly.

[0042] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element or connected to the other element through an intervening element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc., if there is transmission of electrical signals or data between the connected objects.

[0043] As used herein, the singular forms "a," "an," and "the" may also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include," "comprising," "having," and the like specify the presence of stated features, integers, steps, operations, components, parts, or combinations thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof. Furthermore, the term "and / or" as used in this specification includes any and all combinations of the relevant listed items.

[0044] Existing AI edge detection technology is mainly aimed at detecting medium-to-large-sized tiny foreign objects that are relatively obvious in images. The image spatial data volume of such tiny foreign objects is relatively large, and the color or brightness features available for identification are relatively obvious. Therefore, the accuracy of tiny foreign object type identification and location positioning is relatively high, such as R-CNN (Region-based Convolution Neural Network), FAST RCNN (Fast Region-based Convolution Neural Network), FASTER RCNN (Faster Region-based Convolution Neural Network), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), etc. Figure 1 As shown in the figure, it is a schematic diagram of the convolutional structure of FASTER RCNN. The number of channels of each convolutional layer can be 128, 512, 1024, etc.

[0045] Currently, the type recognition and location positioning technology for tiny foreign objects is still in the exploratory and development stage. Tiny foreign objects are common on transmission lines and have relatively strong intrusion capabilities. They often invade transmission equipment along the transmission lines, not only affecting the normal power supply of the lines, but also causing the lines to trip. Therefore, for the type recognition and location positioning of tiny objects in transmission lines, further improvements are needed based on existing edge detection. The improved method can be adapted to process images of tiny objects and effectively identify tiny foreign objects on transmission lines. This is conducive to the automatic identification and alarm of line damage types, improving the level of line safety protection, and thus reducing the damage trip rate.

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] The present application provides a method for identifying small foreign objects in a power transmission line. Figure 2 Shown, including:

[0048] S210, acquiring a transmission line image, performing convolution processing on the transmission line image, and obtaining features of tiny foreign matter;

[0049] Specifically, the transmission line image may include a static image or a frame image in a video frame sequence; FASTER RCNN may be used to perform convolution processing on the transmission line image, and multiple convolution modules are output after the convolution processing. Each convolution module includes multiple convolution layers, and the convolution layers extract different features from the transmission line image; the tiny foreign body feature may be the convolution module output after the convolution processing.

[0050] In some examples, transmission line images can be acquired by aerial surveillance, crawling robots, mobile cameras, fixed cameras, etc. For example, each frame in the frame sequence of the entire video can be intercepted to form each image; in some examples, the convolution processing of the transmission line image to obtain the characteristics of tiny foreign objects can adopt the convolution architecture of VGG16 (Visual Geometry Group Network, which has 13 convolutional layers and 3 fully connected layers in the structure); Figure 3 As shown in the figure, after convolution processing, the transmission line image outputs multiple convolution modules: Conv1, Conv2, Conv3, Conv4 and Conv5, etc.

[0051] S220, using a preset extraction rule to process the characteristics of the tiny foreign matter to obtain first data to be input;

[0052] Specifically, the preset extraction rules may include selecting a convolution module for extracting data from multiple convolution modules, and selecting multiple convolution layers from each selected convolution module, instead of using the last convolution layer of each convolution module for the next step of processing, so as to reduce the number of convolution modules used and improve the efficiency of the processing process; the first data to be input can be a convolution layer obtained by merging the selected multiple convolution layers, which is used to input the multi-layer step processing model and the small foreign body position enhancement model.

[0053] In some examples, such as Figure 3 As shown, Represents cascade, It represents point-by-point summation, selects Conv4 and Conv5 from the convolution module, that is, the fourth and fifth convolution modules in the convolution module; selects multiple convolution layers from Conv4 and Conv5 respectively, and merges them to obtain the first input data.

[0054] S230, processing the first input data using a multi-layer ladder processing model to obtain low-level feature data of tiny foreign objects output by a direct processing layer and high-level feature data to be processed by a high-level processing layer; the multi-layer ladder processing model is a plurality of processing layers each including a corresponding number of convolution kernels; the direct processing layer is the first processing layer in the multi-layer ladder processing model; and the high-level processing layer is the processing layer in the multi-layer ladder processing model other than the direct processing layer;

[0055] Specifically, the multi-layer ladder processing model includes a direct processing layer and an advanced processing layer. Each layer is set with a corresponding number of convolution kernels. The direct processing layer processes the first data to be input and outputs low-level feature data of tiny foreign objects. The advanced processing layer processes the first data to be input and outputs high-level feature data to be processed. The multi-layer ladder processing model uses a product (for example, point-by-point product or dot product) method to process the first data to be input. The product method can characterize the sharpness of the image. The multi-layer ladder processing model enhances the utilization efficiency of the input data and improves the accuracy of tiny foreign object identification by constructing a ladder-like data expression form. The multi-layer ladder processing model is used to process image data, which can not only process the semantic information of the high-level image data, but also include the detailed information of the low-level image data, so as to achieve the purpose of detecting tiny foreign objects.

[0056] In some examples, such as Figure 3As shown, the first data to be input is respectively input into the three processing layers of the multi-layer ladder processing model, and each processing layer has a corresponding number of orders; wherein, the first layer of the multi-layer ladder processing model is a direct processing layer, which can directly output the low-level feature data of tiny foreign matter without setting the convolution kernel; the direct processing layer can also include a group of convolution kernels, which output the low-level feature data of tiny foreign matter after being processed by the convolution kernel; the second and third layers of the multi-layer ladder processing model are high-level processing layers; the number of convolution kernel groups included in the second layer of the multi-layer ladder processing model can be 2, and the number of convolution kernel groups included in the third layer of the multi-layer ladder processing model can be 3; each layer of the high-level processing layer outputs the high-level feature data to be processed through dot product operation after being processed by the convolution kernel; in some examples, the convolution kernel size is 1×1; the number of channels output by the direct processing layer can be equal to or less than the number of channels of the input, and the number of channels output by the high-level processing layer after being processed by the convolution kernel can be equal to or greater than the number of channels of the input, or a multiple of the number of channels of the input, and the number of channels output by each high-level processing layer is different.

[0057] In some examples, the high-level feature data to be processed can be obtained based on the following formula:

[0058]

[0059] The multi-layer ladder processing model includes N processing layers, where n is the number of high-level processing layers, 2≤n≤N. is the convolution layer obtained after the first input data is processed by the i-th group of convolution kernels of the n-th layer, P n It is the high-level feature data to be processed output by the nth layer of the multi-layer ladder processing model.

[0060] S240, processing the first input data using a micro foreign body position enhancement model to obtain micro foreign body position enhancement data; obtaining micro foreign body high-level feature data output by the high-level processing layer based on the high-level feature data to be processed and the micro foreign body position enhancement data; the micro foreign body position enhancement model is a model for extracting micro foreign body position features that includes a plurality of cross-convolution operations and data rectification operations;

[0061] Specifically, the multi-layer staircase processing model brings additional convolution operations, which reduces the gradient features of the image to a certain extent. In order to compensate for this loss, a small foreign body position enhancement model is used to process the first input data; the small foreign body position enhancement model and the multi-layer staircase processing model are in a parallel processing relationship and can provide evaluation data for position recognition; the small foreign body position enhancement model includes multiple convolution operations, and combines multiple convolution operations in a serial form, and performs data rectification processing after each level of convolution operation; the convolution kernel of each convolution operation preferably selects a smaller convolution kernel, such as 1×1 or 3×3, and the number of channels is also controlled at a small level, for example, the number of channels is 1 to 4; the data rectification operation may not be applied to the last level of convolution operation; the data rectification operation may use a rectification function, such as a linear rectification function (Rectified Linear Unit, ReLU); the dot product operation can be used on the high-level feature data to be processed and the small foreign body position enhancement data to obtain the small foreign body high-level feature data.

[0062] In some examples, such as Figure 3 As shown, the tiny foreign body position enhancement model includes a set of convolution operations with a convolution kernel size of 1×1, a data rectification operation, a set of convolution operations with a convolution kernel size of 3×3, a data rectification, a set of convolution operations with a convolution kernel size of 1×1, and an output extension (copy) operation connected in series; the number of channels output after convolution processing of each set of convolution kernels is 1; the Pn of the nth layer in the multi-layer ladder processing model is multiplied point by point with the tiny foreign body position enhancement data output by the tiny foreign body position enhancement model to obtain tiny foreign body high-level feature data, and the number of layers of the tiny foreign body high-level feature data is the same as the number of layers of the high-level feature data to be processed.

[0063] S250 , merging the low-level feature data of the tiny foreign matter and the high-level feature data of the tiny foreign matter to obtain data to be pooled; and obtaining the location and type of the tiny foreign matter based on the data to be pooled.

[0064] Specifically, a cascaded approach is used to combine low-level and high-level feature data of tiny foreign objects to obtain the data to be pooled. Based on the data to be pooled, the location and type of tiny foreign objects are determined through matching or training operations. The combination of the multi-layer ladder processing model and the tiny foreign object location enhancement model is equivalent to a feature enhancement module that can enhance the characteristics of tiny foreign objects, facilitate target positioning, and effectively identify tiny foreign objects on power transmission lines. In some examples, the data to be pooled is sequentially subjected to pooling operations, full connection operations, and matching operations to determine the location and type of tiny foreign objects.

[0065] It should be understood that although Figure 2The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0066] In one embodiment, the preset extraction rules include: selecting two groups of feature convolution layers from the tiny foreign body features, convolving the two groups of feature convolution layers separately and then merging them to obtain the first data to be input; the feature convolution layer is composed of adjacent convolution layers cascaded together.

[0067] Specifically, the preset extraction rules may include selecting a convolution module for extracting data from multiple convolution modules, and selecting two sets of feature convolution layers from each selected convolution module, instead of using the last convolution layer of each convolution module for the next step of processing, so as to reduce the number of convolution modules used and improve the efficiency of the processing process; the first data to be input can be a convolution layer obtained by merging the two selected sets of feature convolution layers, which is used to input a multi-layer step processing model and a small foreign body position enhancement model.

[0068] In some examples, such as Figure 3 As shown, Conv4 and Conv5, that is, the fourth and fifth convolution modules in the convolution module, are selected from the convolution module; two sets of feature convolution layers are selected from Conv4 and Conv5 respectively, and the first input data is obtained by merging them; for example, the outputs of the second convolution layer (Conv4_2) and the third convolution layer (Conv4_3) of Conv4 are cascaded, and the data is expanded by convolution, and the size of the convolution kernel can be 1×1; similarly, the outputs of the second convolution layer (Conv5_2) and the third convolution layer (Conv5_3) of Conv5 are cascaded. The output is cascaded and processed by the convolution kernel and the inverse convolution kernel in turn. The size of the convolution kernel can be 1×1 and the step size can be 1. The size of the inverse convolution kernel can be 1×1 and the convolution step size can be 2, so as to double the length and width of the feature map. Finally, the convolution-processed data is merged by point-by-point summation and convolution-processed again. The convolution kernel size can be 1×1 and the step size can be 2, so as to reduce the length and width of the image to half, and obtain data for inputting the multi-layer step processing model and the tiny foreign body position enhancement model. The number of channels of the data obtained by each convolution process can be 1024.

[0069] In one embodiment, the step of processing the first input data using the micro foreign body location enhancement model to obtain micro foreign body location enhancement data further includes:

[0070] The characteristics of tiny foreign matter are selected to obtain second data to be input; the first data to be input and the second data to be input are processed respectively using a tiny foreign matter position enhancement model to obtain a first processing result and a second processing result; the first processing result and the second processing result are combined to obtain tiny foreign matter position enhancement data.

[0071] Specifically, in the process of using the tiny foreign body position enhancement model to process the first data to be input, convolution modules of different levels are combined. In order to avoid the loss of feature information and to reduce the interference that may be introduced by the inverse convolution process in some examples, the second data to be input is obtained by selecting the tiny foreign body features, that is, the tiny foreign body position is enhanced for the data of the lower level (that is, larger size) (the processing process for the first data to be input is a position enhancement bypass, and the processing process for the second data to be input is equivalent to adding a position enhancement bypass) to retain valid data as much as possible; using the tiny foreign body position enhancement model to process the first data to be input and using the tiny foreign body position enhancement model to process the second data to be input are in a parallel processing relationship; using the tiny foreign body position enhancement model to process the first data to be input and the multi-layer ladder processing model are in a parallel processing relationship; using the tiny foreign body position enhancement model to process the first data to be input to obtain a first processing result, and using the tiny foreign body position enhancement model to process the second data to be input to obtain a second processing result; the first processing result and the second processing result can be merged by summing them point by point to obtain tiny foreign body position enhanced data.

[0072] In some examples, such as Figure 4 As shown, position enhancement bypass 1 and position enhancement bypass 2 are in a parallel processing relationship; position enhancement bypass 1 and the multi-layer ladder processing model are in a parallel processing relationship; in order to maximize the possibility of preventing data loss, Cov4 with a lower level and a larger amount of data is selected for separate output. For example, the outputs of the second and third convolutional layers (Conv4_3, Conv4_2) of Conv4 are cascaded and input into position enhancement bypass 2; the existing position enhancement bypass 1 remains basically unchanged and is parallel to the multi-layer ladder processing model; in order to improve data utilization, the structures of position enhancement bypass 1 and position enhancement bypass 2 may not be exactly the same, and the size of the convolution kernel, the number of channels, and the arrangement order of the convolution kernel may be different; position enhancement bypass 1 is used to process the first data to be input to obtain a first processing result, and position enhancement bypass 2 is used to process the second data to be input to obtain a second processing result; ⊕ represents point-by-point summation, and the first processing result and the second processing result are merged to obtain the position enhancement data of the tiny foreign body.

[0073] In one embodiment, the tiny foreign body feature includes a plurality of convolution modules connected in series;

[0074] The step of selecting the features of the tiny foreign matter to obtain the second data to be input includes: selecting a group of adjacent convolution layers in cascade from the convolution module as the second data to be input.

[0075] Specifically, the characteristics of tiny foreign matter can be several series-connected convolution modules output after convolution processing; a convolution module for extracting data is selected from multiple convolution modules, and a group of adjacent convolution layers are selected from the selected convolution modules for cascading to reduce the number of convolution modules used, thereby improving the efficiency of the processing process, and using data with a lower level (i.e., larger size) as the second data to be input to retain valid data as much as possible and avoid loss of feature information.

[0076] In some examples, such as Figure 4 As shown, the tiny foreign body feature includes a series of convolution modules: Conv1, Conv2, Conv3, Conv4 and Conv5...; the outputs of the second and third convolution layers (Conv4_3, Conv4_2) of Conv4 are selected for cascade as the second input data, and the input position is enhanced by bypass 2.

[0077] In one embodiment, the micro foreign body location enhancement model further includes an output extension operation; the output extension operation is located after a plurality of cross-series convolution operations and data rectification operations; and the previous operation of the output extension operation is a convolution operation.

[0078] Specifically, in order to make the dimension of the high-level feature data to be processed output by the multi-layer ladder processing model and the dimension of the micro-foreign matter position enhancement data output by the micro-foreign matter position enhancement model consistent, it is necessary to stretch the output data of the first data to be input after several cross-series convolution operations and data rectification operations, and extend the length of the output data of the first data to be input after several cross-series convolution operations and data rectification operations to be equal to the output length of the multi-layer ladder processing model, so as to obtain the micro-foreign matter high-level feature data, for example, by extending it by copying.

[0079] In some examples, such as Figure 3 As shown, the tiny foreign body position enhancement model also includes an output extension (copy) operation; the output extension (copy) operation is located after a group of convolution operations with a convolution kernel size of 1×1, a data rectification operation, a group of convolution operations with a convolution kernel size of 3×3, data rectification, and a group of convolution operations with a convolution kernel size of 1×1, which are sequentially connected in series.

[0080] In one embodiment, the high-level feature data to be processed and the enhanced data of the position of the tiny foreign matter are processed by dot product to obtain the high-level feature data of the tiny foreign matter output by the advanced processing layer.

[0081] Specifically, dot product operations are performed on the micro-foreign matter position enhancement data and the high-level feature data to be processed in each layer to obtain the micro-foreign matter high-level feature data output by each high-level processing layer.

[0082] In some examples, such as Figure 3 As shown, the Pn of the nth layer in the multi-layer ladder processing model is multiplied point by point with the tiny foreign matter position enhancement data output by the tiny foreign matter position enhancement model to obtain tiny foreign matter high-level feature data. The number of layers of the tiny foreign matter high-level feature data is the same as the number of layers of the high-level feature data to be processed.

[0083] In one embodiment, the number of groups of convolution kernels included in the direct processing layer is 0 or 1; the number of groups of convolution kernels included in each advanced processing layer is greater than 1; the data to be pooled are the low-level feature data of tiny foreign objects and the high-level feature data of tiny foreign objects obtained by merging in a cascade manner; the position and type of the tiny foreign objects are the data to be pooled obtained by pooling, full connection and feature matching in sequence.

[0084] Specifically, the first layer of the multi-layer ladder processing model is a direct processing layer, which is used to extract the low-level features of the first data to be input. The direct processing layer may not set the convolution kernel and directly output the low-level feature data of tiny foreign matter; the direct processing layer may also include a group of convolution kernels, which output the low-level feature data of tiny foreign matter after processing by the convolution kernel; the multi-layer ladder processing model except the first layer is a high-level processing layer, which is used to extract the complex features of the first data to be input. The number of convolution kernel groups contained in each layer of the high-level processing layer is greater than 1. For example, the number of convolution kernel groups contained in the second layer of the multi-layer ladder processing model can be 2, the number of convolution kernel groups contained in the third layer of the multi-layer ladder processing model can be 3... The number of convolution kernel groups contained in the nth layer can be n; feature matching can also be performed by training.

[0085] In some examples, such as Figure 3 As shown, the number of convolution kernel groups included in the direct processing layer is 0, the number of convolution kernel groups included in the second high-level processing layer in the multi-layer ladder processing model is 2, and the number of convolution kernel groups included in the third high-level processing layer in the multi-layer ladder processing model is 3; the low-level feature data of tiny foreign objects and the high-level feature data of tiny foreign objects are merged in a cascade manner to obtain the data to be pooled; the data to be pooled are subjected to pooling operations, full connection operations, and matching or training operations in sequence to obtain the location and type of the tiny foreign objects.

[0086] The embodiment of the present application provides a device for identifying small foreign objects in a power transmission line, such as Figure 5 As shown, the device includes:

[0087] An image acquisition module 510 is used to acquire a transmission line image and perform convolution processing on the transmission line image to obtain features of tiny foreign objects;

[0088] A feature extraction module 520 is configured to process features of tiny foreign bodies using preset extraction rules to obtain first data to be input;

[0089] The multi-layer ladder processing module 530 is configured to process the first input data using a multi-layer ladder processing model to obtain low-level feature data of tiny foreign objects output by a direct processing layer and high-level feature data to be processed by a high-level processing layer. The multi-layer ladder processing model comprises multiple processing layers each including a corresponding number of convolution kernels. The direct processing layer is the first processing layer in the multi-layer ladder processing model, and the high-level processing layer is the processing layer in the multi-layer ladder processing model other than the direct processing layer.

[0090] Position enhancement module 540 is configured to process the first input data using a micro-foreign matter position enhancement model to obtain micro-foreign matter position enhancement data; and obtain micro-foreign matter high-level feature data output by the high-level processing layer based on the high-level feature data to be processed and the micro-foreign matter position enhancement data. The micro-foreign matter position enhancement model is a model that includes a plurality of cross-convolution operations and data rectification operations for extracting micro-foreign matter position features.

[0091] The result output module 550 is used to merge the low-level feature data of the tiny foreign matter and the high-level feature data of the tiny foreign matter to obtain the data to be pooled; and obtain the location and type of the tiny foreign matter based on the data to be pooled.

[0092] In one embodiment, the feature extraction module 520 is also used to select the features of tiny foreign matter to obtain the second data to be input; the position enhancement module 540 is also used to use the tiny foreign matter position enhancement model to process the first data to be input and the second data to be input respectively to obtain the first processing result and the second processing result; and the first processing result and the second processing result are combined to obtain the tiny foreign matter position enhanced data.

[0093] In one embodiment, the tiny foreign body features obtained by the image acquisition module 510 include a plurality of convolution modules connected in series; the feature extraction module 520 is used to select a group of adjacent convolution layers cascaded from the convolution modules as the second input data.

[0094] In one embodiment, the micro-foreign matter position enhancement model adopted by the position enhancement module 540 further includes an output extension operation; the output extension operation is located after a plurality of cross-series convolution operations and data rectification operations; and the previous operation of the output extension operation is a convolution operation.

[0095] In one embodiment, the multi-layer staircase processing module 530 processes the high-level feature data to be processed and the micro-foreign matter position enhancement data by performing dot product processing to obtain the micro-foreign matter high-level feature data output by the advanced processing layer.

[0096] In one embodiment, the direct processing layer of the multi-layer ladder processing module 530 includes 0 or 1 groups of convolution kernels; the number of groups of convolution kernels included in each advanced processing layer is greater than 1; the data to be pooled is the low-level feature data of tiny foreign matter and the high-level feature data of tiny foreign matter obtained by merging in a cascade manner; the position and type of tiny foreign matter are obtained by sequentially pooling, fully connecting and feature matching the data to be pooled.

[0097] For the specific definition of the transmission line tiny foreign body identification device, please refer to the definition of the transmission line tiny foreign body identification method above, which will not be repeated here. The various modules in the above-mentioned transmission line tiny foreign body identification device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0098] An embodiment of the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0099] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store transmission line image data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for identifying small foreign objects in a transmission line is implemented.

[0100] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for identifying tiny foreign objects in a power transmission line is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0101] Those skilled in the art will understand that Figure 6-7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0102] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0103] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0104] Throughout this specification, references to terms such as "some embodiments," "other embodiments," and "desired embodiments" indicate that a particular feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present application. The schematic descriptions of these terms throughout this specification do not necessarily refer to the same embodiment or example.

[0105] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for identifying tiny foreign objects in a power transmission line, characterized in that: include: Acquire a transmission line image, and perform convolution processing on the transmission line image to obtain features of tiny foreign objects; Processing the tiny foreign body features using a preset extraction rule to obtain first data to be input; The minute foreign body feature includes the output of each convolution layer in a plurality of convolution modules connected in series; the preset extraction rule includes: selecting two sets of feature convolution layer outputs from the minute foreign body feature, convolving the two sets of feature convolution layer outputs separately and then merging them to obtain the first input data; the feature convolution layer is formed by cascading two adjacent convolution layers in the same convolution module; Processing the first data to be input using a multi-layer ladder processing model to obtain low-level feature data of tiny foreign objects output by a direct processing layer and high-level feature data to be processed by a high-level processing layer; the multi-layer ladder processing model is a plurality of processing layers each including a corresponding number of convolution kernels; wherein the direct processing layer is the first processing layer in the multi-layer ladder processing model; and the high-level processing layer is the processing layer in the multi-layer ladder processing model other than the direct processing layer; The first input data is processed using a micro foreign body position enhancement model to obtain micro foreign body position enhancement data; the micro foreign body high-level feature data output by the high-level processing layer is obtained based on the high-level feature data to be processed and the micro foreign body position enhancement data; the micro foreign body position enhancement model is a model for extracting the position features of the micro foreign body and includes a plurality of cross-convolution operations and data rectification operations; the data rectification operation is performed by a rectification function; The low-level feature data of the tiny foreign matter and the high-level feature data of the tiny foreign matter are merged to obtain data to be pooled; and the position and type of the tiny foreign matter are obtained according to the data to be pooled.

2. The method for identifying small foreign objects in a power transmission line according to claim 1, characterized in that: The step of using the micro foreign body position enhancement model to process the first input data to obtain micro foreign body position enhancement data further includes: Selecting the characteristics of the tiny foreign matter to obtain second data to be input; Using the micro foreign body position enhancement model to process the first data to be input and the second data to be input respectively to obtain a first processing result and a second processing result; The first processing result and the second processing result are combined to obtain the tiny foreign body position enhancement data.

3. The method for identifying small foreign objects in a power transmission line according to claim 2, characterized in that: The tiny foreign body feature includes the output of each convolution layer in a plurality of convolution modules connected in series; The step of selecting the characteristics of the tiny foreign matter to obtain the second data to be input includes: Outputs of two adjacent convolutional layers in the convolution module are selected and cascaded as the second input data.

4. The method for identifying tiny foreign objects in a power transmission line according to claim 1, characterized in that: The micro-foreign matter position enhancement model also includes an output extension operation; the output extension operation is located after several cross-series convolution operations and data rectification operations; the previous operation of the output extension operation is a convolution operation; the output extension operation is used to extend the length of the output data of the first to-be-input data after several cross-series convolution operations and data rectification operations to a length equal to the length of the to-be-processed high-level feature data output by the multi-layer ladder processing model.

5. The method for identifying tiny foreign objects in a power transmission line according to claim 1, characterized in that: The high-level feature data to be processed and the micro-foreign matter position enhancement data are processed by dot product to obtain the micro-foreign matter high-level feature data output by the advanced processing layer.

6. The method for identifying tiny foreign objects in a power transmission line according to claim 1, characterized in that: The number of groups of convolution kernels included in the direct processing layer is 0 or 1; the number of groups of convolution kernels included in each high-level processing layer is greater than 1; the data to be pooled is obtained by merging the low-level feature data of tiny foreign matter and the high-level feature data of tiny foreign matter in a cascade manner; the position and type of the tiny foreign matter are obtained by sequentially pooling, fully connecting and matching the data to be pooled; when the number of groups of convolution kernels included in the direct processing layer is 0, the low-level feature data of tiny foreign matter is the first data to be input.

7. A device for identifying small foreign objects in a power transmission line, characterized in that: The device comprises: An image acquisition module is used to acquire a transmission line image and perform convolution processing on the transmission line image to obtain features of tiny foreign objects; a feature extraction module configured to process the minute foreign body features using a preset extraction rule to obtain first data to be input; the minute foreign body features include the output of each convolution layer in a plurality of convolution modules connected in series; the preset extraction rule includes: selecting two sets of feature convolution layer outputs from the minute foreign body features, convolving the two sets of feature convolution layer outputs separately and then combining them to obtain the first data to be input; the feature convolution layer is formed by cascading two adjacent convolution layers in the same convolution module; a multi-layer ladder processing module, configured to process the first data to be input using a multi-layer ladder processing model to obtain low-level feature data of tiny foreign matter output by a direct processing layer and high-level feature data to be processed by a high-level processing layer; the multi-layer ladder processing model is a plurality of processing layers each including a corresponding number of convolution kernels; wherein the direct processing layer is the first processing layer in the multi-layer ladder processing model; and the high-level processing layer is the processing layer in the multi-layer ladder processing model other than the direct processing layer; a position enhancement module, configured to process the first input data using a micro-foreign matter position enhancement model to obtain micro-foreign matter position enhancement data; and obtain micro-foreign matter high-level feature data output by the high-level processing layer based on the high-level feature data to be processed and the micro-foreign matter position enhancement data; the micro-foreign matter position enhancement model is a model comprising a plurality of cross-concatenated convolution operations and data rectification operations for extracting the position features of the micro-foreign matter; the data rectification operation is performed by a rectification function; The result output module is used to merge the low-level feature data of the tiny foreign matter and the high-level feature data of the tiny foreign matter to obtain the data to be pooled; and obtain the position and type of the tiny foreign matter according to the data to be pooled.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images