Line detection method and device, storage medium and computer program product
By equipping drones with ultraviolet imaging equipment and image processing technology, combined with 5G real-time data transmission, the problems of low efficiency in traditional manual inspections and insufficient drone inspections have been solved, achieving efficient and accurate location of partial discharge fault points and ensuring the stability of the power system.
Patent Information
- Application Number
- CN202411087080.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-29
AI Technical Summary
Traditional manual inspection methods are inefficient, labor-intensive, and costly, and pose safety hazards in complex geographical environments. Existing drone inspection technology still has shortcomings in terms of environmental adaptability, real-time processing capabilities, applicable scope, data transmission stability, and system cost, resulting in low efficiency in partial discharge detection of high-voltage overhead transmission lines.
UAVs equipped with ultraviolet imaging devices are used to collect images of power transmission lines. Through image preprocessing, feature extraction, and target detection algorithms, partial discharge fault points can be accurately and quickly located. Real-time data transmission and analysis are achieved by combining 5G technology.
It enables efficient and accurate location of partial discharge fault points, improves line detection efficiency, and ensures the safe and stable operation of the power system.
Smart Images

Figure CN119048445A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a line detection method, device, storage medium and computer program product. Background Art
[0002] High-voltage overhead transmission lines are the main mode of regional power transmission, and transmission line inspection is crucial to the safe operation of local power transmission and distribution networks. During operation, various abnormal conditions may occur in power transmission and transformation equipment, among which partial discharge is a common phenomenon that causes serious hidden dangers. Partial discharge is one of the main reasons for the final insulation breakdown of high-voltage transmission equipment and an important sign of insulation degradation. Especially when overhead transmission lines operate in harsh environments for a long time, high-frequency abnormal conditions of transmission equipment are prone to occur, which require timely monitoring and processing to ensure the normal operation of the equipment and the safety and stability of the transmission system. However, the traditional manual inspection method is inefficient, has a large workload, high investment costs, and poses safety risks in complex geographical environments.
[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0004] The embodiments of the present application provide a line detection method, device, storage medium and computer program product to at least solve the technical problem of low line detection efficiency caused by the use of manual inspection in related technologies when performing line inspection.
[0005] According to one aspect of an embodiment of the present application, a line detection method is provided, comprising: acquiring a line image and performing image preprocessing on the line image, wherein the line image is an ultraviolet imaging image including a power transmission line collected by a drone; performing feature extraction on the line image after image preprocessing to obtain a feature map; determining an anchor point frame containing a target to be detected in the feature map, and generating a candidate frame based on the anchor point frame containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor point frame is a set of frames of different sizes preset in the feature map, and the candidate frame is used to frame the area where the target to be detected is located in the feature map; extracting an image area corresponding to the position of the candidate frame in the feature map, and determining a line fault type corresponding to the image area, wherein the line fault type includes: a partial discharge fault.
[0006] Optionally, generating a candidate frame based on an anchor frame containing the target to be detected includes: using a classification layer in a regional candidate network to determine the probability that the anchor frame in the feature map contains the target to be detected, wherein each point in the feature map corresponds to a set of anchor frames; using a regression layer in the regional candidate network to determine the offset of the target to be detected relative to the anchor frame; determining position information of the candidate frame based on the probability determined by the classification layer and the offset determined by the regression layer, wherein the position information includes: the horizontal and vertical coordinates of the center point of the candidate frame, and the length and width of the candidate frame.
[0007] Optionally, determining the position information of the candidate frame based on the probability determined by the classification layer and the offset determined by the regression layer includes: correcting the anchor frame in the feature map based on the offset, wherein the offset includes: the horizontal coordinate offset and the vertical coordinate offset of the center point of the anchor frame, and the length offset and the width offset of the anchor frame; obtaining a preset number of anchor frames with the highest probability of containing the target to be detected in the corrected anchor frame, and obtaining a set of candidate anchor frames; deleting the anchor frames in the candidate anchor frame set whose sizes are smaller than a preset size threshold, and determining the remaining anchor frames in the candidate anchor frame set as candidate frames.
[0008] Optionally, the region candidate network is trained by the following steps: using the initial region candidate network to predict the probability that the anchor box corresponding to the training image contains the target to be detected, and the offset of the target to be detected relative to the anchor box; calculating a first loss value based on the predicted probability and the true probability value corresponding to the anchor box in the training image, wherein when the anchor box contains the target to be detected, the true probability value corresponding to the anchor box is one, and when the anchor box does not contain the target to be detected, the true probability value corresponding to the anchor box is zero; calculating a second loss value based on the predicted offset and the true offset between the anchor box and the target to be detected in the training image; determining a target loss value corresponding to the region candidate network based on the first loss value and the second loss value, and updating the model parameters of the region candidate network based on the target loss value, and repeating the above training steps using the updated region candidate network until the target loss value is no greater than a preset loss threshold.
[0009] Optionally, extracting the image area corresponding to the position of the candidate box in the feature map, and determining the line fault type corresponding to the image area includes: using a pooling layer to perform a pooling operation on the feature map based on the position information of the candidate box to obtain the image area corresponding to the position of the candidate box in the feature map, wherein the size of the image area after the pooling operation is consistent; using a fully connected layer and a classifier to determine the fault type corresponding to the image area.
[0010] Optionally, image preprocessing of the line image includes: performing a mean filtering operation on the line image, wherein the mean filtering operation is used to update the pixel average value of all pixels in the line image located in the neighborhood of a first pixel point to a new pixel value of the first pixel point; performing a grayscale operation on the line image that has undergone the mean filtering operation, wherein the grayscale operation is used to determine the grayscale value corresponding to the pixel point based on the red separation, green separation and blue components of each pixel point in the line image; performing a binarization operation on the line image that has undergone the grayscale operation, wherein the binarization operation is used to set the pixel value of the pixel point in the line image whose grayscale value is greater than a preset grayscale value to one, and to set the pixel value of the pixel point in the line image whose grayscale value is not greater than the preset grayscale value to zero.
[0011] Optionally, image preprocessing of the line image also includes: performing morphological processing on the line image that has undergone a binarization operation, wherein the morphological processing includes: an erosion operation and a dilation operation, and when the origin of the erosion operator is aligned with a pixel point of the line image, if the erosion operator is completely contained by a portion of the line image with a pixel value of one, the pixel point is determined to be a pixel point retained after the erosion operation; and when the origin of the dilation operator is aligned with a pixel point of the line image, if the dilation operator intersects with a portion of the line image with a pixel value of one, the pixel point is determined to be a pixel point retained after the dilation operation.
[0012] According to another aspect of the embodiment of the present application, a line detection device is also provided, including: an image acquisition module, used to acquire a line image and perform image preprocessing on the line image, wherein the line image is an ultraviolet imaging image containing a power transmission line collected by a drone; a feature extraction module, used to perform feature extraction on the line image after image preprocessing to obtain a feature map; an area candidate module, used to determine an anchor point box containing a target to be detected in the feature map, and generate a candidate box based on the anchor point box containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor point box is a set of borders of different sizes preset in the feature map, and the candidate box is used to frame the area position of the target to be detected in the feature map; a fault classification module, used to extract the image area corresponding to the position of the candidate box in the feature map, and determine the line fault type corresponding to the image area, wherein the line fault type includes: partial discharge fault.
[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the line detection method by running the computer program.
[0014] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the line detection method are implemented.
[0015] In an embodiment of the present application, a line image is acquired and image preprocessed, wherein the line image is an ultraviolet imaging image containing a power transmission line collected by a drone; feature extraction is performed on the line image after image preprocessing to obtain a feature map; an anchor box containing a target to be detected in the feature map is determined, and a candidate box is generated based on the anchor box containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor box is a set of frames of different sizes preset in the feature map, and the candidate box is used to frame the area where the target to be detected is located in the feature map; an image area corresponding to the position of the candidate box in the feature map is extracted, and a line fault type corresponding to the image area is determined, wherein the line fault type includes: a partial discharge fault mode, the ultraviolet impact of the power transmission line is collected by an ultraviolet imaging device carried by the drone, the image features are enhanced in combination with the image preprocessing step, and the target detection algorithm is used to detect the fault, so as to achieve the purpose of accurately and quickly locating the discharge fault point, thereby solving the technical problem of low line detection efficiency caused by the use of manual inspection in related technologies during line inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0017] Figure 1 It is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a line detection method provided in an embodiment of the present application;
[0018] Figure 2 It is a schematic diagram of a method flow of line detection provided according to an embodiment of the present application;
[0019] Figure 3 It is a schematic diagram of a method flow for partial discharge detection of UAV power transmission lines based on ultraviolet imaging method provided in an embodiment of the present application;
[0020] Figure 4 It is a structural schematic diagram of a line detection device provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0023] High-voltage overhead transmission lines are the main mode of regional power transmission, and transmission line inspections are crucial to the safe operation of local power transmission and distribution networks. During operation, a variety of abnormal conditions may occur in power transmission and transformation equipment, among which partial discharge is a common phenomenon that causes serious hidden dangers. Partial discharge is one of the main reasons for the final insulation breakdown of high-voltage transmission equipment and an important sign of insulation degradation. Especially when overhead transmission lines operate in harsh environments for a long time, high-frequency abnormal conditions of transmission equipment are prone to occur, which need to be monitored and handled in a timely manner to ensure the normal operation of the equipment and the safety and stability of the transmission system.
[0024] However, the traditional manual inspection method is inefficient, labor-intensive, and requires high investment, and there are safety risks in complex geographical environments. The related technology of using drones for inspection still has deficiencies in environmental adaptability, real-time processing capabilities, scope of application, data transmission stability, system cost and complexity, and needs further optimization and improvement.
[0025] In order to solve the above problems, relevant solutions are provided in the embodiments of the present application, which are described in detail below.
[0026] According to an embodiment of the present application, a method embodiment of line detection is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or electronic device) for implementing a line detection method. Figure 1 As shown, the computer terminal 10 (or electronic device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0028] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or electronic device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the line detection method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned line detection method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0031] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or electronic device).
[0032] In the above operating environment, the embodiment of the present application provides a line detection method. Figure 2 is a schematic diagram of a line detection method flow according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:
[0033] Step S202, obtaining a line image and performing image preprocessing on the line image, wherein the line image is an ultraviolet imaging image of the power transmission line collected by a drone;
[0034] Step S204, extracting features from the line image after image preprocessing to obtain a feature map;
[0035] Step S206, determining an anchor box containing the target to be detected in the feature map, and generating a candidate box based on the anchor box containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor box is a set of frames of different sizes preset in the feature map, and the candidate box is used to select the area where the target to be detected is located in the feature map;
[0036] Step S208, extracting the image area corresponding to the position of the candidate box in the feature map, and determining the line fault type corresponding to the image area, wherein the line fault type includes: partial discharge fault.
[0037] Through the above steps, the ultraviolet impact of the transmission line is collected by the ultraviolet imaging equipment carried by the drone, the image features are enhanced by combining the image preprocessing steps, and the target detection algorithm is used to detect the fault, so as to achieve the purpose of accurately and quickly locating the discharge fault point, thereby solving the technical problem of low line detection efficiency caused by the use of manual inspection methods in related technologies during line inspections.
[0038] The line detection method in steps S202 to S208 of the embodiment of the present application is further introduced below.
[0039] Figure 3 is a schematic diagram of a method flow for detecting partial discharge of a UAV power transmission line based on ultraviolet imaging according to an embodiment of the present application, such as Figure 3 shown.
[0040] The image preprocessing module obtains the ultraviolet imaging image containing the transmission line collected by the drone, that is, the above-mentioned line image, and then performs image preprocessing on the line image to make the line image adapt to the format required by the subsequent feature extraction module; wherein, the image preprocessing includes but is not limited to: mean filtering, image grayscale, image binarization, morphological filtering and other steps, as follows.
[0041] In some embodiments of the present application, image preprocessing of a line image includes the following steps: performing a mean filtering operation on the line image, wherein the mean filtering operation is used to update the pixel average value of all pixels in the line image located in the neighborhood of a first pixel point to a new pixel value of the first pixel point; performing a grayscale operation on the line image that has undergone the mean filtering operation, wherein the grayscale operation is used to determine the grayscale value corresponding to the pixel point based on the red separation, green separation and blue components of each pixel point in the line image; performing a binarization operation on the line image that has undergone the grayscale operation, wherein the binarization operation is used to set the pixel value of the pixel point in the line image whose grayscale value is greater than a preset grayscale value to one, and to set the pixel value of the pixel point in the line image whose grayscale value is not greater than the preset grayscale value to zero.
[0042] Specifically, the line image is firstly subjected to mean filtering, wherein the basic principle of mean filtering is to calculate the average value of the pixel values of all pixels in the neighborhood of a certain pixel point in the image, and use the average value as the new pixel value of the point, as shown in the following formula:
[0043]
[0044] Among them, g(x,y) is the pixel value of the updated pixel, fi(x,y) is the pixel value of the i-th pixel to be processed in the neighborhood of the pixel to be updated, n is the number of pixels in the template, and the mean filter kernel corresponding to the template is shown as follows:
[0045]
[0046] Among them, M is the normalization constant and N is the order of the square matrix.
[0047] After the mean filtering, the line image is grayed out. In this embodiment, the gray value corresponding to the pixel point can be determined by weighted averaging the three RGB components with different weights according to the following formula, as shown in the following formula:
[0048] F(x,y)=0.299*R(x,y)+0.587*G(x,y)+0.144*B(x,y)
[0049] Among them, F represents the grayscale value of the pixel (x, y), R represents the red component, G represents the green component, and B represents the blue component.
[0050] After that, the line image is binarized. The core idea of binarization is to determine the value, compare the gray value of each pixel of the grayscale processed image with the set value (i.e. the preset gray value), and divide the pixels into two categories: target and background according to the comparison result. Taking f(x,y) as the original image, the model formula of the binarization method is as follows:
[0051]
[0052] Among them, f(x, y) represents the pixel value of each point in the original grayscale image, g(x, y) represents the pixel value of each point after binarization processing, and T is the set threshold (preset grayscale value).
[0053] In order to further enhance the image details, the embodiment of the present application may also perform morphological processing on the line image after the binarization processing, and the specific steps are as follows.
[0054] In some embodiments of the present application, image preprocessing of the line image also includes the following steps: performing morphological processing on the line image that has undergone a binarization operation, wherein the morphological processing includes: an erosion operation and a dilation operation, and when the origin of the erosion operator is aligned with a pixel point of the line image, if the erosion operator is completely contained by a portion of the line image with a pixel value of one, the pixel point is determined to be a pixel point retained after the erosion operation; and when the origin of the dilation operator is aligned with a pixel point of the line image, if the dilation operator intersects with a portion of the line image with a pixel value of one, the pixel point is determined to be a pixel point retained after the dilation operation.
[0055] Specifically, the basis of mathematical morphology is mathematical geometry and set theory. It can use structural elements of different shapes to identify and judge image structure and shape, and is widely used in the field of image processing. On the one hand, mathematical morphology simplifies image data by calculating the relationship between elements and between parts, and can maintain the shape characteristics of geometric shapes in the image; on the other hand, the mathematical morphology algorithm is simple and has the advantage of parallel processing structure. The basic operations of mathematical morphology processing include: erosion and dilation.
[0056] For example, A represents the image set to be processed, B represents the processing structure element (ie, the erosion operator or the dilation operator), and morphological processing is to use B to operate on A. In the erosion operation, A is eroded by B, which is counted as AΘB, where B is the erosion operator. The definition of erosion is: AΘB = {x|(B) x A}, which means: for each point x in image A, if B is completely inside A after the origin of B is aligned with x, then x is considered to be part of the eroded image, that is, the result of A being eroded by the erosion operator B is the set of the origin positions of B when A completely includes B. In the dilation operation, A is dilated by B, counted as B is the expansion operator, and the expansion is defined as: This means that for each point x in image A, if B and A intersect at least partially after the origin of B is aligned with x, then x is considered to be part of the expanded image.
[0057] For the line image obtained through the above image preprocessing step, feature extraction can be performed on the line image after image preprocessing, and converted into an input suitable for the target detection module. Specifically, in this embodiment, the convolution layer in Faster R-CNN (Faster Region-based Convolutional Neural Network, a faster region-based convolutional neural network) can be used to extract features from the line image, as shown in the following formula:
[0058]
[0059] in, and Represent the i-th output in the l-th convolutional layer and a convolution kernel in the l-th layer, * and They represent the convolution operation and convolution layer bias of the power line image, respectively, and f(·) represents the ReLU activation function.
[0060] After obtaining the feature map corresponding to the line image, the RPN (Region Proposal Network) in Faster R-CNN can be used to determine the target candidate region, that is, the above-mentioned candidate box. The specific steps are as follows.
[0061] In some embodiments of the present application, generating a candidate frame based on an anchor frame containing a target to be detected includes the following steps: using a classification layer in a regional candidate network to determine the probability that an anchor frame in a feature map contains the target to be detected, wherein each point in the feature map corresponds to a set of anchor frames; using a regression layer in the regional candidate network to determine the offset of the target to be detected relative to the anchor frame; determining the position information of the candidate frame based on the probability determined by the classification layer and the offset determined by the regression layer, wherein the position information includes: the horizontal and vertical coordinates of the center point of the candidate frame, and the length and width of the candidate frame.
[0062] In some embodiments of the present application, determining the position information of the candidate frame based on the probability determined by the classification layer and the offset determined by the regression layer includes the following steps: correcting the anchor frame in the feature map based on the offset, wherein the offset includes: the horizontal coordinate offset and the vertical coordinate offset of the center point of the anchor frame, the length offset and the width offset of the anchor frame; obtaining a preset number of anchor frames with the highest probability of containing the target to be detected in the corrected anchor frame, and obtaining a set of candidate anchor frames; deleting the anchor frames in the candidate anchor frame set whose sizes are smaller than the preset size threshold, and determining the remaining anchor frames in the candidate anchor frame set as candidate frames.
[0063] Specifically, as a fully convolutional network, the RPN network shares the convolution layer with Faster R-CNN, which maps d-dimensional feature vectors through sliding windows. K region proposals are jointly judged at different sliding window positions, so the regression layer and classification layer contain 4k and 2k outputs respectively. The output of the classification layer is the probability that different candidate boxes belong to the target or not, that is, to determine whether the anchor box contains the target to be detected. The K candidate boxes are parameterized by the corresponding k anchor boxes, and different anchors use the center of the current sliding window as the reference point.
[0064] That is, the classification layer in the RPN network is used to determine whether there is a target to be detected in each anchor box, and the regression layer is used to generate the offset corresponding to each anchor box, which is used to correct the anchor box to obtain a more accurate candidate box. The above offset is shown in the following formula:
[0065]
[0066] Among them, x a ,y a 、w a 、h a Respectively represent the horizontal coordinate, vertical coordinate, width and height of the center point of the anchor box, t x ,t y ,t w ,t h The offset predicted by the regression layer, x, y, w, h are the horizontal coordinate, vertical coordinate, width and height of the center point of the anchor box corrected by the offset.
[0067] The following is an introduction to the training process of the region candidate network. The specific steps are as follows.
[0068] In some embodiments of the present application, the region candidate network is trained by the following steps: using the initial region candidate network to predict the probability that the anchor box corresponding to the training image contains the target to be detected, and the offset of the target to be detected relative to the anchor box; calculating a first loss value based on the predicted probability and the true probability value corresponding to the anchor box in the training image, wherein when the anchor box contains the target to be detected, the true probability value corresponding to the anchor box is one, and when the anchor box does not contain the target to be detected, the true probability value corresponding to the anchor box is zero; calculating a second loss value based on the predicted offset and the true offset between the anchor box and the target to be detected in the training image; determining a target loss value corresponding to the region candidate network based on the first loss value and the second loss value, and updating the model parameters of the region candidate network based on the target loss value, and repeating the above training steps using the updated region candidate network until the target loss value is no greater than a preset loss threshold.
[0069] Specifically, the loss function of the RPN network joint training is as follows:
[0070]
[0071] It contains the losses corresponding to the classification layer and the regression layer respectively. and Represent the classification loss function and box regression loss function respectively, where p i Indicates that the i-th anchor box predicted by the network contains the category of the target to be detected. Represents the corresponding Ground Truth (i.e., the above true probability value). In this embodiment, when the anchor box contains the target to be detected, Take 1, otherwise take 0, t i Represents the predicted offset corresponding to the i-th anchor box, Indicates the actual offset, N ds and N reg are the number of anchor boxes involved in the classification and regression loss calculations, respectively, and λ is a hyperparameter for adjusting the classification and regression prediction weights. cls (p i ,p * i ) can use the softmax function, It can be obtained by performing linear regression processing based on the obtained anchor box size and its center coordinates.
[0072] After determining the position of the candidate box, the pooling layer and the fully connected layer in Faster R-CNN can be used to extract the image area corresponding to the position of the candidate box in the feature map and determine the line fault type corresponding to the image area. The specific steps are as follows.
[0073] In some embodiments of the present application, extracting an image area in a feature map corresponding to the position of a candidate box and determining a line fault type corresponding to the image area includes the following steps: using a pooling layer to perform a pooling operation on the feature map based on the position information of the candidate box to obtain an image area in the feature map corresponding to the position of the candidate box, wherein the size of the image area after the pooling operation is consistent; using a fully connected layer and a classifier to determine the fault type corresponding to the image area.
[0074] Specifically, the mth layer in Faster R-CNN is a pooling layer, and its output feature vector is shown as follows:
[0075]
[0076] in, and f(·) represent the input and softmax function of the pooling layer respectively. and down(·) represent the summation operation of the connection weight and the input matrix, respectively. Represents the pooling layer bias.
[0077] The embodiment of the present application adopts ultraviolet imaging technology to efficiently detect partial discharge of power transmission lines through drones. The ultraviolet imaging equipment carried by the drone can capture the discharge phenomenon, enhance the image features in combination with the image preprocessing step, and then use the Faster R-CNN algorithm for target detection. The fast regional convolutional neural network is applied to the recognition of transmission line defects, which can realize rapid defect recognition and accurate positioning of discharge points. In addition, data can be transmitted to the edge computing end in real time for analysis through 5G technology, ensuring real-time monitoring and management of discharge activities.
[0078] Compared with the optical detection of partial discharge based on visible light in the related art, the present invention adopts a drone equipped with precise ultraviolet imaging equipment to comprehensively scan the transmission line during flight, and uses an ultraviolet camera to capture a series of phenomena caused by partial discharge of the transmission line, thereby generating clear image information. Through professional processing and in-depth analysis of image data, the location and degree of the discharge point can be accurately determined, providing accurate assessment and timely processing of the health status of the power line, thereby effectively preventing power grid failure accidents and ensuring stable and safe operation of the power system.
[0079] According to an embodiment of the present application, an embodiment of a line detection device is also provided. Figure 4 is a schematic diagram of the structure of a line detection device provided according to an embodiment of the present application. Figure 4 As shown, the device comprises:
[0080] An image acquisition module 40 is used to acquire a line image and perform image preprocessing on the line image, wherein the line image is an ultraviolet imaging image of the power transmission line collected by a drone;
[0081] A feature extraction module 42 is used to extract features from the line image after image preprocessing to obtain a feature map;
[0082] The region candidate module 44 is used to determine the anchor box containing the target to be detected in the feature map, and generate a candidate box based on the anchor box containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor box is a set of frames of different sizes preset in the feature map, and the candidate box is used to select the region where the target to be detected is located in the feature map;
[0083] The fault classification module 46 is used to extract the image area corresponding to the position of the candidate box in the feature map, and determine the line fault type corresponding to the image area, wherein the line fault type includes: partial discharge fault.
[0084] Optionally, generating a candidate frame based on an anchor frame containing the target to be detected includes: using a classification layer in a regional candidate network to determine the probability that the anchor frame in the feature map contains the target to be detected, wherein each point in the feature map corresponds to a set of anchor frames; using a regression layer in the regional candidate network to determine the offset of the target to be detected relative to the anchor frame; determining position information of the candidate frame based on the probability determined by the classification layer and the offset determined by the regression layer, wherein the position information includes: the horizontal and vertical coordinates of the center point of the candidate frame, and the length and width of the candidate frame.
[0085] Optionally, determining the position information of the candidate frame based on the probability determined by the classification layer and the offset determined by the regression layer includes: correcting the anchor frame in the feature map based on the offset, wherein the offset includes: the horizontal coordinate offset and the vertical coordinate offset of the center point of the anchor frame, and the length offset and the width offset of the anchor frame; obtaining a preset number of anchor frames with the highest probability of containing the target to be detected in the corrected anchor frame, and obtaining a set of candidate anchor frames; deleting the anchor frames in the candidate anchor frame set whose sizes are smaller than a preset size threshold, and determining the remaining anchor frames in the candidate anchor frame set as candidate frames.
[0086] Optionally, the region candidate network is trained by the following steps: using the initial region candidate network to predict the probability that the anchor box corresponding to the training image contains the target to be detected, and the offset of the target to be detected relative to the anchor box; calculating a first loss value based on the predicted probability and the true probability value corresponding to the anchor box in the training image, wherein when the anchor box contains the target to be detected, the true probability value corresponding to the anchor box is one, and when the anchor box does not contain the target to be detected, the true probability value corresponding to the anchor box is zero; calculating a second loss value based on the predicted offset and the true offset between the anchor box and the target to be detected in the training image; determining a target loss value corresponding to the region candidate network based on the first loss value and the second loss value, and updating the model parameters of the region candidate network based on the target loss value, and repeating the above training steps using the updated region candidate network until the target loss value is no greater than a preset loss threshold.
[0087] Optionally, extracting the image area corresponding to the position of the candidate box in the feature map, and determining the line fault type corresponding to the image area includes: using a pooling layer to perform a pooling operation on the feature map based on the position information of the candidate box to obtain the image area corresponding to the position of the candidate box in the feature map, wherein the size of the image area after the pooling operation is consistent; using a fully connected layer and a classifier to determine the fault type corresponding to the image area.
[0088] Optionally, image preprocessing of the line image includes: performing a mean filtering operation on the line image, wherein the mean filtering operation is used to update the pixel average value of all pixels in the line image located in the neighborhood of a first pixel point to a new pixel value of the first pixel point; performing a grayscale operation on the line image that has undergone the mean filtering operation, wherein the grayscale operation is used to determine the grayscale value corresponding to the pixel point based on the red separation, green separation and blue components of each pixel point in the line image; performing a binarization operation on the line image that has undergone the grayscale operation, wherein the binarization operation is used to set the pixel value of the pixel point in the line image whose grayscale value is greater than a preset grayscale value to one, and to set the pixel value of the pixel point in the line image whose grayscale value is not greater than the preset grayscale value to zero.
[0089] Optionally, image preprocessing of the line image also includes: performing morphological processing on the line image that has undergone a binarization operation, wherein the morphological processing includes: an erosion operation and a dilation operation, and when the origin of the erosion operator is aligned with a pixel point of the line image, if the erosion operator is completely contained by a portion of the line image with a pixel value of one, the pixel point is determined to be a pixel point retained after the erosion operation; and when the origin of the dilation operator is aligned with a pixel point of the line image, if the dilation operator intersects with a portion of the line image with a pixel value of one, the pixel point is determined to be a pixel point retained after the dilation operation.
[0090] It should be noted that the various modules in the above-mentioned line detection device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.
[0091] It should be noted that the line detection device provided in this embodiment can be used to perform Figure 2 The line detection method shown, therefore, the relevant explanations and descriptions of the above line detection method are also applicable to the embodiments of the present application and will not be repeated here.
[0092] An embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following line detection method by running the computer program: obtaining a line image and performing image preprocessing on the line image, wherein the line image is an ultraviolet imaging image containing a power transmission line collected by a drone; performing feature extraction on the line image after image preprocessing to obtain a feature map; determining an anchor point frame containing a target to be detected in the feature map, and generating a candidate frame based on the anchor point frame containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor point frame is a set of frames of different sizes preset in the feature map, and the candidate frame is used to frame the area where the target to be detected is located in the feature map; extracting the image area corresponding to the position of the candidate frame in the feature map, and determining the line fault type corresponding to the image area, wherein the line fault type includes: partial discharge fault.
[0093] The embodiments of the present application also provide a computer program product, including a computer program, which, when executed by a processor, implements the steps of the line detection method described in each embodiment of the present application: obtaining a line image and performing image preprocessing on the line image, wherein the line image is an ultraviolet imaging image containing a power transmission line collected by a drone; performing feature extraction on the line image after image preprocessing to obtain a feature map; determining an anchor point frame containing a target to be detected in the feature map, and generating a candidate frame based on the anchor point frame containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor point frame is a set of frames of different sizes preset in the feature map, and the candidate frame is used to frame the area where the target to be detected is located in the feature map; extracting the image area corresponding to the position of the candidate frame in the feature map, and determining the line fault type corresponding to the image area, wherein the line fault type includes: partial discharge fault.
[0094] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0095] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0096] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0098] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0100] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A line detection method, It is characterized in that include: Acquire a line image and perform image preprocessing on the line image, wherein the line image is an ultraviolet imaging image of the power transmission line collected by a drone; Performing feature extraction on the line image after image preprocessing to obtain a feature map; Determine an anchor box containing a target to be detected in the feature map, and generate a candidate box based on the anchor box containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor box is a set of frames of different sizes preset in the feature map, and the candidate box is used to select the area where the target to be detected is located in the feature map; An image area corresponding to the position of the candidate box in the feature map is extracted, and a line fault type corresponding to the image area is determined, wherein the line fault type includes: partial discharge fault.
2. The line detection method according to claim 1, It is characterized in that Generating a candidate frame according to the anchor frame containing the target to be detected includes: Using a classification layer in a region candidate network, determining a probability that the anchor box in the feature map contains the target to be detected, wherein each point in the feature map corresponds to a set of anchor boxes; Using the regression layer in the region candidate network, determining the offset of the target to be detected relative to the anchor box; The position information of the candidate box is determined according to the probability determined by the classification layer and the offset determined by the regression layer, wherein the position information includes: the horizontal coordinate and the vertical coordinate of the center point of the candidate box, and the length and width of the candidate box.
3. The line detection method according to claim 2, It is characterized in that Determining the position information of the candidate box according to the probability determined by the classification layer and the offset determined by the regression layer includes: According to the offset, the anchor point frame in the feature map is corrected, wherein the offset includes: a horizontal coordinate offset and a vertical coordinate offset of a center point of the anchor point frame, and a length offset and a width offset of the anchor point frame; Obtaining a preset number of anchor point frames having the highest probability of containing the target to be detected from among the corrected anchor point frames, to obtain a set of candidate anchor point frames; The anchor frames whose sizes are smaller than a preset size threshold in the candidate anchor frame set are deleted, and the remaining anchor frames in the candidate anchor frame set are determined as the candidate frames.
4. The line detection method according to claim 2, It is characterized in that The region candidate network is trained by the following steps: Using the initial region candidate network, predicting the probability that the anchor box corresponding to the training image contains the target to be detected, and the offset of the target to be detected relative to the anchor box; Calculating a first loss value according to the predicted probability and a true probability value corresponding to the anchor box in the training image, wherein, when the anchor box contains the target to be detected, the true probability value corresponding to the anchor box is one, and when the anchor box does not contain the target to be detected, the true probability value corresponding to the anchor box is zero; Calculating a second loss value according to the predicted offset and a real offset between the anchor point box and the target to be detected in the training image; According to the first loss value and the second loss value, a target loss value corresponding to the regional candidate network is determined, and according to the target loss value, the model parameters of the regional candidate network are updated, and the above training steps are repeated using the updated regional candidate network until the target loss value is no greater than a preset loss threshold.
5. The line detection method according to claim 2, It is characterized in that Extracting an image area corresponding to the position of the candidate frame in the feature map, and determining a line fault type corresponding to the image area includes: Using a pooling layer, according to the position information of the candidate box, a pooling operation is performed on the feature map to obtain an image area in the feature map corresponding to the position of the candidate box, wherein the size of the image area after the pooling operation is consistent; A fully connected layer and a classifier are used to determine the fault type corresponding to the image area.
6. The line detection method according to claim 1, It is characterized in that Performing image preprocessing on the line image includes: Performing a mean filtering operation on the line image, wherein the mean filtering operation is used to update the pixel average value of all pixels in the line image located in the neighborhood of the first pixel point to a new pixel value of the first pixel point; Performing a grayscale operation on the line image after the mean filtering operation, wherein the grayscale operation is used to determine the grayscale value corresponding to each pixel in the line image according to the red separation, green separation and blue component of each pixel in the line image; A binarization operation is performed on the line image that has undergone the grayscale operation, wherein the binarization operation is used to set the pixel values of the pixel points in the line image whose grayscale values are greater than the preset grayscale value to one, and to set the pixel values of the pixel points in the line image whose grayscale values are not greater than the preset grayscale value to zero.
7. The line detection method according to claim 6, It is characterized in that Performing image preprocessing on the line image further includes: The line image after the binarization operation is subjected to morphological processing, wherein the morphological processing includes: an erosion operation and a dilation operation. When the origin of the erosion operator is aligned with the pixel point of the line image, if the erosion operator is completely contained by the portion of the line image whose pixel value is one, the pixel point is determined to be a pixel point retained after the erosion operation; when the origin of the dilation operator is aligned with the pixel point of the line image, if the dilation operator intersects with the portion of the line image whose pixel value is one, the pixel point is determined to be a pixel point retained after the dilation operation.
8. A line detection device, It is characterized in that include: An image acquisition module, used to acquire a line image and perform image preprocessing on the line image, wherein the line image is an ultraviolet imaging image of the power transmission line collected by a drone; A feature extraction module, used to extract features from the line image after image preprocessing to obtain a feature map; A region candidate module, used to determine an anchor box containing a target to be detected in the feature map, and generate a candidate box based on the anchor box containing the target to be detected, wherein the target to be detected includes: a power transmission line, the anchor box is a set of frames of different sizes preset in the feature map, and the candidate box is used to select the region where the target to be detected is located in the feature map; A fault classification module is used to extract the image area corresponding to the position of the candidate box in the feature map, and determine the line fault type corresponding to the image area, wherein the line fault type includes: partial discharge fault.
9. A non-volatile storage medium, It is characterized in that The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the line detection method according to any one of claims 1 to 7 by running the computer program.
10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the line detection method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Rotary transformation wire harness processing quality detection method and system based on machine vision
CN120070426A
