Multi-light fusion adaptive lightweight inspection method, storage medium and equipment

Abstract

The invention discloses a multi-light fusion adaptive lightweight inspection method, a storage medium and equipment, and relates to the field of fault inspection, and the method comprises the steps: determining an inspection target on a power transmission line, dividing the importance and type of the inspection target, and lightening the inspection target; determining a to-be-inspected target according to the importance of the inspection target, and determining an initial coordinate of the to-be-inspected target; according to the coordinates of the to-be-inspected target, a path planning algorithm is called for path planning; shooting a visible light image of the to-be-inspected target according to the planned path, identifying the type of the to-be-inspected target in real time by using the visible light image, repositioning the initial coordinate of the to-be-inspected target, and adaptively scheduling and shooting an infrared image and / or an ultraviolet image according to the type of the to-be-inspected target to obtain an image of the to-be-inspected target; and analyzing the image of the to-be-inspected target by using the lightweight neural network model, and identifying a power transmission line fault. According to the invention, different types of fault hidden dangers can be accurately positioned and identified.

Description

Technical Field

[0001] The present invention relates to the technical field of fault inspection, and in particular to a multi-light fusion adaptive lightweight inspection method, storage medium, and equipment. Background Art

[0002] With the rapid development of the power industry, a large number of power lines have been erected. In order to ensure the stability and safety of the power grid operation, it is necessary to conduct regular inspections of the power lines. Line inspection is to find and eliminate problems by inspecting each link of the line, prevent accidents in advance, and ensure the quality of power supply.

[0003] Since power lines span long distances, in order to improve inspection efficiency and reduce inspection costs, it is necessary to obtain the inspection target and target location in advance during line inspection. Under different climates and air humidity, power lines face different safety hazards. For example, insulator strings will be corroded due to humidity in rainy and humid weather, increasing the risk of ultraviolet discharge. Therefore, they need to be listed as inspection targets in rainy and humid weather. Components or areas that had problems in the last inspection are listed as problem targets, and they also need to be listed as inspection targets in this inspection.

[0004] Ultraviolet and infrared signals are commonly found in fire point sources, electrical discharges, and natural light sources. Multi-light fusion detection equipment has gradually developed and is widely used in the fire protection and power industries, and is favored by people.

[0005] In the power-related industry, ultraviolet detection technology has been widely used in corona discharge, arc discharge, etc., and relevant fault information is obtained by acquiring ultraviolet signals. Infrared light detection is mainly used for heating fault information of power equipment. These technical products have become standard configuration equipment in the power-related industry, and are gradually used in the fields of power generation, power transmission, railways, etc., and can be equipped with drones, helicopters, robots and other means to achieve effective inspection of equipment through a full close-range perspective.

[0006] In the field of fire detection, ultraviolet light and infrared light signals detect fires by detecting ultraviolet light and infrared thermal imaging produced by the burning of substances.

[0007] The Chinese invention patent with announcement number: CN102694351B discloses a photoelectric detection device for high-voltage overhead power transmission line inspection drones, which carries a visible light camera, an ultraviolet imager, a fully digital dynamic infrared thermal imager in any combination, and a laser rangefinder, and combines the interchangeability of multiple sensors to achieve a comprehensive and accurate high-voltage overhead power transmission line inspection task. However, the scheduling scheme between various cameras (sensors) is not described, and it is impossible to accurately locate and identify inspection targets and hidden dangers of the targets. Summary of the invention

[0008] The purpose of the present invention is to solve the problem of inaccurate inspection target positioning and identification in the prior art, and to propose a multi-light fusion adaptive lightweight inspection method, comprising the following steps:

[0009] S1. Determine the inspection targets on the transmission line, classify the importance and types of the inspection targets, and reduce the inspection targets;

[0010] S2. Determine the target to be inspected according to its importance and determine the initial coordinates of the target to be inspected;

[0011] S3. Calling a path planning algorithm to plan a path according to the coordinates of the target to be inspected;

[0012] S4. According to the planned path, a visible light image is taken of the inspection target, and the type of the inspection target is identified in real time using the visible light image, and the initial coordinates of the inspection target are relocated. According to the type of the inspection target, an infrared image and / or an ultraviolet image are adaptively scheduled to be taken to obtain an image of the inspection target;

[0013] S5. Build a neural network model based on YOLOv8, and perform training and verification. Perform lightweight and pruning operations on the trained and verified model to obtain a lightweight model. Use the lightweight model to analyze the image of the inspection target and identify transmission line faults.

[0014] Furthermore, the types of inspection targets include: discharge targets, heating targets, and damaged targets.

[0015] Furthermore, the importance of inspection targets is divided according to the following formula:

[0016] G f (o,t,T,H)=a×G1(o,t)+b×G2(o,T)+c×G3(o,H),

[0017] Among them G f (o, t, T, H) is the importance score of the inspection target, a, b, c are weights, o is the type of target, t is time, T is temperature, H is humidity, G i () is the importance score associated with the two variables in brackets, i=1,2,3.

[0018] Furthermore, the coordinate value is read through the RTK positioning technology. The coordinate value is the position information under the WGS-85 coordinate system. The Y coordinate in the coordinate value represents the latitude, the X coordinate represents the longitude, and H represents the altitude.

[0019] Furthermore, the path planning algorithm uses a simulated annealing algorithm and uses the Metropolis criterion to determine whether the optimal solution is obtained. The Metropolis criterion is shown in the formula:

[0020]

[0021] in, represents the probability of an object transferring from state i to state j at temperature T, E(i) and E(j) represent the internal energy of the object in state i and state j respectively, and k represents the Boltzmann constant;

[0022] The above is converted to the scenario of the present invention, It represents the probability of the UAV transferring from state i to state i at temperature T, E(i) and E(j) represent the objective functions of the UAV at coordinate i and state j, respectively, and the total length of the UAV inspection is set as the objective function.

[0023] Furthermore, the ultraviolet image is used to identify the state of the discharge target, and the infrared camera is used to identify the state of the heating target.

[0024] Furthermore, the YOLOv8 neural network is used to recognize the images taken by the visible light camera. The images taken by the visible light camera are input into the YOLOv8 neural network in real time, and the types of targets to be inspected in the images taken by the visible light camera are output and anchor frames are generated. The photo position is re-located according to the position of the anchor frame, and the camera combination is called to take photos according to the types of targets to be inspected.

[0025] Furthermore, a temperature threshold of the infrared image is set, the infrared image is converted into a binary image, all connected areas of the infrared image binary image and the ultraviolet image binary image are identified and marked, and the area of ​​all connected domains is calculated in pixels. If the area is greater than the set threshold, it means that there is a hidden danger in the shooting component.

[0026] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned power transmission line fault inspection method is implemented.

[0027] The present invention also proposes an electronic device, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the above-mentioned transmission line fault inspection method.

[0028] The beneficial effects brought by the technical solution provided by the present invention are:

[0029] The power transmission line fault inspection method proposed by the present invention first divides the importance and type of the inspection target; determines the target to be inspected, and determines the initial coordinates of the target to be inspected; performs initial positioning of the target to be inspected according to the planned path and initial coordinates, captures the visible light image of the target to be inspected after the initial positioning, processes the visible light image in real time, identifies the type of the target to be inspected in real time, repositions the initial coordinates of the target to be inspected according to the processed position, and schedules the shooting of infrared images and / or ultraviolet images according to the type of the target to be inspected. The present invention first performs initial positioning of the target, accurately repositions it according to the visible light image processing result, makes the photo clearer and more accurate, and schedules the shooting of infrared images and / or ultraviolet images according to the target type that can identify the optical image in real time, and accurately identifies different types of fault hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of a multi-light fusion adaptive lightweight inspection method according to an embodiment of the present invention;

[0031] Figure 2 It is a block diagram of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0033] A flowchart of a multi-light fusion adaptive lightweight inspection method according to an embodiment of the present invention is as follows: Figure 1 , specifically including the following steps:

[0034] S1. Determine the inspection targets on the transmission line, classify the importance and types of the inspection targets, and reduce the number of inspection targets.

[0035] Transmission line hidden dangers generally include discharge hidden dangers, heating hidden dangers, and rupture and damage hidden dangers. The target that is prone to discharge hidden dangers is a discharge target, the target that is prone to heating hidden dangers is a heating target, and the target that is prone to rupture and damage hidden dangers is a damage target. The types of inspection targets in the embodiments of the present invention include: discharge targets, heating targets, and damage targets.

[0036] The components of the transmission line include: insulators, poles, crossarms, uprights, hardware, conductors, wires, circuit breakers, disconnectors, lightning arresters, transformers, transformer platforms, distribution boxes, power cables, and grounding devices. Circuit breakers, disconnectors, transformers, transformer platforms, and distribution boxes are generally installed indoors and are not prone to failure. Insulators, poles, crossarms, uprights, hardware, conductors, wires, lightning arresters, power cables, and grounding devices are installed outdoors and are easily affected by the environment. Under different weather and air humidity, components that are prone to hidden dangers include: insulators, crossarms, lightning arresters, and grounding devices. Under different environments, different parts of the transmission line produce different hidden dangers. Please refer to Table 1 for the relationship between the environment and the type of inspection targets.

[0037] Table 1

[0038]

[0039] The importance of inspection targets is divided according to the following formula:

[0040] G f =a×G1(o,t)+b×G2(o,T)+c×G3(o,H)+d×G4(o,B)+e×G5(o,S)

[0041] Among them G f is the importance score of the inspection target, a, b, c, d, e are weights, o is the type of target, t is the duration, T is the temperature, H is the humidity, B is the bird strike, S is the lightning strike, G is the i () is the importance score associated with the two variables in brackets, i=1, 2, 3.

[0042] The arrester becomes the discharge target, the duration t and the lightning strike affect the importance score of the arrester, and the weights of temperature, humidity, and bird strike are 0. f >G t (importance threshold), it is classified as a high-risk target (for inspection), otherwise it is a normal target (not for inspection).

[0043] S2. According to the importance of the inspection target, determine the target to be inspected and determine the initial coordinates of the target to be inspected.

[0044] After determining the target to be inspected, the coordinate value is read through RTK positioning technology. The coordinate value is the location information in the WGS-85 coordinate system. The Y coordinate in the coordinate value represents the latitude, the X coordinate represents the longitude, and H represents the altitude.

[0045] S3. According to the coordinates of the target to be inspected, a path planning algorithm is called to perform path planning.

[0046] The path planning algorithm uses the simulated annealing algorithm and uses the Metropolis criterion to determine whether the optimal solution is obtained. The Metropolis criterion is shown in the formula:

[0047]

[0048] in, represents the probability of an object transferring from state i to state j at temperature T, E(i) and E(j) represent the internal energy of the object in state i and state j respectively, and k represents the Boltzmann constant;

[0049] The above is converted to the scenario of the present invention, It represents the probability that the UAV transfers from state i to state j (such as coordinate i transfers to coordinate j) at temperature T (UAV system temperature). E(i) and E(j) represent the objective functions of the UAV at coordinate i and state j, respectively. The total length of the UAV inspection is set as the objective function.

[0050] Assume d ij is the distance from the drone's hovering point i to j. Since the drone's hovering point is in mid-air, it is necessary to use three-dimensional coordinates to display the drone's position. The coordinate points i and j are defined as (x i ,y i ,z i ) and (x j ,y j ,z j ). Then d ij It can be expressed by the formula:

[0051]

[0052] The Metropolis criterion is then used to determine whether the optimal solution, that is, the shortest distance, is obtained. If the distance is the shortest, it proves that the optimal solution is obtained. The drone can conduct inspections according to the planned route, which can effectively save time and improve inspection efficiency.

[0053] S4. According to the planned path, visible light images are taken of the inspection targets, the types of the inspection targets are identified in real time using the visible light images, and the initial coordinates of the inspection targets are relocated. According to the types of the inspection targets, infrared images and / or ultraviolet images are adaptively scheduled for shooting.

[0054] S5. Build a neural network model based on YOLOv8, and perform training and verification. Perform lightweight and pruning operations on the trained and verified model to obtain a lightweight model. Use the lightweight model to analyze the image of the inspection target and identify transmission line faults.

[0055] The drone begins to inspect the transmission lines according to the planned route. When the drone arrives at the initial shooting location, the images taken by the visible light camera are input into the trained YOLOv8 neural network in real time. The YOLOv8 neural network is used to recognize the images of the visible light camera, output the type of the target to be inspected in the image taken by the visible light camera and the anchor frame, and re-locate the shooting position according to the position of the anchor frame. The onboard camera is controlled to accurately track the target components, and the camera combination is called to take pictures according to the type of the target to be inspected.

[0056] Use bilinear interpolation to adjust the image size to be consistent, specifically:

[0057]

[0058] in: Represents the value of a two-dimensional function at the integer coordinates (i+u,j+v), where u and v represent the offset relative to the integer coordinates (i,j) and are used to specify the position of the interpolation point. The function values ​​representing four adjacent points are weighted according to certain weights. It means that by interpolating the four points near the integer coordinates (i, j), the function value at the coordinates (i+u,j+v) is obtained.

[0059] Image graying is performed to eliminate redundant information and convert the color image into a gray image, specifically: Gray = 0.299×R+0.578×G+0.114×B.

[0060] Among them: R is the red channel component of the image; G is the green channel component of the image; B is the blue channel component of the image; Gray is the grayscale image.

[0061] The YOLOv8 neural network training process is as follows:

[0062] 1. Prepare a large number of labeled images of different components of the transmission line and combine them into a data set, which are preprocessed to obtain training sets and test sets for neural network model training and performance testing.

[0063] 2. Perform model compression operations such as quantization and pruning on the trained model to minimize the model's memory usage and increase the model's running speed while ensuring a certain level of accuracy.

[0064] S5. Analyze the image of the inspection target and identify the transmission line fault.

[0065] Each image of the infrared camera is W×H×T, where the T value of each pixel represents the temperature of the area (the strength of thermal radiation). The temperature threshold of the infrared image is set and the infrared image is converted into a binary image.

[0066] Each image taken by the UV camera is W×H×D, where D is 0 or 1, and is a binary image.

[0067] Identify and mark all connected areas of the infrared image binary map and the ultraviolet image binary map, and calculate the area of ​​all connected domains in pixels. If the area is greater than the set threshold, it means that there is a hidden danger in the shooting component.

[0068] In an exemplary embodiment, a computer-readable storage medium is included, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned power transmission line fault inspection method is implemented.

[0069] See also Figure 2 In an exemplary embodiment, an electronic device is also included, including at least one processor, at least one memory, and at least one communication bus.

[0070] Wherein, a computer program is stored in the memory, and the computer program includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the above-mentioned power transmission line fault inspection method.

[0071] When the drone of the present invention arrives at the observation point according to the planned path, the visible light camera is first turned on, and the captured video is cut into video frames for processing. The network predicts the bounding box coordinates (including the center point coordinates and width and height), the bounding box confidence (indicating whether there is an object in the box) and the category probability distribution. After decoding, the above three sets of predicted values ​​in the input image are obtained and attached to the original image. According to the relative position of the output target component boundary box center coordinate position in the figure, the drone position can be dynamically, real-time and finely adjusted to achieve accurate tracking of the component. According to the type of the output target component and the artificially preset component-camera combination comparison table in Table 2, the red\ultraviolet camera can be quickly and adaptively switched on.

[0072] Table 2

[0073] Target Type Fever target Discharge target Damage Target Heating and discharging targets Open the camera Infrared, visible light UV, visible light Visible light Infrared, ultraviolet, visible light

[0074] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-light fusion adaptive lightweight inspection method, characterized in that: The following steps are involved: S1. Determine the inspection targets on the transmission line, classify the importance and types of the inspection targets, and reduce the inspection targets; S2. Determine the target to be inspected according to its importance and determine the initial coordinates of the target to be inspected; S3. Calling a path planning algorithm to plan a path according to the coordinates of the target to be inspected; S4. According to the planned path, a visible light image is taken of the inspection target, and the type of the inspection target is identified in real time using the visible light image, and the initial coordinates of the inspection target are relocated. According to the type of the inspection target, an infrared image and / or an ultraviolet image are adaptively scheduled to be taken to obtain an image of the inspection target; S5. Build a neural network model based on YOLOv8, and perform training and verification. Perform lightweight and pruning operations on the trained and verified model to obtain a lightweight model. Use the lightweight model to analyze the image of the inspection target and identify transmission line faults.

2. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: The types of inspection targets include: discharge targets, heating targets, and damaged targets.

3. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: The importance of inspection targets is divided according to the following formula: G f (o,t,T,H)=a×G1(o,t)+b×G2(o,T)+c×G3(o,H), Among them G f (o, t, T, H) is the importance score of the inspection target, a, b, c are weights, o is the type of target, t is time, T is temperature, H is humidity, G i () is the importance score associated with the two variables in brackets, i=1,2,3.

4. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: The coordinate value is read through RTK positioning technology. The coordinate value is the location information in the WGS-85 coordinate system. The Y coordinate in the coordinate value represents the latitude, the X coordinate represents the longitude, and the H represents the altitude.

5. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: The path planning algorithm uses the simulated annealing algorithm and uses the Metropolis criterion to determine whether the optimal solution is obtained. The Metropolis criterion is shown in the formula: in, represents the probability of an object transferring from state i to state j at temperature T, E(i) and E(j) represent the internal energy of the object in state i and state j respectively, and k represents the Boltzmann constant; The above is converted to the scenario of the present invention, It represents the probability of the UAV transferring from state i to state j at temperature T. E(i) and E(j) represent the objective functions of the UAV at coordinate i and state j, respectively. The total length of the UAV inspection is set as the objective function.

6. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: The ultraviolet image is used to identify the state of the discharge target, and the infrared camera is used to identify the state of the heating target.

7. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: The YOLOv8 neural network is used to recognize images from the visible light camera. The images taken by the visible light camera are input into the YOLOv8 neural network in real time. The types of targets to be inspected in the images taken by the visible light camera are output and anchor frames are generated. The photo position is re-located according to the anchor frame position, and the camera combination is called to take photos according to the types of targets to be inspected.

8. The multi-light fusion adaptive lightweight inspection method according to claim 1 is characterized in that: Set the temperature threshold of the infrared image, convert the infrared image into a binary image, identify and mark all connected areas of the infrared image binary image and the ultraviolet image binary image, and calculate the area of ​​all connected domains in pixels. If the area is greater than the set threshold, it means that there is a hidden danger in the shooting component.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the method according to any one of claims 1 to 8.

Images