Electric power inspection method and system based on image recognition
The image recognition-based method optimizes drone shooting strategies using multi-objective genetic algorithms to reduce data redundancy and enhance accuracy in power inspection.
Patent Information
- Application Number
- CN202411200033.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-03
AI Technical Summary
Existing drone-based power inspection methods face challenges in optimizing shooting strategies to reduce data redundancy while ensuring accuracy, as single angles may miss safety hazards and full shots generate excessive data, straining storage and processing capabilities.
An image recognition-based method using multi-objective genetic algorithms to optimize shooting coordinates, combining orientation and equipment area information to generate discrete and optimized shooting positions for drones, reducing redundant data capture.
Ensures comprehensive and accurate power inspection by minimizing redundant data capture and improving efficiency through targeted drone shooting.
Smart Images

Figure CN119068368A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electric power inspection, and in particular to an electric power inspection method and system based on image recognition. Background Art
[0002] Using drones for power inspection is an efficient and accurate technical means that integrates navigation, remote sensing and image recognition technologies. During the inspection process, the drone first flies autonomously along the preset route through path planning to cover the entire power line area. During the flight, the drone is equipped with high-definition cameras and infrared thermal imagers and other sensor equipment to take directional photos of key parts of power equipment such as towers, insulators, and wires, and capture the status of power equipment and changes in the surrounding environment in real time, thereby realizing power inspection.
[0003] However, when drones are photographing power equipment, a single shooting angle may not fully reflect the status of the power equipment, resulting in potential safety hazards being missed; and if a full shot is taken, a large amount of data will be generated, which puts higher demands on the storage capacity, data transmission speed, and subsequent data processing and analysis capabilities of the drone. Therefore, how to optimize the shooting strategy to reduce data redundancy while ensuring the accuracy of drone inspections has become an urgent problem to be solved in the current field of drone inspections of power lines. Summary of the invention
[0004] In order to ensure the accuracy of drone inspections while optimizing shooting strategies to reduce data redundancy, the present application provides an image recognition-based power inspection method and system.
[0005] In the first aspect, the present application provides a power inspection method based on image recognition, which adopts the following technical solution: An electric power inspection method based on image recognition, applied to a UAV, comprises: Acquire image data of power inspection; Extracting orientation information of each power line based on the image data, and respectively calculating first offset coordinates between the orientation information and the preset route information of the drone; Extracting equipment area information of the electric power equipment to be inspected based on the image data; Perform one-to-one matching on the device area information and the power lines to obtain a matching result; wherein the matching result includes a corresponding relationship between each of the power lines and its device area information; Based on the matching result, a second offset coordinate between the preset key inspection area in the equipment area information and the corresponding power line is obtained; generating a plurality of discrete shooting coordinates based on each group of the first offset coordinates and the second offset coordinates; Inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm; The drone is controlled to move to each optimized shooting coordinate in the coordinate array in sequence, and performs shooting to obtain an inspection image.
[0006] Optionally, extracting the orientation information of each power line based on the image data, and respectively calculating the first offset coordinates between the orientation information and the preset route information of the drone, includes: Extracting the edge contour of each power line in the image data to obtain multiple line areas; Determine whether each line area conforms to a preset line shape template; If they meet the requirements, multiple line areas are marked as independent areas respectively; If not, dividing the plurality of line areas into a plurality of independent areas by using the line shape template; Obtain the center coordinates of each pixel in each of the independent areas; Performing linear regression processing on the central coordinates of all pixel points in each of the independent regions respectively to obtain a regression line corresponding to each of the independent regions, and generating orientation information of the regression line; The first offset coordinates between the orientation information and the preset route information of the UAV are calculated respectively.
[0007] Optionally, the performing one-to-one matching on the device area information and the power lines to obtain a matching result includes: Get the independent area corresponding to each power line; The device region information closest to the independent region is selected as the target device region information corresponding to the independent region, and the independent region and the target device region information are associated to obtain a matching result.
[0008] Optionally, the step of inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm, comprises: Inputting a plurality of the discrete shooting coordinates into a multi-objective genetic algorithm model, generating an initial population based on the discrete shooting coordinates; wherein the initial population includes a plurality of original chromosome data, and each of the original chromosome data is a coordinate array composed of one or more discrete shooting coordinates; Performing a crossover mutation operation on the original chromosome data in the initial population to obtain first chromosome data; Constructing a fitness function based on the coordinates of the key inspection area; Based on the fitness function, respectively calculating fitness values of the original chromosome data and the first chromosome data; Performing Pareto sorting based on the fitness value to obtain an optimized population, and repeating the crossover mutation operation on the optimized population until a preset condition is met, so as to generate second chromosome data; A coordinate array including one or more optimized shooting coordinates is determined from the second staining data through Pareto sorting.
[0009] Optionally, constructing a fitness function based on the coordinates of the key inspection area includes: Constructing a first objective function based on the distance information between the shooting coordinates and the coordinates of the key inspection area; Based on the difference between the preset standard shooting angle of the key inspection area and the actual shooting angle, a second objective function is constructed; wherein the actual shooting angle is the deviation angle between the shooting coordinates and the coordinates of the key inspection area; Based on the first objective function and the second objective function, constructing an effective shooting value for each of the key inspection areas, and generating a coverage function for predicting that the key inspection areas can be effectively photographed based on the effective shooting values; Constructing a penalty function based on the number of optimized shooting coordinates included in the coordinate array; The fitness function is obtained based on the coverage function and the penalty function.
[0010] Optionally, also include: Extracting type information of the electric power equipment to be inspected based on the image data; The preset standard shooting angle of the key inspection area of the power equipment is obtained based on the type information.
[0011] Optionally, the step of calculating fitness values of the original chromosome data and the first chromosome data based on the fitness function respectively includes: If the original chromosome data or the first chromosome data contains one of the shooting coordinates, substituting the shooting coordinate into the fitness function to obtain the fitness value of the original chromosome data or the first chromosome data; If the original chromosome data or the first chromosome data contains multiple shooting coordinates, the multiple shooting coordinates are respectively substituted into the fitness function to generate a coverage value for each key inspection area based on each discrete shooting coordinate, and the extreme value among all coverage values is taken as the valid shooting value.
[0012] In the second aspect, the present application provides a power inspection system based on image recognition, which adopts the following technical solutions: An electric power inspection system based on image recognition, comprising: An image acquisition module, used to acquire image data of power inspection; a first offset recognition module, configured to extract orientation information of each power line based on the image data, and respectively calculate first offset coordinates between the orientation information and the preset route information of the drone; An equipment area identification module, used for extracting equipment area information of the power equipment to be inspected based on the image data; A matching module, used for performing one-to-one matching between the device area information and the power lines to obtain a matching result; wherein the matching result includes a corresponding relationship between each of the power lines and its device area information; A second offset identification module is used to obtain a second offset coordinate between a preset key inspection area in the equipment area information and a corresponding power line based on the matching result; A discrete coordinate generation module, used for generating a plurality of discrete shooting coordinates based on each group of the first offset coordinates and the second offset coordinates; A result generation module, used for inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm; The shooting control module is used to control the drone to move to each optimized shooting coordinate in the coordinate array in turn and perform shooting to obtain an inspection image.
[0013] In a third aspect, the present application provides a control device, which adopts the following technical solution: A control device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program according to any one of the above methods.
[0014] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes a computer program that can be loaded by a processor and executed in any of the above methods. In summary, the present application includes the following beneficial technical effects: First, the orientation information of each line is extracted through image data, and the first offset coordinates between the preset route of the drone are calculated, which provides a basis for the drone to initially locate near the power line. Subsequently, the specific area information of the power equipment to be inspected is identified and extracted, and an accurate correspondence is established between the equipment area and the corresponding power line. On this basis, the second offset coordinates between the key inspection area in the equipment area and the power line are further calculated, and the first offset coordinates and the second offset coordinates are combined to generate multiple discrete shooting coordinates for each key inspection area for preliminary positioning of the drone shooting position. Finally, these discrete shooting coordinates are optimized using a multi-objective genetic algorithm model to generate a set of coordinate arrays containing the optimal shooting coordinates, reducing the positions and times that the drone needs to perform shooting, so that the drone moves and performs shooting in sequence according to the optimized coordinate array, effectively avoiding invalid or repeated shooting, thereby ensuring the comprehensiveness and accuracy of shooting during the inspection, while significantly improving the inspection efficiency, optimizing the shooting strategy, and reducing the redundancy of shooting data. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 This is a flow chart of an electric power inspection method based on image recognition according to one embodiment of the present application.
[0016] Figure 2 It is a flow chart of a method for generating a first offset coordinate according to one embodiment of the present application.
[0017] Figure 3 It is a flow chart of a method for generating coordinate data using a multi-objective genetic algorithm model in one embodiment of the present application.
[0018] Figure 4 It is a flow chart of a method for generating a fitness function in a multi-objective genetic algorithm model in one embodiment of the present application.
[0019] Figure 5 It is a block diagram of an electric power inspection system based on image recognition according to one embodiment of the present application. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0021] The present application embodiment discloses a power inspection method based on image recognition. Figure 1 , a power inspection method based on image recognition, applied to UAV, including: Step S101: Acquire image data of power inspection.
[0022] Among them, the image data is taken by one or more cameras carried on the drone.
[0023] Step S102: extracting the orientation information of each power line based on the image data, and respectively calculating the first offset coordinates between the orientation information and the preset route information of the drone.
[0024] The orientation information refers to the direction or extension trend of the power line; the preset route information is the flight path set by the drone before performing the inspection task. It should be understood that since the preset route information may not coincide with the orientation information of each power line, calculating the first offset coordinate between the orientation information and the preset route of the drone helps the drone locate each power line so that the drone can inspect each power line in a targeted manner according to actual needs.
[0025] Step S103: extracting equipment area information of the power equipment to be inspected based on the image data.
[0026] Among them, the equipment area information refers to the specific location and range of the power equipment in the image data. The power equipment can be insulators, anti-vibration hammers, hardware, etc. Insulators are electrical equipment used to support and insulate power lines, usually made of insulating materials such as ceramics, glass fiber reinforced plastics, etc. The main function of the insulator is to electrically insulate the conductors of the power line from the brackets, prevent the current from flowing to the ground through the brackets, and ensure the safety of power transmission. Anti-vibration hammers are devices used to reduce shock and protect power lines. They are usually installed at the bottom of the insulator string to reduce the vibration and impact force on the line, reduce the impact on the insulators and brackets, and thus extend the service life of the power line. Hardware is a metal part used to connect, fix and support power lines. Hardware includes crossarms, hangers, wire clamps, etc. on pole towers, which are used to support and fix conductors, insulator strings and other line equipment.
[0027] Step S104: Perform one-to-one matching on the device area information and the power lines to obtain a matching result.
[0028] The matching result includes the correspondence between each power line and its equipment area information. It should be understood that since each power line corresponds to a set of corresponding power equipment, such as insulators, anti-vibration hammers, hardware, etc., these power equipment need to be set at the junction of each power line and the tower, so the distribution positions of the power equipment are different, and it is difficult to use a unified shooting position for shooting equipment such as towers or transformers. Therefore, in this embodiment, the power equipment corresponding to each power line is extracted and matched separately to obtain the equipment area information uniquely corresponding to the power line.
[0029] Step S105: Based on the matching result, a second offset coordinate between the key inspection area preset in the equipment area information and the corresponding power line is obtained.
[0030] Specifically, based on the matching results, the preset key inspection area in the equipment area information is further determined. The key inspection area can be the key parts or potential fault points of the power equipment, or the power equipment as a whole. For example, in the equipment area information of the insulator, the key inspection area can be the lower half of the insulator used to suspend the line, which is more vulnerable to damage. In this case, the lower half of the equipment area information can be used as the key inspection area. By calculating the second offset coordinates between the key inspection area and the corresponding power line, the flight position that the drone should take when shooting the key inspection area can be determined more accurately, thereby improving the shooting effect.
[0031] Step S106: generating a plurality of discrete shooting coordinates based on each group of the first offset coordinates and the second offset coordinates.
[0032] It should be understood that in this embodiment, the calculation of the first offset coordinate and the second offset coordinate will not adjust the flight route of the drone in real time, that is, the drone will not change its preset route information when inspecting the power lines. The preset route information will only be changed when the drone flies to the power equipment at the tower for targeted inspection. This is because currently drones usually use an indiscriminate repeated flight sampling method to inspect at the tower, which makes it easy to capture a large number of redundant images. That is, this embodiment is designed to more accurately sample and inspect the power equipment at the tower.
[0033] Step S107: inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm; Among them, the discrete shooting coordinates are the optimal shooting coordinates corresponding to each key inspection area. However, since there are many key inspection areas and some key inspection areas can be inspected by shooting at the same position, in order to reduce the amount of data, inspection images are not taken at each discrete shooting coordinate point. Instead, the discrete shooting coordinate points are input into the multi-objective genetic algorithm model as the initial population to generate optimized shooting coordinates with less shooting.
[0034] Step S108: Control the drone to move to each optimized shooting coordinate in the coordinate array in turn, and perform shooting to obtain an inspection image.
[0035] As an example, the inspection image can be sent to the inspection management center for reference by the inspection personnel. In addition, step S108 can also include image recognition of the inspection image to obtain the inspection results of the power equipment. The inspection results can include whether the power equipment is normal, the risks of the power equipment, etc.
[0036] In the above implementation, the orientation information of each line is first extracted through the image data, and the first offset coordinates between the preset route of the drone are calculated, which provides a basis for the drone to initially locate near the power line. Subsequently, the specific area information of the power equipment to be inspected is identified and extracted, and an accurate correspondence is established between the equipment area and the corresponding power line. On this basis, the second offset coordinates between the key inspection area in the equipment area and the power line are further calculated, and the first offset coordinates and the second offset coordinates are combined to generate multiple discrete shooting coordinates for each key inspection area for preliminary positioning of the drone shooting position. Finally, these discrete shooting coordinates are optimized using a multi-objective genetic algorithm model to generate a set of coordinate arrays containing the optimal shooting coordinates, reducing the positions and times that the drone needs to perform shooting, so that the drone moves and performs shooting in sequence according to the optimized coordinate array, effectively avoiding invalid or repeated shooting, thereby ensuring the comprehensiveness and accuracy of shooting during the inspection, significantly improving the inspection efficiency, optimizing the shooting strategy, and reducing the redundancy of shooting data.
[0037] Reference Figure 2 As an implementation of step S102, step S102 includes: Step S1021: extracting the edge contour of each power line in the image data to obtain multiple line areas; In some embodiments, after step S1021, the step may further include converting the line area into a binary image; and performing morphological processing on the binary image. The binary image is a black and white image. In this embodiment, the morphological processing may adopt a morphological closing operation. The binary image records the pixels corresponding to the multiple line areas as 1, and the pixels corresponding to the remaining areas as 0. The binary image is then subjected to a morphological closing operation. The morphological closing operation can exclude small holes in the multiple line areas, connect the narrow discontinuities and gullies in the multiple line areas, and exclude small holes in the multiple line areas.
[0038] Step S1022: Determine whether each line area conforms to a preset line shape template; if so, execute step S1023; if not, execute step S1024; It should be understood that power lines usually have clear shape features, such as straight line segments or regular curve segments, and the line shape template is a template that includes the shape features of the power line. Although the method of extracting the line area by edge contour is simple, there may be a situation where the line areas overlap or intersect, and the line areas corresponding to multiple power circuits will be identified as a whole area. Therefore, by comparing the shape features of the line area with the preset template, it can be determined whether the line area is a line area corresponding to an independent power line.
[0039] Step S1023: Marking multiple line areas as independent areas respectively; The independent area refers to a line area corresponding to only one power circuit.
[0040] Step S1024: Divide the plurality of line regions into a plurality of independent regions using a line shape template.
[0041] As an example, a plurality of independent regions may be distinguished by a sliding window method, for example, after scaling and rotating the line shape template in different ways, the line region is traversed to identify a plurality of independent regions in the line region.
[0042] Step S1025: Obtain the center coordinates of each pixel in each independent area; Step S1026: Perform linear regression processing on the central coordinates of all pixel points in each independent area respectively, obtain the regression line corresponding to each independent area, and generate the orientation information of the regression line.
[0043] Among them, linear regression is a statistical method for finding the best fitting straight line between a set of data points. In this embodiment, linear regression processing is performed on the central coordinates of all pixel points in each independent area to find the best fitting straight line of the area, that is, the regression line. This is because power lines usually appear in the shape of straight lines or curves, so the main trends and directions of these independent areas can be found through linear regression, thereby obtaining the orientation information of the regression line, so as to plan the subsequent calculation of the first offset coordinates and the shooting path.
[0044] Step S1027: respectively calculating the first offset coordinates between the orientation information and the preset route information of the drone.
[0045] It should be understood that the first offset coordinates represent the degree of offset between the preset route information of the drone and the actual direction of the power line. It should be noted that the calculation of the first offset coordinates does not change the preset route of the drone, but is used to locate the shooting position of the drone in the key inspection area in the future.
[0046] In the above implementation, the line area is obtained by edge contour extraction. Since the shape of the power line is relatively certain, a relatively simple template matching method is used to distinguish independent areas from the line areas with overlapping or intersecting areas. Then, linear regression processing is performed on the independent areas to convert the curved power line into a regression line that can represent its characteristics, and the orientation information of the regression line is used as the orientation information of the power line, so that the first offset coordinate can be calculated more quickly.
[0047] As an implementation of step S103, step S103 includes: Step S1031: Obtaining the independent area corresponding to each power line; Step S1032: selecting the device region information closest to the independent region as the target device region information corresponding to the independent region, and associating the independent region with the target device region information to obtain a matching result.
[0048] The target device area information may be the device area information that is closest to the independent area. This is because the power equipment corresponding to each power line is usually installed at the connection between the power line and the tower, so the power equipment used to fix the power line is closest to the power line, so that the independent area and the target device area information can be matched more quickly.
[0049] Reference Figure 3 As an implementation of step S107, step S107 includes: Step S1071: inputting the plurality of discrete shooting coordinates into a multi-objective genetic algorithm model, and generating an initial population based on the discrete shooting coordinates; Among them, the multi-objective genetic algorithm is an evolutionary algorithm used to solve optimization problems with multiple objectives. Different from the single-objective genetic algorithm, the multi-objective genetic algorithm needs to consider multiple optimization objectives at the same time and try to find a solution set that strikes a balance between different objectives. It should be understood that in this embodiment, the purpose is to find a set of shooting coordinates that can both meet the shooting effect and reduce the amount of shooting data.
[0050] The initial population includes a number of original chromosome data, and each of the original chromosome data is a coordinate array composed of one or more discrete shooting coordinates, so that the number of discrete shooting coordinates contained in the original chromosome is different. The initial population is a candidate solution in the multi-objective genetic algorithm, that is, the operation starting point of the multi-objective genetic algorithm. In this embodiment, the coordinate array composed of the calculated discrete shooting coordinates is used as the initial population, so that the multi-objective genetic algorithm has a relatively excellent population from the beginning of the operation, so that the multi-objective genetic algorithm can generate optimized shooting coordinates more efficiently.
[0051] Step S1072: performing a crossover mutation operation on the original chromosome data in the initial population to obtain first chromosome data.
[0052] Among them, crossover mutation is a different operation in the genetic algorithm. Crossover is used to combine some genes of two parent chromosomes to generate new offspring chromosomes. Mutation randomly changes certain genes in the chromosome data with a certain probability to prevent the multi-objective genetic algorithm from falling into the local optimal solution, thereby generating the first chromosome data that is better than the initial chromosome data.
[0053] Step S1073: constructing a fitness function based on the coordinates of the key inspection area; Step S1074: Based on the fitness function, respectively calculate the fitness values of the original chromosome data and the first chromosome data.
[0054] The fitness value is a value obtained after the fitness function evaluates the original chromosome data and the first chromosome data, reflecting the degree to which the solutions represented by the original chromosome data and the first chromosome data satisfy the optimization problem.
[0055] Step S1075: Perform Pareto sorting based on fitness values to obtain an optimized population, and repeat the crossover and mutation operation on the optimized population until a preset condition is met to generate second chromosome data.
[0056] Among them, the optimization population is composed of the Pareto frontier obtained by Pareto sorting the first chromosome data by fitness value. In multi-objective optimization, Pareto (Pareto Ranking) sorting is used to sort solutions according to their merits. If a solution is not inferior to another solution in all objectives and is better than another solution in at least one objective, then the solution is said to Pareto dominate the other solution. Through Pareto sorting, a set of non-dominated solutions (i.e., Pareto frontier) can be obtained. Non-dominated solutions achieve a balance between multiple objectives, that is, the second chromosome data contained in the optimization population are all non-dominated solutions.
[0057] The preset condition may be the number of crossover mutations performed, for example, performing 50 or 100 crossover mutation operations, and repeating step S1075 each time to continuously re-determine the optimized population.
[0058] Step S1076: Determine a coordinate array including one or more optimized shooting coordinates from the second staining data through Pareto sorting.
[0059] It should be understood that by performing Pareto sorting on the second chromosome data again, it is ensured that the coordinate array finally selected is optimal in the current optimization population, that is, the coordinate array not only meets the coverage requirements of the key inspection area, but also reduces the amount of captured data.
[0060] In the above implementation, first, multiple discrete shooting coordinates are input as initial information into the multi-objective genetic algorithm model as original chromosome data to construct an initial population. Subsequently, the first generation of mutated chromosome data, i.e., the first chromosome data, is generated through crossover mutation operation. Then, a fitness function is constructed based on the coordinates of the key inspection area, and the fitness value is calculated to evaluate the efficiency and accuracy of the drone inspection under different shooting strategies, i.e., the fitness value of the original chromosome data and the first chromosome data. Then, the Pareto sorting method is used to select a non-dominated solution set from the original chromosome data and the first chromosome data as the optimized population to achieve a good balance on multiple objectives. By repeatedly repeating crossover mutation and Pareto sorting until the preset conditions are met, the second chromosome data containing the optimized shooting coordinates is generated. Finally, from the second chromosome data, the coordinate array of the optimal shooting position is determined again through Pareto sorting. It ensures that the drone can accurately cover all key inspection areas, and also minimizes invalid shooting and data redundancy, thereby significantly improving the inspection efficiency while ensuring the inspection quality.
[0061] Reference Figure 4 As an implementation of step S1073, step S1073 includes: Step S10731: constructing a first objective function based on the distance information between the shooting coordinates and the coordinates of the key inspection area; Among them, the shooting coordinates are the intermediate coordinates generated by the multi-objective genetic algorithm model during the optimization process. The shooting coordinates can be the coordinates represented in the initial population or the optimized population, that is, the shooting coordinates include the coordinates contained in the original chromosome data, the first chromosome data, and the second chromosome data, that is, the shooting coordinates include discrete shooting coordinates, optimized shooting coordinates, and other intermediate coordinates generated in the multi-objective genetic algorithm model. The first objective function can be the superposition value between the shooting coordinates and the coordinates of each key inspection area.
[0062] Step S10732: constructing a second objective function based on the difference between the preset standard shooting angle and the actual shooting angle of the key inspection area; The actual shooting angle is the deviation angle between the shooting coordinates and the coordinates of the key inspection area. Specifically, the current shooting orientation of the camera on the drone, that is, the focus position, can be obtained through the shooting coordinates, and the deviation angle can be obtained by comparing the focus position with the coordinates of the key inspection area.
[0063] Among them, the standard shooting angle refers to the shooting angle of the drone when shooting the key inspection area. For example, the key inspection area of the hardware can be photographed at a certain tilt angle to achieve the best shooting effect, while the shooting angle of the insulator can be shot at the angle that the camera is facing directly to achieve the best shooting effect.
[0064] Step S10733: Based on the first objective function and the second objective function, construct an effective shooting value for each of the key inspection areas, and generate a coverage function for predicting that the key inspection areas can be effectively photographed based on the effective shooting values.
[0065] As an example, since the first objective function calculates the distance information between the shooting coordinates and the coordinates of each key inspection area, the second objective function calculates the difference between the preset standard shooting angle and the actual shooting angle of each key inspection area, that is, the effective shooting value can be the weighted value between the difference between the standard shooting angle and the actual shooting angle and the distance information.
[0066] Among them, by setting a preset threshold, it can be determined whether the effective shooting value of each key inspection area meets the preset threshold. If it meets the preset threshold, the key inspection area is determined as effective shooting, and if it does not meet the preset threshold, the corresponding key inspection area is determined as invalid shooting. By judging whether each key inspection area is effectively photographed based on the effective shooting value of each key inspection area, a coverage function is obtained, which is the ratio of the key inspection areas with effective shooting and invalid shooting.
[0067] Step S10734: constructing a penalty function based on the number of optimized shooting coordinates included in the coordinate array; It should be understood that the original chromosome data in the initial population is composed of discrete shooting coordinates of different numbers. Therefore, in the optimization process using the multi-objective genetic algorithm model, the first chromosome data or the second chromosome data containing different numbers of shooting coordinates will be obtained. Taking the selection of the second chromosome data as an example, in order to avoid redundancy of shooting data, it is necessary to try to select the second chromosome data containing fewer shooting coordinates, and the number of shooting points cannot be increased indefinitely. Therefore, a penalty function is constructed to penalize the shooting points that exceed the preset number. The penalty function can give a negative score according to the number of shooting points that exceed the number, thereby guiding the reduction of unnecessary number of shooting points during the optimization process.
[0068] As an example, the preset number can be set to 3. When the shooting coordinates contained in the second chromosome data are equal to or less than 3, the second chromosome data with the optimal coverage function can be selected. If the shooting coordinates contained in the second chromosome data are greater than 3, it is necessary to balance the coverage of effective shooting of key inspection areas and the number of shooting coordinates required for the drone to perform shooting.
[0069] Step S10735: Obtain the fitness function based on the coverage function and the penalty function.
[0070] In the above implementation, firstly, based on the distance information between the shooting coordinates and the coordinates of the key inspection area, a first objective function is constructed to optimize the selection of the shooting position and ensure that the drone can get as close to the target area as possible. Subsequently, the difference between the preset standard shooting angle and the actual shooting angle of the key inspection area is used to construct the second objective function to ensure the accuracy of the shooting angle and avoid image distortion or information omission caused by angle deviation. Then, combining the first objective function and the second objective function, the effective shooting value is calculated for each key inspection area, and a coverage function is generated to comprehensively evaluate the shooting situation of the key inspection area under different shooting strategies. At the same time, in order to balance the shooting quality and efficiency, a penalty term function is introduced, which is adjusted according to the number of optimized shooting coordinates contained in the coordinate array, effectively avoiding data redundancy caused by excessive shooting. Finally, the coverage function is combined with the penalty term function to obtain the fitness function, which not only reflects the coverage effect of the shooting strategy on the key inspection area, but also takes into account the shooting efficiency and cost.
[0071] As a further implementation of the power inspection method based on image recognition, the power inspection method based on image recognition also includes: Step S201: extracting data on the type information of the power equipment to be inspected; Step S202 obtains the preset standard shooting angle of the key inspection area of the power equipment based on the type information.
[0072] Furthermore, after obtaining the type information, the effective shooting value can be redefined according to the type information, and different weighted values can be set for the distance information according to different type information. For example, for larger power equipment, the weighted value of its distance information can be appropriately lowered, because these power equipment can also achieve better shooting effects at a farther distance, and for smaller power equipment, the weighted value of its distance information can be appropriately increased, because smaller power equipment cannot capture its details when the distance information is slightly far away, and thus cannot achieve a better shooting effect.
[0073] As an implementation of step S1074, step S1074 further includes: Step S10741: if the original chromosome data or the first chromosome data contains a shooting coordinate, substituting the shooting coordinate into the fitness function to obtain the fitness value of the original chromosome data or the first chromosome data; Step S10741: If the original chromosome data or the first chromosome data contains multiple shooting coordinates, the multiple shooting coordinates are substituted into the fitness function respectively to generate a coverage value for each key inspection area based on each discrete shooting coordinate, and the extreme value among all coverage values is taken as the valid shooting value.
[0074] In the above implementation, in order to ensure that each key inspection area can meet the standard of valid shooting value in at least one inspection image when the drone shoots in sequence according to the shooting coordinates, for the case of multiple shooting coordinates, any key inspection area is valid as long as it is in the inspection image taken at any shooting coordinate. Therefore, the extreme value with the best shooting effect among all coverage values can be used as the valid shooting value.
[0075] In addition, an embodiment of the present application discloses a power inspection system based on image recognition. The power inspection system based on image recognition can be applied to a control device, which is a schematic diagram of the architecture of a control device for implementing the above method provided by an embodiment of the present invention. In this embodiment, the control device may include a power inspection system based on image recognition, a machine-readable storage medium, and a processor.
[0076] In this embodiment, the machine-readable storage medium and the processor may be located in the control device and the two may be separately arranged. The machine-readable storage medium may also be independent of the control device and accessed by the processor. The power inspection system based on image recognition may include multiple functional modules stored in the machine-readable storage medium, such as the various software functional modules included in the power inspection system based on image recognition. When the processor executes the computer program corresponding to the software functional module in the power inspection system based on image recognition, the power inspection system based on image recognition provided by the aforementioned method embodiment is implemented.
[0077] In this embodiment, the control device may include one or more processors. The processor may process information and / or data related to the service request to perform one or more functions described in the present invention. In some embodiments, the processor may include one or more processing engines (e.g., a single-core processor or a multi-core processor). Just to give an example, the processor may include one or more hardware processors, such as a central processing unit CPU, an application-specific integrated circuit ASIC, a dedicated instruction set processor ASIP, a graphics processor GPU, a physical operation processing unit PPU, a digital signal processor DSP, a field programmable gate array FPGA, a programmable logic device PLD, a controller, a microcontroller unit, a reduced instruction set computer RISC, a microprocessor, etc., or similar or any combination thereof.
[0078] The machine-readable storage medium may store data and / or instructions. In some embodiments, the machine-readable storage medium may store acquired data or information. In some embodiments, the machine-readable storage medium may store data and / or instructions for execution or use by the control device, and the control device may implement the exemplary method described in the present application by executing or using the data and / or instructions. In some embodiments, the machine-readable storage medium may include a mass storage, a removable storage, a volatile read-write memory, a read-only memory ROM, etc., or any combination of the above examples. Exemplary mass storage may include a disk, an optical disk, a solid-state disk, etc. Exemplary removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a compressed disk, a magnetic tape, etc. Exemplary volatile read-write memory may include a random access memory RAM. Exemplary random access memory may include dynamic RAM, double rate synchronous dynamic RAM, static RAM, thyristor RAM, and zero capacitance RAM, etc. Exemplary ROM may include a masked ROM, a programmable ROM, an erasable programmable ROM, an electronically erasable programmable ROM, a compressed disk ROM, and a digital universal disk ROM, etc.
[0079] The power inspection system based on image recognition included in the control device may include one or more software function modules. The software function modules may be programs and instructions stored in the machine-readable storage medium, and these software function modules are used to implement the above method when executed by the corresponding processor, for example, when executed by the processor of the drone, they are used to implement the method steps executed by the drone, or when executed by the control device, they are used to implement the method steps executed by the control device.
[0080] Refer to the detailed Figure 5 The present application embodiment discloses a power inspection system based on image recognition. A power inspection system based on image recognition includes: An image acquisition module, used to acquire image data of power inspection; a first offset recognition module, configured to extract orientation information of each power line based on the image data, and respectively calculate first offset coordinates between the orientation information and the preset route information of the drone; An equipment area identification module, used for extracting equipment area information of the power equipment to be inspected based on the image data; A matching module, used for performing one-to-one matching between the device area information and the power lines to obtain a matching result; wherein the matching result includes a corresponding relationship between each of the power lines and its device area information; A second offset identification module is used to obtain a second offset coordinate between a preset key inspection area in the equipment area information and a corresponding power line based on the matching result; A discrete coordinate generation module, used for generating a plurality of discrete shooting coordinates based on each group of the first offset coordinates and the second offset coordinates; A result generation module, used for inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm; The shooting control module is used to control the drone to move to each optimized shooting coordinate in the coordinate array in turn and perform shooting to obtain an inspection image.
[0081] The image recognition-based power inspection system provided in the present application can implement the above-mentioned image recognition-based power inspection method, and the specific working process of the image recognition-based power inspection system can refer to the corresponding process in the above-mentioned method embodiment.
[0082] It should be noted that in the above embodiments, the description of each embodiment has different emphases, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0083] The present invention also discloses a computer-readable storage medium, which includes a computer program that can be loaded by a processor and executed as in any of the above methods.
[0084] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0085] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0086] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.
Claims
1. A power inspection method based on image recognition, applied to unmanned aerial vehicles, characterized in that: include: Acquire image data of power inspection; Extracting orientation information of each power line based on the image data, and respectively calculating first offset coordinates between the orientation information and the preset route information of the drone; Extracting equipment area information of the electric power equipment to be inspected based on the image data; Perform one-to-one matching on the device area information and the power lines to obtain a matching result; wherein the matching result includes a corresponding relationship between each of the power lines and its device area information; Based on the matching result, a second offset coordinate between the preset key inspection area in the equipment area information and the corresponding power line is obtained; generating a plurality of discrete shooting coordinates based on each group of the first offset coordinates and the second offset coordinates; Inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm; The drone is controlled to move to each optimized shooting coordinate in the coordinate array in sequence, and performs shooting to obtain an inspection image.
2. The method according to claim 1, characterized in that The extracting the orientation information of each power line based on the image data, and respectively calculating the first offset coordinates between the orientation information and the preset route information of the drone, includes: Extracting the edge contour of each power line in the image data to obtain multiple line areas; Determine whether each line area conforms to a preset line shape template; If they meet the requirements, multiple line areas are marked as independent areas respectively; If not, dividing the plurality of line areas into a plurality of independent areas by using the line shape template; Obtain the center coordinates of each pixel in each of the independent areas; Performing linear regression processing on the central coordinates of all pixel points in each of the independent regions respectively to obtain a regression line corresponding to each of the independent regions, and generating orientation information of the regression line; The first offset coordinates between the orientation information and the preset route information of the UAV are calculated respectively.
3. The method according to claim 2, characterized in that The one-to-one matching of the device area information and the power line to obtain a matching result includes: Get the independent area corresponding to each power line; The device region information closest to the independent region is selected as the target device region information corresponding to the independent region, and the independent region and the target device region information are associated to obtain a matching result.
4. The method according to claim 1, characterized in that: The step of inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm, comprises: Inputting a plurality of the discrete shooting coordinates into a multi-objective genetic algorithm model, generating an initial population based on the discrete shooting coordinates; wherein the initial population includes a plurality of original chromosome data, and each of the original chromosome data is a coordinate array composed of one or more discrete shooting coordinates; Performing a crossover mutation operation on the original chromosome data in the initial population to obtain first chromosome data; Constructing a fitness function based on the coordinates of the key inspection area; Based on the fitness function, respectively calculating fitness values of the original chromosome data and the first chromosome data; Performing Pareto sorting based on the fitness value to obtain an optimized population, and repeating the crossover mutation operation on the optimized population until a preset condition is met, so as to generate second chromosome data; A coordinate array including one or more optimized shooting coordinates is determined from the second staining data through Pareto sorting.
5. The method according to claim 4, characterized in that The step of constructing a fitness function based on the coordinates of the key inspection area includes: Constructing a first objective function based on the distance information between the shooting coordinates and the coordinates of the key inspection area; Based on the difference between the preset standard shooting angle of the key inspection area and the actual shooting angle, a second objective function is constructed; wherein the actual shooting angle is the deviation angle between the shooting coordinates and the coordinates of the key inspection area; Based on the first objective function and the second objective function, constructing an effective shooting value for each of the key inspection areas, and generating a coverage function for predicting that the key inspection areas can be effectively photographed based on the effective shooting values; Constructing a penalty function based on the number of optimized shooting coordinates included in the coordinate array; The fitness function is obtained based on the coverage function and the penalty function.
6. The method according to claim 5, characterized in that Also includes: Extracting type information of the electric power equipment to be inspected based on the image data; The preset standard shooting angle of the key inspection area of the power equipment is obtained based on the type information.
7. The method according to claim 4, characterized in that The step of calculating fitness values of the original chromosome data and the first chromosome data based on the fitness function respectively includes: If the original chromosome data or the first chromosome data contains one of the shooting coordinates, substituting the shooting coordinate into the fitness function to obtain the fitness value of the original chromosome data or the first chromosome data; If the original chromosome data or the first chromosome data contains multiple shooting coordinates, the multiple shooting coordinates are respectively substituted into the fitness function to generate a coverage value for each key inspection area based on each discrete shooting coordinate, and the extreme value among all coverage values is taken as the valid shooting value.
8. An electric power inspection system based on image recognition, applied to drones, characterized in that: include: An image acquisition module, used to acquire image data of power inspection; a first offset recognition module, configured to extract orientation information of each power line based on the image data, and respectively calculate first offset coordinates between the orientation information and the preset route information of the drone; An equipment area identification module, used for extracting equipment area information of the power equipment to be inspected based on the image data; A matching module, used for performing one-to-one matching between the device area information and the power lines to obtain a matching result; wherein the matching result includes a corresponding relationship between each of the power lines and its device area information; A second offset identification module is used to obtain a second offset coordinate between a preset key inspection area in the equipment area information and a corresponding power line based on the matching result; A discrete coordinate generation module, used for generating a plurality of discrete shooting coordinates based on each group of the first offset coordinates and the second offset coordinates; A result generation module, used for inputting the plurality of discrete shooting coordinates and the coordinates of the key inspection area into a multi-objective genetic algorithm model, and generating a coordinate array including one or more optimized shooting coordinates through the multi-objective genetic algorithm; The shooting control module is used to control the drone to move to each optimized shooting coordinate in the coordinate array in turn and perform shooting to obtain an inspection image.
9. A control device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program according to any one of the methods of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The method comprises storing a computer program which can be loaded by a processor and execute the method according to any one of claims 1 to 7.
Citation Information
Cited By
Substation three-dimensional inspection system based on image analysis
CN120297533A
