Power inspection method, device and equipment based on binocular camera and laser radar

Abstract

The invention provides an electric power inspection method, device and equipment based on a binocular camera and a laser radar, and relates to the technical field of electric power inspection. The method comprises the steps of determining at least one target electric power facility based on an electric power inspection task, selecting a plurality of reference points on the target electric power facility, and obtaining a reference point set according to all the reference points; determining vibration information of the inspection unmanned aerial vehicle according to an acquisition result of the laser radar in a preset time period and the position coordinate of each reference point in the reference point set; determining shaking information of the binocular camera based on an initial real-time image collected by the binocular camera, the position coordinates of each reference point in the reference point set and the vibration information of the inspection unmanned aerial vehicle, and fusing the shaking information with the initial real-time image to obtain a real-time image of the binocular camera; and obtaining an inspection result of the at least one target electric power facility according to the real-time image after image processing. According to the invention, the consistency of the real-time image and the actual situation can be ensured, and the accuracy of the inspection result is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power inspection, and in particular to a power inspection method, device and equipment based on a binocular camera and a laser radar. BACKGROUND

[0002] Complex weather conditions such as strong winds, smog, rain and snow can greatly affect the visual perception ability and inspection accuracy of the unmanned aerial vehicle. The binocular camera can provide depth information and three-dimensional spatial perception ability by simulating the stereoscopic vision principle of the human eye, thereby helping the unmanned aerial vehicle to more accurately identify the details and defects of the power equipment under complex weather conditions, and effectively improving the reliability and safety of the inspection even in low visibility or poor lighting conditions. In addition, the stereoscopic imaging function of the binocular camera can also help the unmanned aerial vehicle to better avoid obstacles and ensure flight stability in complex environments, which is crucial for ensuring the efficiency and accuracy of power inspection.

[0003] However, complex weather conditions can easily cause the binocular camera to shake during power inspection, affecting image quality and inspection effect. In the existing solution, in order to effectively reduce the problem of image blurring and inspection accuracy decline caused by shaking of the binocular camera in complex environments, advanced sensor fusion technology is adopted, which can more accurately perceive the motion state of the unmanned aerial vehicle and the surrounding environment by comprehensively processing the visual data of the binocular camera and the data of other sensors, for example, an inertial measurement unit can monitor the acceleration and angular velocity of the unmanned aerial vehicle in real time to provide motion compensation information for the images of the binocular camera, and a GPS can provide accurate position data to help the unmanned aerial vehicle maintain a stable flight trajectory. Through the fusion of multi-sensor data, the system can more effectively offset the shaking caused by wind, air flow and other factors.

[0004] However, due to the errors of each sensor itself and the error accumulation during data fusion, these reasons can cause the compensation effect of shaking to be not accurate enough, thereby affecting the image quality of the binocular camera and the accuracy of the inspection results. SUMMARY

[0005] The embodiments of the present application provide a power inspection method, device and equipment based on a binocular camera and a laser radar to solve the problem that in the existing sensor fusion technology, due to the errors of each sensor itself and the error accumulation during data fusion, these reasons can cause the compensation effect of shaking to be not accurate enough.

[0006] In a first aspect, the embodiments of the present application provide a power inspection method based on a binocular camera and a laser radar, comprising:

[0007] The power inspection task is used to determine at least one target power facility, and a plurality of reference points are selected on the target power facility, and a reference point set is obtained according to all the reference points on all the target power facilities;

[0008] According to the collection result of the laser radar on the inspection unmanned aerial vehicle within a preset time period and the position coordinates of each reference point in the reference point set, vibration information of the inspection unmanned aerial vehicle is determined.

[0009] Based on the initial real-time image collected by the binocular camera on the inspection unmanned aerial vehicle, the position coordinates of each reference point in the reference point set and the vibration information of the inspection unmanned aerial vehicle, the shaking information of the binocular camera is determined, the shaking information is fused with the initial real-time image, and a real-time image of the binocular camera is obtained.

[0010] The real-time image is image-processed, and an inspection result of at least one target power facility is obtained according to the image-processed real-time image.

[0011] In a second aspect, an embodiment of the present application provides a power inspection device based on a binocular camera and a laser radar, comprising:

[0012] The selection module is configured to determine at least one target power facility based on a power inspection task, and select a plurality of reference points on the target power facility, and obtain a reference point set according to all the reference points on all the target power facilities.

[0013] The determination module is configured to determine vibration information of the inspection unmanned aerial vehicle according to the collection result of the laser radar on the inspection unmanned aerial vehicle within a preset time period and the position coordinates of each reference point in the reference point set.

[0014] The fusion module is configured to determine the shaking information of the binocular camera based on the initial real-time image collected by the binocular camera on the inspection unmanned aerial vehicle, the position coordinates of each reference point in the reference point set and the vibration information of the inspection unmanned aerial vehicle, fuse the shaking information with the initial real-time image, and obtain a real-time image of the binocular camera.

[0015] The processing module is configured to image-process the real-time image, and obtain an inspection result of at least one target power facility according to the image-processed real-time image.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0017] In the embodiment of the present application, the power facilities are determined according to the power inspection task, a plurality of reference points are selected on each power facility, a reference point set is further obtained according to all the reference points, the vibration information of the inspection unmanned aerial vehicle is determined in combination with the collection result of the laser radar on the inspection unmanned aerial vehicle within a preset time period and the position coordinates of each reference point in the reference point set, the shaking information of the binocular camera is determined in combination with the initial real-time image collected by the binocular camera, the vibration information of the inspection unmanned aerial vehicle and the position coordinates of each reference point, the real-time image of the binocular camera is obtained by fusing the shaking information and the real-time image, and finally the real-time image is processed, and the inspection result of the target power facility is obtained according to the processed real-time image. The vibration information of the inspection unmanned aerial vehicle is obtained by combining the collection result of the laser radar within the preset time period, so that the accuracy of the vibration information can be ensured. The shaking information of the binocular camera is obtained by combining the vibration information, the initial real-time image and the position coordinates of the reference points, and the real-time image is obtained by fusing the shaking information and the initial real-time image, so that the consistency of the real-time image with the actual situation can be ensured, and the accuracy of the inspection result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is an implementation flowchart of the power inspection method based on the binocular camera and the laser radar provided by the embodiment of the present application;

[0019] Figure 2 is an implementation flowchart of step S130 of the power inspection method based on the binocular camera and the laser radar provided by the embodiment of the present application;

[0020] Figure 3 is an implementation flowchart of step S140 of the power inspection method based on the binocular camera and the laser radar provided by the embodiment of the present application;

[0021] Figure 4 is a structural schematic diagram of the power inspection device based on the binocular camera and the laser radar provided by the embodiment of the present application;

[0022] Figure 5 is a schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0024] Reference Figure 1 , which shows an implementation flowchart of the power inspection method based on the binocular camera and the laser radar provided by the embodiment of the present application, and is described in detail as follows:

[0025] In step S110, at least one target power facility is determined based on the power inspection task, a plurality of reference points are selected on the target power facility, and a reference point set is obtained according to all the reference points on all the target power facilities.

[0026] In some embodiments, the power inspection task is a specific task in power system maintenance work, which aims to check and monitor power facilities to ensure the safe and stable operation of the power system, to periodically troubleshoot a section of power transmission line, to detect the operating state of substation equipment, etc. The target power facility refers to a specific power facility selected for inspection in the power inspection task. These facilities are the specific objects of the inspection work. Towers in power transmission lines, transformers in substations, high-voltage cables, etc. can all be target power facilities. The reference point is a point selected on the target power facility that has a landmark and can be used as a reference. The position coordinates and other information of these points will be used for subsequent inspection data processing and analysis, for example, selecting multiple points at different positions on the transformer shell. The reference point set is a collection of all reference points on all target power facilities, forming a set containing all information of these reference points. For example, the position coordinates and other information of each reference point are integrated together to form a reference point set, that is, a reference point set.

[0027] In a possible implementation, the specific processing of step S110 is as follows: based on the power inspection task, at least one target power facility is determined, and multiple reference points are selected on each target power facility according to the surface features of each target power facility; all reference points on all target power facilities are combined to obtain a reference point set.

[0028] In some embodiments, the surface feature refers to a unique shape, structure or identification of the target power facility surface that can be identified. These features can be used to accurately locate the reference points, such as the cross arm end point of the power tower, the edge profile of the insulator, the bolt hole position of the transformer shell, etc. These parts with fixed shapes can all be used as surface features for reference point selection. Combining all reference points on all target power facilities means that the reference points scattered on various target power facilities are integrated into a whole. For example, the position coordinates and other information of the reference points selected on the first tower, the reference points on the second tower, and the reference points on the transformer are summarized together to form a set containing all reference point information, so as to facilitate subsequent unified data processing and analysis based on the set.

[0029] Step S120, according to the collection results of the laser radar on the inspection unmanned aerial vehicle within the preset time period and the position coordinates of each reference point in the reference point set, the vibration information of the inspection unmanned aerial vehicle is determined.

[0030] In some embodiments, the inspection unmanned aerial vehicle is an unmanned aerial vehicle used to perform a power inspection task, usually equipped with a laser radar, a binocular camera and the like, and can collect data of a target power facility in the air according to a preset flight route. The laser radar is a sensor that detects the position and distance of a target by emitting a laser beam, and can generate three-dimensional point cloud data of the environment. The laser radar installed on the inspection unmanned aerial vehicle can continuously scan the target power facility during flight to obtain its spatial position information. The laser radar emits a laser beam and receives a reflected signal, and can construct a three-dimensional point cloud model of the power tower, transformer and other facilities. The preset time period refers to a time interval for data collection set in advance before the inspection, which is used to limit the time range of data collection by the laser radar. It should be noted that the last moment of the preset time period is the current moment. The collection result refers to the data obtained by the laser radar after scanning the target power facility in the preset time period, which usually records the spatial position, distance and other information of the target in the form of point cloud. The position data of the points on the power facility scanned by the laser radar at each moment, as well as the distance between these points and the radar, all belong to the collection result. The position coordinates are numerical values used to represent the specific position of a reference point in a three-dimensional space, usually in the form of (x, y, z) coordinates. Each reference point in the reference point set has a unique position coordinate, such as the specific position of a reference point in the coordinate system with the inspection area as the origin, which can be used as a reference for judging the vibration of the unmanned aerial vehicle. The vibration information refers to the vibration parameters of the inspection unmanned aerial vehicle during flight due to factors such as air flow and equipment operation, including the amplitude, frequency and direction of vibration. By analyzing the changes in the collection result of the laser radar and the position coordinates of the reference points, the vibration of the unmanned aerial vehicle can be determined. For example, if the position of the reference point collected by the laser radar periodically deviates from its fixed coordinates, the amplitude and frequency of the vibration of the unmanned aerial vehicle can be calculated accordingly.

[0031] In a possible implementation, the specific processing manner of step S120 is as follows: determining, according to the collection result of the laser radar on the inspection unmanned aerial vehicle in the preset time period, a plurality of second reference points contained in the collection result at each moment in the preset time period; obtaining, according to the collection result in the preset time period, the position coordinates of each second reference point at each moment, and performing data fitting on the position coordinates of each second reference point at different moments to obtain the motion trajectory of each second reference point; matching each second reference point with each reference point in the reference point set to obtain a matching result, and determining the vibration information of the inspection unmanned aerial vehicle based on the matching result, the motion trajectory of each second reference point and the position coordinates of each reference point in the reference point set.

[0032] In some embodiments, the second reference point is a point on the target power facility identified by the laser radar in the acquisition result of each time within a preset time period, which is a real-time data point obtained by laser radar scanning and needs to be matched and associated with the known reference points in the reference point set. For example, the laser radar scans a certain angle steel end point on the power tower at a certain time, and this point is the second reference point. Data fitting is a technology of processing multiple discrete data points through mathematical methods to generate a continuous curve or function to describe the data variation law. In this application, it refers to mathematical modeling of the position coordinates of the second reference point at different times, such as using polynomial fitting or curve interpolation method, to obtain an expression that can reflect the motion law. The motion trajectory is the moving path of the second reference point in three-dimensional space obtained by data fitting, which is used to intuitively show the position change of the point with time. For example, the position of a certain second reference point at different times forms a fluctuating curve after fitting, which is the motion trajectory of the point, and can reflect the influence of unmanned aerial vehicle vibration on laser radar data acquisition. Matching each second reference point with each reference point in the reference point set means determining the corresponding relationship between the second reference point and the reference point in the reference point set, that is, judging which preset reference point on the target power facility the second reference point collected by the laser radar corresponds to, for example, through point cloud feature comparison, determining that a certain second reference point corresponds to the reference point marked as "top of tower A" in the reference point set. The matching result refers to the information of the corresponding relationship between the second reference point and the reference point set, such as the identification or position coordinates of the reference point matched by each second reference point. This result is used to analyze the position deviation of the second reference point in combination with the fixed position coordinates of the reference point, and then deduce the vibration of the unmanned aerial vehicle.

[0033] In step S130, based on the initial real-time image collected by the binocular camera on the inspection unmanned aerial vehicle, the position coordinates of each reference point in the reference point set, and the vibration information of the inspection unmanned aerial vehicle, the shaking information of the binocular camera is determined. The shaking information is fused with the initial real-time image to obtain the real-time image of the binocular camera.

[0034] In some embodiments, the binocular camera is an imaging device mounted on the inspection UAV, which can obtain three-dimensional environmental information by simulating the principle of human binocular vision, and can provide depth data and stereoscopic vision effect. For example, in power inspection, the binocular camera can take stereoscopic images of the power transmission tower to help identify details such as broken insulator and broken wire. The initial real-time image is the image data collected by the binocular camera during the inspection process without processing, which reflects the real-time visual information of the target power facility, but may be blurred or distorted due to UAV vibration or camera shaking, for example, when the UAV flies in strong wind, the initial image of the transformer taken by the binocular camera may have tower profile deviation or pixel misplacement. The shaking information is used to describe the dynamic parameters such as position deviation and angle rotation of the binocular camera during image acquisition, including translation, rotation angle, vibration amplitude, etc. For example, by analyzing the position change of the reference point when the camera is shooting, the shaking amplitude and rotation angle of the camera in the X, Y and Z axis directions can be calculated. The real-time image is the output image of the binocular camera after fusion and correction of the shaking information, which can truly reflect the actual state of the target power facility and provide accurate visual data for subsequent inspection analysis.

[0035] Referring to Figure 2 The specific processing method of step S130 includes steps S1301-S1306, and the specific contents are as follows:

[0036] In step S1301, based on the initial real-time image collected by the binocular camera on the inspection UAV, a plurality of first reference points contained in the initial real-time image are determined.

[0037] In some embodiments, the first reference point refers to a feature point on the target power facility identified from the initial real-time image collected by the binocular camera, which has a corresponding relationship with the preset reference point in the reference point set and can be used as a reference for image analysis. For example, in the initial real-time image, the endpoints of the cross arm of the power transmission tower and the edges of the insulator have obvious contours, and the points formed after image recognition algorithm extraction are the first reference points, which are used for subsequent matching with the reference point set and calculation of camera shaking parameters.

[0038] In step S1302, the matching relationship between the first reference point and each reference point in the reference point set is determined, and the first shaking information of the binocular camera is calculated according to the matching relationship and the position coordinates of each reference point.

[0039] In some embodiments, the matching relationship is corresponding mapping information of the first reference point and the reference points in the reference point set, such as the identification or spatial coordinate attribute of the reference point matched by each first reference point. For example, a certain first reference point is determined to correspond to a transformer shell bolt hole reference point numbered A01 in the corresponding reference point set through feature matching, and the corresponding relationship is the matching relationship. The first shaking information is the shaking parameter of the binocular camera obtained through image matching and coordinate calculation, mainly reflecting the position and angle changes of the camera in the imaging process due to the vibration of the unmanned aerial vehicle or its own movement, such as translation displacement, rotation Euler angle, etc. For example, according to the coordinate deviation of multiple matching reference points, it is calculated that the binocular camera has a certain distance of translation along the X axis and a certain angle of rotation around the Y axis when shooting, and these parameters jointly constitute the first shaking information.

[0040] In step S1303, the second shaking information of the binocular camera is determined based on the vibration information of the inspection unmanned aerial vehicle and the relative position relationship between the binocular camera and the inspection unmanned aerial vehicle.

[0041] In some embodiments, the relative position relationship refers to the spatial correlation constituted by the installation position and orientation of the binocular camera on the inspection unmanned aerial vehicle, such as the horizontal distance and vertical height from the camera installation position to the center of gravity of the unmanned aerial vehicle, and the pitch and yaw angles of the lens, etc. This position relationship is a key parameter for transmitting the vibration of the unmanned aerial vehicle to the shaking calculation of the camera, for example, when the unmanned aerial vehicle vibrates along the horizontal axis of the fuselage, the camera installed at the end of the arm will have a larger shaking amplitude than the camera installed in the middle of the fuselage due to the farther distance from the center of gravity. The second shaking information is the shaking parameter of the binocular camera calculated by the vibration information of the unmanned aerial vehicle and the relative position relationship of the camera, for example, according to the vibration frequency and amplitude of the unmanned aerial vehicle in a certain period of time, combined with the distance and angle from the camera installation point to the center of gravity of the unmanned aerial vehicle, the translation amount and rotation angle of the camera in three-dimensional space are calculated. This part of information and the first shaking information obtained based on image matching form a complement, for example, when the unmanned aerial vehicle pitches due to air flow, the second shaking information can directly derive the pitch angle change of the camera through the relative position relationship, providing more comprehensive dynamic parameters for image correction.

[0042] In step S1304, the shaking information of the binocular camera is determined according to the first shaking information and the second shaking information.

[0043] In some embodiments, the shaking information of the binocular camera refers to the final shaking parameter obtained by fusing the first shaking information and the second shaking information, which is used to accurately describe the position offset, angle rotation and other dynamic changes of the binocular camera when collecting images. If the first shaking information shows that the camera has a translation in the X axis direction, and the second shaking information shows that the camera has a rotation in the Y axis direction, then the final shaking information will integrate these two parameters to form a complete correction basis.

[0044] In a possible implementation, the specific processing manner of step S1304 is: comparing the first shake information and the second shake information to obtain a difference value of each shake parameter in the shake information; determining at least one target shake parameter according to the difference value of each shake parameter, and adjusting the target shake parameter to obtain a parameter value of the target shake parameter; and determining the shake information of the binocular camera based on the first value of each shake parameter in the first shake information, the second value of each shake parameter in the second shake information, and the parameter value of the target shake parameter.

[0045] In some embodiments, comparing the first shake information and the second shake information means comparing corresponding shake parameters in the first shake information and the second shake information one by one, and analyzing the difference between the two, for example, comparing the translation amount of the X axis in the first shake information with the translation amount of the X axis in the second shake information, and determining the deviation of the two in this dimension. The difference value of the shake parameter is the numerical difference of each shake parameter in the first and second shake information obtained by comparison. If a rotation angle parameter in the first shake information is A and the corresponding parameter in the second shake information is B, the difference value is |A-B|. The difference value is used to measure the inconsistency of the two kinds of information in this parameter. The target shake parameter is a shake parameter that needs to be adjusted according to the difference value. For example, when the difference value of a certain translation parameter exceeds a preset threshold, it is determined as the target shake parameter, and it is preferentially corrected to improve the overall accuracy.

[0046] In some embodiments, adjusting the target shake parameter means the process of correcting the numerical value of the target shake parameter. The target shake parameter can be adjusted by algorithms such as weighted fusion and error compensation. For example, the target parameters in the first shake information and the second shake information are weighted and calculated according to the difference value proportion to obtain a parameter value closer to the actual situation. The parameter value of the target shake parameter is the specific numerical value of the target shake parameter after adjustment, which is used for subsequent comprehensive calculation. For example, after adjustment, a certain rotation angle parameter is integrated from two different original values to a new parameter value C, which is used as the effective data for final calculation. The shake information is obtained by the above comparison, adjustment and comprehensive calculation, and is a complete parameter set that accurately describes the shake state of the binocular camera. That is, all adjusted parameters and unadjusted parameters are integrated to form the final shake information containing translation, rotation and other dimensions, which is used for image correction.

[0047] In step S1305, based on the shake information, the distortion information and the displacement information of each pixel point in the initial real-time image are determined.

[0048] In some embodiments, each pixel point refers to the smallest unit constituting the image in the initial real-time image, each point has independent position coordinates and color information. For example, in the initial image of the transmission tower, a certain pixel point at the top of the tower corresponds to a specific position in the actual scene, and the coordinates will deviate from the true position due to camera shaking. Distortion information is a parameter that describes the geometric deformation of pixel points in the initial real-time image due to camera shaking, such as stretching, twisting or local shape change of pixel points. Camera shaking may cause the originally straight edge pixels of the tower in the initial image to appear curved, and the distortion information is used to describe the degree and direction of the shape deviation. The displacement information is used to represent the offset of each pixel point in the initial real-time image relative to its ideal position, including the moving distance and direction in the horizontal and vertical directions. If the camera shifts to the right due to shaking, the displacement information of all pixel points in the image will reflect the overall rightward position deviation, and the pixel point at the end of a certain tower cross arm may be displaced from the left side of the image to the middle area.

[0049] In step S1306, each pixel point in the initial real-time image is adjusted according to the distortion information and the displacement information to obtain the real-time image of the binocular camera.

[0050] In some embodiments, adjusting each pixel point in the initial real-time image refers to correcting the position, shape and other attributes of each pixel point in the initial real-time image through an algorithm to eliminate the distortion caused by camera shaking. For example, if a certain pixel point has a rightward offset and shape distortion due to camera shaking, the adjustment process will move the point back to the correct position to the left and correct its geometric shape. It should be noted that all pixels in the initial real-time image need to be processed one by one to ensure that every detail of the image is corrected. For example, in the initial image of the transmission tower, each pixel of the tower body, insulator or background needs to be adjusted individually according to its corresponding distortion and displacement information. The real-time image refers to the binocular camera output image obtained after adjustment of the distortion and displacement information. This image can truly reflect the actual state of the target power facility and can be directly used for subsequent defect identification and inspection analysis. For example, in the adjusted real-time image, the cracks of the insulator and the wear of the conductor can be clearly presented to provide accurate visual data for the inspection system.

[0051] By combining the initial real-time image collected by the binocular camera, the reference point position coordinates and the unmanned aerial vehicle vibration information, the camera shaking information can be accurately determined and fused with the initial image to effectively eliminate the interference of camera shaking caused by the vibration of the inspection unmanned aerial vehicle or the influence of air flow on the image quality, so that the corrected real-time image can truly restore the actual state of the target power facility and provide high-precision visual data for subsequent image feature extraction and defect identification, thereby improving the accuracy and reliability of the power inspection results and avoiding the problems of misjudgment or omission caused by image distortion.

[0052] Step S140, image processing is performed on the real-time image, and an inspection result of at least one target power facility is obtained according to the real-time image after image processing.

[0053] In some embodiments, image processing refers to a series of operations of optimizing and analyzing the real-time image by using algorithms, such as eliminating interference points in the image by a noise reduction algorithm, highlighting the contour of the power facility by edge detection, or separating the target power facility from the background by image segmentation. By analyzing and judging the result after image processing, the final conclusion about the state of the target power facility is obtained. For example, after image enhancement and feature extraction are completed, the system generates a judgment result of the facility condition based on the processed image data, that is, an inspection result, which includes an operation state evaluation or a fault detection conclusion of the target power facility, such as judging whether a power transmission tower has loose bolts, whether insulators have cracks, whether conductors have broken strands, etc. If the system identifies that there is an oil leakage trace on the transformer shell after image processing, this conclusion is one of the inspection results of the target power facility.

[0054] Referring to Figure 3 , the specific processing mode of step S140 includes steps S1401-S1402, and the specific content is shown as follows:

[0055] Step S1401, image processing is performed on the real-time image, and feature extraction is performed on the real-time image after image processing to obtain a feature information set of the real-time image.

[0056] In some embodiments, feature extraction is to identify and extract representative key information such as shape, texture, color, edge, etc. from the real-time image after image processing, for example, in the processed insulator image, the contour shape of the insulator is extracted by an edge detection algorithm, or the feature pattern of the surface crack is extracted by texture analysis, which can be used as a basis for judging whether the facility has defects. The feature information set is a collection formed by integrating all extracted image features, which contains a variety of feature parameters describing the appearance state of the target power facility, for example, for the real-time image of a section of power transmission line, the feature information set may include the sag shape feature of the conductor, the edge feature of the tower bolt, and the texture feature of the insulator, etc. These information collectively constitute the data basis for subsequent inspection result judgment.

[0057] Step S1402, determining an inspection result of at least one target power facility based on the feature information set.

[0058] In some embodiments, the feature information set is a collection of all key features extracted from the real-time image, containing various parameters describing the appearance, structure, and other attributes of the target power facility, such as contour shape, texture pattern, color distribution, etc. For example, the feature information set for a power transmission tower can include the geometric shape features of the cross arm, the surface texture features of the insulator, the sag curve features of the conductor, etc., which together constitute the data basis for determining whether the facility has defects. It should be noted that the number of target power facilities involved in the inspection result can be one or more, i.e., each analysis can generate a corresponding inspection conclusion for a single facility or multiple facilities.

[0059] In one possible implementation, the specific processing of step S1402 is as follows: extracting a plurality of key feature information from the feature information set, determining at least one target power facility corresponding to the feature information set according to the key feature information; comparing each feature information in the feature information set with the facility information of each target power facility corresponding to the feature information set to obtain the inspection result of the target power facility.

[0060] In some embodiments, the key feature information refers to image features that play a decisive role in identifying power facility defects, such as shape, texture, color mutation, etc. For example, in power inspection for a transformer, the key feature information can include the arrangement shape features of the shell bolts, the texture features of the cooling fins, the color boundary features of the oil level gauge, etc., according to which the target power facility corresponding to the feature information set can be determined. There is a matching association between the feature information set and the target power facility, i.e., a specific feature information set corresponds to one or more specific power facilities. For example, the tower number identification and insulator model features contained in a certain feature information set can explicitly indicate that they correspond to the No. 1 main transformer and the associated equipment in the substation. The facility information is the standard attribute, design parameter, or feature data of the target power facility in the normal operating state, such as the structural size of the tower, the standard texture pattern of the insulator, the color parameter of the transformer shell, etc. The inspection result is a conclusion about the operating state of the target power facility generated by comparing the feature information with the facility information, including whether there is a defect, the type and severity of the defect, etc. For example, after comparing the feature information of the transformer shell with the standard color in the facility information, and combining the abnormal texture features, the inspection result "there is an oil leakage trace on the transformer shell, which needs to be further detected" is obtained.

[0061] The target power facilities are determined through the power inspection task, reference points on each target power facility are selected to form a reference point set, vibration information of the inspection unmanned aerial vehicle is obtained through data fitting according to the reference point set and the collection result of the laser radar on the inspection unmanned aerial vehicle within a preset time period, the accuracy of the vibration information can be improved, the shaking information of the binocular camera is further determined by using the vibration information and the initial real-time image collected by the binocular camera, the shaking information is fused with the initial real-time image, the real-time image of the binocular camera is obtained, finally, the real-time image is processed, and the inspection result of the target power facility is obtained according to the processed real-time image, the consistency between the real-time image and the actual situation can be ensured, and the accuracy of the inspection result is further improved.

[0062] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0063] The following is a device embodiment of the application, and for details not described in detail, reference can be made to the corresponding method embodiments described above.

[0064] Figure 4 The structure schematic diagram of the power inspection device based on the binocular camera and the laser radar provided by the embodiment of the application is shown, only the parts related to the embodiment of the application are shown for the convenience of description, and the details are as follows:

[0065] As shown in Figure 4 The power inspection device based on the binocular camera and the laser radar 4 comprises:

[0066] The selection module 41 is configured to determine at least one target power facility based on the power inspection task, select a plurality of reference points on the target power facility, and obtain a reference point set according to all reference points on all target power facilities.

[0067] The determination module 42 is configured to determine vibration information of the inspection unmanned aerial vehicle according to the collection result of the laser radar on the inspection unmanned aerial vehicle within a preset time period and the position coordinates of each reference point in the reference point set.

[0068] The fusion module 43 is configured to determine shaking information of the binocular camera based on the initial real-time image collected by the binocular camera on the inspection unmanned aerial vehicle, the position coordinates of each reference point in the reference point set and the vibration information of the inspection unmanned aerial vehicle, fuse the shaking information with the initial real-time image, and obtain a real-time image of the binocular camera.

[0069] The processing module 44 is configured to process the real-time image, and obtain an inspection result of at least one target power facility according to the real-time image after image processing.

[0070] In a possible implementation, the selecting module 41 specifically includes: determining at least one target power facility based on the power inspection task, and selecting a plurality of reference points on each target power facility according to the surface features of each target power facility; and combining all the reference points on all the target power facilities to obtain a reference point set.

[0071] In a possible implementation, the determining module 42 specifically includes: determining a plurality of second reference points contained in the collection result of each time in the preset time period according to the collection result of the laser radar on the inspection unmanned aerial vehicle in the preset time period; obtaining the position coordinates of each second reference point at different times, and performing data fitting on the position coordinates of each second reference point at different times to obtain the motion trajectory of each second reference point according to the collection result in the preset time period; matching each second reference point with each reference point in the reference point set to obtain a matching result, and determining the vibration information of the inspection unmanned aerial vehicle based on the matching result, the motion trajectory of each second reference point, and the position coordinates of each reference point in the reference point set.

[0072] In a possible implementation, the fusion module 43 specifically includes: determining a plurality of first reference points contained in the initial real-time image collected by the binocular camera on the inspection unmanned aerial vehicle based on the initial real-time image; determining the matching relationship between the first reference points and each reference point in the reference point set, and calculating the first shaking information of the binocular camera according to the matching relationship and the position coordinates of each reference point; determining the second shaking information of the binocular camera based on the vibration information of the inspection unmanned aerial vehicle and the relative position relationship between the binocular camera and the inspection unmanned aerial vehicle; and determining the shaking information of the binocular camera according to the first shaking information and the second shaking information.

[0073] In a possible implementation, the fusion module 43 further includes: comparing the first shaking information and the second shaking information to obtain the difference value of each shaking parameter in the shaking information; determining at least one target shaking parameter according to the difference value of each shaking parameter, and adjusting the target shaking parameter to obtain the parameter value of the target shaking parameter; and determining the shaking information of the binocular camera based on the first value of each shaking parameter in the first shaking information, the second value of each shaking parameter in the second shaking information, and the parameter value of the target shaking parameter.

[0074] In a possible implementation, the fusion module 43 further includes: determining the distortion information and the displacement information of each pixel point in the initial real-time image based on the shaking information; and adjusting each pixel point in the initial real-time image according to the distortion information and the displacement information to obtain the real-time image of the binocular camera.

[0075] In a possible implementation, the processing module 44 specifically includes: performing image processing on the real-time image, and performing feature extraction on the real-time image after the image processing to obtain a feature information set of the real-time image; and determining the inspection result of the at least one target power facility based on the feature information set.

[0076] In a possible implementation, the processing module 44 further includes: extracting a plurality of key feature information from the feature information set, and determining the at least one target power facility corresponding to the feature information set according to the key feature information; and comparing each feature information in the feature information set with facility information of each target power facility corresponding to the feature information set to obtain the inspection result of the target power facility.

[0077] Figure 5 is a schematic diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 5 the electronic device 5 of this embodiment includes a processor 50 and a memory 51. The memory 51 stores a computer program 52. The processor 50 implements the steps in each of the above method embodiments when executing the computer program 52. Alternatively, the processor 50 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 52.

[0078] For example, the computer program 52 can be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 52 in the electronic device 5.

[0079] The electronic device 5 can include, but is not limited to, the processor 50 and the memory 51. Those skilled in the art can understand that Figure 5 The electronic device 5 is only an example and does not constitute a limitation on the electronic device 5, and can include more or fewer components than those shown, or combine certain components, or different components, for example, the electronic device 5 can also include an input / output device, a network access device, a bus, etc.

[0080] For the convenience and brevity of description, only the above-mentioned division of functional modules / units is exemplified, and in actual application, the above-mentioned functions can be completed by different functional modules / units according to needs. The above-mentioned modules / units can be realized in the form of hardware, software, or a combination of hardware and software.

[0081] In the above embodiments, the description of each embodiment is focused on, and the part not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments. If there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be mutually referred to, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0082] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A power line inspection method based on binocular cameras and lidar, characterized in that, include: Based on the power inspection task, at least one target power facility is identified, and multiple reference points are selected on the target power facility. A reference point set is obtained based on all reference points on all target power facilities. Based on the data collected by the lidar on the inspection drone within a preset time period and the position coordinates of each reference point in the reference point set, the vibration information of the inspection drone is determined. Based on the initial real-time image collected by the binocular camera on the inspection drone, the position coordinates of each reference point in the reference point set, and the vibration information of the inspection drone, the shaking information of the binocular camera is determined, and the shaking information is fused with the initial real-time image to obtain the real-time image of the binocular camera. The real-time image is processed, and the inspection results of at least one target power facility are obtained based on the processed real-time image.

2. The power line inspection method based on binocular camera and lidar according to claim 1, characterized in that, The determination of the binocular camera's sway information based on the initial real-time images acquired by the binocular camera on the inspection drone, the position coordinates of each reference point in the reference point set, and the vibration information of the inspection drone includes: Based on the initial real-time images captured by the binocular camera on the inspection drone, a number of first reference points contained in the initial real-time images are determined. Determine the matching relationship between the first reference point and each reference point in the reference point set, and calculate the first shake information of the binocular camera based on the matching relationship and the position coordinates of each reference point; Based on the vibration information of the inspection drone and the relative positional relationship between the binocular camera and the inspection drone, the second shaking information of the binocular camera is determined; The shake information of the binocular camera is determined based on the first shake information and the second shake information.

3. The power line inspection method based on binocular camera and lidar according to claim 2, characterized in that, Determining the shake information of the binocular camera based on the first shake information and the second shake information includes: The first sway information is compared with the second sway information to obtain the difference of each sway parameter in the sway information; Based on the difference of each swaying parameter, at least one target swaying parameter is determined, and the target swaying parameter is adjusted to obtain the parameter value of the target swaying parameter. The shake information of the binocular camera is determined based on the first value of each shake parameter in the first shake information, the second value of each shake parameter in the second shake information, and the parameter value of the target shake parameter.

4. The power line inspection method based on binocular camera and lidar according to claim 1, characterized in that, The step of fusing the shaking information with the initial real-time image to obtain the real-time image from the binocular camera includes: Based on the shaking information, the distortion information and displacement information of each pixel in the initial real-time image are determined; Based on the distortion and displacement information, each pixel in the initial real-time image is adjusted to obtain the real-time image from the binocular camera.

5. The power line inspection method based on binocular camera and lidar according to claim 1, characterized in that, The step of determining the vibration information of the inspection drone based on the data collected by the lidar on the inspection drone within a preset time period and the position coordinates of each reference point in the reference point set includes: Based on the collection results of the lidar on the inspection drone within a preset time period, determine the multiple second reference points contained in each collection result at each moment within the preset time period; Based on the collection results within a preset time period, the position coordinates of each second reference point at each time are obtained, and the position coordinates of each second reference point at different times are fitted to obtain the motion trajectory of each second reference point. Each second reference point is matched with each reference point in the reference point set to obtain a matching result. Based on the matching result, the motion trajectory of each second reference point, and the position coordinates of each reference point in the reference point set, the vibration information of the inspection drone is determined.

6. The power line inspection method based on binocular camera and lidar according to claim 1, characterized in that, The process, based on power line inspection tasks, involves identifying at least one target power facility, selecting multiple reference points on that target power facility, and obtaining a reference point set based on all reference points on all target power facilities, including: Based on the power inspection task, at least one target power facility is identified, and multiple reference points are selected on each target power facility according to the surface characteristics of each target power facility. The reference point set is obtained by combining all reference points on all target power facilities.

7. The power line inspection method based on binocular camera and lidar according to claim 1, characterized in that, The step of processing the real-time image and obtaining the inspection results of at least one target power facility based on the processed real-time image includes: The real-time image is processed, and features are extracted from the processed real-time image to obtain a feature information set of the real-time image. Based on the feature information set, the inspection results of at least one target power facility are determined.

8. The power line inspection method based on binocular camera and lidar according to claim 7, characterized in that, The determination of the inspection results of at least one target power facility based on the feature information set includes: Multiple key feature information is extracted from the feature information set, and at least one target power facility corresponding to the feature information set is determined based on the key feature information. Each feature in the feature information set is compared with the facility information of each target power facility corresponding to the feature information set to obtain the inspection result of the target power facility.

9. A power line inspection device based on a binocular camera and lidar, characterized in that, include: The selection module is used to determine at least one target power facility based on the power inspection task, select multiple reference points on the target power facility, and obtain a reference point set based on all reference points on all target power facilities. The determination module is used to determine the vibration information of the inspection drone based on the collection results of the lidar on the inspection drone within a preset time period and the position coordinates of each reference point in the reference point set. The fusion module is used to determine the shaking information of the binocular camera based on the initial real-time image collected by the binocular camera on the inspection drone, the position coordinates of each reference point in the reference point set, and the vibration information of the inspection drone, and to fuse the shaking information with the initial real-time image to obtain the real-time image of the binocular camera. The processing module is used to perform image processing on the real-time image and obtain the inspection results of at least one target power facility based on the processed real-time image.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 8.

Images