A method and device for identifying blind areas of power equipment inspection

Abstract

The application discloses a kind of identification method and device of power equipment inspection blind area, identification method specifically includes the following steps: obtaining the point cloud data of power equipment, forms the point cloud grid model of single power equipment, and establishes global coordinate system;Give the inspection visual coordinate system of carrying out power equipment inspection, and establish the mapping relationship between inspection visual coordinate system and global coordinate system;Through global coordinate system, based on the interactive analysis of the intersection of inspection projection light and point cloud grid model, give the inspection blind area of power equipment.The application realizes the identification of automatic, efficient, accurate and comprehensive power equipment inspection blind area by fusing three-dimensional point cloud, visual inspection and coordinate transformation, and then carries out quantitative evaluation for power equipment inspection blind area, to optimize the inspection route, improve the inspection efficiency, inspection point coverage, reduce the operation risk of power equipment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power inspection, and particularly relates to a method and device for identifying blind areas of power equipment inspection. BACKGROUND

[0002] Power equipment, such as substations and transmission lines, is usually located in a vast and complex environment. Effective monitoring of these devices is essential to ensure the stable operation of the power system. Current monitoring methods mainly rely on fixed pan-tilt cameras, ground robots, and unmanned aerial vehicles to conduct regular inspection operations to monitor the operating status of the equipment.

[0003] However, due to the limitations of the physical installation location and field of view of the fixed pan-tilt camera, the height and inspection path of the ground robot, it is often impossible to cover all key components of the power equipment for inspection. These visual blind spots may result in missed monitoring of critical failures or damage, thereby affecting the safe operation of the power system.

[0004] At the same time, although unmanned aerial vehicles are flexible in location and have a wide range of views, they can provide multi-angle visual inspection and effectively supplement the monitoring gaps in high places and difficult-to-access areas, but still face challenges in actual operation. Especially in cases where accurate blind area location information cannot be provided, unmanned aerial vehicles need to use full-coverage point shooting planning, which not only reduces flight efficiency, but also makes it difficult to achieve three-dimensional collaborative inspection in the station, resulting in repeated shooting and inspection blind areas.

[0005] Existing identification and evaluation of power equipment inspection blind areas have evolved from early manual drawing and simulation to complex computer modeling and simulation, as well as advanced computer vision technology. A detailed three-dimensional model of the substation is constructed using computer modeling software, and visual blind areas are identified and optimized through line-of-sight analysis and patrol path simulation, significantly improving the accuracy of blind area management.

[0006] For example, patent CN112487916A discloses a power equipment binocular three-dimensional recognition system, which includes an image acquisition and fusion module, an identification and positioning model training module, and a target positioning correction module. The image acquisition and fusion module, the identification and positioning model training module, and the target positioning correction module are connected in sequence, and finally output the type of power equipment and the boundary box of the power equipment, completing the identification of the power equipment. This scheme has the advantages of high recognition accuracy, strong anti-shielding capability, and high utilization rate of training samples.

[0007] In addition, existing technologies also include the application of laser scanning and point cloud technology, especially the use of LiDAR equipment to quickly obtain three-dimensional data of substations, which accelerates the construction speed of substation models and provides higher spatial analysis capabilities for blind area identification and evaluation.

[0008] However, the above identification and evaluation methods generally have high requirements for computing resources, are costly, and are inefficient when dealing with large amounts of data. In addition, accuracy in complex environments still faces challenges, especially in insufficient light or severe weather conditions, and the error rate can increase. Furthermore, although the prior art has made progress in identifying blind areas for inspection, it still shows certain limitations in providing quantitative analysis of blind areas for inspection, which limits the scientificity and systematicness of blind area management for inspection.

[0009] Therefore, how to identify blind areas for inspection of power equipment to achieve automatic, efficient, accurate and comprehensive identification of blind areas for inspection, and to realize quantitative analysis of blind areas for inspection is a problem to be solved by those skilled in the art. SUMMARY

[0010] In view of the defects in the prior art, the present application provides a method and device for identifying blind areas for inspection of power equipment. The identification method specifically includes the following steps: obtaining point cloud data of the power equipment, forming a point cloud grid model of the single power equipment, and establishing a global coordinate system; giving an inspection visual coordinate system for power equipment inspection, and establishing a mapping relationship between the inspection visual coordinate system and the global coordinate system; and giving the blind area for inspection of the power equipment based on the interactive analysis of the inspection projection light and the point cloud grid model through the global coordinate system. The present application realizes automatic, efficient, accurate and comprehensive identification of blind areas for inspection of power equipment by fusing three-dimensional point cloud, visual inspection and coordinate transformation, and then quantitatively evaluates the blind areas for inspection of power equipment to optimize the inspection route, improve the inspection efficiency and inspection point coverage rate, and reduce the operation risk of power equipment.

[0011] In a first aspect, the present application provides a method for identifying blind areas for inspection of power equipment, specifically including the following steps:

[0012] Obtaining point cloud data of the power equipment, forming a point cloud grid model of the single power equipment, and establishing a global coordinate system;

[0013] Giving an inspection visual coordinate system for power equipment inspection, and establishing a mapping relationship between the inspection visual coordinate system and the global coordinate system;

[0014] Giving the blind area for inspection of the power equipment based on the interactive analysis of the inspection projection light and the point cloud grid model through the global coordinate system.

[0015] Further, the point cloud grid model of the single power equipment specifically includes the following steps:

[0016] Obtaining initial three-dimensional point cloud data of the power equipment;

[0017] Based on the Gaussian model, the initial three-dimensional point cloud data is preprocessed to obtain the point cloud data of the power equipment;

[0018] The point cloud data is classified by clustering processing, and the point cloud data of each single power equipment is constructed;

[0019] The point cloud data of each single power equipment is simplified and converted into voxel data;

[0020] An isosurface extraction algorithm is used to extract the surface of the point cloud grid model from the voxel data;

[0021] The point cloud grid model of the single power equipment is formed by combining the occupancy state of the voxel data and the surface of the point cloud grid model.

[0022] Further, a global coordinate system is established, specifically including:

[0023] The vertical direction of the point cloud grid model is taken as the Z axis, and the coordinate system of the point cloud grid model is represented as (x r , y r , z r ), and the coordinate system of the point cloud grid model is defined as the global coordinate system.

[0024] Further, a patrol visual coordinate system for power equipment inspection is given, and a mapping relationship between the patrol visual coordinate system and the global coordinate system is established, specifically including the following steps:

[0025] The patrol visual coordinate system of the power equipment inspection device is determined;

[0026] The translation vector and the rotation matrix are obtained, and the affine transformation matrix is constructed;

[0027] The mapping relationship between the patrol visual coordinate system and the global coordinate system is established through the affine transformation matrix.

[0028] Further, the translation vector and the rotation matrix are obtained, and the affine transformation matrix is constructed, specifically including the following steps:

[0029] The vector from the origin of the patrol visual coordinate system to the origin of the global coordinate system is obtained, and the translation vector is given;

[0030] The rotation matrix is given by analyzing the included angle between each coordinate axis of the patrol visual coordinate system and the corresponding coordinate axis of the global coordinate system;

[0031] The affine transformation matrix is constructed by combining the translation vector and the rotation matrix.

[0032] Further, the mapping relationship between the patrol visual coordinate system and the global coordinate system is established, specifically represented as:

[0033]

[0034] Wherein, (xa, ya, za) is the point coordinate in the inspection vision coordinate system, Q(x c ,y c ,z c ) is the point and point coordinate in the global coordinate system corresponding to the point coordinate in the inspection coordinate system, R is the rotation matrix, and t is the translation vector.

[0035] Further, through the global coordinate system, based on the interaction analysis of the inspection projection light and the point cloud grid model, the inspection blind area of the power equipment is given, specifically including the following steps:

[0036] Defining the inspection projection light, the corresponding target point cloud grid model is obtained;

[0037] Analyzing the intersection of the projection light and the target point cloud grid model;

[0038] Based on the shape position relationship of the intersection and the target point cloud grid model, the visual blind area is marked.

[0039] Further, the intersection of the projection light and the target point cloud grid model is analyzed, which is specifically expressed as:

[0040] P(x P ,y P ,z P )=(x0+td x ,y0+td y ,z0+td z )

[0041] Wherein, (x0,y0,z0) is the initial point coordinate of the projection light, t is the direction vector parameter, (d x ,d y ,d z ) is the component of the projection light on the normal vector of the target point cloud grid model, and P is the intersection.

[0042] Further, based on the shape position relationship of the intersection and the target point cloud grid model, the visual blind area is marked, specifically including the following steps:

[0043] Based on the direction of the inspection projection light, the planar figure of the target point cloud grid model is determined;

[0044] The coordinates of the center of gravity of the planar figure under the global coordinate system are given;

[0045] Combined with the connection vector analysis of the intersection and each vertex in the planar figure, a vector cross product group is given, which is specifically expressed as:

[0046] C i =(x i+1 -x i )·(y P -y i) · (x i+1 -y i ) · (x P -x i )

[0047] wherein, C i is the vector cross product of the corresponding i-th vertex in the vector cross product group, (x i , y i ) is the coordinate of the i-th vertex in the planar graph, (x i+1 , y i+1 ) is the coordinate of the i+1-th vertex in the planar graph, (x P , y P ) is the coordinate of the intersection point P;

[0048] Based on the sign of each cross product and the comparison of the Euclidean distance between the intersection point and the barycenter, the intersection point located in the planar graph of the target point cloud grid model is given, which is specifically represented as:

[0049]

[0050] wherein, n is the total number of vector cross products in the vector cross product group, d() is the Euclidean distance between points, (x G , y G ) is the coordinate of the barycenter G of the planar graph of the target point cloud grid model;

[0051] Based on the intersection point located in the planar graph of the target point cloud grid model, the intersection points of multiple inspection projection light rays and the grid model are marked for visible discrimination, and the visual blind area is given.

[0052] In the second aspect, the application further provides an identification device for a power equipment inspection blind area, which adopts the above-mentioned identification method for a power equipment inspection blind area, and specifically comprises:

[0053] A collection unit is configured to acquire point cloud data of the power equipment.

[0054] A construction unit is configured to form a point cloud grid model of a single power equipment, establish a global coordinate system, and give an inspection visual coordinate system for the power equipment inspection.

[0055] An analysis unit is configured to map the inspection visual coordinate system to the global coordinate system, and based on the interactive analysis of the inspection projection light rays and the point cloud grid model, give the inspection blind area of the power equipment through the global coordinate system.

[0056] The identification method and device for a power equipment inspection blind area provided by the application at least have the following beneficial effects:

[0057] (1) The application realizes automatic, efficient, accurate and comprehensive identification of power equipment inspection blind area by fusing three-dimensional point cloud, visual inspection and coordinate transformation, and then quantitatively evaluates the power equipment inspection blind area to optimize the inspection route, improve the inspection efficiency and inspection point coverage rate, and reduce the operation risk of power equipment.

[0058] (2) The application ensures accurate and seamless conversion from the inspection visual coordinate system to the global coordinate system through the affine transformation matrix composed of the translation vector and the rotation matrix, so that the vision of the inspection device can be effectively mapped into the point cloud grid model, so that the vision data of the inspection device can be used for further processing, blind area analysis and evaluation in the point cloud grid model. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 A flowchart of a power equipment inspection blind area identification method provided by the application is provided.

[0060] Figure 2 A point cloud grid model construction flowchart of a single power equipment in an embodiment provided by the application is provided.

[0061] Figure 3 A flowchart of establishing the mapping relationship between the inspection visual coordinate system and the global coordinate system in an embodiment provided by the application is provided.

[0062] Figure 4 A flowchart of constructing an affine transformation matrix in an embodiment provided by the application is provided.

[0063] Figure 5 A schematic diagram of affine transformation in an embodiment provided by the application is provided.

[0064] Figure 6 A flowchart of giving the inspection blind area of the power equipment in an embodiment provided by the application is provided.

[0065] Figure 7 A schematic diagram of the field of view angle of the inspection device in an embodiment provided by the application is provided.

[0066] Figure 8 A multi-inspection device point map in an embodiment provided by the application is provided.

[0067] Figure 9 An inspection blind area example diagram under the multi-inspection device in an embodiment provided by the application is provided.

[0068] Figure 10 A structural schematic diagram of a power equipment inspection blind area identification device provided by the application is provided. DETAILED DESCRIPTION

[0069] For better understanding of the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings of the specification and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor belong to the scope of protection of the present application.

[0070] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Multiple" generally includes at least two.

[0071] It should also be noted that the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the goods or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such goods or devices. Without more limitation, the element defined by the sentence "including a…" does not exclude the presence of other identical elements in the goods or devices including the element.

[0072] In order to further improve the efficiency and accuracy of the blind area identification and quantitative evaluation of power equipment inspection, computer vision and artificial intelligence technology are introduced. By installing cameras at key positions and using occlusion area detection algorithm, the blind area in the image is automatically identified and the inspection strategy is optimized.

[0073] Based on the fusion of three-dimensional point cloud, visual inspection and coordinate transformation, and through the blind area detection algorithm, the photographing coverage of various inspection mode devices such as fixed pan-tilt camera, ground robot and unmanned aerial vehicle platform is calculated, realizing automatic, efficient, accurate and comprehensive identification of power equipment inspection blind area.

[0074] As shown in Figure 1 The present application provides a method for identifying the blind area of power equipment inspection, which specifically comprises the following steps:

[0075] Obtain the point cloud data of the power equipment, form the point cloud grid model of the single power equipment, and establish a global coordinate system;

[0076] Give the inspection visual coordinate system for power equipment inspection, and establish the mapping relationship between the inspection visual coordinate system and the global coordinate system;

[0077] Through the global coordinate system, based on the interactive analysis of the inspection projection light and the point cloud grid model, the inspection blind area of the power equipment is given.

[0078] The identification of the power equipment inspection blind area given by the application covers key technical points such as comprehensive processing of three-dimensional point cloud data, attitude calibration of inspection equipment (such as a camera holder), coordinate transformation, blind area detection, multi-inspection platform coordination coverage, and quantitative evaluation of inspection blind areas, and finally realizes automatic, efficient, accurate, and comprehensive identification of the power equipment inspection blind area.

[0079] The point cloud data of the power equipment can be obtained by using various scanning technologies, such as a ground static laser scanner, a mobile scanning system, or a laser scanning technology carried by a drone, a depth camera, etc. The power equipment (such as high-voltage equipment in a substation, a camera, and other structures) is scanned in detail from multiple angles and positions to ensure that all key areas are covered. During the scanning process, the equipment emits laser and receives its reflected light, thereby calculating the distance and forming point cloud data. After the collection is completed, data synchronization, correction, and feature point matching are performed to ensure that the data from different scanning angles are accurately aligned. Through these steps, a high-quality, accurate three-dimensional point cloud data of the power equipment corresponding to the entity facility is finally formed.

[0080] After obtaining the point cloud data of the power equipment, noise removal processing is also included for the point cloud data. A noise removal algorithm based on a Gaussian mathematical model can be used, or related operations of point cloud processing software. The noise removal algorithm based on the Gaussian mathematical model mainly uses the statistical characteristics of Gaussian distribution (normal distribution) to identify and remove noise points caused by random errors. The specific steps include the following:

[0081] A neighborhood is defined for each point A in the point cloud data, where the neighborhood is a spherical neighborhood with a preset radius.

[0082] The mean vector and covariance matrix of all points in the neighborhood are given, which is specifically represented as:

[0083]

[0084] X i =(x i ,y i ,z i )

[0085]

[0086] Where μ is the mean vector, Σ is the covariance matrix, (μ x ,μ y ,μ z ) is the partial vector of the mean vector, m is the total number of points in the neighborhood, X i is the i-th point in the neighborhood, (x i ,y i ,z i ) is the coordinate of the i-th point in the neighborhood.

[0087] By analyzing the Mahalanobis distance of each point in the neighborhood from the mean vector, combined with the threshold for determining the closeness, the point cloud data is denoised.

[0088] The Mahalanobis distance of each point in the neighborhood from the mean vector is specifically represented as:

[0089]

[0090] Where D M (X i ) is the Mahalanobis distance of the i-th point in the field from the mean vector.

[0091] According to the value of the Mahalanobis distance D M (X i ), a threshold τ is set. If D M (X i ) is greater than τ, the point X i is considered as a noise point. Through the above method, the point cloud surface can be effectively smoothed, thereby reducing random noise in the data and improving the overall quality of the data.

[0092] Of course, in addition to the above denoising algorithm, ready-made denoising tools provided by point cloud processing software can also be used to remove abnormal points (outliers), such as SOR Filter or Noise Filter provided by CloudCompare. These tools are specifically designed to detect and remove noise and outliers in point cloud data, thereby improving data quality and analysis accuracy, providing cleaner and more accurate data sets for subsequent data processing and analysis. SOR Filter is based on statistical analysis method, which identifies possible noise or outliers by comparing the distance distribution of each point with its surrounding points, suitable for removing irregular points caused by random measurement errors. Noise Filter focuses on the local geometric properties of points, evaluating the geometric consistency of each point with its surrounding points, such as the smoothness of the surface or the spatial continuity, suitable for processing errors or outliers caused by environmental interference or scanning device limitations. The combination of these tools can significantly improve the processing quality and usability of point cloud data.

[0093] For the individualization of power equipment, clustering algorithms (or related operations in point cloud processing software) and cutting classification of point cloud data can be combined to construct point cloud data and point cloud mesh models of individual power equipment.

[0094] As shown in Figure 2 , the point cloud mesh model of the individual power equipment is formed, specifically including the following steps:

[0095] Obtain the initial three-dimensional point cloud data of the power equipment;

[0096] Based on the Gaussian model, the initial three-dimensional point cloud data is preprocessed to obtain the point cloud data of the power equipment;

[0097] Through clustering processing, the point cloud data is classified to construct the point cloud data of each single power equipment;

[0098] The point cloud data of each single power equipment is simplified and converted into voxel data;

[0099] An isosurface extraction algorithm is used to extract the surface of the point cloud grid model from the voxel data;

[0100] Combining the occupancy state of the voxel data and the surface of the point cloud grid model, the point cloud grid model of the single power equipment is formed.

[0101] For clustering processing, specifically including:

[0102] Identify each core point in the three-dimensional point cloud data, wherein the core point is at least MinPts points in the neighborhood; thus, the core point can show a high enough density, so it can be considered as part of a cluster.

[0103] Define the boundary points and noise points for clustering with the core points, wherein the boundary points are less than MinPts points in the neighborhood but located in the neighborhood of at least one core point, and the noise points are less than MinPts points in the neighborhood and do not belong to any core point neighborhood.

[0104] Starting from any unclustered core point, find all density-reachable core points to form a new cluster. Density-reachable means that a core point can be reached from another core point through a series of core points in the neighborhood.

[0105] All density-reachable points from the core point are included in the same cluster, wherein the density-reachable points include core points and boundary points.

[0106] Repeat the above process until all point cloud data is processed.

[0107] Through the above clustering processing process, the point cloud can be effectively organized into independent clusters according to its density, and each cluster corresponds to a single device. After completing the clustering process, careful analysis and verification of the clustering results are needed to ensure that each cluster accurately represents a specific device. In addition, manual intervention may be needed to correct any classification errors caused by the clustering algorithm to ensure the accuracy and reliability of the data.

[0108] In addition to the above clustering, spatial cutting and classification of point cloud data can also be performed through the plane cutting, polygon cutting and voxelization cutting tools provided by point cloud processing software such as CloudCompare, effectively isolating and analyzing specific equipment or structural areas.

[0109] Specifically, plane cutting divides the point cloud by setting one or more planes, which is suitable for isolating simple structures. For objects with more complex or irregular shapes, the polygon cutting tool provides a custom cutting path that can tailor the point cloud as needed. These cutting methods greatly simplify the process of extracting detailed information of specific equipment or facilities from massive point cloud data, laying the foundation for subsequent fine processing of individualized equipment.

[0110] Of course, after the rough cutting and classification of point cloud data is completed through the above steps and the individualized equipment point cloud data is obtained, the individualized equipment point cloud data can be further denoised to improve the quality of the individualized equipment point cloud.

[0111] The point cloud data of each individual power equipment is simplified and converted into voxel data, which specifically includes:

[0112] The size of the voxel is determined, and each voxel is a cube in space, with the size determined by the length of the cube side;

[0113] Each point in the point cloud data of each individual power equipment is assigned to the corresponding voxel according to its coordinates, and in each voxel, all points are merged into a representative point, usually the geometric center of all points in the voxel.

[0114] An isosurface extraction algorithm is used to extract the surface of the point cloud mesh model from the voxel data, which is specifically represented as:

[0115] The size of the voxel mesh is determined, and the three-dimensional mesh size is set according to the distribution of the point cloud and the required target resolution, with each mesh element defined as a voxel; wherein the size of the voxel will directly affect the level of detail and computational efficiency of the model.

[0116] Each point in the point cloud is mapped to the nearest voxel by calculating the distance from the center of the voxel.

[0117] A threshold is set to determine whether a voxel is occupied based on the number of points in the voxel. If the number of points in the voxel is greater than the threshold, it indicates that the voxel is occupied, otherwise it is not occupied. For empty voxels, interpolation or morphological operations are used to fill them to ensure the continuity and integrity of the point cloud mesh model and avoid empty holes.

[0118] The surface of the point cloud mesh model is extracted from the voxelized data by using a Marching Cubes algorithm. The algorithm connects the boundaries of voxels according to the voxel occupancy state to form a continuous surface, thereby obtaining the point cloud mesh model of the single-atomization device.

[0119] A global coordinate system is established, specifically including:

[0120] The vertical direction of the point cloud mesh model is taken as the Z axis, and the coordinate system of the point cloud mesh model is represented as (x r , y r , z r ). The coordinate system of the point cloud mesh model is defined as the global coordinate system.

[0121] In an embodiment, the coordinate system of the three-dimensional point cloud model in the substation scene is defined as the global coordinate system. In this global coordinate system, the spatial relationship of point cloud data, camera position, gimbal setting and other sensors is accurately calibrated and matched. Using a unified coordinate system not only promotes efficient integration and processing of data, but also supports accurate object positioning, spatial relationship measurement between devices and complex scene analysis, ensuring consistency and accuracy of data processing. The coordinate system of the three-dimensional point cloud model has high flexibility and can be customized according to the configuration of the point cloud acquisition device. For example, when using a ground scanner to collect point cloud data, the Z axis of the point cloud model can be set to be perpendicular to the ground, i.e. vertically upward, and the X and Y axes can be adjusted according to the layout of the scanning environment to best capture and describe the spatial characteristics of the scanned object. The origin of the coordinate system of the three-dimensional point cloud model can be represented as (x r , y r , z r ).

[0122] As shown in Figure 3 , a visual coordinate system for power equipment inspection is given, and a mapping relationship between the visual coordinate system and the global coordinate system is established, specifically including the following steps:

[0123] Determine the visual coordinate system of the power equipment inspection device;

[0124] Obtain a translation vector and a rotation matrix to construct an affine transformation matrix;

[0125] Establish the mapping relationship between the visual coordinate system and the global coordinate system through the affine transformation matrix.

[0126] The power equipment inspection device covers fixed gimbal camera, ground mobile robot and unmanned aerial vehicle, each of which has its unique coordinate system, the accurate definition and mutual relationship of which are crucial for coordinate transformation. Specifically, the camera coordinate system usually takes the camera optical center as the origin, the camera line of sight direction as the Y axis, the Z axis perpendicular to the Y axis upward, and the X axis perpendicular to the YZ plane, forming a standard right-handed coordinate system. For unmanned aerial vehicles and ground robots, usually equipped with advanced navigation systems, the coordinate system takes the unmanned aerial vehicle itself as the origin, and the direction is set according to the specific configuration of the device. For example, the unmanned aerial vehicle can define the current shooting point as the coordinate origin, and the heading angle as the Y axis, the Z axis perpendicular upward, and the X axis perpendicular to the YZ plane.

[0127] The camera's intrinsic parameters such as focal length f can be calibrated by two main methods. First, the traditional checkerboard calibration method, which is a widely used technique, captures checkerboard patterns placed at different angles to calculate the camera's intrinsic parameters. Second, the camera manufacturer's software development kit (SDK) can be used to directly obtain the focal length and other intrinsic parameter settings from the software interface. The checkerboard calibration can provide high-precision results, while using the SDK is more convenient and faster.

[0128] The camera's extrinsic parameters include its installation position in the world coordinate system and the gimbal attitude (pitch angle and horizontal angle of the camera gimbal). Among them, the installation position of the camera can be determined by matching with the known three-dimensional point cloud model of the substation to determine the accurate position of the camera. The gimbal attitude calibration includes the accurate determination of the pitch angle, yaw angle and roll angle. If the gimbal supports remote control, the camera manufacturer's software development kit (SDK) can be used to directly obtain these extrinsic parameter settings from the software interface. In addition, when the camera adjusts its gimbal attitude according to the preset point, these angle values should be re-acquired to ensure that they accurately reflect the actual attitude of the camera.

[0129] According to the above calibration method, the installation position, coordinate system, intrinsic parameters (focal length) and extrinsic parameters (gimbal attitude) of the camera can be accurately obtained. Among them, the camera coordinate system origin (installation position) is represented as (x c ,y c ,z c ). The camera coordinate system takes the camera optical center as the origin, the camera line of sight direction as the Y axis, the Z axis perpendicular to the Y axis upward, and the X axis perpendicular to the YZ plane, forming a standard right-handed coordinate system. The camera intrinsic parameters such as focal length are represented as f. The three-dimensional attitude of the gimbal can be represented as Euler angles (Pitch, Yaw, Roll), where Pitch is the pitch angle, Yaw is the yaw angle, and Roll is the roll angle.

[0130] After determining the installation position of the fixed PTZ camera, the inspection route of the robot and the UAV, and the PTZ attitude of all cameras, the camera coordinate system and attitude are accurately mapped into the point cloud model coordinate system through coordinate transformation next. This conversion involves an affine transformation matrix composed of a translation transformation matrix and a rotation transformation matrix to ensure accurate and seamless conversion of the camera coordinate system to the point cloud coordinate system.

[0131] As shown in Figure 4 , the translation vector and the rotation matrix are obtained, and the affine transformation matrix is constructed, which specifically includes the following steps:

[0132] The vector from the origin of the inspection visual coordinate system to the origin of the global coordinate system is obtained, and the translation vector is given, which is specifically represented as:

[0133]

[0134] Where t x , t y , t z are the coordinates of the origin of the inspection visual coordinate system in the global coordinate system.

[0135] The rotation matrix is given by analyzing the included angle between each coordinate axis of the inspection visual coordinate system and the corresponding coordinate axis of the global coordinate system; wherein the origin of the inspection visual coordinate system has been aligned with the origin of the global coordinate system through the translation vector t, in order to further synchronize the directions of the two coordinate systems, a rotation matrix R needs to be constructed, which is determined based on the included angle relationship between the axes of the camera coordinate system and the global coordinate system.

[0136] In a certain scene, as shown in Figure 5 , the inspection visual coordinate system is rotated around the Z axis of the global coordinate system until the directions of the two coordinate systems are completely aligned, then the rotation matrix R can be represented as:

[0137]

[0138] Where θ is the rotation angle around the Z axis of the inspection visual coordinate system.

[0139] If the rotation involves multiple axes (for example, first rotate θ around the Z axis, then rotate φ around the X axis), the rotation matrix R can be combined by matrix multiplication of the rotation of a single axis:

[0140] R=R x (φ)·R z (θ)

[0141] Where the rotation matrix R x (φ) around the X axis is:

[0142]

[0143] By combining the translation vector and the rotation matrix, the affine transformation matrix is ​​constructed.

[0144] By combining the rotation matrix R with the translation vector t, we ensure that the transformation from the inspection vision coordinate system to the global coordinate system involves not only rotational alignment but also positional alignment. The formula for the affine transformation matrix T is as follows:

[0145]

[0146] The transformation matrix T ensures a precise mapping from the inspection vision coordinate system to the global coordinate system, achieving complete spatial alignment. By applying the transformation matrix T, any point and vector in the inspection vision coordinate system can be accurately transformed into the global coordinate system.

[0147] Establish the mapping relationship between the inspection vision coordinate system and the global coordinate system, specifically represented as follows:

[0148]

[0149] Where (xa, ya, za) are the coordinates of a point in the inspection vision coordinate system, Q(x c ,y c ,z c ) represents the point and its coordinates in the global coordinate system corresponding to the point coordinates in the inspection coordinate system, R is the rotation matrix, and t is the translation vector.

[0150] By combining the field of view and light projection technology of the inspection device with other blind spot detection algorithms, the visual coverage of each camera on a single device under various pan-tilt postures is accurately calculated, including the precise marking of the visible and invisible parts (i.e., blind spots) in the point cloud model of each device.

[0151] like Figure 6 As shown, based on the interaction analysis of the inspection projection rays and the point cloud mesh model using a global coordinate system, the blind spots of power equipment are identified, specifically including the following steps:

[0152] Define the inspection projection ray and obtain the corresponding target point cloud mesh model;

[0153] Analyze the intersection points of the projected rays and the target point cloud mesh model;

[0154] Based on the shape and positional relationship between the intersection point and the target point cloud mesh model, visual blind spots are marked.

[0155] like Figure 7 As shown, the inspection projection ray is defined as multiple rays emitted from the camera position to cover the entire field of view. This is usually achieved by uniformly generating ray vectors within the camera's field of view, specifically represented as:

[0156]

[0157] where FOV is the field of view of the camera, usually in degrees. d is the size (width or height) of the camera sensor, usually in millimeters (mm). f is the focal length of the camera, also usually in millimeters (mm).

[0158] The number and distribution density of these light vectors can be adjusted according to specific needs to achieve the best balance between computational efficiency and coverage accuracy. The light can be represented as:

[0159] R(t) = O + tD

[0160] where O is the origin of the light (camera position), D is the direction vector of the light (unit vector), t is the parameter along the light direction, t ≥ 0.

[0161] Analyzing the intersection of the projected light and the target point cloud grid model, specifically represented as:

[0162] P(x P ,y P ,z P ) = (x0+ td x ,y0+ td y ,z0+ td z )

[0163] where (x0, y0, z0) is the initial point coordinate of the projected light, t is the direction vector parameter, (d x ,d y ,d z ) is the component of the projected light on the normal vector of the target point cloud grid model, and P is the intersection point.

[0164] Taking the point cloud grid model of a triangle as an example, for each light, calculate the intersection of its point cloud model surface triangle with each single device in the substation scene. The condition for determining whether the light intersects with the triangle can be determined by solving the corresponding linear equation system.

[0165] For a triangle model, its surface can be defined by the following vertices:

[0166] Triangle = (P1, P2, P3)

[0167] where P1, P2, P3 are the three vertices of the triangle.

[0168] The condition for determining whether the light intersects with the triangle can be determined by solving the corresponding linear equation system. Substitute the equation of the light into the equation of the plane where the triangle is located:

[0169] N·(O + tD) + d = 0

[0170] where N is the normal vector of the triangle, obtained by the following cross product:

[0171] N = (P2 - P1) x (P3 - P1)

[0172] d is the constant term of the plane equation, which can be calculated by the distance from a point to a plane formula.

[0173] Solve t to find the potential intersection point P:

[0174] P = O + tD

[0175] Based on the shape position relationship between the intersection point and the target point cloud grid model, mark the visual blind area, which specifically includes the following steps:

[0176] Based on the direction of the inspection projection light, determine the planar graph of the target point cloud grid model;

[0177] Give the coordinates of the barycenter of the planar graph under the global coordinate system;

[0178] Combine the connection vector analysis of the intersection point and each vertex in the planar graph, and give the vector cross product group, which is specifically represented as:

[0179] C i = (x i+1 -x i ) · (y P -y i ) - (y i+1 -y i ) · (x P -x i )

[0180] where C i is the vector cross product of the corresponding i-th vertex in the vector cross product group, (x i , y i ) is the coordinates of the i-th vertex in the planar graph, (x i+1 , y i+1 ) is the coordinates of the i+1-th vertex in the planar graph, and (x P , y P ) is the coordinates of the intersection point P;

[0181] Based on the signs of each cross product and the comparison of the Euclidean distance between the intersection point and the barycenter, the intersection point located in the planar graph of the target point cloud grid model is given, which is specifically represented as:

[0182]

[0183] where n is the total number of vector cross products in the vector cross product group, d() is the Euclidean distance between points, (x G , y G) coordinates of the center of gravity G of the planar graph of the target point cloud mesh model;

[0184] Based on the intersection points located in the planar graph of the target point cloud mesh model, the intersection points of the multiple inspection projection rays and the mesh model are marked for visible discrimination, and the visual blind area is given.

[0185] As Figure 8 shown, the arrangement of multiple inspection devices is given, wherein, Figure 8 (a) is a video monitoring point map, Figure 8 (b) is a robot inspection outdoor point map.

[0186] From Figure 8 (a) video monitoring, the #1 main transformer is covered by the 2nd camera below and the 1st camera on the right, Figure 8 (b) robot inspection, the #1 main transformer is covered by the camera carried by the robot in the left and right directions, according to the blind area detection algorithm proposed in the foregoing, the visible points and invisible points of the 1# main transformer point cloud model under the specific visual angle of the 1st, 2nd and robot cameras can be calculated respectively. By fusing the visible points and invisible points under the visual angle of multiple cameras, the union set of all visible points is obtained. Finally, the visible points and invisible points of the 1# main transformer point cloud model can be obtained, as Figure 9 shown.

[0187] After the visual blind area is given, the following steps are included:

[0188] The area of the grid unit of the point cloud mesh model is calculated and accumulated to obtain the total area of the power equipment;

[0189] The visible point cloud and invisible point cloud of the power equipment point cloud mesh model are marked;

[0190] By accumulating the blind area coverage rate of all single equipment, the blind area quantitative evaluation of the power equipment is realized.

[0191] Taking the triangular point cloud mesh model as an example, the triangle has vertices A, B and C, then two vectors and can be calculated by the following coordinate difference:

[0192]

[0193] The area S i of the i-th triangle can be obtained by calculating half the length of the cross product of the two vectors:

[0194]

[0195] Where the cross product gives a vector whose length is equal to the area of the triangle formed by and The area of the parallelogram formed, the direction being perpendicular to the plane formed by the two vectors.

[0196] The area of all the triangles in the model is added up, that is, the total surface area S.

[0197]

[0198] Calculate the surface area s of the visible point cloud cover , and the blind area coverage rate P of each power equipment is calculated by the following formula i :

[0199] P i = 1-s cover / s

[0200] Where s cover is the coverage area of the single device under the multi-patrol platform multi-camera field of view (the surface area of the visible point cloud in the power equipment point cloud model), and s is the total area of the front, back, left and right and top of the power equipment.

[0201] By adding up the blind area coverage rate P of all single devices, the quantitative evaluation of the blind area of the substation equipment is realized, and the formula is as follows:

[0202]

[0203] Where n is the number of single devices in the substation, and i is the i-th single device.

[0204] As Figure 10 shown, the application also provides an identification device for a power equipment inspection blind area, which adopts the identification method for a power equipment inspection blind area as described above, and specifically comprises:

[0205] The acquisition unit is configured to acquire point cloud data of the power equipment.

[0206] The construction unit is configured to form a point cloud grid model of the single power equipment, establish a global coordinate system, and give a patrol visual coordinate system for the power equipment inspection.

[0207] The analysis unit is configured to map the patrol visual coordinate system to the global coordinate system, and based on the interactive analysis of the patrol projection light and the point cloud grid model, give the patrol blind area of the power equipment.

[0208] The identification method and device for a power equipment inspection blind area provided by the application at least have the following beneficial effects:

[0209] (1) The application realizes automatic, efficient, accurate and comprehensive identification of power equipment inspection blind area by fusing three-dimensional point cloud, visual inspection and coordinate transformation, and then carries out quantitative evaluation on the power equipment inspection blind area to optimize the inspection route, improve the inspection efficiency and inspection point coverage rate, and reduce the operation risk of power equipment.

[0210] (2) The application ensures accurate and seamless conversion from the inspection visual coordinate system to the global coordinate system through the affine transformation matrix composed of the translation vector and the rotation matrix, and the vision of the inspection device can be effectively mapped into the point cloud grid model, so that the vision data of the inspection device is used for further processing, blind area analysis and evaluation in the point cloud grid model.

[0211] Although the preferred embodiments of the present application have been described, those skilled in the art, once they know the basic creative concept, can make additional changes and modifications to these embodiments. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the present application. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include these modifications and variations.

Claims

1. A method for identifying blind spots in power equipment inspection, characterized in that, Specifically, the steps include the following: Acquire point cloud data of power equipment, form a point cloud mesh model of individual power equipment, and establish a global coordinate system; The inspection visual coordinate system for power equipment inspection is given, and the mapping relationship between the inspection visual coordinate system and the global coordinate system is established. Based on the interactive analysis of the inspection projection ray and the point cloud mesh model using a global coordinate system, the blind spots of power equipment are identified. The specific steps include: defining the inspection projection ray and obtaining the corresponding target point cloud mesh model; analyzing the intersection points of the projection ray and the target point cloud mesh model; and marking the visual blind spots based on the shape and positional relationship between the intersection points and the target point cloud mesh model. Specifically, this involves: determining the planar shape of the target point cloud mesh model based on the direction of the inspection projection ray; providing the coordinates of the centroid of the planar shape in the global coordinate system; combining the analysis of the connection vectors of the intersection points and each vertex in the planar shape to provide a set of vector cross products; and based on the signs of each cross product and a comparison of the Euclidean distance between the intersection points and the centroid, identifying the intersection points located in the planar shape of the target point cloud mesh model. Based on the intersection points in the planar graphics of the target point cloud mesh model, the intersection points of multiple inspection projection rays and the mesh model are identified by visible marking, and the visual blind spots are given.

2. The method for identifying blind spots in power equipment inspection as described in claim 1, characterized in that, The process of creating a point cloud mesh model of a single power device includes the following steps: Acquire initial 3D point cloud data of the power equipment; Based on the Gaussian model, the initial 3D point cloud data is preprocessed to obtain the point cloud data of the power equipment. By performing clustering, the point cloud data is classified and point cloud data of each individual power device is constructed. The point cloud data of each individual power device is simplified and converted into voxel data; An isosurface extraction algorithm is used to extract the surface of the point cloud mesh model from the voxel data; By combining the occupancy state of voxel data and the surface of the point cloud mesh model, a point cloud mesh model of a single power device is formed.

3. The method for identifying blind spots in power equipment inspection as described in claim 2, characterized in that, Establishing a global coordinate system specifically includes: With the vertical direction of the point cloud mesh model as the Z-axis, the coordinate system of the point cloud mesh model is represented as follows: The coordinate system of the point cloud mesh model is defined as the global coordinate system.

4. The method for identifying blind spots in power equipment inspection as described in claim 1, characterized in that, The inspection visual coordinate system for power equipment inspection is given, and the mapping relationship between the inspection visual coordinate system and the global coordinate system is established. The specific steps include the following: Determine the visual coordinate system for the power equipment inspection device; Obtain the translation vector and rotation matrix, and construct the affine transformation matrix; By using an affine transformation matrix, a mapping relationship between the inspection vision coordinate system and the global coordinate system is established.

5. The method for identifying blind spots in power equipment inspection as described in claim 4, characterized in that, To obtain the translation vector and rotation matrix, and construct the affine transformation matrix, the specific steps include: Obtain the vector from the origin of the inspection vision coordinate system to the origin of the global coordinate system, and provide the translation vector; By analyzing the angles between each coordinate axis of the inspection vision coordinate system and the corresponding coordinate axes of the global coordinate system, a rotation matrix is ​​given. By combining the translation vector and the rotation matrix, the affine transformation matrix is ​​constructed.

6. The method for identifying blind spots in power equipment inspection as described in claim 4, characterized in that, Establish the mapping relationship between the inspection vision coordinate system and the global coordinate system, specifically represented as follows: ； Among them, (x) a y a , z a () represents the coordinates of a point in the inspection visual coordinate system. R represents the point and its coordinates in the global coordinate system corresponding to the point coordinates in the inspection coordinate system, where R is the rotation matrix and t is the translation vector.

7. The method for identifying blind spots in power equipment inspection as described in claim 1, characterized in that, The intersection of the projected ray and the target point cloud mesh model is analyzed, specifically represented as follows: ； Where (x0, y0, z0) are the initial coordinates of the projected ray, t is the direction vector parameter, and (d x ,d y ,d z ) represents the component of the projected ray on the normal vector of the target point cloud mesh model, and P is the intersection point.

8. The method for identifying blind spots in power equipment inspection as described in claim 1, characterized in that, Based on the analysis of the intersection points and the connection vectors of each vertex in the planar figure, the cross product set of vectors is given, specifically as follows: ； Among them, C i The cross product of the vectors corresponding to the i-th vertex in the cross product set, (x i ,y i Let (x) be the coordinates of the i-th vertex in the planar figure. i+1 ,y i+1 Let (x) be the coordinates of the (i+1)th vertex in the planar figure. P ,y P Let P be the coordinates of the intersection point. Based on the signs of each cross product and a comparison of the Euclidean distance between the intersection point and the centroid, the intersection points located in the planar graph of the target point cloud mesh model are given, specifically represented as follows: ； Where n is the total number of cross products in the cross product group, d() is the Euclidean distance between points, (x G ,y G ) represents the coordinates of the centroid G of the planar graphic of the target point cloud mesh model.

9. A device for identifying blind spots in power equipment inspection, characterized in that, The method for identifying blind spots in power equipment inspection as described in any one of claims 1-8 specifically includes: The acquisition unit is used to acquire point cloud data of power equipment; The building unit is used to form a point cloud mesh model of a single power equipment and establish a global coordinate system, providing an inspection visual coordinate system for power equipment inspection; The analysis unit is used to map the inspection visual coordinate system to the global coordinate system. Based on the interactive analysis of the inspection projection rays and the point cloud mesh model, the blind spots of the power equipment are given through the global coordinate system.

Images