A method and system for detecting power equipment that integrates camera and lidar

Abstract

A method and system for detecting substation equipment integrating camera and lidar, relating to the field of power detection, is disclosed. It solves the problem of personal injury or death caused by maintenance personnel accidentally entering live areas due to improper operation. The method includes: acquiring images and point cloud data of the substation equipment and its surrounding environment; performing two-dimensional target detection on the substation equipment, extracting the detection frame information of the substation equipment, and correspondingly acquiring the point cloud data of the substation equipment; jointly calibrating the camera and lidar to obtain the extrinsic parameters of the joint calibration; filtering the acquired point cloud data; clustering the filtered point cloud data to obtain the spatial pose information of the substation equipment; projecting the point cloud onto the image through the joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment; matching the spatial pose information of the substation equipment with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized. This method has applications in the field of substation equipment maintenance.

Description

Technical Field

[0001] This invention relates to the field of power detection, and more particularly to a method for detecting substation equipment that integrates camera and lidar. Background Technology

[0002] In the operation of a power system, substation equipment is a crucial component, ensuring its normal operation. With socio-economic development, the demand for electricity is increasing, placing immense pressure on the entire power system. Because substation equipment is relatively expensive and generally has a long service life, it is rarely replaced. Furthermore, many power companies lack the financial capacity to upgrade their substation equipment in a timely manner. Against this backdrop, substation equipment faces the challenges of long operating hours and heavy loads, making it highly susceptible to aging and other malfunctions, impacting the speed and quality of power transmission.

[0003] Therefore, substation operation and maintenance requires the operation of substation equipment. However, maintenance personnel may accidentally enter the live area due to non-standard operation and need to conduct close-range contact testing of the substation equipment to check whether it is live, which brings the risk of electric shock and a serious psychological burden to the maintenance personnel.

[0004] With the continuous expansion of the power grid and the deepening of power system reform, the traditional operation and maintenance (O&M) method, which mainly relies on close-range contact testing of substation equipment to determine if it is energized, has had a certain impact on improving the safety, quality, and efficiency of substation O&M. Traditional manual O&M methods typically require O&M personnel to climb power poles or use suspended platforms for testing. This high-altitude working environment poses serious safety risks, and the workload for O&M personnel is high, making them prone to fatigue and injury. Furthermore, some substation equipment, such as busbars, is often located in complex geographical environments, making it impossible for manual O&M to cover certain hard-to-reach areas, leading to missed inspections. Summary of the Invention

[0005] This invention addresses the problem of personal injury or death caused by maintenance personnel accidentally entering live areas due to improper operation. It proposes a method for detecting substation equipment that integrates cameras and lidar, the method comprising:

[0006] S1: Acquire images and point cloud data of the substation equipment and its surrounding environment using RGB cameras and LiDAR;

[0007] S2: Use a target detection algorithm to perform two-dimensional target detection on the substation equipment, extract the detection box information of the substation equipment, and obtain the point cloud data of the substation equipment accordingly;

[0008] S3: Perform joint calibration of the camera and LiDAR to obtain the external parameters of the joint calibration; and calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters;

[0009] S4: Filter the acquired point cloud data;

[0010] S5: Cluster the filtered point cloud data to obtain the spatial pose information of the substation;

[0011] S6: The processed point cloud is projected onto the image through a joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. The spatial pose information of the substation equipment is matched with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

[0012] Furthermore, a preferred embodiment is proposed, wherein in step S1, the field of view of the RGB camera is 69°*42°; the frame rate is 1920*1080; the range of the lidar is 0.1~70m, and the ranging accuracy is 2cm.

[0013] Furthermore, a preferred method is proposed, wherein the target detection algorithm is used to identify substation equipment in step S2, including: using an improved YOLOv8 target detection network model to identify three types of substation equipment: busbars, disconnectors, and switches; and adding a convolutional attention module after the C2f module in the neck layer of the improved YOLOv8 target detection network model.

[0014] Furthermore, a preferred embodiment is proposed, wherein step S3 includes:

[0015] The lidar coordinate system is transformed to the camera coordinate system using the rotation matrix R and the translation matrix T.

[0016]

[0017] Among them, X c Y c Z c Let X be the camera coordinate system. L Y L Z L For the lidar coordinate system;

[0018] Transform from camera coordinate system to image coordinate system:

[0019]

[0020] Where f is the focal length of the camera, and (x, y) is the coordinate of a pixel in the image coordinate system;

[0021] The transformation relationship between the image coordinate system xoy and the pixel coordinate system uov:

[0022]

[0023] Where dx and dy represent the dimensions of each pixel in the x-axis and y-axis directions of the image coordinate system; (u0, v0) represent the coordinates of the origin of the image coordinate system in the pixel coordinate system.

[0024] The transformation relationship between the lidar coordinate system and the pixel coordinate system is as follows:

[0025]

[0026] Where K is the camera intrinsic parameter matrix, and M is the transformation matrix from the lidar coordinate system to the camera coordinate system.

[0027] Furthermore, a preferred embodiment is proposed, wherein step S4 includes:

[0028] Voxel filtering is performed on the original point cloud data, and a new point cloud dataset is obtained after downsampling.

[0029] A pass-through filter is used to extract regions of interest from a new point cloud dataset while simultaneously segmenting the ground point cloud data.

[0030] Outliers in ground point cloud data are eliminated.

[0031] Furthermore, a preferred embodiment is proposed, wherein step S5 includes:

[0032] Euclidean clustering algorithm is used to segment point cloud data into different clusters;

[0033] The center point of the power equipment is obtained by fitting the bounding box method.

[0034] Spatial pose data of the substation equipment is obtained through the center point.

[0035] Furthermore, an optimal approach is proposed, employing neighborhood search and region growing methods to achieve Euclidean clustering.

[0036] Based on the same inventive concept, this invention also proposes a substation equipment detection system that integrates a camera and a lidar, the system comprising:

[0037] The data acquisition unit is used to acquire images and point cloud data of the substation equipment and its surrounding environment through an RGB camera and a LiDAR.

[0038] The detection unit is used to perform two-dimensional target detection on the substation equipment using a target detection algorithm, extract the detection box information of the substation equipment, and obtain the point cloud data of the substation equipment accordingly.

[0039] The calibration unit is used to jointly calibrate the camera and the lidar to obtain the external parameters of the joint calibration; and to calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters.

[0040] The filtering unit is used to filter the acquired point cloud data;

[0041] Clustering unit is used to cluster the filtered point cloud data to obtain the spatial pose information of the substation equipment;

[0042] The judgment unit is used to project the processed point cloud onto the image through a joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. It then matches the spatial pose information of the substation equipment with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

[0043] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a substation equipment detection method based on a fusion camera and lidar according to any one of the above-mentioned methods.

[0044] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of a substation equipment detection method that integrates a camera and a lidar as described in any of the preceding claims.

[0045] The advantages of this invention are:

[0046] Traditional methods of inspecting power equipment largely rely on manual judgment, requiring maintenance personnel to manually check the energized status of the equipment. Because power equipment is often widely distributed and located in complex environments, maintenance personnel are prone to entering energized areas due to ignoring safety signs or misjudging the equipment's condition, leading to electric shock accidents. This invention proposes a visualized energized status detection method for power equipment. Through automated energized status detection, it utilizes devices such as RGB cameras, LiDAR, and electric field sensors, combined with target detection algorithms, to automatically identify and determine the energized status of power equipment. This effectively reduces the possibility of human error, thereby improving maintenance safety.

[0047] The energized state of power equipment is often difficult to determine directly, especially when the equipment is located in complex or harsh environments. Traditional detection methods struggle to provide real-time and accurate assessments of equipment energization. Manual inspections are also affected by environmental factors (such as insufficient light or obstructions), leading to inaccurate identification. This invention proposes a power equipment detection method that integrates cameras and LiDAR. It acquires image and point cloud data using a combination of RGB cameras and LiDAR, then combines this with a target detection algorithm to identify the equipment's location. Simultaneously, it performs cluster analysis on the equipment's spatial pose to further improve detection accuracy and robustness. Finally, by matching the data with field strength data collected by an electric field sensor, the energized state of the power equipment can be accurately determined under different environments, overcoming the complexity and error problems of traditional methods.

[0048] During the operation and maintenance of substation equipment, the equipment status may change rapidly. Traditional methods often rely on static inspections, making it difficult to achieve dynamic tracking and real-time feedback. Furthermore, maintenance personnel need to constantly monitor the equipment's energized status to respond promptly and avoid misoperation. This invention proposes a substation equipment detection method integrating cameras and LiDAR. It employs a real-time detection system that uses an RGB camera and LiDAR to acquire equipment images and point cloud data in real time, and utilizes an electric field sensor to detect the equipment's energized status in real time. This ensures that maintenance personnel can promptly understand the equipment's safety status.

[0049] The spatial location and orientation of power equipment often significantly influence the determination of its energized state. Traditional methods may fail to accurately locate the equipment or lack precise spatial orientation information, leading to judgment errors. This invention proposes a power equipment detection method that integrates camera and lidar data. By clustering point cloud data acquired through lidar, the spatial orientation information of the power equipment can be accurately obtained. Through data matching with electric field sensors, the system can more accurately determine whether the equipment is energized and precisely locate its position. This fusion of spatial orientation and electric field data provides high-precision support for determining the energized state.

[0050] This invention is applied to the field of power equipment maintenance. Attached Figure Description

[0051] Figure 1 This is a flowchart of a substation equipment detection method that integrates camera and lidar as described in Embodiment 1.

[0052] Figure 2 This is a schematic diagram of the network model after the improvement of the YOLOv8 algorithm described in Implementation Method 3;

[0053] Figure 3 A schematic diagram of the CBAM module added to the YOLOv8 algorithm described in Implementation Method 3;

[0054] Figure 4 A schematic diagram of the channel attention module of the CBAM module added to the YOLOv8 algorithm described in Implementation Method 3;

[0055] Figure 5 A schematic diagram of the spatial attention module of the CBAM module added to the YOLOv8 algorithm described in Implementation Method 3;

[0056] Figure 6 This is a schematic diagram of the mAP@0.5 metric before the improvement of the YOLOv8 algorithm described in Implementation Method 3, where Precision represents precision and Recall represents recall.

[0057] Figure 7 This is a schematic diagram of the mAP@0.5 index after the improvement of the YOLOv8 algorithm described in Implementation Method 3;

[0058] Figure 8 This is a schematic diagram of the F1 score of the YOLOv8 algorithm before its improvement as described in Implementation Method 3;

[0059] Figure 9 This is a schematic diagram of the F1 score after the improvement of the YOLOv8 algorithm described in Implementation Method 3;

[0060] Figure 10 This is a flowchart of the Euclidean clustering described in Implementation Method Seven;

[0061] Figure 11 This is a schematic diagram of the point cloud 3D detection effect described in Implementation Method 7;

[0062] Figure 12 This is a schematic diagram of the improved YOLOv8 detection effect described in Implementation Method 7. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0064] Implementation Method 1, see [link] Figure 1 This embodiment describes a method for detecting power equipment that integrates a camera and a lidar sensor. The method includes:

[0065] S1: Acquire images and point cloud data of the substation equipment and its surrounding environment using RGB cameras and LiDAR;

[0066] S2: Use a target detection algorithm to perform two-dimensional target detection on the substation equipment, extract the detection box information of the substation equipment, and obtain the point cloud data of the substation equipment accordingly;

[0067] S3: Perform joint calibration of the camera and LiDAR to obtain the external parameters of the joint calibration; and calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters;

[0068] S4: Filter the acquired point cloud data;

[0069] S5: Cluster the filtered point cloud data to obtain the spatial pose information of the substation;

[0070] S6: The processed point cloud is projected onto the image through a joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. The spatial pose information of the substation equipment is matched with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

[0071] Traditional methods of inspecting substation equipment largely rely on manual judgment, requiring maintenance personnel to manually check the energized status of the equipment. Because substation equipment is often widely distributed and located in complex environments, maintenance personnel are prone to entering energized areas due to ignoring safety signs or misjudging equipment status, leading to electric shock accidents. This embodiment proposes a substation equipment detection method integrating cameras and LiDAR. Through automated energized status detection, it utilizes devices such as RGB cameras, LiDAR, and electric field sensors, combined with target detection algorithms, to automatically identify and determine the energized status of substation equipment. This effectively reduces the possibility of human error, thereby improving maintenance safety.

[0072] The energized state of substation equipment is often difficult to determine directly, especially when the equipment is located in complex or harsh environments. Traditional detection methods struggle to provide real-time and accurate assessments of equipment energization. Manual inspections are also affected by environmental factors (such as insufficient light or obstructions), leading to inaccurate identification. This embodiment proposes a substation equipment detection method that integrates cameras and LiDAR. It acquires image and point cloud data using a combination of RGB cameras and LiDAR, then combines this with a target detection algorithm to identify the equipment location. Simultaneously, it performs cluster analysis on the equipment's spatial pose to further improve detection accuracy and robustness. Finally, by matching the data with field strength data collected by an electric field sensor, the energized state of substation equipment can be accurately determined under different environments, overcoming the complexity and error problems of traditional methods.

[0073] During the operation and maintenance of substation equipment, the equipment status may change rapidly. Traditional methods often rely on static inspections, making it difficult to achieve dynamic tracking and real-time feedback. Furthermore, maintenance personnel need to constantly monitor the equipment's energized status to respond promptly and avoid misoperation. This embodiment proposes a substation equipment detection method that integrates cameras and LiDAR. It uses RGB cameras and LiDAR to acquire real-time equipment images and point cloud data, and utilizes electric field sensors to detect the equipment's energized status in real time, ensuring that maintenance personnel can promptly understand the equipment's safety status.

[0074] The spatial location and orientation of power equipment often significantly influence the determination of its energized state. Traditional methods may fail to accurately locate the equipment or lack precise spatial orientation information, leading to judgment errors. In this embodiment, point cloud data clustering acquired by lidar can accurately obtain the spatial orientation information of the power equipment. By matching this data with that of electric field sensors, the system can more accurately determine whether the equipment is energized and precisely locate its position. This fusion of spatial orientation and electric field data provides high-precision support for determining the energized state.

[0075] Implementation Method 2: This implementation method further defines the substation equipment detection method that integrates a camera and a lidar as described in Implementation Method 1. In step S1, the RGB camera has a field of view of 69°*42° and a frame rate of 1920*1080. The lidar has a range of 0.1 to 70 m and a ranging accuracy of 2 cm.

[0076] This embodiment employs an RGB camera with a field of view of 69°*42°, which can cover a wider area, thus ensuring that image information of the power equipment can be acquired more comprehensively during the inspection process. A larger field of view allows the camera to capture a wider area, enabling more effective identification of the equipment's status and avoiding missed detections. A high-resolution camera provides clearer and more detailed images, facilitating accurate analysis of the power equipment's energized state.

[0077] Furthermore, in this embodiment, the lidar can accurately measure the position, size, and relative position of the equipment to other objects. Especially within a range of 70 meters, lidar can help detect the status of power equipment at greater distances; the 2cm ranging accuracy of the lidar provides more precise identification, which is crucial for identifying the shape, spatial configuration, and electrical status of the equipment.

[0078] In this embodiment, data from an RGB camera and a LiDAR system are combined to effectively integrate visual and spatial information. In practical applications, RGB images can provide surface information, while LiDAR provides depth and spatial location information. This fusion effectively improves the robustness and accuracy of the detection system, providing more efficient detection capabilities, especially in complex environments (such as multiple transformers in a substation).

[0079] In practical applications, the RGB camera can be Intel's RealSense D435i camera, and the LiVox Mid-360 LiDAR can be used to effectively ensure the accuracy and efficiency of spatial pose data measurement for power equipment.

[0080] Implementation Method 3, see below Figures 2 to 9 This embodiment further defines the method for detecting substation equipment that integrates camera and lidar as described in Embodiment 1. It employs an improved YOLOv8 target detection network model to identify three types of substation equipment: busbars, disconnectors, and switches. A convolutional attention module is added after the C2f module in the neck layer of the improved YOLOv8 target detection network model.

[0081] The Convolutional Attention Module (CBAM) described in this embodiment integrates channel attention and spatial attention. Furthermore, the CBAM proposed in this embodiment is lightweight and plug-and-play. The overall flow of the CBAM module is as follows: Figure 3 As shown, firstly, the input is an intermediate feature map. This feature map is fed into the Channel Attention Module to obtain channel attention, and then the attention weights are applied to the intermediate feature map. Next, the feature map with channel attention applied is fed into the Spatial Attention Module to obtain spatial attention, and then the attention weights are applied to the feature map. Finally, through the sequential operation of these two attention modules, the initial feature map has undergone processing by both channel and spatial attention mechanisms, adaptively refining the features. The final improved YOLOv8 object detection network model is as follows: Figure 2 As shown.

[0082] Channel attention module such as Figure 4As shown, the feature map is pooled in the spatial dimension to preserve channel feature information. In the Convolutional Attention Module (CBAM), both global average pooling and global max pooling are used. First, the input feature map is subjected to global max pooling and global average pooling respectively, compressing the spatial dimension to 1 while preserving channel information. Then, the two pooled features are fed into a shared Multilayer Perceptron (MLP) to extract features. Finally, the pooled features from the MLP are summed and activated by sigmoid to obtain the final channel attention weights.

[0083] Spatial attention module such as Figure 5 As shown, the feature map is pooled along the channel dimension to preserve spatial feature information. First, the feature map (after channel attention calculation) is subjected to max pooling and average pooling along the channel dimension, compressing the channel dimension to 1 and preserving spatial information. Then, the pooled features are concatenated and passed through a convolutional layer to extract features, while simultaneously reducing the channel dimension to 1. Finally, it is activated by a sigmoid layer to obtain the final spatial attention weights (which include channel attention).

[0084] After performing these two steps in sequence, the spatial attention features are multiplied by the original feature map.

[0085] The improved YOLOv8 object detection network model performs as shown in the Map@0.5 benchmark comparison. Figure 6 and Figure 7 As shown, the comparison effect of the F1 index is as follows: Figure 8 and Figure 9 As shown, it is evident that the improved YOLOv8 object detection network model achieves a 1.7% increase in the Map0.5 metric and a 1% increase in the F1 metric, demonstrating higher detection accuracy compared to the original YOLOv8 algorithm.

[0086] Implementation Method Four: This implementation method further defines the substation equipment detection method that integrates camera and lidar as described in Implementation Method One. Step S3 includes:

[0087] The lidar coordinate system is transformed to the camera coordinate system using the rotation matrix R and the translation matrix T.

[0088]

[0089] Among them, X c Y c Z c Let X be the camera coordinate system. L Y L Z L For the lidar coordinate system;

[0090] Transform from camera coordinate system to image coordinate system:

[0091]

[0092] Where f is the focal length of the camera, and (x, y) is the coordinate of a pixel in the image coordinate system;

[0093] The transformation relationship between the image coordinate system xoy and the pixel coordinate system uov:

[0094]

[0095] Where dx and dy represent the dimensions of each pixel in the x-axis and y-axis directions of the image coordinate system; (u0, v0) represent the coordinates of the origin of the image coordinate system in the pixel coordinate system.

[0096] The transformation relationship between the lidar coordinate system and the pixel coordinate system is as follows:

[0097]

[0098] Where K is the camera intrinsic parameter matrix, and M is the transformation matrix from the lidar coordinate system to the camera coordinate system.

[0099] In this embodiment, the LiDAR coordinate system is transformed to the camera coordinate system using a rotation matrix R and a translation matrix T, ensuring spatial consistency between LiDAR and camera data. In practical applications, the LiDAR and camera sensors are typically in different coordinate systems. This transformation allows the measurement results from both the LiDAR and camera to be processed within the same coordinate system. Through coordinate transformation, the 3D point cloud data captured by the LiDAR can be precisely matched with the 2D image data captured by the camera. This not only enables target recognition using visual information from the images but also improves the accuracy of spatial information by combining LiDAR data, especially in complex environments such as low light or inclement weather, where the camera's visual capabilities may be limited, while the LiDAR can still provide stable data.

[0100] Furthermore, the transformation from the camera coordinate system to the image coordinate system can map the 3D data of the LiDAR onto the 2D image using focal length and pixel coordinates (x, y), allowing the depth information obtained by the LiDAR to be directly represented in the image. In this way, the system can directly mark the distance and position of the target object detected by the LiDAR on the 2D image. This enhances the depth perception capability. Moreover, when transforming between the image coordinate system and the pixel coordinate system, the physical scale of each pixel and the position of the origin of the image coordinate system in the pixel coordinate system are considered, ensuring accurate mapping to the actual size of the image.

[0101] Implementation Method 5: This implementation method further defines the substation equipment detection method that integrates camera and lidar as described in Implementation Method 1. Step S4 includes:

[0102] Voxel filtering is performed on the original point cloud data, and a new point cloud dataset is obtained after downsampling.

[0103] A pass-through filter is used to extract regions of interest from a new point cloud dataset while simultaneously segmenting the ground point cloud data.

[0104] Outliers in ground point cloud data are eliminated.

[0105] The original point cloud data is subjected to voxel filtering and downsampling to obtain a new point cloud dataset, including:

[0106] Since raw point cloud data typically contains a large amount of redundant information, this embodiment utilizes voxel filtering to effectively reduce the density of the point cloud data by segmenting it into small voxel units and replacing all points in that region with the average value within each voxel. Combined with the new point cloud dataset obtained after downsampling, this significantly reduces computational load, improves subsequent processing speed, and preserves the main features of the point cloud. Furthermore, this embodiment also uses a pass-through filter to extract the region of interest and segment the ground point cloud data. This effectively focuses on the relevant areas of the substation equipment, avoiding the processing of irrelevant areas. By eliminating outliers in the ground point cloud data, the robustness and reliability of the detection are significantly improved.

[0107] Implementation Method Six: This implementation method further defines the substation equipment detection method that integrates camera and lidar as described in Implementation Method One. Step S5 includes:

[0108] Euclidean clustering algorithm is used to segment point cloud data into different clusters;

[0109] The center point of the power equipment is obtained by fitting the bounding box method.

[0110] Spatial pose data of the substation equipment is obtained through the center point.

[0111] This embodiment employs Euclidean clustering to cluster point cloud data, effectively removing noise and separating the substation equipment from its surrounding environment. Clustering improves the accuracy and robustness of substation equipment detection, reducing false positives and false negatives. Furthermore, this embodiment uses the bounding box method to determine the center point of the substation equipment, which is simple, efficient, and accurately captures the equipment's boundary and spatial position. Since the bounding box method does not rely on complex geometric calculations, it has a smaller computational load and faster processing speed, making it suitable for real-time or large-scale substation equipment detection tasks. Obtaining the spatial pose data of the substation equipment through its center point allows for accurate localization of its three-dimensional spatial position and orientation (including rotation angle). This embodiment, through the combination of Euclidean clustering and the bounding box method, efficiently and accurately extracts the spatial pose information of the substation equipment from point cloud data, exhibiting strong robustness, high processing efficiency, and strong adaptability, making it suitable for automatic detection and localization of substation equipment in various environments.

[0112] Implementation Method Seven, see below Figure 10 , Figure 11 and Figure 12 This embodiment describes a further refinement of the substation equipment detection method fusion of camera and lidar described in Embodiment 1. It employs neighborhood search and region growing to achieve Euclidean clustering, including:

[0113] The neighborhood of point pi is defined as the set of points whose Euclidean distance is less than a certain threshold ∈.

[0114]

[0115] The Euclidean distance between points pi and pj is:

[0116]

[0117] Use the region growing method to group points into different clusters:

[0118] Randomly select an unassigned point pi from the point cloud as the seed point and initialize a new cluster Ck;

[0119] Add all points in the neighborhood of the seed point to cluster Ck;

[0120] For each newly added point to the cluster, continue searching its neighborhood until no more points meet the criteria;

[0121] Process the remaining unassigned points until all points are assigned to a cluster or marked as noise points.

[0122] In this embodiment, point cloud data is divided into different clusters using neighborhood search and region growing. Neighborhood search ensures that all points surrounding each point are effectively identified, avoiding the omission of clusters due to sparse or complex data point distribution. Region growing, starting from a seed point, gradually expands the cluster's range, maintaining cluster compactness and integrity while ensuring segmentation clarity. By setting a Euclidean distance threshold ∈, the system can automatically adapt to the characteristics of different point cloud data within a reasonable range, making cluster allocation more flexible and accurate. Different threshold settings can effectively handle various noise and data unevenness conditions. Region growing demonstrates strong adaptability when handling clusters of different sizes and shapes, making it particularly suitable for detection tasks in complex environments. Furthermore, in this embodiment, all points that fail to be assigned to a cluster are marked as noise points, effectively distinguishing between valid and noisy data, avoiding false detections caused by noise data, and improving detection accuracy.

[0123] Implementation Method 8: A substation equipment detection system integrating camera and lidar as described in this implementation method, the system comprising:

[0124] The data acquisition unit is used to acquire images and point cloud data of the substation equipment and its surrounding environment through an RGB camera and a LiDAR.

[0125] The detection unit is used to perform two-dimensional target detection on the substation equipment using a target detection algorithm, extract the detection box information of the substation equipment, and obtain the point cloud data of the substation equipment accordingly.

[0126] The calibration unit is used to jointly calibrate the camera and the lidar to obtain the external parameters of the joint calibration; and to calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters.

[0127] The filtering unit is used to filter the acquired point cloud data;

[0128] Clustering unit is used to cluster the filtered point cloud data to obtain the spatial pose information of the substation equipment;

[0129] The judgment unit is used to project the processed point cloud onto the image through a joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. It then matches the spatial pose information of the substation equipment with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

[0130] Implementation Method Nine: A computer device according to this implementation method includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the method for visually detecting the energized state of power equipment according to any one of Implementation Methods One to Seven.

[0131] Implementation Method 10: A computer-readable storage medium according to this implementation method, wherein a computer program is stored on the computer program, and the computer program is executed by a processor to perform the steps of the power equipment energization status visualization detection method as described in any one of Implementation Methods 1 to 7.

[0132] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure and not to limit its protection scope. Although this disclosure has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading this disclosure, they can still make various changes, modifications or equivalent substitutions to the specific implementation of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the published pending claims.

Claims

1. A method for detecting substation equipment that integrates camera and lidar, characterized in that, The method includes: S1: Acquire images and point cloud data of the substation equipment and its surrounding environment using RGB cameras and LiDAR; S2: Use a target detection algorithm to perform two-dimensional target detection on the substation equipment, extract the detection box information of the substation equipment, and obtain the point cloud data of the substation equipment accordingly; S3: Perform joint calibration of the camera and LiDAR to obtain the external parameters of the joint calibration; and calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters; S4: Filter the acquired point cloud data; S5: Cluster the filtered point cloud data to obtain the spatial pose information of the substation; S6: The processed point cloud is projected onto the image through a joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. The spatial pose information of the substation equipment is matched with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized. Step S2 uses a target detection algorithm to identify substation equipment, including: using an improved YOLOv8 target detection network model to identify three types of substation equipment: busbars, disconnectors, and switches; a convolutional attention module is added after the C2f module in the neck layer of the improved YOLOv8 target detection network model. Step S4 includes: Voxel filtering is performed on the original point cloud data, and a new point cloud dataset is obtained after downsampling. A pass-through filter is used to extract regions of interest from a new point cloud dataset while simultaneously segmenting the ground point cloud data. Eliminate outliers in ground point cloud data; Step S5 includes: Euclidean clustering algorithm is used to segment point cloud data into different clusters; The center point of the power equipment is obtained by fitting the bounding box method. The spatial pose data of the substation is obtained through the center point; Euclidean clustering is achieved using neighborhood search and region growing methods.

2. The method for detecting substation equipment by fusing camera and lidar according to claim 1, characterized in that, In step S1, the RGB camera has a field of view of 69°*42° and a frame rate of 1920*1080; the lidar has a range of 0.1~70m and a ranging accuracy of 2cm.

3. A substation equipment detection system integrating camera and lidar, characterized in that, The system includes: The data acquisition unit is used to acquire images and point cloud data of the substation equipment and its surrounding environment through an RGB camera and a LiDAR. The detection unit is used to perform two-dimensional target detection on the substation equipment using a target detection algorithm, extract the detection box information of the substation equipment, and obtain the point cloud data of the substation equipment accordingly; the target detection algorithm is used to identify the substation equipment, including: using an improved YOLOv8 target detection network model to identify three types of substation equipment: busbars, disconnectors, and switches; a convolutional attention module is added after the C2f module in the neck layer of the improved YOLOv8 target detection network model; The calibration unit is used to jointly calibrate the camera and the lidar to obtain the external parameters of the joint calibration; and to calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters. A filtering unit is used to filter the acquired point cloud data; it includes: Voxel filtering is performed on the original point cloud data, and a new point cloud dataset is obtained after downsampling. A pass-through filter is used to extract regions of interest from a new point cloud dataset while simultaneously segmenting the ground point cloud data. Eliminate outliers in ground point cloud data; Clustering units are used to cluster the filtered point cloud data to obtain the spatial pose information of the substation equipment; including: Euclidean clustering algorithm is used to segment point cloud data into different clusters; The center point of the power equipment is obtained by fitting the bounding box method. The spatial pose data of the substation is obtained through the center point; Euclidean clustering is achieved using neighborhood search and region growing methods; The judgment unit is used to project the processed point cloud onto the image through the joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. It then matches the spatial pose information of the substation equipment with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

4. A computer device, characterized in that: The device includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a method for detecting substation equipment by fusing a camera and a lidar according to any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the substation equipment detection method according to any one of claims 1-2.

Images