Power transformation equipment detection method and system integrating camera and laser radar

Abstract

The invention discloses a power transformation equipment detection method and system integrating a camera and a laser radar, and relates to the field of power detection. The problem that the operation and maintenance personnel mistakenly enter a live-line area to cause casualties due to the fact that the operation does not conform to the specification is solved. The method comprises the following steps: acquiring images and point cloud data of power transformation equipment and a surrounding environment; performing two-dimensional target detection on the power transformation equipment, extracting detection frame information of the power transformation equipment, and correspondingly obtaining point cloud data of the power transformation equipment; performing joint calibration on the camera and the laser radar to obtain external parameters of joint calibration; filtering the obtained point cloud data; clustering the filtered point cloud data to obtain spatial pose information of the power transformation equipment; the point cloud is projected to the image through the joint calibration external parameter matrix, classification information and space pose information of the power transformation equipment are obtained, the space pose information of the power transformation equipment is matched with field intensity data detected by the electric field sensor, and whether the power transformation equipment is electrified or not is judged. The method is applied to the substation equipment maintenance field.

Description

Technical Field

[0001] The present invention relates to the field of electric power detection, and in particular to a method for detecting substation equipment by integrating a camera and a laser radar. Background Art

[0002] Substation equipment is a crucial component of power system operation, ensuring its proper functioning. With socioeconomic development, electricity demand is increasing, placing immense pressure on the entire power system. Due to the high cost and long service life of substation equipment, replacement is often a rarity. Furthermore, many power companies lack the financial resources to upgrade substation equipment in a timely manner. Consequently, substation equipment faces challenges from long operating times and heavy loads, making it prone to equipment aging and other failures, impacting the speed and quality of power transmission.

[0003] Therefore, substation operation and maintenance requires operation on substation equipment, but operation and maintenance personnel may mistakenly enter the energized area due to non-compliant operation and need to conduct close contact detection on the substation equipment to see if it is energized, which brings safety hazards of electric shock and serious psychological burden on operation and maintenance personnel.

[0004] With the continuous expansion of the power grid and the deepening of power system reform, the traditional operation and maintenance method, which relies primarily on close-up inspection of substation equipment to determine if it is energized, has had a certain impact on improving the safety, quality, and efficiency of substation operations. Traditional manual operation and maintenance methods often require operators to climb power towers or use hanging baskets for inspection. This high-altitude working environment poses serious safety risks and is also very intensive, making operators prone to fatigue and injury. Furthermore, certain substation equipment, such as busbars, are often located in complex geographical environments. This makes manual operation and maintenance impossible to cover certain hard-to-reach areas, leading to missed inspections. Summary of the Invention

[0005] To address the issue of personal injury and death caused by operation and maintenance personnel mistakenly entering live areas due to non-compliant operations, the present invention proposes a substation equipment detection method that integrates cameras and lidar, the method comprising:

[0006] S1: Acquires images and point cloud data of the substation and its surroundings through RGB cameras and lidar;

[0007] S2: Use the target detection algorithm to perform two-dimensional target detection on the substation equipment, extract the detection frame information of the substation equipment, and obtain the corresponding point cloud data of the substation equipment;

[0008] S3: Jointly calibrate the camera and lidar to obtain the external parameters of the joint calibration; and calibrate the camera using the Zhang Zhengyou calibration method to obtain the intrinsic parameters of the camera;

[0009] S4: filtering the acquired point cloud data;

[0010] S5: Clustering the filtered point cloud data to obtain the spatial pose information of the substation equipment;

[0011] S6: The processed point cloud is projected onto the image through the joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. The spatial pose information of the substation equipment is matched with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

[0012] Furthermore, a preferred method is proposed, in which the field of view of the RGB camera in step S1 is 69°*42°; the frame rate is 1920*1080; the range of the laser radar is 0.1~70m, and the ranging accuracy is 2cm.

[0013] Furthermore, a preferred method is proposed, in which a target detection algorithm is used in step S2 to identify substation equipment, including: using an improved YOLOv8 target detection network model to identify three types of substation equipment: busbar, knife switch and switch; and adding a convolutional attention module after the C2f module of the neck layer in the improved YOLOv8 target detection network model.

[0014] Furthermore, a preferred embodiment is proposed, wherein step S3 includes:

[0015] The laser radar is transformed to the camera coordinate system through the rotation matrix R and the translation matrix T:

[0016]

[0017] Among them, X c Y c Z c is the camera coordinate system, X L Y L Z L is the laser radar coordinate system;

[0018] Convert the camera coordinate system to the image coordinate system:

[0019]

[0020] Where f is the focal length of the camera, (x, y) is the coordinate of a pixel in the image coordinate system;

[0021] The conversion relationship between the image coordinate system xoy and the pixel coordinate system uov is:

[0022]

[0023] Where dx and dy represent the dimensions of each pixel in the x-axis and y-axis directions of the image coordinate system; (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system;

[0024] The conversion relationship between the lidar coordinate system and the pixel coordinate system is as follows:

[0025]

[0026] Among them, K is the camera intrinsic parameter matrix, and M is the transformation matrix from the lidar coordinate system to the camera coordinate system.

[0027] Furthermore, a preferred embodiment is proposed, wherein step S4 includes:

[0028] Perform voxel filtering on the original point cloud data and obtain a new point cloud dataset after downsampling;

[0029] Use the straight-through filter to extract the region of interest in the new point cloud data set and segment the ground point cloud data;

[0030] Eliminate outliers in ground point cloud data.

[0031] Furthermore, a preferred embodiment is proposed, wherein step S5 includes:

[0032] Use the Euclidean clustering algorithm to divide the point cloud data into different clusters;

[0033] The center point is fitted as the center point of the substation equipment by using the external bounding box method;

[0034] The spatial position data of the substation equipment is obtained through the center point.

[0035] Furthermore, an optimal method is proposed to realize Euclidean clustering by using neighborhood search and region growing method.

[0036] Based on the same inventive concept, the present invention also proposes a substation equipment detection system integrating a camera and a lidar, the system comprising:

[0037] A data acquisition unit, used to acquire images and point cloud data of the substation and its surroundings through an RGB camera and lidar;

[0038] The detection unit is used to perform two-dimensional target detection on the substation equipment using a target detection algorithm, extract the detection frame information of the substation equipment, and obtain the corresponding point cloud data of the substation equipment;

[0039] The calibration unit is used to jointly calibrate the camera and lidar to obtain the external parameters of the joint calibration; and to calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters;

[0040] A filtering unit, used for filtering the acquired point cloud data;

[0041] A clustering unit is used to cluster the filtered point cloud data to obtain the spatial posture information of the substation equipment;

[0042] The judgment unit is used to project the processed point cloud onto the image through the joint calibration external parameter matrix, obtain the classification information and spatial posture information of the substation equipment, match the spatial posture information of the substation equipment with the field strength data detected by the electric field sensor, and judge whether the substation equipment is energized.

[0043] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a substation equipment detection method that integrates a camera and a lidar according to any one of the above-mentioned methods.

[0044] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes the steps of a substation equipment detection method that integrates a camera and a lidar as described in any one of the above items.

[0045] The present invention is beneficial in that:

[0046] Traditional substation equipment inspection methods mostly rely on manual judgment, and operators must manually check the equipment's live status. Because substation equipment is often widely distributed and located in complex environments, operators can easily enter live areas due to ignoring safety signs or misjudging the equipment's status, leading to electric shock accidents. The present invention proposes a method for visually detecting the live status of substation equipment. Through automated live status detection, the method utilizes devices such as RGB cameras, lidar, and electric field sensors, combined with target detection algorithms to automatically identify and determine the live status of substation equipment. This effectively reduces the possibility of human error and thus improves operation and maintenance safety.

[0047] The energized state of substation equipment is usually not easy to judge directly, especially when the equipment is located in a complex or harsh environment. Traditional detection methods make it difficult to accurately judge whether the equipment is energized in real time. Manual inspection is affected by environmental factors (such as insufficient light, obstruction, etc.), resulting in inaccurate identification. The present invention proposes a substation equipment detection method that integrates cameras and lidars. It acquires images and point cloud data through a combination of RGB cameras and lidars, and then identifies the location of the equipment in combination with a target detection algorithm. At the same time, it performs cluster analysis on the spatial posture of the equipment to further improve the accuracy and robustness of the detection. Finally, by matching with the field strength data collected by the electric field sensor, it can accurately judge the energized state of the substation equipment in different environments, overcoming the complexity and error problems in traditional methods.

[0048] During the operation and maintenance of substation equipment, the equipment status may change rapidly. Traditional methods often rely on static inspections, making it difficult to achieve dynamic tracking and real-time feedback. Furthermore, operators need to constantly monitor the equipment's energized status in order to respond promptly and avoid misoperation. This invention proposes a method for detecting substation equipment that integrates cameras and lidar. This method uses a real-time detection system to acquire device images and point cloud data in real time using an RGB camera and lidar, and utilizes an electric field sensor to detect the equipment's energized status in real time. This ensures that operators can promptly understand the safety status of the equipment.

[0049] The spatial position and posture of substation equipment often have a significant impact on determining its energized state. Traditional methods may be unable to accurately locate the equipment or lack accurate spatial posture information, leading to judgment errors. In the substation equipment detection method proposed in the present invention that integrates cameras and lidar, clustering of point cloud data obtained by lidar can accurately obtain the spatial posture information of the substation equipment. By matching the data with the electric field sensor, the system can more accurately determine whether the equipment is energized and precisely locate its position. This fusion of spatial posture and electric field data provides high-precision support for determining the energized state.

[0050] The present invention is applied to the field of maintenance of power transformation equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a method for detecting substation equipment by integrating a camera and a laser radar as described in Embodiment 1;

[0052] Figure 2 Schematic diagram of the improved network model of the YOLOv8 algorithm described in Implementation Method 3;

[0053] Figure 3 This is a schematic diagram of the CBAM module added to the YOLOv8 algorithm described in Implementation Method 3;

[0054] Figure 4 Schematic diagram of the channel attention module of the CBAM module added to the YOLOv8 algorithm described in Implementation 3;

[0055] Figure 5 Schematic diagram of the spatial attention module of the CBAM module added to the YOLOv8 algorithm described in Implementation 3;

[0056] Figure 6 This is a schematic diagram of the mAP@0.5 indicator before the improvement of the YOLOv8 algorithm described in Implementation Method 3, where Precision represents accuracy and Recall represents recall;

[0057] Figure 7 This is a schematic diagram of the mAP@0.5 indicator after the improvement of the YOLOv8 algorithm described in Implementation Method 3;

[0058] Figure 8 This is a schematic diagram of the F1 indicator before the improvement of the YOLOv8 algorithm described in Implementation Method 3;

[0059] Figure 9 This is a schematic diagram of the improved F1 indicator of the YOLOv8 algorithm described in Implementation Method 3;

[0060] Figure 10 This is a flowchart of the Euclidean clustering described in Embodiment 7;

[0061] Figure 11 This is a schematic diagram of the effect of point cloud three-dimensional detection according to the seventh embodiment;

[0062] Figure 12 This is a schematic diagram of the improved YOLOv8 detection effect described in Implementation Example 7. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0064] Implementation method 1, see Figure 1 This embodiment describes a method for detecting substation equipment by integrating a camera and a laser radar, the method comprising:

[0065] S1: Acquires images and point cloud data of the substation and its surroundings through RGB cameras and lidar;

[0066] S2: Use the target detection algorithm to perform two-dimensional target detection on the substation equipment, extract the detection frame information of the substation equipment, and obtain the corresponding point cloud data of the substation equipment;

[0067] S3: Jointly calibrate the camera and lidar to obtain the external parameters of the joint calibration; and calibrate the camera using the Zhang Zhengyou calibration method to obtain the intrinsic parameters of the camera;

[0068] S4: filtering the acquired point cloud data;

[0069] S5: Clustering the filtered point cloud data to obtain the spatial pose information of the substation equipment;

[0070] S6: The processed point cloud is projected onto the image through the joint calibration extrinsic parameter matrix to obtain the classification information and spatial pose information of the substation equipment. The spatial pose information of the substation equipment is matched with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

[0071] Traditional substation equipment inspection methods mostly rely on manual judgment, requiring operators to manually check the equipment's live status. Because substation equipment is often widely distributed and located in complex environments, operators can easily enter live areas due to ignoring safety signs or misjudging the equipment's status, leading to electric shock accidents. This embodiment proposes a substation equipment inspection method that integrates cameras and lidar. This method uses an automated live status detection method, utilizing RGB cameras, lidar, and electric field sensors, combined with target detection algorithms, to automatically identify and determine the live status of substation equipment. This effectively reduces the possibility of human error and thus improves operation and maintenance safety.

[0072] The energized state of substation equipment is usually not easy to judge directly, especially when the equipment is located in a complex or harsh environment. Traditional detection methods make it difficult to accurately judge whether the equipment is energized in real time. Manual inspection is affected by environmental factors (such as insufficient light, obstruction, etc.), resulting in inaccurate identification. This embodiment proposes a substation equipment detection method that integrates cameras and lidars. It acquires images and point cloud data through a combination of RGB cameras and lidars, and then identifies the location of the equipment in combination with the target detection algorithm. At the same time, it performs cluster analysis on the spatial posture of the equipment to further improve the accuracy and robustness of the detection. Finally, by matching with the field strength data collected by the electric field sensor, it can accurately judge the energized state of the substation equipment in different environments, overcoming the complexity and error problems in traditional methods.

[0073] During the operation and maintenance of substation equipment, the equipment status may change rapidly. Traditional methods often rely on static inspections, making it difficult to achieve dynamic tracking and real-time feedback. Furthermore, operators need to constantly monitor the equipment's energized status in order to respond promptly and avoid misoperation. This embodiment proposes a substation equipment detection method that integrates cameras and lidar. RGB cameras and lidar can be used to acquire equipment images and point cloud data in real time, while electric field sensors can be used to detect the equipment's energized status in real time, ensuring that operators can promptly understand the equipment's safety status.

[0074] The spatial position and posture of substation equipment often have a significant impact on determining its energized state. Traditional methods may be unable to accurately locate the equipment or lack precise spatial posture information, leading to errors in judgment. In this implementation, clustering point cloud data acquired through LiDAR precisely determines the spatial position of substation equipment. By matching this with data from electric field sensors, the system can more accurately determine whether the equipment is energized and precisely locate its position. This fusion of spatial position and electric field data provides high-precision support for determining the energized state.

[0075] Implementation method 2. This implementation method further limits the substation equipment detection method that integrates a camera and a lidar described in implementation method 1. The field of view angle of the RGB camera in step S1 is 69°*42°; the frame rate is 1920*1080; the range of the lidar is 0.1~70m, and the ranging accuracy is 2cm.

[0076] This implementation utilizes an RGB camera with a 69° x 42° field of view, enabling coverage of a wider area, ensuring comprehensive image information of the substation during inspection. A wider field of view allows the camera to capture a wider area, enabling more effective identification of equipment status and preventing missed inspections. High-resolution cameras provide clearer, more detailed images, enabling precise analysis of the energized state of substation equipment.

[0077] Furthermore, this embodiment uses LiDAR to accurately measure the device's position, dimensions, and relative position to other objects. Specifically, within a 70-meter range, LiDAR can help detect the status of substation equipment at greater distances. LiDAR's 2cm ranging accuracy provides more precise identification, which is crucial for identifying device shape, spatial configuration, and energized status.

[0078] In this implementation, the RGB camera and LiDAR data are combined to effectively fuse visual and spatial information. In practical applications, the RGB image provides surface information, while the LiDAR provides depth information and spatial location. This fusion can effectively improve the robustness and accuracy of the detection system, especially in complex environments (such as substations with multiple transformers), providing more efficient detection capabilities.

[0079] In practical applications, Intel's RealSense D435i camera can be used as the RGB camera, and Livox's Mid-360 lidar can be used as the lidar, effectively ensuring the accuracy and efficiency of spatial posture data measurement of substation equipment.

[0080] Implementation method three, see Figures 2 to 9 This embodiment further defines the camera and lidar fusion method for detecting substation equipment described in Embodiment 1. It employs an improved YOLOv8 object detection network model to identify busbars, switches, and other substation equipment. A convolutional attention module is added after the C2f module in the neck layer of the improved YOLOv8 object detection network model.

[0081] The convolutional attention module CBAM described in this embodiment integrates channel attention and spatial attention. At the same time, the convolutional attention module proposed in this embodiment has the characteristics of being lightweight and plug-and-play. The overall process of the convolutional attention module CBAM is as follows: Figure 3 As shown in the figure, first, the input is an intermediate feature map. The feature map is input into the Channel Attention Module to obtain channel attention, and then the attention weight is applied to the intermediate feature map. Then, the feature map with channel attention is input into the Spatial Attention Module to obtain spatial attention, and then the attention weight is applied to the feature map. Finally, after the serial operation of these two attention modules, the original feature map has been processed by both channel and spatial attention mechanisms, and the features are adaptively refined. The final improved YOLOv8 target detection network model is shown as follows: Figure 2 shown.

[0082] Channel attention module such as Figure 4As shown in the figure, the feature map is pooled in the spatial dimension to preserve channel feature information. In the convolutional attention module (CBAM), both global average pooling and global maximum pooling are used. First, the input feature map undergoes global maximum pooling and global average pooling, respectively, compressing the spatial dimension to 1 while preserving channel information. The two pooled features are then fed into a shared multi-layer perceptron (MLP) for feature extraction. Finally, the pooled features from the MLP are summed and activated with a sigmoid to obtain the final channel attention weights.

[0083] The spatial attention module is as follows Figure 5 As shown in the figure, the feature map is pooled in the channel dimension to preserve spatial feature information. First, the feature map (after channel attention calculation) is subjected to maximum pooling and average pooling in the channel dimension, compressing the channel dimension to 1 and preserving spatial information. The pooled features are then concatenated and passed through a convolutional layer to extract features while reducing the channel dimension to 1. Finally, a sigmoid layer is activated to obtain the final spatial attention weights (including channel attention).

[0084] After performing these two steps in series, the spatial attention feature is multiplied with the original feature map.

[0085] The comparison of the improved YOLOv8 target detection network model in the Map@0.5 index is as follows: Figure 6 and Figure 7 As shown in the F1 indicator, the comparison effect is as follows Figure 8 and Figure 9 It can be clearly seen that the improved YOLOv8 target detection network model has increased by 1.7% in the Map0.5 index and 1% in the F1 index, and has higher detection accuracy than the original YOLOv8 algorithm.

[0086] Implementation 4: This implementation further limits the method for detecting substation equipment by integrating a camera and a laser radar as described in Implementation 1. Step S3 includes:

[0087] The laser radar is transformed to the camera coordinate system through the rotation matrix R and the translation matrix T:

[0088]

[0089] Among them, X c Y c Z c is the camera coordinate system, X L Y L Z L is the laser radar coordinate system;

[0090] Convert the camera coordinate system to the image coordinate system:

[0091]

[0092] Where f is the focal length of the camera, (x, y) is the coordinate of a pixel in the image coordinate system;

[0093] The conversion relationship between the image coordinate system xoy and the pixel coordinate system uov is:

[0094]

[0095] Where dx and dy represent the dimensions of each pixel in the x-axis and y-axis directions of the image coordinate system; (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system;

[0096] The conversion relationship between the lidar coordinate system and the pixel coordinate system is as follows:

[0097]

[0098] Among them, K is the camera intrinsic parameter matrix, and M is the transformation matrix from the lidar coordinate system to the camera coordinate system.

[0099] In this embodiment, the laser radar coordinate system is transformed into the camera coordinate system by the rotation matrix R and the translation matrix T to ensure the spatial consistency of the laser radar data and the camera data. In practical applications, the sensors of the laser radar and the camera are usually in different coordinate systems. Through this transformation, the measurement results of the laser radar and the camera can be processed in the same coordinate system. Through coordinate transformation, the three-dimensional point cloud data captured by the laser radar can be accurately connected with the two-dimensional image data taken by the camera. In this way, not only can the target be identified by the visual information in the image, but the accuracy of the spatial information can also be improved by combining the laser radar data, especially in complex environments, such as low light or bad weather, where the visual effect of the camera may be limited, but the laser radar can still provide stable data.

[0100] Furthermore, the conversion from the camera coordinate system to the image coordinate system can map the three-dimensional data of the lidar to a two-dimensional image through the focal length and pixel coordinates (x, y), so that the depth information obtained by the lidar can be directly represented in the image. In this way, the system can directly mark the distance and position of the target object detected by the lidar on the two-dimensional image. In this way, the ability of depth perception can be enhanced. Furthermore, when converting between the image coordinate system and the pixel coordinate system, the physical scale of each pixel and the position of the origin of the image coordinate system in the pixel coordinate system are taken into account, which can accurately map to the actual size of the image.

[0101] Implementation 5: This implementation further limits the method for detecting substation equipment by integrating a camera and a laser radar as described in Implementation 1. Step S4 includes:

[0102] Perform voxel filtering on the original point cloud data and obtain a new point cloud dataset after downsampling;

[0103] Use the straight-through filter to extract the region of interest in the new point cloud data set and segment the ground point cloud data;

[0104] Eliminate outliers in ground point cloud data.

[0105] Perform voxel filtering on the original point cloud data and obtain a new point cloud dataset after downsampling, including:

[0106] Since the original point cloud data usually contains a large amount of redundant information, the use of voxel filtering in this embodiment can effectively reduce the density of the point cloud data by dividing the point cloud data into small voxel units and replacing all points in the area with the average value within the voxel. Combined with downsampling to obtain a new point cloud data set, it can greatly reduce the amount of calculation, improve the subsequent processing speed, and retain the main features of the point cloud. Furthermore, in this embodiment, a pass filter is also used to extract the area of ​​interest and segment the ground point cloud data. This can effectively focus on the relevant areas of the substation equipment and avoid processing irrelevant areas. By eliminating outliers in the ground point cloud data, the robustness and reliability of the detection are significantly improved.

[0107] Implementation 6: This implementation further limits the method for detecting substation equipment by integrating a camera and a laser radar as described in Implementation 1. Step S5 includes:

[0108] Use the Euclidean clustering algorithm to divide the point cloud data into different clusters;

[0109] The center point is fitted as the center point of the substation equipment by using the external bounding box method;

[0110] The spatial position data of the substation equipment is obtained through the center point.

[0111] In this embodiment, the Euclidean clustering algorithm is used to cluster the point cloud data, which can effectively remove noise and separate the different parts of the substation equipment from the surrounding environment. Through the clustering algorithm, the accuracy and robustness of the substation equipment detection can be improved, and false detection and missed detection can be reduced. Furthermore, in this embodiment, the external bounding box method is used to determine the center point of the substation equipment, which is both simple and efficient, and can accurately capture the boundary and spatial position of the substation equipment. Since the external bounding box method does not rely on complex geometric calculations, the computational complexity is small and the processing speed is fast, which is suitable for real-time or large-scale substation equipment detection tasks. The spatial position data of the substation equipment is obtained through the center point, and the three-dimensional spatial position and posture (including rotation angle) of the substation equipment can be accurately located. In this embodiment, through the combination of the Euclidean clustering algorithm and the external bounding box method, the spatial position information of the substation equipment can be efficiently and accurately extracted from the point cloud data. It has strong robustness, high processing efficiency, and strong adaptability, and is suitable for automatic detection and positioning of substation equipment in a variety of environments.

[0112] Implementation method seven, see Figure 10 、 Figure 11 and Figure 12 This embodiment further defines the method for detecting substation equipment using a camera and lidar integrated with the first embodiment, and employs neighborhood search and region growing methods to implement Euclidean clustering, including:

[0113] The neighborhood of a point pi is defined as the set of points whose Euclidean distance is less than a certain threshold ∈:

[0114]

[0115] The Euclidean distance between points pi and pj is:

[0116]

[0117] Use region growing to group points into clusters:

[0118] Randomly select an unassigned point pi from the point cloud as a seed point and initialize a new cluster Ck;

[0119] Add all points in the neighborhood of the seed point to cluster Ck;

[0120] For each point newly added to the cluster, continue searching its neighborhood until there are no points that meet the conditions;

[0121] The remaining unassigned points are processed until all points are assigned to a cluster or marked as noise points.

[0122] In this embodiment, point cloud data is divided into different clusters through neighborhood search and region growing method. Neighborhood search ensures that the points around each point can be effectively identified, avoiding the omission of clusters due to sparse or complex distribution of data points. The region growing method starts from the seed point and gradually expands the range of the cluster, which can ensure the clarity of the segmentation while maintaining the compactness and integrity of the cluster. By setting the Euclidean distance threshold ∈, it can automatically adapt to the characteristics of different point cloud data within a reasonable range, making the cluster allocation more flexible and accurate. The setting of different thresholds can effectively deal with various noise and data unevenness. The region growing method shows strong adaptability when processing clusters of different sizes and shapes, and is particularly suitable for detection tasks in complex environments. Furthermore, in this embodiment, all points that cannot be classified into the cluster will be marked as noise points, which can effectively distinguish between valid data and noise data, avoid false detection caused by noise data, and improve the accuracy of detection.

[0123] Embodiment 8: A substation detection system integrating a camera and a laser radar according to this embodiment includes:

[0124] A data acquisition unit, used to acquire images and point cloud data of the substation and its surroundings through an RGB camera and lidar;

[0125] The detection unit is used to perform two-dimensional target detection on the substation equipment using a target detection algorithm, extract the detection frame information of the substation equipment, and obtain the corresponding point cloud data of the substation equipment;

[0126] The calibration unit is used to jointly calibrate the camera and lidar to obtain the external parameters of the joint calibration; and to calibrate the camera using the Zhang Zhengyou calibration method to obtain the camera's internal parameters;

[0127] A filtering unit, used for filtering the acquired point cloud data;

[0128] A clustering unit is used to cluster the filtered point cloud data to obtain the spatial posture information of the substation equipment;

[0129] The judgment unit is used to project the processed point cloud onto the image through the joint calibration external parameter matrix, obtain the classification information and spatial posture information of the substation equipment, match the spatial posture information of the substation equipment with the field strength data detected by the electric field sensor, and judge whether the substation equipment is energized.

[0130] Embodiment 9. A computer device described in this embodiment includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method for visually detecting the energized state of the substation equipment according to any one of embodiments 1 to 7.

[0131] Implementation method 10. A computer-readable storage medium described in this implementation method stores a computer program, and when the computer program is executed by a processor, the steps of the method for visually detecting the energized state of a substation are executed as described in any one of implementation methods 1 to 7.

[0132] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and are not intended to limit its scope of protection. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the disclosed claims.

Claims

1. A method for detecting substation equipment integrating camera and laser radar, characterized in that: The method comprises: S1: Acquire images and point cloud data of substation equipment and surrounding environment through RGB camera and LiDAR; S2: Use the target detection algorithm to perform two-dimensional target detection on the substation equipment, extract the detection frame information of the substation equipment, and obtain the corresponding point cloud data of the substation equipment; S3: Jointly calibrate the camera and the lidar to obtain the external parameters of the joint calibration; and calibrate the camera using the Zhang Zhengyou calibration method to obtain the internal parameters of the camera; S4: filtering the acquired point cloud data; S5: clustering the filtered point cloud data to obtain the spatial posture information of the substation equipment; S6: The processed point cloud is projected onto the image through the joint calibration external parameter matrix to obtain the classification information and spatial posture information of the substation equipment, and the spatial posture information of the substation equipment is matched with the field strength data detected by the electric field sensor to determine whether the substation equipment is energized.

2. The method for detecting substation equipment by integrating camera and laser radar according to claim 1, characterized in that: The field of view of the RGB camera in step S1 is 69°*42°; the frame rate is 1920*1080; the range of the laser radar is 0.1~70m, and the ranging accuracy is 2cm.

3. The method for detecting substation equipment by integrating camera and laser radar according to claim 1, characterized in that: In the step S2, a target detection algorithm is used to identify substation equipment, including: using an improved YOLOv8 target detection network model to identify three types of substation equipment: busbar, knife switch and switch; adding a convolutional attention module after the C2f module of the neck layer in the improved YOLOv8 target detection network model.

4. The method for detecting substation equipment by integrating camera and laser radar according to claim 1, characterized in that: The step S3 comprises: The laser radar is transformed to the camera coordinate system through the rotation matrix R and the translation matrix T: Among them, X c Y c Z c is the camera coordinate system, X L Y L Z L is the laser radar coordinate system; Convert the camera coordinate system to the image coordinate system: Where f is the focal length of the camera, (x, y) is the coordinate of a pixel in the image coordinate system; The conversion relationship between the image coordinate system xoy and the pixel coordinate system uov is: Where, dx, dy represent the size of each pixel in the x-axis and y-axis directions of the image coordinate system; (u0, v0) represent the coordinates of the origin of the image coordinate system in the pixel coordinate system; The conversion relationship between the laser radar coordinate system and the pixel coordinate system is as follows: Among them, K is the camera intrinsic parameter matrix, and M is the transformation matrix from the lidar coordinate system to the camera coordinate system.

5. The method for detecting substation equipment integrating camera and laser radar according to claim 1, characterized in that: The step S4 comprises: Perform voxel filtering on the original point cloud data and obtain a new point cloud data set after downsampling; Use the straight-through filter to extract the region of interest in the new point cloud data set and segment the ground point cloud data; Eliminate outliers in ground point cloud data.

6. The method for detecting substation equipment by integrating camera and laser radar according to claim 1, characterized in that: The step S5 comprises: The Euclidean clustering algorithm is used to divide the point cloud data into different clusters; The center point is fitted by the external bounding box method as the center point of the substation equipment; The spatial position data of the substation equipment is obtained through the center point.

7. The method for detecting substation equipment by integrating camera and laser radar according to claim 1, characterized in that: Neighborhood search and region growing methods are used to implement Euclidean clustering.

8. A substation equipment detection system integrating camera and laser radar, characterized in that: The system comprises: A data acquisition unit, used to obtain images and point cloud data of the substation and its surroundings through an RGB camera and a lidar; A detection unit, used to perform two-dimensional target detection on the substation equipment using a target detection algorithm, extract detection frame information of the substation equipment, and correspondingly obtain point cloud data of the substation equipment; The calibration unit is used to jointly calibrate the camera and the laser radar to obtain the external parameters of the joint calibration; and to calibrate the camera using the Zhang Zhengyou calibration method to obtain the internal parameters of the camera; A filtering unit, used for filtering the acquired point cloud data; A clustering unit, used for clustering the filtered point cloud data to obtain the spatial posture information of the substation equipment; The judgment unit is used to project the processed point cloud onto the image through the joint calibration external parameter matrix, obtain the classification information and spatial posture information of the substation equipment, match the spatial posture information of the substation equipment with the field strength data detected by the electric field sensor, and judge whether the substation equipment is energized.

9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a substation equipment detection method integrating a camera and a laser radar according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a method for detecting substation equipment by integrating a camera and a laser radar as described in any one of claims 1 to 7.

Images