A method for inspecting a power system

Abstract

The application discloses a kind of power system's inspection method in the technical field of power system monitoring and maintenance, to solve the technical problem that the comprehensiveness and accuracy of inspection result are limited.It includes: first, the original point cloud data of target object is obtained, and the preliminary detection confidence is obtained by the original point cloud data, and the target that obviously does not meet the requirements is quickly filtered out, so as to improve the efficiency of overall inspection;According to the image data of target object, the second detection confidence of each target object is obtained, and the target that obviously does not meet the requirements is filtered out again, the efficiency of overall inspection is improved again, and the nature of image data and original point cloud data is not compatible, the comprehensiveness of screening is improved;Using convolutional neural network to fuse multiple images can combine the advantages of point cloud data and image data, and can adapt to different inspection scenes and needs;So as to reduce the risk of equipment failure and accident, and then meet the comprehensiveness and accuracy requirements of inspection result.

Description

Technical Field

[0001] This invention relates to a method for inspecting power systems, belonging to the field of power system monitoring and maintenance technology. Background Technology

[0002] As a crucial component of the power system, the safe and stable operation of substations is vital to ensuring the reliability of the power grid. Traditional substation inspection methods mainly rely on manual labor, which is not only time-consuming and labor-intensive, but also, due to limitations in human resources, makes it difficult to conduct a comprehensive and detailed inspection of the substation, easily overlooking potential fault points.

[0003] With the development of technology, automated inspection technology has begun to be applied to the daily maintenance of substations in order to improve inspection efficiency and accuracy. However, existing automated inspection technologies often rely on only a single information source, such as using only image data for fault diagnosis, which limits the comprehensiveness and accuracy of inspection results. Summary of the Invention

[0004] The purpose of this disclosure is to overcome the shortcomings of the prior art and provide a power system inspection method that improves the comprehensiveness and accuracy of inspection results.

[0005] To achieve the above objectives, this disclosure employs the following technical solution:

[0006] Firstly, this disclosure provides a method for inspecting a power system, including:

[0007] Obtain the original point cloud data of target objects in the power system scene to be inspected, and perform feature detection on the original point cloud data to obtain the preliminary detection confidence of each target object;

[0008] In response to the initial detection confidence reaching the first threshold, image data of the target object is acquired, and a second detection confidence of each target object is obtained based on the feature detection results of the image data of the target object.

[0009] In response to the second detection confidence reaching the second threshold, a pre-constructed and trained convolutional neural network is used to fuse multiple images of the target object to generate a fused feature map of the target object; the multiple images include at least point cloud data and image data;

[0010] The trained recognition model is used to perform behavior recognition on the fused feature map, thereby obtaining the inspection results.

[0011] In some embodiments of the first aspect, obtaining the preliminary detection confidence of each target object based on the raw point cloud data includes,

[0012] The raw point cloud data is filtered.

[0013] The DBSCAN clustering algorithm is used to segment the filtered point cloud data into different point cloud clusters, with each point cloud cluster representing a target object.

[0014] Use the PointNet++ model to extract features of point cloud clusters;

[0015] The initial detection confidence of the target object is labeled based on the features of each point cloud cluster.

[0016] In some embodiments of the first aspect, the step of acquiring image data of the target object in response to the preliminary detection confidence level reaching a first threshold includes:

[0017] Each target object is classified into its corresponding category based on the characteristics of the point cloud clusters;

[0018] Continuously scan the classified target objects and obtain their motion trajectories;

[0019] The initial trajectory of the target object is estimated by fusing filtered point cloud data from different times in the motion trajectory using an unscented Kalman filter.

[0020] The initial trajectory is optimized using the sliding window averaging method;

[0021] Estimate the new position of the target object based on the optimized trajectory;

[0022] Based on the new position of the target object, obtain the image data of the target object.

[0023] In some embodiments of the first aspect, the segmentation of the filtered point cloud data into different point cloud clusters using the DBSCAN clustering algorithm includes,

[0024] The optimal clustering radius and minimum number of points are found using an adaptive density radius algorithm;

[0025] Different point cloud clusters are obtained based on the optimal clustering radius and the minimum number of points;

[0026] Wherein: finding the optimal clustering radius and minimum number of points through the adaptive density radius algorithm includes:

[0027] The core object is obtained from the filtered point cloud data, and the optimal clustering radius is obtained by calculating the average distance from the core object to the nearest neighbor point cloud data point.

[0028] The local density of each core object is calculated using a kernel density estimation method. The formula for calculating the local density is as follows:

[0029] ;

[0030] In the formula, This refers to the i-th core object, where i is the core object's index. Let be the local density of the i-th core object, and j be the index of the point cloud data points of other non-core objects within the point cloud cluster where the core object is located. For the j-th non-core object point cloud data point, is the optimal clustering radius for the i-th core object, and n is the number of other non-core object point cloud data points within the point cloud cluster where the core object is located.

[0031] The minimum number of points is set as a function related to the local density, and the formula for obtaining the minimum number of points is as follows:

[0032] ;

[0033] in, The minimum number of points is obtained based on a function related to local density. This is a constant used to control the magnitude of changes in local density, so that the minimum number of points is positively correlated with the local density.

[0034] In some embodiments of the first aspect, prior to extracting features of point cloud clusters using the PointNet++ model,

[0035] The kernel size is adjusted based on the comparison between point cloud density and density threshold.

[0036] Using pyramid convolution, the PointNet++ model can perceive the point cloud structure within different ranges;

[0037] We use a weighted average to fuse the convolution results at different scales.

[0038] In some embodiments of the first aspect, the method of fusing multiple types of images of the target object using a pre-constructed and trained convolutional neural network to generate a fused feature map of the target object includes,

[0039] Obtain the original feature map of each image class, and extract fused features from the original feature map of each image class through independent convolution channels;

[0040] Spatial location features are obtained from the fused features extracted from the convolutional channels;

[0041] Map spatial location features to weights;

[0042] The original feature maps of each image class are weighted and fused with their corresponding weights to generate a fused feature map.

[0043] In some embodiments of the first aspect, the optimization of the initial trajectory using the sliding window averaging method includes,

[0044] Get the pre-set size of the sliding window;

[0045] Input the initial trajectory points from the initial trajectory into the sliding window, obtain the average trajectory points, and obtain the optimized trajectory based on the average trajectory points;

[0046] The formula for obtaining the average trajectory point is:

[0047] ;

[0048] In the formula, Let k be the average trajectory point, k be the pre-set size of the sliding window, and m be the range of the sliding window. ; ig is the index of the trajectory point, jj is the count within the sliding window, The initial trajectory point within the sliding window.

[0049] In some embodiments of the first aspect, obtaining the core object from the filtered point cloud data includes,

[0050] Starting from the bounding box of the entire filtered point cloud data, create a root node of a tree. The root node is recursively divided into eight child nodes, each representing an eighth of the subspace.

[0051] Continue segmenting until the number of point cloud data points contained in each leaf node is below a set value or reaches a set depth;

[0052] Assign each point cloud data point to its corresponding leaf node;

[0053] Non-leaf nodes are represented by the number and average of points within the leaf nodes;

[0054] Construct a grid structure where each grid cell corresponds to a leaf node of an octree;

[0055] Traverse the filtered point cloud data and check the leaf node to which each point cloud data point belongs;

[0056] Find the leaf node with the largest number of nodes and use the K-nearest neighbor algorithm to search for the core object.

[0057] In some embodiments of the first aspect, the convolutional neural network employs a residual path constructed from multiple layers of convolution and a single layer of convolution;

[0058] The original feature maps of each image class are used to extract fusion features through independent convolution channels, including extracting fusion features using the following expression:

[0059] ;

[0060] In the formula, As a feature of fusion, For activation function, The feature output is the original feature map after passing through multiple convolutional paths. is the feature output of the original feature map after a single-layer convolution, and i is the index of the core object in the filtered point cloud data.

[0061] In a second aspect, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the steps of the power system inspection method described in any embodiment of the first aspect.

[0062] Compared with the prior art, the beneficial effects achieved by this disclosure are as follows:

[0063] First, the raw point cloud data of the target objects is acquired, and an initial detection confidence score is obtained from the raw point cloud data to quickly filter out targets that obviously do not meet the requirements, thereby improving the overall inspection efficiency. Then, based on the image data of the target objects, a second detection confidence score is obtained for each target object, further filtering out obviously unqualified targets, improving the overall inspection efficiency again. Furthermore, the incompatibility between image data and raw point cloud data enhances the comprehensiveness of the filtering. Finally, a convolutional neural network is used to fuse multiple types of images, combining the advantages of point cloud data and image data to generate a more comprehensive and accurate fused feature map, thereby improving the accuracy of subsequent behavior recognition and adapting to different inspection scenarios and needs. This reduces the risk of equipment failure and accidents, improves the safety of the power system, and ultimately meets the requirements for the comprehensiveness and accuracy of the inspection results. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart illustrating the power system inspection method provided in this embodiment;

[0066] Figure 2 This is a schematic diagram of the structure of the power system inspection device provided in this embodiment;

[0067] In the diagram: 1. 3D LiDAR; 2. Gimbal camera; 3. Fusion module; 4. Control module; 5. Recognition module; 6. Image processing and confidence assessment module. Detailed Implementation

[0068] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0069] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0070] Example 1:

[0071] Figure 1 This is a flowchart of a power system inspection method according to Embodiment 1 of the present invention. This flowchart only illustrates the logical sequence of the method described in this embodiment. Provided there are no conflicts, different methods may be used in other possible embodiments of the present invention. Figure 1 Complete the steps shown or described in the order indicated.

[0072] The power system inspection method provided in this embodiment can be applied to a robot terminal and can be executed by an inspection robot. See also... Figure 1 and Figure 2 The method implemented in this way specifically includes the following steps:

[0073] Obtain the original point cloud data of target objects in the power system scene to be inspected, and perform feature detection on the original point cloud data to obtain the preliminary detection confidence of each target object;

[0074] In response to the initial detection confidence reaching the first threshold, image data of the target object is acquired, and a second detection confidence of each target object is obtained based on the feature detection results of the image data of the target object.

[0075] In response to the second detection confidence reaching the second threshold, a pre-constructed and trained convolutional neural network is used to fuse multiple images of the target object to generate a fused feature map of the target object; the multiple images include at least point cloud data and image data;

[0076] The trained recognition model is used to perform behavior recognition on the fused feature map, thereby obtaining the inspection results.

[0077] In existing technologies, robots performing automated power system inspections generally rely solely on gimbal vision cameras to identify targets. The output data volume of gimbal cameras is large, and maintaining high recognition accuracy at all times places high demands on computation. Furthermore, pure vision-based hazard identification solutions have poor scene adaptability. If the computational load of visual recognition is reduced, the accuracy of detection and identification will decrease.

[0078] The power system inspection method provided in this embodiment first acquires the original point cloud data of the target object for preliminary detection, and then acquires the image data of the target object for a second detection. Based on the two-layer target screening path, continuous inspection can be ensured in an environment with low computing resources. Furthermore, the power system inspection method provided in this embodiment uses feature maps generated by fusing multiple types of images to identify hazard sources, giving it strong scene adaptability.

[0079] refer to Figure 2 Furthermore, in terms of efficiency, the raw point cloud data of the target object is generally acquired through a 3D LiDAR 1. The rapid scanning capability of the 3D LiDAR 1 can acquire a large amount of raw point cloud data of the target object in a short time, thus greatly shortening the initial detection time. Point cloud data is also relatively easy to process, allowing for rapid acquisition of initial detection confidence levels. Based on preset first and second thresholds, target objects requiring further detection can be quickly filtered out. Image data of the target object is generally acquired through a pan-tilt camera 2, which can quickly respond to and capture image data of the target object, further improving the inspection response speed. Moreover, the raw point cloud data and image data have different properties, allowing for more comprehensive filtering of target objects. The pre-built and trained convolutional neural network enables rapid fusion of multiple image types, making the generation of fused feature maps more efficient and comprehensive.

[0080] In terms of accuracy, raw point cloud data contains precise three-dimensional information of the target object, providing an accurate data foundation for preliminary detection. Combined with image data of the target object, it can accurately assess the confidence levels of both the initial and secondary detections. Generating a fused feature map of the target object combines the advantages of both point cloud and image data, making behavior recognition based on the fused feature map more accurate and comprehensive, and accurately determining the behavioral state of the target object.

[0081] Furthermore, by combining various advanced technologies and algorithms, intelligent inspection of power systems has been achieved. The system can automatically complete tasks such as target object detection, image capture, data processing, and behavior recognition, reducing manual intervention and error rates. Through continuous learning and optimization, the system can gradually improve the accuracy and efficiency of inspections, adapting to different inspection scenarios and needs.

[0082] From a cost perspective, automated and intelligent inspection methods can reduce the workload and difficulty of manual inspections, thereby lowering operation and maintenance costs. Timely detection and handling of potential problems or anomalies can prevent equipment failures and accidents, reducing repair and replacement costs associated with equipment malfunctions. Through accurate and efficient inspections, this method can promptly identify potential problems or anomalies in the power system, preventing safety accidents caused by equipment failures. Intelligent inspections can also improve the operational efficiency and stability of the power system, further ensuring its safety.

[0083] Therefore, the power system inspection method provided in this embodiment, through efficient, accurate, and intelligent inspection methods, not only improves the operating efficiency and safety of the power system but also reduces operation and maintenance costs, providing a strong guarantee for the stable operation of the power system. It also meets the requirements for the comprehensiveness and accuracy of the inspection results.

[0084] As one embodiment, the initial detection confidence and the second detection confidence of each target object can be obtained by using the softmax function. In each of the two steps, only the output value of the softmax function is defined as the initial detection confidence or the second detection confidence.

[0085] Example 2

[0086] This embodiment is an optimization based on the above embodiments. For details not described in this embodiment, please refer to Embodiment 1.

[0087] The power system inspection method provided in this embodiment is applicable to substations, especially unattended substations located in remote areas or suburban areas.

[0088] In some embodiments, a 3D LiDAR 1 is used to acquire raw 3D point cloud data of the substation site. Optionally, the scanning range of the 3D LiDAR 1 is 0°~360°. By analyzing the raw point cloud data, target detection and tracking can be performed. The target detection stage identifies target objects in the LiDAR point cloud, such as personnel, vehicles, animals, etc.

[0089] As one embodiment, the preliminary detection confidence of each target object is obtained based on the raw point cloud data, including:

[0090] The raw point cloud data acquired from the 3D LiDAR 1 is filtered by using curvature downsampling to remove noise, outliers, and other useless information. The filtered point cloud data is then used as input, and an improved DBSCAN clustering algorithm is employed to segment it into different point cloud clusters, each representing a potential target object. The PointNet++ model is used to extract features from each point cloud cluster to describe the shape and attributes of the target object. Target detection is then performed based on the extracted point cloud cluster features, and the preliminary detection confidence score for each target object is marked.

[0091] The entire process, from data preprocessing to object detection, has been meticulously designed to ensure efficiency and accuracy. Through curvature downsampling filtering, improved DBSCAN clustering, PointNet++ feature extraction, and detection, this method effectively extracts target object information from complex raw point cloud data and performs accurate detection and confidence assessment. If multiple target objects exist within the detected region, each point cloud cluster is classified into its corresponding category, achieving semantic segmentation of the scene. Since target objects may be continuously moving, tracking analysis is required to trace their motion trajectory during continuous scanning.

[0092] The initial trajectory of the target object is estimated by fusing filtered point cloud data from different times within the motion trajectory using an unscented Kalman filter (UKF). For example, by fusing current point cloud observation data with previous point cloud tracking information, the new position of the target object can be estimated using an unscented Kalman filter (UKF), thus formulating an initial trajectory. Since the initial trajectory has poor accuracy, a sliding window averaging method is needed for post-processing and optimization. This involves removing unreasonable trajectories, adjusting the target's position, handling occlusions, and recording the location of dangerous situations and the initial detection confidence level. The target object is tracked based on the optimized trajectory, and an alarm is issued when the initial detection confidence level exceeds a threshold. The new position of the target object is estimated based on the optimized trajectory, and image data of the target object is acquired based on this new position.

[0093] As one embodiment, the sliding window averaging method can be improved by pre-setting the size of the sliding window; inputting the initial trajectory points from the initial trajectory into the sliding window to obtain the average trajectory points, and optimizing the trajectory based on the average trajectory points; wherein, the formula for obtaining the average trajectory points is:

[0094] ;

[0095] ;

[0096] In the formula, Let be the average trajectory points, k be the pre-set sliding window size, m be the range of the sliding window, ig be the count index of the trajectory points, and jj be the count index within the sliding window. The initial trajectory points are within the sliding window. After averaging within the sliding window, unreasonable trajectories are removed, the target position is adjusted, occlusion is handled, and the optimized trajectory is output. The root mean square error of the optimized trajectory and the initial trajectory is compared, and this is used as the trajectory confidence score.

[0097] It is also worth noting that the DBSCAN clustering algorithm processes the filtered point cloud data to obtain the core objects, dataset, minimum number of points, and optimal cluster radius. Then, based on the core objects, dataset, minimum number of points, and optimal cluster radius, it segments the filtered point cloud data into different point cloud clusters. As one embodiment, the method for obtaining the optimal cluster radius and minimum number of points can be improved: the core objects are pre-obtained from the filtered point cloud data, and the optimal cluster radius is obtained by calculating the average distance from the core objects to their nearest neighbor point cloud data points.

[0098] The local density of each core object is calculated using a kernel density estimation method. The formula for calculating the local density is as follows:

[0099] ;

[0100] In the formula, This refers to the i-th core object, where i is the core object's index. Let be the local density of the i-th core object, and j be the index of the point cloud data points of other non-core objects within the point cloud cluster where the core object is located. For the j-th non-core object point cloud data point, is the optimal clustering radius for the i-th core object, and n is the number of other non-core object point cloud data points within the point cloud cluster where the core object is located.

[0101] The minimum number of points is set as a function related to the local density, and the formula for obtaining the minimum number of points is as follows:

[0102] ;

[0103] in, The minimum number of points is obtained based on a function related to local density. This is a constant used to control the magnitude of changes in local density, so that the minimum number of points is positively correlated with the local density.

[0104] As one embodiment, the DBSCAN clustering algorithm can be improved to some extent:

[0105] The point cloud data is meshed using an octree structure to determine the initial core objects and point cloud sample set, and then the optimal clustering radius and minimum number of points are found through an adaptive density radius algorithm.

[0106] Specifically, starting from the bounding box of the entire filtered point cloud data, a root node of a tree is created, which is recursively divided into eight child nodes, each representing an eighth of a subspace.

[0107] Continue segmenting until the number of point cloud data points contained in each leaf node is below a set value or reaches a set depth;

[0108] Assign each point cloud data point to its corresponding leaf node;

[0109] Non-leaf nodes are represented by the number and average of points within the leaf nodes;

[0110] Construct a grid structure where each grid cell corresponds to a leaf node of an octree;

[0111] Traverse the filtered point cloud data and check the leaf node to which each point cloud data point belongs;

[0112] Find the leaf node with the largest number of nodes and use the K-nearest neighbor algorithm to search for the core object.

[0113] The octree structure provides an efficient way to organize and index point cloud data by recursively dividing the space into smaller subspaces (one-eighths of a subspace). This method allows the system to quickly locate specific regions in the point cloud data, thereby accelerating data retrieval and processing. Furthermore, the combination of the octree structure and the K-nearest neighbor algorithm improves the accuracy of object detection and clustering.

[0114] As mentioned earlier, the PointNet++ model is used to label the initial detection confidence of the target object based on the cluster features of the point cloud of each target object. As one embodiment, a multi-scale convolutional structure can be added to the PointNet++ model to replace the original SAM component with an MS-SAM component. Specifically, the convolutional kernel size is adjusted according to the comparison result of the point cloud density and the density threshold. Optionally, when the point cloud density is less than the density threshold, the convolutional size is smaller, while when the point cloud density reaches the density threshold, the convolutional size is larger.

[0115] Using pyramid convolution, the PointNet++ model can perceive the point cloud structure within different ranges;

[0116] We use a weighted average to fuse the convolution results at different scales.

[0117] By adjusting the convolutional kernel size based on a comparison between point cloud density and a density threshold, the model can more effectively process point cloud data of varying densities. In high-density regions, smaller convolutional kernels can be used to reduce computational cost and the risk of overfitting; in low-density regions, larger kernels can be used to capture more contextual information. The pyramidal convolutional structure enables the PointNet++ model to perceive point cloud structures across different ranges simultaneously.

[0118] This is particularly important for target object detection in power system inspections, as target objects may have different sizes and shapes. A multi-scale structure ensures that the model does not overlook details when detecting small objects, while also capturing the overall structure when dealing with large objects. By fusing the convolution results from different scales using a weighted average, a comprehensive feature representation containing multi-scale information can be generated. This feature representation includes not only the local details of the target object but also its overall structure and contextual information.

[0119] This is crucial for improving the accuracy and robustness of target detection. Because the MS-SAM component can generate richer and more accurate feature representations, it provides more reliable results when labeling the initial detection confidence of target objects. This helps reduce the possibility of false positives and false negatives, improving the efficiency and accuracy of power system inspections.

[0120] This significantly improves the accuracy and robustness of the PointNet++ model in target object detection during power system inspections. These technical features enable the model to better adapt to point cloud data of varying densities, perceive multi-scale structures, generate rich feature representations, and ultimately improve the accuracy of initial detection confidence.

[0121] Example 1 mentions using a convolutional neural network to fuse multiple types of images. As one example, based on this example, the original feature map of each type of image is obtained, and the original feature map of each type of image is used to extract fused features through independent convolution channels.

[0122] Spatial location features are obtained from the features extracted from the convolutional channels;

[0123] Map spatial location features to weights;

[0124] The original feature maps of each image class are weighted and fused with their corresponding weights to generate a fused feature map.

[0125] By setting up independent convolution channels for feature extraction in each image class, the features of each class can be effectively extracted based on its own characteristics. This avoids interference between features from different image classes and improves the accuracy and specificity of feature extraction.

[0126] Further spatial location features are obtained from the features extracted from the convolutional channels. This means that the fusion process considers not only the content information of the image but also the spatial relationships between various elements in the image. This is crucial for understanding the overall structure and layout of the target object, especially in power system inspection, where spatial location information can help distinguish different devices and components.

[0127] By mapping spatial location features to weights and then weighting and fusing the original feature maps of each image class with their corresponding weights, this method allows the system to adaptively adjust the contributions of different image classes during the fusion process based on the image content and spatial location information. This increases the flexibility and adaptability of feature fusion, enabling the system to better handle multi-class images under different conditions.

[0128] By weighted fusion of the original feature maps from different image types, the generated fused feature map contains richer and more comprehensive information. This fused feature map not only includes the content information of various images but also incorporates their spatial location information, providing a more reliable and accurate basis for subsequent object detection and behavior recognition.

[0129] The above technical solutions enable more effective integration of information from multiple image types, generating more accurate and comprehensive fusion feature maps. This helps improve the efficiency and accuracy of inspection methods, allowing the system to more quickly detect potential problems or anomalies in the power system and accurately determine their nature and scope of impact.

[0130] The above mentioned obtaining spatial location features from the features extracted from the convolutional channels. As one embodiment, the convolutional neural network constructs the residual path in the form of a main path consisting of multiple layers of convolution and a single layer of convolution.

[0131] The fusion formula for the extracted features from multi-layer convolution and single-layer convolution is as follows:

[0132] ;

[0133] In the formula, For the fusion result, For activation function, The feature output is the original feature map after multiple convolutions. is the feature output of the original feature map after a single-layer convolution, and i is the index of the core object in the point cloud data.

[0134] By designing a main path containing multiple convolutions, complex features in an image can be extracted progressively and deeply. Multi-level convolutions can capture multi-layered information in an image, from low-level edges and textures to high-level shapes and structures, thereby improving the accuracy and richness of feature extraction.

[0135] The design of residual paths in single-layer convolutions allows the network to learn residual mappings—that is, to learn the difference between the input and output, rather than directly learning the output itself. This approach accelerates the convergence speed during network training while reducing the risk of overfitting. Because residual learning allows the network to skip certain layers, it is easier to find the optimal solution.

[0136] Due to the introduction of residual paths, the network can retain more original information during propagation. Even as the network depth increases, gradient information can be effectively backpropagated due to the presence of residual connections, avoiding the problems of gradient vanishing or exploding. This makes the extracted features more robust and adaptable to various complex scenarios and lighting conditions that may occur in power system inspections.

[0137] By combining the main path and the residual path, a more efficient and high-performance convolutional neural network model can be constructed. This model structure maintains high performance while reducing computational complexity and the number of parameters, thus improving the model's real-time performance and generalization ability.

[0138] In summary, the power system inspection method provided in this embodiment improves detection efficiency and accuracy compared to Embodiment 1, thereby enhancing the safety of substations.

[0139] The target objects described in this embodiment include people, vehicles, animals, etc.

[0140] Optionally, the point cloud cluster feature extraction method includes: extracting the average position of all points in the cluster to determine the centroid, used to represent the approximate position of the target; describing the overall shape of the target using a rectangle or cube surrounding the target; calculating the volume and surface area of ​​the point cloud cluster to provide information on the size and shape of the target; and obtaining information describing the target's orientation by calculating the normal vectors of the points in the point cloud cluster.

[0141] The behavior recognition mentioned in Example 1 includes not wearing a safety helmet correctly, crossing the line / intruding, not wearing long-sleeved work clothes, smoke / fire on the site / equipment, and high temperature of the equipment. The pre-trained recognition model used by recognition module 5 includes a real-time detection, embedding, and tracking RT-DETR model, and is trained using a labeled multimodal image dataset.

[0142] Summary of this embodiment:

[0143] Example 1 mentions obtaining the preliminary detection confidence of each target object based on the original point cloud data. In this example, the original point cloud data is filtered; the DBSCAN clustering algorithm is used to divide the filtered point cloud data into different point cloud clusters, each representing a target object; the PointNet++ model is used to extract the features of the point cloud clusters; and the preliminary detection confidence of the target object is marked according to the features of each point cloud cluster.

[0144] Example 1 mentions that in response to the initial detection confidence reaching a first threshold, image data of the target object is acquired. In this example, each target object is classified into a corresponding category based on the characteristics of the point cloud clusters; the classified target objects are continuously scanned and their motion trajectories are obtained; the initial trajectory of the target object is estimated by fusing the filtered point cloud data from different times in the motion trajectory using an unscented Kalman filter; the initial trajectory is optimized using a sliding window averaging method; the new position of the target object is estimated based on the optimized trajectory; and the image data of the target object is acquired based on the new position of the target object.

[0145] As mentioned above, the DBSCAN clustering algorithm is used to segment the filtered point cloud data into different point cloud clusters. Specifically, the optimal cluster radius and minimum number of points are found using an adaptive density radius algorithm; different point cloud clusters are obtained based on the optimal cluster radius and minimum number of points. Finding the optimal cluster radius and minimum number of points using the adaptive density radius algorithm includes: extracting core objects from the filtered point cloud data; obtaining the optimal cluster radius by calculating the average distance from the core object to its nearest neighbor point cloud data points; and calculating the local density of each core object using a kernel density estimation method. The local density calculation formula is as follows: In the formula, This refers to the i-th core object, where i is the core object's index. Let be the local density of the i-th core object, and j be the index of the point cloud data points of other non-core objects within the point cloud cluster where the core object is located. For the j-th non-core object point cloud data point, Let be the optimal clustering radius for the i-th core object, and n be the number of other non-core object point cloud data points within the point cloud cluster where the core object is located. The minimum number of points is set as a function related to local density, and the formula for obtaining the minimum number of points is as follows: ;in, The minimum number of points is obtained based on a function related to local density. This is a constant used to control the magnitude of changes in local density, so that the minimum number of points is positively correlated with the local density.

[0146] As mentioned above, the PointNet++ model is used to extract features of point cloud clusters. Before this step, the convolution kernel size is adjusted based on the comparison between point cloud density and density threshold. Pyramid convolution is used to enable the PointNet++ model to perceive point cloud structures within different ranges. Weighted averaging is used to fuse the convolution results at different scales.

[0147] As mentioned above, a pre-built and trained convolutional neural network is used to fuse multiple types of images of the target object to generate a fused feature map of the target object. Specifically, the original feature map of each type of image is obtained, and the fused features of the original feature map of each type of image are extracted through independent convolution channels. Spatial location features are obtained from the fused features extracted from the convolution channels. The spatial location features are mapped to weights. The original feature map of each type of image is weighted and fused with the corresponding weights to generate a fused feature map.

[0148] The above text mentions using a sliding window averaging method to optimize the initial trajectory. Optionally, the size of a pre-set sliding window can be obtained; the initial trajectory points are input into the sliding window to obtain the average trajectory points, and the optimized trajectory is obtained based on the average trajectory points; the formula for obtaining the average trajectory points is as follows: In the formula, Let k be the average trajectory point, k be the pre-set size of the sliding window, and m be the range of the sliding window. ; ig is the index of the trajectory point, jj is the count within the sliding window, The initial trajectory point within the sliding window.

[0149] The above section mentioned extracting the core object from the filtered point cloud data. Optionally, starting from the bounding box of the entire filtered point cloud data, a root node of a tree is created. The root node is recursively divided into eight child nodes, each representing an eighth of a subspace. This division continues until the number of point cloud data points contained in each leaf node is below a set value or reaches a set depth. Each point cloud data point is assigned to its corresponding leaf node. Non-leaf nodes are represented using the number and average value of points within the leaf nodes. A mesh structure is constructed, where each mesh cell corresponds to a leaf node of the octree. The filtered point cloud data is traversed, and the leaf node to which each point cloud data point belongs is checked. The leaf node with the largest number of points is found, and the core object is determined using the K-nearest neighbor algorithm.

[0150] Furthermore, the convolutional neural network employs a residual path constructed from multiple convolutional layers and a single convolutional layer. The original feature maps of each image class are used to extract fused features through independent convolutional channels, including the extraction of fused features using the following expression: In the formula, As a feature of fusion, For activation function, The feature output is the original feature map after passing through multiple convolutional paths. is the feature output of the original feature map after a single-layer convolution, and i is the index of the core object in the filtered point cloud data.

[0151] Example 3

[0152] refer to Figure 2 This embodiment provides a power system inspection device, including:

[0153] 3D LiDAR 1 is used to acquire the raw point cloud data of the target object;

[0154] Gimbal camera 2 is used to acquire image data of the target object;

[0155] The fusion module 3 is used to fuse multiple types of images using a convolutional neural network to generate a fused feature map. The sources of the multiple types of images include the 3D LiDAR 1 and the gimbal camera 2.

[0156] Control module 4 is used to control the sensor to acquire images of the target object. The sensor includes gimbal camera 2.

[0157] Recognition module 5 is used to perform behavior recognition on the fused feature map using the trained recognition model;

[0158] The image processing and confidence judgment module 6 is used to obtain the initial detection confidence of each target object based on the original point cloud data, and to obtain the second detection confidence of each target object based on the image data.

[0159] Optionally, in a further, but non-limiting, embodiment of this application, the PTZ camera 2 includes a camera, an infrared thermal imaging camera, an ultraviolet camera, and an inertial measurement unit. The infrared thermal imaging camera is used for high-temperature detection, and the inertial measurement unit is used to provide PTZ rotation angle information, enabling the PTZ to move to the correct position and correcting the cumulative error caused by long-term PTZ movement. Laser point cloud technology can provide high-precision three-dimensional spatial data, which helps to accurately identify the location and shape of substation equipment; image information can be used to capture detailed surface conditions of equipment; and temperature information can reflect abnormal hot spots in the equipment's operating state, serving as an important indicator for predicting equipment failures. Therefore, integrating this multi-modal information and analyzing and processing it through multi-modal information fusion technology will greatly improve the accuracy and comprehensiveness of substation inspections.

[0160] Optionally, all images acquired by the sensors undergo preprocessing operations, including image correction, denoising, alignment, and normalization.

[0161] In one application method, the power system inspection device is mounted on a robot. During inspection, the mobile robot moves along a preset route and drives a 3D LiDAR 1 to perform preliminary detection of dangerous behaviors. When a dangerous behavior is detected, the mobile robot stops moving and drives a gimbal camera 2 to direct the image acquisition unit's field of view towards the location of the dangerous behavior for a second detection. After confirming the presence of a dangerous behavior, an alarm message is issued, and the location and trajectory of the danger are monitored in real time. Movement resumes after the danger has subsided.

[0162] The power system inspection device provided in this embodiment of the invention can execute the power system inspection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method, which will not be described in detail here.

[0163] Example 4

[0164] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the power system inspection method provided in Embodiment 1 or 2:

[0165] Obtain the original point cloud data of target objects in the power system scene to be inspected, and obtain the preliminary detection confidence of each target object based on the original point cloud data;

[0166] In response to the initial detection confidence reaching the first threshold, image data of the target object is acquired, and a second detection confidence of each target object is obtained based on the image data of the target object.

[0167] In response to the second detection confidence reaching the second threshold, a pre-built and trained convolutional neural network is used to fuse multiple images of the target object to generate a fused feature map of the target object; the multiple images include at least point cloud data and image data;

[0168] The trained recognition model is used to perform behavior recognition on the fused feature map, thereby obtaining the inspection results.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0174] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0175] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for inspecting a power system, characterized in that, include: Obtain the original point cloud data of target objects in the power system scene to be inspected, and perform feature detection on the original point cloud data to obtain the preliminary detection confidence of each target object; In response to the initial detection confidence reaching the first threshold, image data of the target object is acquired, and a second detection confidence of each target object is obtained based on the feature detection results of the image data of the target object. In response to the second detection confidence reaching the second threshold, a pre-constructed and trained convolutional neural network is used to fuse multiple images of the target object to generate a fused feature map of the target object; the multiple images include at least point cloud data and image data; The trained recognition model is used to perform behavior recognition on the fused feature map, thereby obtaining the inspection results; The preliminary detection confidence score for each target object is obtained based on the original point cloud data, including: The raw point cloud data is filtered; The DBSCAN clustering algorithm is used to segment the filtered point cloud data into different point cloud clusters, with each point cloud cluster representing a target object. Use the PointNet++ model to extract features of point cloud clusters; The initial detection confidence of the target object is marked based on the features of each point cloud cluster; The DBSCAN clustering algorithm is used to segment the filtered point cloud data into different point cloud clusters, including: The optimal clustering radius and minimum number of points are found using an adaptive density radius algorithm; Different point cloud clusters are obtained based on the optimal clustering radius and the minimum number of points; Wherein: finding the optimal clustering radius and minimum number of points through the adaptive density radius algorithm includes: The core object is obtained from the filtered point cloud data, and the optimal clustering radius is obtained by calculating the average distance from the core object to the nearest neighbor point cloud data point. The local density of each core object is calculated using a kernel density estimation method. The formula for calculating the local density is as follows: ； In the formula, This refers to the i-th core object, where i is the core object's index. Let be the local density of the i-th core object, and j be the index of the point cloud data points of other non-core objects within the point cloud cluster where the core object is located. For the j-th non-core object point cloud data point, is the optimal clustering radius for the i-th core object, and n is the number of other non-core object point cloud data points within the point cloud cluster where the core object is located. The minimum number of points is set as a function related to the local density, and the formula for obtaining the minimum number of points is as follows: ； in, The minimum number of points is obtained based on a function related to local density. This is a constant used to control the magnitude of changes in local density, so that the minimum number of points is positively correlated with the local density.

2. The power system inspection method according to claim 1, characterized in that, The step of acquiring image data of the target object in response to the initial detection confidence level reaching a first threshold includes: Each target object is classified into its corresponding category based on the characteristics of the point cloud clusters; Continuously scan the classified target objects and obtain their motion trajectories; The initial trajectory of the target object is estimated by fusing filtered point cloud data from different times in the motion trajectory using an unscented Kalman filter. The initial trajectory is optimized using the sliding window averaging method; Estimate the new position of the target object based on the optimized trajectory; Based on the new position of the target object, obtain the image data of the target object.

3. The power system inspection method according to claim 1, characterized in that, Before using the PointNet++ model to extract features of point cloud clusters... The kernel size is adjusted based on the comparison between point cloud density and density threshold. Using pyramid convolution, the PointNet++ model can perceive the point cloud structure within different ranges; We use a weighted average to fuse the convolution results at different scales.

4. The power system inspection method according to claim 1, characterized in that, The method involves fusing multiple types of images of the target object using a pre-constructed and trained convolutional neural network to generate a fused feature map of the target object, including: Obtain the original feature map of each image class, and extract fused features from the original feature map of each image class through independent convolution channels; Spatial location features are obtained from the fused features extracted from the convolutional channels; Map spatial location features to weights; The original feature maps of each image class are weighted and fused with their corresponding weights to generate a fused feature map.

5. The power system inspection method according to claim 2, characterized in that, The initial trajectory is optimized using the sliding window averaging method. include, Get the pre-set size of the sliding window; Input the initial trajectory points from the initial trajectory into the sliding window, obtain the average trajectory points, and obtain the optimized trajectory based on the average trajectory points; The formula for obtaining the average trajectory point is: ； In the formula, Let k be the average trajectory point, k be the pre-set size of the sliding window, and m be the range of the sliding window. ; ig is the index of the trajectory point, jj is the count within the sliding window, The initial trajectory point within the sliding window.

6. The power system inspection method according to claim 1, characterized in that, The process of obtaining the core object from the filtered point cloud data includes: Starting from the bounding box of the entire filtered point cloud data, create a root node of a tree. The root node is recursively divided into eight child nodes, each representing an eighth of the subspace. Continue segmenting until the number of point cloud data points contained in each leaf node is below a set value or reaches a set depth; Assign each point cloud data point to its corresponding leaf node; Non-leaf nodes are represented by the number and average of points within the leaf nodes; Construct a grid structure where each grid cell corresponds to a leaf node of an octree; Traverse the filtered point cloud data and check the leaf node to which each point cloud data point belongs; Find the leaf node with the largest number of nodes and use the K-nearest neighbor algorithm to search for the core object.

7. The power system inspection method according to claim 4, characterized in that, The convolutional neural network employs residual paths constructed from multiple layers of convolution and single layers of convolution. The original feature maps of each image class are used to extract fusion features through independent convolution channels, including extracting fusion features using the following expression: ； In the formula, As a feature of fusion, For activation function, The feature output is the original feature map after passing through multiple convolutional paths. is the feature output of the original feature map after a single-layer convolution, and i is the index of the core object in the filtered point cloud data.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the power system inspection method according to any one of claims 1 to 7.

Images