Infrared thermal image diagnosis and intelligent inspection robot control method and system for power equipment

Abstract

The invention provides a power equipment infrared thermal image diagnosis and intelligent inspection robot control method and system, and relates to the technical field of intelligent inspection, and the method comprises the steps: obtaining infrared thermal image data and robot pose information, extracting temperature anomaly features, and building a three-dimensional mapping relation; a robot track and posture sequence giving consideration to diagnosis precision and inspection efficiency is generated through multi-target optimization, and an anomaly recognition threshold and a constraint weight are adaptively adjusted according to feedback information, so that accurate positioning and efficient intelligent inspection of the anomaly of the power equipment are realized, and the safe operation level of the power equipment is improved.

Description

Technical Field

[0001] This invention relates to intelligent inspection technology, and more particularly to infrared thermal imaging diagnostics of power equipment and intelligent inspection robot control methods and systems. Background Technology

[0002] Condition monitoring and fault diagnosis of power equipment have always been core tasks of power system operation and maintenance. In recent years, with the rapid development of infrared thermal imaging technology, infrared thermal imaging has become an important means of condition monitoring and fault early warning for power equipment. Infrared thermal imaging technology can detect the surface temperature distribution of equipment non-contactly. By analyzing areas of abnormal temperature, it can effectively identify potential faults in power equipment, such as poor contact, insulation aging, and overload operation.

[0003] Traditional infrared inspection of power equipment mainly relies on manual handheld infrared thermal imagers, requiring inspectors to be on-site to photograph and analyze the equipment. With the continuous expansion of power systems and their increasing intelligence, intelligent inspection robots have been widely used in the field of power equipment inspection. Intelligent inspection robots can replace personnel in hazardous areas, working around the clock, improving inspection efficiency and safety. Intelligent inspection robots are typically equipped with multiple sensors such as infrared thermal imagers and visible light cameras, enabling them to navigate autonomously, collect data, and perform preliminary analysis.

[0004] In existing technologies, the integration of infrared thermal imaging data processing with robot pose information is not close enough, making it difficult to establish an accurate spatial mapping relationship for abnormal areas. This results in low positioning accuracy for areas with abnormal temperatures, affecting subsequent accurate diagnosis and treatment.

[0005] Traditional inspection path planning is mainly based on single-objective optimization such as coverage and path length. It lacks differentiated inspection strategies for abnormal areas and cannot effectively balance diagnostic accuracy and inspection efficiency. In particular, when anomalies are detected, it is difficult to automatically adjust the inspection strategy for focused observation.

[0006] Existing technologies lack effective feedback mechanisms and adaptive adjustment capabilities, making it impossible to dynamically optimize anomaly identification thresholds and constraint parameters based on actual inspection results. This results in difficulty in guaranteeing diagnostic accuracy and inspection efficiency in complex and ever-changing power equipment operating environments. Summary of the Invention

[0007] This invention provides a method and system for controlling an infrared thermal imaging diagnostic and intelligent inspection robot for power equipment, which can solve the problems in the prior art.

[0008] A first aspect of the present invention provides a control method for an infrared thermal imaging diagnostic and intelligent inspection robot for power equipment, comprising: Infrared thermal image data of the target power equipment and robot pose information are acquired. The temperature field distribution of the infrared thermal image data is analyzed, and the spatial coordinates and temperature gradient features of the temperature anomaly area are extracted to obtain an anomaly feature set. The spatial coordinates in the abnormal feature set are transformed and spatially registered with the robot pose information to establish a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the spatial association result is obtained. Based on the spatial correlation results and the preset equipment safety constraint rules, a robot motion trajectory and shooting posture sequence that takes into account both diagnostic accuracy and inspection efficiency are generated through multi-objective optimization. The multi-objective optimization uses the coverage integrity of the abnormal area as a hard constraint and the smoothness of robot joint motion and energy consumption as soft constraints to solve together and obtain control commands. The robot is driven to perform inspection actions according to the control instructions, collect supplementary infrared thermal image data, calculate the actual coverage and temperature change trend of abnormal areas, and generate feedback information. The feedback information is used to adaptively adjust the anomaly identification threshold in the temperature field distribution analysis and the constraint weights in the multi-objective optimization.

[0009] Infrared thermal image data of the target power equipment and robot pose information are acquired. Temperature field distribution analysis is performed on the infrared thermal image data to extract the spatial coordinates and temperature gradient features of temperature anomaly regions, resulting in a set of anomaly features including: Acquire infrared thermal image data of the target power equipment and simultaneously record the robot's pose information when acquiring the infrared thermal image data; The infrared thermal image data is analyzed for temperature field distribution. The temperature field distribution data is combined with the robot pose information to construct a three-dimensional temperature distribution model of the target power equipment. Based on the three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly regions, and the spatial coordinates and temperature gradient features of the temperature anomaly regions are extracted to generate an anomaly feature set.

[0010] Based on the aforementioned three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly regions, and the spatial coordinates and temperature gradient features of the temperature anomaly regions are extracted, including: Based on the three-dimensional temperature distribution data, a temperature gradient matrix is ​​constructed, and an adaptive region growing algorithm is used to analyze the temperature gradient matrix to determine the initial seed points for temperature anomaly regions. Using the initial seed point as the center, the temperature gradient difference value of the adjacent regions is calculated recursively and iteratively. When the temperature gradient difference value is greater than the dynamic threshold, the adjacent regions are included in the temperature anomaly region, thus obtaining the complete temperature anomaly region boundary. Extract the spatial coordinate information of the temperature anomaly region and calculate the temperature gradient characteristics within the temperature anomaly region.

[0011] The spatial coordinates in the abnormal feature set are transformed and spatially registered with the robot pose information to establish a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, resulting in spatial association results including: Obtain spatial coordinates and robot pose information from the abnormal feature set, convert the robot pose information into a homogeneous transformation matrix, and calculate the transformation parameters of the spatial coordinates from the image coordinate system to the robot coordinate system based on the homogeneous transformation matrix. The spatial coordinates are transformed according to the transformation parameters, and the transformed spatial coordinates are corrected for errors using a spatial registration algorithm to obtain the accurate spatial position in the robot coordinate system. The corrected spatial location is then correlated with the feature data in the set of abnormal features to form a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the spatial association result is output.

[0012] The spatial coordinates are transformed according to the transformation parameters, and the transformed spatial coordinates are corrected for errors using a spatial registration algorithm to obtain the accurate spatial position in the robot coordinate system, including: Calculate the rotation matrix and translation vector based on the transformation parameters, substitute the spatial coordinates into the rotation matrix and translation vector to perform coordinate system transformation, and obtain the initial transformed coordinates; A spatial point cloud is established based on the initial transformed coordinates. An error correction is performed on the spatial point cloud using a spatial registration algorithm to generate a corrected spatial position. The corrected spatial position is then mapped to the robot coordinate system to obtain the accurate spatial position in the robot coordinate system.

[0013] Based on the spatial correlation results and preset equipment safety constraints, a robot motion trajectory and shooting posture sequence that balances diagnostic accuracy and inspection efficiency is generated through multi-objective optimization, including: Based on the spatial correlation results, the detection point sequence of the abnormal area is extracted, the spatial distance between adjacent detection points in the detection point sequence is calculated, and the reachable path between detection points is determined in combination with the equipment safety constraint rules. Construct an inspection path graph based on the reachable path, calculate the node connectivity and path length in the inspection path graph, and generate an initial inspection path; The initial inspection path is optimized through multi-objective optimization, with detection coverage and total inspection time as optimization indicators, and a set of candidate paths is generated through cross-mutation operation. The fitness of the candidate path set is evaluated, the Pareto optimal solution is selected as the optimization result, and the robot motion trajectory and shooting posture sequence are output.

[0014] A second aspect of the present invention provides a control system for an infrared thermal imaging diagnostic and intelligent inspection robot for power equipment, comprising: The first module is used to acquire infrared thermal image data of the target power equipment and robot pose information, analyze the temperature field distribution of the infrared thermal image data, extract the spatial coordinates and temperature gradient features of the temperature anomaly area, and obtain an anomaly feature set. The second module is used to perform coordinate system transformation and spatial registration between the spatial coordinates in the abnormal feature set and the robot pose information, establish the three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and obtain the spatial association result. The third module is used to generate a robot motion trajectory and shooting posture sequence that takes into account both diagnostic accuracy and inspection efficiency based on the spatial correlation results and the preset equipment safety constraint rules through multi-objective optimization. The multi-objective optimization uses the coverage integrity of the abnormal area as a hard constraint and the smoothness of robot joint movement and energy consumption as soft constraints to solve together to obtain control commands. The fourth module is used to drive the robot to perform inspection actions according to the control instructions, collect supplementary infrared thermal image data, calculate the actual coverage and temperature change trend of abnormal areas, and generate feedback information. The fifth module is used to adaptively adjust the anomaly identification threshold in the temperature field distribution analysis and the constraint weights in the multi-objective optimization using the feedback information.

[0015] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0017] The beneficial effects of this application are as follows: By collaboratively processing infrared thermal imaging data and robot pose information, precise spatial positioning and feature extraction of abnormal temperature areas were achieved, improving the accuracy of power equipment fault diagnosis. Based on the three-dimensional mapping relationship between abnormal areas and the robot coordinate system, a spatial correlation model was established, solving the problem of difficulty in matching thermal imaging data with actual equipment positions in traditional inspection systems. Innovatively, multi-objective optimization is applied to inspection path planning. The integrity of abnormal area coverage is used as a hard constraint, while soft constraints such as robot joint motion smoothness and energy consumption are taken into account, thus achieving a balance between diagnostic accuracy and inspection efficiency. A real-time feedback mechanism is introduced to adaptively adjust the anomaly identification threshold and constraint weight by calculating the actual coverage and temperature change trend, giving the system environmental adaptability and continuous optimization capabilities. The overall solution organically combines infrared thermal imaging diagnostic technology with intelligent robot control, which significantly improves the intelligence level and fault early warning capability of power equipment inspection, reduces the cost of manual inspection, and enhances the operational safety of the power system. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the control method for infrared thermal imaging diagnosis and intelligent inspection robot of power equipment according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0021] Figure 1 This is a flowchart illustrating the control method for infrared thermal imaging diagnosis and intelligent inspection robot of power equipment according to an embodiment of the present invention. Figure 1 As shown, the method includes: Infrared thermal image data of the target power equipment and robot pose information are acquired. The temperature field distribution of the infrared thermal image data is analyzed, and the spatial coordinates and temperature gradient features of the temperature anomaly area are extracted to obtain an anomaly feature set. The spatial coordinates in the abnormal feature set are transformed and spatially registered with the robot pose information to establish a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the spatial association result is obtained. Based on the spatial correlation results and the preset equipment safety constraint rules, a robot motion trajectory and shooting posture sequence that takes into account both diagnostic accuracy and inspection efficiency are generated through multi-objective optimization. The multi-objective optimization uses the coverage integrity of the abnormal area as a hard constraint and the smoothness of robot joint motion and energy consumption as soft constraints to solve together and obtain control commands. The robot is driven to perform inspection actions according to the control instructions, collect supplementary infrared thermal image data, calculate the actual coverage and temperature change trend of abnormal areas, and generate feedback information. The feedback information is used to adaptively adjust the anomaly identification threshold in the temperature field distribution analysis and the constraint weights in the multi-objective optimization.

[0022] In one optional implementation, infrared thermal image data of the target power equipment and robot pose information are acquired. Temperature field distribution analysis is performed on the infrared thermal image data to extract the spatial coordinates and temperature gradient features of temperature anomaly regions, resulting in an anomaly feature set including: Acquire infrared thermal image data of the target power equipment and simultaneously record the robot's pose information when acquiring the infrared thermal image data; The infrared thermal image data is analyzed for temperature field distribution. The temperature field distribution data is combined with the robot pose information to construct a three-dimensional temperature distribution model of the target power equipment. Based on the three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly regions, and the spatial coordinates and temperature gradient features of the temperature anomaly regions are extracted to generate an anomaly feature set.

[0023] Monitoring the operational status of power equipment is a crucial aspect of ensuring the safe and stable operation of the power grid. Infrared thermal imaging technology, as a non-contact detection method, can effectively detect abnormal temperatures in power equipment. This embodiment provides a method for detecting abnormalities in power equipment based on infrared thermal imaging, enabling autonomous inspection by a robot, thereby improving detection efficiency and accuracy.

[0024] The system acquires infrared thermal image data of the target power equipment and simultaneously records the robot's pose information during the acquisition of this data. In practical applications, an inspection robot equipped with an infrared thermal imager is used to autonomously inspect the power equipment. The infrared thermal imager employs an uncooled microbolometer with a wavelength range of 8-14 μm, a temperature measurement range of -20°C to +150°C, and a temperature resolution better than 0.05°C. The acquisition frequency is set to 5 Hz to ensure sufficient thermal image data is acquired. Simultaneously, the robot's inertial measurement unit and visual positioning system record the robot's position coordinates (x, y, z) and attitude angles (pitch, yaw, roll) in real time, forming a pose matrix. Each frame of thermal image data corresponds to a set of pose information, and synchronization is achieved through timestamps to ensure a one-to-one correspondence between the thermal image data and the pose information.

[0025] Temperature field distribution analysis was performed on infrared thermal image data. Preprocessing of the raw thermal image data included noise filtering and temperature calibration. A median filtering algorithm was used to remove salt-and-pepper noise from the thermal image, with a filter window size of 3×3. Temperature calibration was based on the thermal imager's radiometric calibration model, combined with ambient temperature, atmospheric transmittance, and target emissivity for correction. Different components of power equipment exhibit varying emissivity; corresponding emissivity values ​​were set according to material characteristics, such as 0.6 for copper busbars, 0.4 for aluminum busbars, and 0.9 for ceramic insulators. A temperature field interpolation algorithm was used to convert discrete temperature points into a continuous temperature distribution field, employing bilinear interpolation to ensure the smoothness of the temperature field representation.

[0026] By combining temperature field distribution data with robot pose information, a three-dimensional temperature distribution model of the target power equipment is constructed. Based on single-view reconstruction technology, the temperature information in the two-dimensional thermal image is projected into three-dimensional space using known camera intrinsic and extrinsic parameter matrices. The camera intrinsic parameter matrix, including parameters such as focal length and principal point coordinates, is obtained through calibration. The extrinsic parameter matrix is ​​derived from the robot pose information and describes the camera's position and orientation in the world coordinate system. For each thermal image pixel (u, v), its three-dimensional coordinates (X, Y, Z) in the world coordinate system are calculated through projection transformation, combining its temperature value and depth information. When thermal image data of the same equipment is acquired from multiple perspectives, point cloud registration technology is used to fuse the temperature distributions from different perspectives. The registration process uses an iterative nearest-point algorithm to achieve accurate alignment by minimizing the distance error between point sets, with a registration accuracy better than 5 mm. The final generated three-dimensional temperature distribution model contains spatial geometric information and temperature attributes, accurately presenting the temperature state of various parts of the equipment.

[0027] Based on a three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly areas. Traditional fixed threshold methods are difficult to adapt to different equipment and environmental conditions; therefore, this method employs an adaptive threshold strategy. First, temperature assessment areas are divided according to the type of power equipment, such as busbar areas, circuit breaker areas, and cable areas for high-voltage switchgear. For each area, temperature statistical characteristics are calculated, including average temperature μ, standard deviation σ, and historical temperature rise data. The dynamic threshold Tth is set as μ + k × σ, where the coefficient k is determined based on the importance of the equipment and historical fault data, generally ranging from 2.0 to 3.5. For critical locations such as busbar connections, a smaller k value, such as 2.0, is set to improve detection sensitivity; for areas with large natural temperature differences, a larger k value, such as 3.5, is set to reduce the false alarm rate.

[0028] Spatial coordinates and temperature gradient features of temperature anomaly regions are extracted to generate an anomaly feature set. For each identified anomaly region, its geometric center coordinates (Xc, Yc, Zc) are first determined as the anomaly location identifier. The maximum temperature Tmax, area S, and average temperature Tavg of the anomaly region are calculated. Temperature gradient features reflect the spatial distribution characteristics of temperature changes. The Sobel operator is used to calculate the first-order temperature gradient within the anomaly region, obtaining the gradient magnitude G and direction θ. Based on the gradient direction distribution, different types of anomalies, such as point-like overheating and strip-like overheating, are distinguished. For anomalies at connecting points, the temperature difference ΔT between the two sides and the temperature transfer characteristics are calculated to analyze whether the anomaly is caused by loosening or oxidation. All features constitute the anomaly feature set, which includes the location, temperature, geometry, and gradient features of the anomaly.

[0029] In a practical application case, during the testing of a substation circuit breaker, the 3D temperature model showed a localized high-temperature point in the contact area, with the highest temperature reaching 78.5°C, while the average temperature of the surrounding equipment was 42.3°C. The dynamic threshold algorithm identified this area as an anomaly, and the extracted features showed: anomaly center coordinates (1.25m, 0.75m, 1.6m), highest temperature 78.5°C, and area of ​​12cm². 2 The maximum temperature gradient is 4.2°C / cm and is concentrated towards the contact joint. This characteristic indicates poor contact, leading to increased contact resistance and localized overheating, requiring a maintenance plan.

[0030] Through the above technologies, this method can accurately identify abnormal temperatures in power equipment, provide detailed information on the location and characteristics of the abnormality, and provide a reliable basis for equipment condition assessment and fault early warning.

[0031] In one optional implementation, based on the three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly regions, and the spatial coordinates and temperature gradient features of the temperature anomaly regions are extracted, including: Based on the three-dimensional temperature distribution data, a temperature gradient matrix is ​​constructed, and an adaptive region growing algorithm is used to analyze the temperature gradient matrix to determine the initial seed points for temperature anomaly regions. Using the initial seed point as the center, the temperature gradient difference value of the adjacent regions is calculated recursively and iteratively. When the temperature gradient difference value is greater than the dynamic threshold, the adjacent regions are included in the temperature anomaly region, thus obtaining the complete temperature anomaly region boundary. Extract the spatial coordinate information of the temperature anomaly region and calculate the temperature gradient characteristics within the temperature anomaly region.

[0032] Based on the acquired three-dimensional temperature distribution data, a temperature gradient matrix is ​​constructed. This matrix reflects the direction and magnitude of temperature changes at each point in space and is a crucial basis for identifying temperature anomaly regions. Specifically, for each point (x, y, z) in the three-dimensional temperature data, the rate of temperature change in the three directions is calculated, denoted as Gx, Gy, and Gz, respectively. The calculation formulas are as follows: For a point (x, y, z), Gx represents the rate of temperature change in the x-direction, equal to T(x+1, y, z) - T(x-1, y, z) divided by 2; Gy and Gz are calculated in a similar manner. Thus, each spatial point corresponds to a temperature gradient vector G(x, y, z) = [Gx, Gy, Gz], and its magnitude |G(x, y, z)| represents the severity of the temperature change at that point.

[0033] An adaptive region growing algorithm is used to analyze the temperature gradient matrix and determine the initial seed points for temperature anomaly regions. The algorithm first calculates the average value μ and standard deviation σ of the temperature gradient magnitude throughout the three-dimensional space. Then, an initial threshold T0 = μ + kσ is set, where k is an empirical coefficient, typically between 2.5 and 3.0. For points with a temperature gradient magnitude greater than T0, the top N points are selected as candidate seed points, sorted by gradient value from largest to smallest. To ensure the representativeness of the seed points, further screening is required. The neighborhood of each candidate seed point is analyzed to calculate the consistency of temperature gradient directions within the neighborhood. When at least 60% of the points in the neighborhood of a candidate point have similar temperature gradient directions, that point is considered a valid initial seed point.

[0034] Centered on an initial seed point, the temperature gradient difference between adjacent regions is recursively calculated to dynamically expand the temperature anomaly region. In each iteration, the 26-neighborhood (adjacent points in 3D space) of each point on the boundary of the current anomaly region is examined. For the boundary point P and its neighboring point Q, the temperature gradient difference value D(P, Q) = |G(P) - G(Q)| is calculated, where G(P) and G(Q) represent the temperature gradient vectors of points P and Q, respectively. Simultaneously, considering the scale variations present in the temperature anomaly region, the dynamic threshold Td adopts an adaptive adjustment strategy: Td = T0 × (1 - α × d / dmax), where d represents the distance from the current examined point to the nearest seed point, dmax represents the preset maximum consideration distance, and α is a decay coefficient with a value range of [0, 1]. When D(P, Q) is greater than the dynamic threshold Td, point Q is included in the temperature anomaly region. This dynamic threshold strategy can adapt to the gradient variations present within the temperature anomaly region, making region growth more accurate.

[0035] The iterative process continues until the termination condition is met: when the number of newly added points in three consecutive iterations is less than 1% of the total number of points in the current region, the region growth is considered to be basically complete, and the iteration terminates. In this way, the complete boundary of the temperature anomaly region is obtained.

[0036] After identifying the temperature anomaly region, its spatial coordinate information is extracted. For each point (x, y, z) within the anomaly region, its coordinates in the original three-dimensional space are recorded, forming a point cloud representation of the anomaly region. Simultaneously, spatial features such as the region's geometric center, volume, and surface area are calculated. These features aid in subsequent classification and evaluation of the anomaly region.

[0037] The temperature gradient characteristics within the temperature anomaly region are calculated. These characteristics include: average temperature gradient magnitude, maximum temperature gradient magnitude, principal components of the temperature gradient direction, and spatial distribution of gradient changes. Specifically, the mean and standard deviation of the temperature gradient magnitudes at all points within the region are calculated to analyze the severity of temperature changes; the principal direction of the temperature gradient vector within the region is calculated using principal component analysis to reflect the main paths of heat conduction; and the spatial distribution characteristics of the temperature gradient, such as the spatial autocorrelation of the gradient magnitudes, are calculated to reflect the homogeneity or complexity of the internal structure of the temperature anomaly region.

[0038] In practical applications, taking power equipment monitoring as an example, when performing infrared thermal imaging monitoring on power equipment in substations, three-dimensional temperature distribution data is acquired through multi-angle thermal imaging equipment. Using the above method, abnormal heating areas in parts such as transformer joints can be accurately identified, and the calculated temperature gradient characteristics can effectively distinguish between temperature distribution under normal operating conditions and abnormal heating states. Practice shows that this method has good identification effects on both large-area abnormal areas with slow temperature changes and small-area abnormal areas with drastic temperature changes, providing a reliable basis for equipment fault early warning.

[0039] In one optional implementation, the spatial coordinates in the abnormal feature set are transformed and spatially registered with the robot pose information to establish a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the resulting spatial association includes: Obtain spatial coordinates and robot pose information from the abnormal feature set, convert the robot pose information into a homogeneous transformation matrix, and calculate the transformation parameters of the spatial coordinates from the image coordinate system to the robot coordinate system based on the homogeneous transformation matrix. The spatial coordinates are transformed according to the transformation parameters, and the transformed spatial coordinates are corrected for errors using a spatial registration algorithm to obtain the accurate spatial position in the robot coordinate system. The corrected spatial location is then correlated with the feature data in the set of abnormal features to form a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the spatial association result is output.

[0040] The system acquires the spatial coordinates and robot pose information from the anomaly feature set. This anomaly feature set, typically generated by a vision inspection system, includes the target region's three-dimensional spatial coordinates (x_i, y_i, z_i) in the image coordinate system and corresponding feature descriptions. The robot pose information includes the position vector T = (t_x, t_y, t_z) of the robot's end effector in the robot's base coordinate system and the pose matrix R, which is provided in real-time by the robot control system.

[0041] Converting robot pose information into a homogeneous transformation matrix is ​​the fundamental step in coordinate system transformation. The homogeneous transformation matrix H is composed of a rotation matrix R and a translation vector T, and has the following form: H consists of a 3x3 rotation matrix R and a 3x1 translation vector T, with the fourth row being [0, 0, 0, 1]. This matrix describes the transformation relationship from the robot base coordinate system to the end effector coordinate system.

[0042] Calculating the transformation parameters from the image coordinate system to the robot coordinate system based on the homogeneous transformation matrix requires considering the relationship between the two coordinate systems. The transformation relationship between the camera coordinate system and the robot end effector coordinate system must be determined, which can be obtained through hand-eye calibration, yielding the transformation matrix H_camera_to_end. The complete transformation process includes transformation from the image coordinate system to the camera coordinate system, then to the robot end effector coordinate system, and finally to the robot base coordinate system.

[0043] Based on the calculated transformation parameters, the spatial coordinates are transformed into a coordinate system. For each spatial coordinate point (x_i, y_i, z_i) in the abnormal feature set, it is first expanded into homogeneous coordinates (x_i, y_i, z_i, 1), and then transformed into the robot base coordinate system through matrix multiplication.

[0044] The transformed coordinates contain systematic or random errors, thus requiring spatial registration algorithms for error correction. An effective method is the Iterative Closest Point (ICP) algorithm, which optimizes transformation parameters by minimizing the distance between two sets of points. In practice, several feature points with known coordinates in the environment are first selected as reference points, and their coordinates in both the image and robot coordinate systems are obtained. Then, the Euclidean distance between the initially transformed points and the reference points is calculated, and the transformation parameters are iteratively optimized to minimize these distances.

[0045] In practical applications, at least three non-collinear calibration points can be set up in the working area, and their coordinate values ​​in different coordinate systems can be measured to establish a correspondence. By solving the least squares problem, the parameters in the transformation matrix can be optimized, thereby improving the accuracy of coordinate transformation.

[0046] After coordinate correction, a correspondence is established between the corrected spatial locations and the feature data in the anomaly feature set. This step involves associating the type, severity, and geometric dimensions of each anomaly feature with its accurate spatial location in the robot coordinate system. The association can be stored as key-value pairs, with the spatial location as the key and the feature data as the value.

[0047] The three-dimensional mapping of the formed abnormal region in the robot coordinate system can be represented as a data structure, containing the spatial boundary, center position, coverage area, and corresponding feature attributes of the abnormal region. This mapping enables the robot to accurately locate the abnormal region and take appropriate processing measures based on the feature attributes.

[0048] The output spatial correlation results include spatial descriptions and feature attributes of the abnormal regions. The spatial descriptions can be in the form of 3D point clouds, 3D meshes, or bounding boxes, while the feature attributes include information such as anomaly type, confidence level, and processing suggestions.

[0049] In practice, filtering algorithms can be introduced to smooth the transformed coordinates and reduce the impact of noise. For example, for time-series acquired data, a Kalman filter can be applied to fuse results from multiple frames, improving the stability of position estimation.

[0050] Through the above steps, the coordinate system transformation and spatial registration of the spatial coordinates in the abnormal feature set and the robot pose information were realized, and an accurate three-dimensional mapping relationship of the abnormal region in the robot coordinate system was established, providing a spatial positioning basis for the robot to perform subsequent precise operations.

[0051] In one optional implementation, the spatial coordinates are transformed according to the transformation parameters, and the transformed spatial coordinates are corrected for errors using a spatial registration algorithm to obtain the accurate spatial position in the robot coordinate system, including: Calculate the rotation matrix and translation vector based on the transformation parameters, substitute the spatial coordinates into the rotation matrix and translation vector to perform coordinate system transformation, and obtain the initial transformed coordinates; A spatial point cloud is established based on the initial transformed coordinates. An error correction is performed on the spatial point cloud using a spatial registration algorithm to generate a corrected spatial position. The corrected spatial position is then mapped to the robot coordinate system to obtain the accurate spatial position in the robot coordinate system.

[0052] Obtain the transformation parameters, including rotation angle parameters and translation parameters. The rotation angle parameters are usually represented by Euler angles, which include the rotation angles around the x-axis, y-axis and z-axis, denoted as α, β and γ respectively. The translation parameters are represented as vectors (tx, ty, tz), which represent the translation amounts in the x, y and z directions respectively.

[0053] Based on the obtained Euler angles α, β, and γ, the rotation matrix R is calculated. R is a 3×3 matrix used to describe the rotation relationship between one coordinate system and another in space. Specifically, the basic rotation matrices Rx, Ry, and Rz about the x-axis, y-axis, and z-axis are calculated separately, and then these three basic rotation matrices are multiplied together to obtain the final rotation matrix R. The translation vector T is directly composed of the translation parameters (tx, ty, tz).

[0054] For a spatial coordinate point P(x, y, z) to be transformed, a coordinate system transformation is performed using a rotation matrix R and a translation vector T to obtain the initial transformed coordinates P'(x', y', z'). The calculation formula is P' = R·P + T, where R·P represents the matrix multiplication of the rotation matrix and the coordinate point. Adding the result to the translation vector T yields the initial transformed coordinates.

[0055] A spatial point cloud is built based on initial transformed coordinates. A spatial point cloud refers to a set of points distributed in three-dimensional space, where each point has its corresponding spatial coordinates. In practical applications, multiple feature points can be selected for transformation to form an initial transformed coordinate point cloud. These feature points can be obvious markers in the scene, corner points of objects, or other easily identifiable feature locations.

[0056] Spatial registration algorithms are used to correct errors in spatial point clouds. Commonly used spatial registration algorithms include the Iterative Closest Point (ICP) algorithm and the Random Sample Consensus (RANSAC) algorithm.

[0057] For each point in the target point cloud, find its nearest neighbor in the source point cloud to form corresponding point pairs. Then, calculate the rigid body transformation (including rotation and translation) that minimizes the sum of squared distances between these corresponding point pairs. Next, transform the source point cloud according to the calculated transformation. Finally, repeat the above steps until the convergence condition is met (e.g., the number of iterations reaches a set value, or the transformation amount is less than a threshold).

[0058] The ICP algorithm yields a corrected transformation matrix, which is used to correct errors in the initial transformation. Applying this corrected transformation matrix to the initial transformed coordinates gives the corrected spatial position.

[0059] The corrected spatial position is mapped to the robot coordinate system, which is typically defined by the robot's base as the origin, thus defining the robot's operating space. By transforming the corrected spatial position back into the robot coordinate system, the accurate spatial position within that system can be obtained.

[0060] In practical applications, such as robot grasping tasks, the spatial coordinates of the target object in the camera coordinate system are first obtained through the vision system. Then, the coordinates are transformed to the robot coordinate system using the method described above. Finally, the robot is controlled to perform the grasping action. Because errors in the coordinate transformation process are considered and corrected using a spatial registration algorithm, the accuracy of robot operations can be improved.

[0061] Another application scenario is robot navigation, which requires integrating environmental information acquired by different sensors (such as LiDAR, vision sensors, etc.) into the same coordinate system. Using the method described above, information from each sensor's coordinate system can be accurately converted to the robot's coordinate system, and spatial registration algorithms can eliminate deviations caused by factors such as sensor installation errors, thereby improving navigation accuracy.

[0062] In robot collaboration scenarios, multiple robots need to work together in the same workspace. The above method can convert the environmental information perceived by each robot into a unified coordinate system, and eliminate positional errors between robots through spatial registration algorithms, thereby achieving precise collaborative operation.

[0063] In summary, the method of calculating rotation matrices and translation vectors to perform initial coordinate transformation, then using spatial registration algorithms for error correction, and finally mapping the corrected spatial position to the robot coordinate system can effectively improve the accuracy of spatial coordinate transformation in robot systems, providing a foundation for precise robot operation.

[0064] In one optional implementation, based on the spatial correlation results and preset equipment safety constraint rules, a robot motion trajectory and shooting posture sequence that balances diagnostic accuracy and inspection efficiency is generated through multi-objective optimization, including: Based on the spatial correlation results, the detection point sequence of the abnormal area is extracted, the spatial distance between adjacent detection points in the detection point sequence is calculated, and the reachable path between detection points is determined in combination with the equipment safety constraint rules. Construct an inspection path graph based on the reachable path, calculate the node connectivity and path length in the inspection path graph, and generate an initial inspection path; The initial inspection path is optimized through multi-objective optimization, with detection coverage and total inspection time as optimization indicators, and a set of candidate paths is generated through cross-mutation operation. The fitness of the candidate path set is evaluated, the Pareto optimal solution is selected as the optimization result, and the robot motion trajectory and shooting posture sequence are output.

[0065] Based on spatial correlation results, a sequence of detection points for abnormal areas is extracted. In practical applications, spatial correlation results typically include the three-dimensional coordinate information of equipment anomaly hotspots. For an equipment inspection scenario in a power substation, several temperature anomaly points are identified from thermal imaging images, and the three-dimensional coordinates of these points are obtained using a depth camera, forming a sequence of detection points P={p1, p2, ..., p...}. n}, where each point p i It includes location coordinates (x, y, z) and anomaly type labels. For example, when an abnormal temperature is detected on the heat sink of a transformer, its three-dimensional coordinates are added to the detection point sequence.

[0066] Calculate the spatial distance between adjacent detection points in the detection point sequence, using Euclidean distance to calculate the distance between any two detection points p. i and p j Spatial distance d between ij Construct a distance matrix D. For inspection tasks involving multiple devices within a large substation, the distance matrix has a large dimension. In this case, spatial index structures such as KD-trees can be used to accelerate the search process for nearest neighbors.

[0067] The reachable paths between detection points are determined by combining equipment safety constraints, which include: equipment safety distance constraints, obstacle avoidance constraints, and robot motion constraints. Equipment safety distance constraints specify the minimum safe distance between the robot and high-voltage equipment; for example, for 110kV equipment, the safe distance is no less than 1.5 meters. Obstacle avoidance constraints ensure that the robot will not collide with obstacles on site. Robot motion constraints consider the robot's kinematic characteristics, such as maximum turning angle and climbing ability. Based on these constraints, it is determined whether a reachable path exists between any two detection points, and this is marked in the reachability matrix R, where R... ij =1 indicates that there is a reachable path from point i to point j, R ij =0 indicates unreachable.

[0068] Construct an inspection path graph G=(V, E) based on reachable paths, where the vertex set V represents the set of inspection points, and the edge set E represents the reachable paths between inspection points. For each edge e... ij Assign weight w ij , representing the path length from point i to point j. By analyzing the topological characteristics of the inspection path graph, the connectivity of each node is calculated, which is the number of edges directly connected to that node. Nodes with high connectivity are usually key turning points in the inspection path and should be given priority.

[0069] Based on the constructed inspection path graph, an initial inspection path is generated, and the nearest neighbor algorithm is used to construct the initial solution: starting from the starting point, the nearest unvisited inspection point is selected as the next inspection point, until all inspection points have been visited. Considering the special characteristics of power equipment inspection, priorities can be set according to the importance of the equipment. For high-priority inspection points, higher weights are assigned during path generation.

[0070] The initial inspection path is optimized through multi-objective optimization, defining two optimization objectives: detection coverage C and total inspection time T. Detection coverage represents the proportion of key detection points that can be observed by the inspection robot; total inspection time includes robot movement time and dwell time for image capture. The non-dominated sorting genetic algorithm NSGA-II is used for multi-objective optimization, with a population size of 50 and 100 iterations. In each iteration, new candidate paths are generated through crossover and mutation operations. The crossover operation uses the partially mapped crossover (PMX) method, and the mutation operation uses reverse mutation, that is, randomly selecting two positions in the path and reversing the order of the detection points between these two positions.

[0071] The fitness of the candidate path set is evaluated. For each candidate path S, its coverage C(S) and total duration T(S) are calculated. Coverage C(S) is defined as the ratio of the number of key detection points observable on path S to the total number of detection points; total duration T(S) is defined as the time required for the robot to complete the entire path, including movement time and time spent at each detection point. The optimal solution is selected from the Pareto front, i.e., the solution that achieves the best balance between coverage and time efficiency.

[0072] The optimization results are converted into a sequence of robot motion trajectories and camera poses. For each detection point in the path, the optimal observation position and camera orientation when the robot reaches that point are calculated. The observation position must consider safety distance constraints, and the camera orientation should ensure that the detection point is located at the center of the camera's field of view. The generated trajectory contains a series of path points {r1, r2, ..., r...}. n}, each path point r i This includes position coordinates (x, y, z), robot orientation angle, and camera parameters (such as pitch angle and focal length). These parameters together constitute a complete robot inspection execution plan, which can be directly used to guide the robot's actual inspection operations.

[0073] In a practical application at a substation, the inspection trajectory generated by the above method saves about 25% of the inspection time compared to manually planned paths, while ensuring a detection coverage rate of over 98%, effectively improving the efficiency and quality of power equipment inspection.

[0074] A second aspect of the present invention provides a control system for an infrared thermal imaging diagnostic and intelligent inspection robot for power equipment, comprising: The first module is used to acquire infrared thermal image data of the target power equipment and robot pose information, analyze the temperature field distribution of the infrared thermal image data, extract the spatial coordinates and temperature gradient features of the temperature anomaly area, and obtain an anomaly feature set. The second module is used to perform coordinate system transformation and spatial registration between the spatial coordinates in the abnormal feature set and the robot pose information, establish the three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and obtain the spatial association result. The third module is used to generate a robot motion trajectory and shooting posture sequence that takes into account both diagnostic accuracy and inspection efficiency based on the spatial correlation results and the preset equipment safety constraint rules through multi-objective optimization. The multi-objective optimization uses the coverage integrity of the abnormal area as a hard constraint and the smoothness of robot joint movement and energy consumption as soft constraints to solve together to obtain control commands. The fourth module is used to drive the robot to perform inspection actions according to the control instructions, collect supplementary infrared thermal image data, calculate the actual coverage and temperature change trend of abnormal areas, and generate feedback information. The fifth module is used to adaptively adjust the anomaly identification threshold in the temperature field distribution analysis and the constraint weights in the multi-objective optimization using the feedback information.

[0075] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0076] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0077] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control method for an infrared thermal imaging diagnostic and intelligent inspection robot for power equipment, characterized in that, include: Infrared thermal image data of the target power equipment and robot pose information are acquired. The temperature field distribution of the infrared thermal image data is analyzed, and the spatial coordinates and temperature gradient features of the temperature anomaly area are extracted to obtain an anomaly feature set. The spatial coordinates in the abnormal feature set are transformed and spatially registered with the robot pose information to establish a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the spatial association result is obtained. Based on the spatial correlation results and the preset equipment safety constraint rules, a robot motion trajectory and shooting posture sequence that takes into account both diagnostic accuracy and inspection efficiency are generated through multi-objective optimization. The multi-objective optimization uses the coverage integrity of the abnormal area as a hard constraint and the smoothness of robot joint motion and energy consumption as soft constraints to solve together and obtain control commands. The robot is driven to perform inspection actions according to the control instructions, collect supplementary infrared thermal image data, calculate the actual coverage and temperature change trend of abnormal areas, and generate feedback information. The feedback information is used to adaptively adjust the anomaly identification threshold in the temperature field distribution analysis and the constraint weights in the multi-objective optimization.

2. The method according to claim 1, characterized in that, Infrared thermal image data of the target power equipment and robot pose information are acquired. Temperature field distribution analysis is performed on the infrared thermal image data to extract the spatial coordinates and temperature gradient features of temperature anomaly regions, resulting in a set of anomaly features including: Acquire infrared thermal image data of the target power equipment and simultaneously record the robot's pose information when acquiring the infrared thermal image data; The infrared thermal image data is analyzed for temperature field distribution. The temperature field distribution data is combined with the robot pose information to construct a three-dimensional temperature distribution model of the target power equipment. Based on the three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly regions, and the spatial coordinates and temperature gradient features of the temperature anomaly regions are extracted to generate an anomaly feature set.

3. The method according to claim 2, characterized in that, Based on the aforementioned three-dimensional temperature distribution model, a dynamic temperature threshold algorithm is used to identify temperature anomaly regions, and the spatial coordinates and temperature gradient features of the temperature anomaly regions are extracted, including: Based on the three-dimensional temperature distribution data, a temperature gradient matrix is ​​constructed, and an adaptive region growing algorithm is used to analyze the temperature gradient matrix to determine the initial seed points for temperature anomaly regions. Using the initial seed point as the center, the temperature gradient difference value of the adjacent regions is calculated recursively and iteratively. When the temperature gradient difference value is greater than the dynamic threshold, the adjacent regions are included in the temperature anomaly region, thus obtaining the complete temperature anomaly region boundary. Extract the spatial coordinate information of the temperature anomaly region and calculate the temperature gradient characteristics within the temperature anomaly region.

4. The method according to claim 1, characterized in that, The spatial coordinates in the abnormal feature set are transformed and spatially registered with the robot pose information to establish a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, resulting in spatial association results including: Obtain spatial coordinates and robot pose information from the abnormal feature set, convert the robot pose information into a homogeneous transformation matrix, and calculate the transformation parameters of the spatial coordinates from the image coordinate system to the robot coordinate system based on the homogeneous transformation matrix. The spatial coordinates are transformed according to the transformation parameters, and the transformed spatial coordinates are corrected for errors using a spatial registration algorithm to obtain the accurate spatial position in the robot coordinate system. The corrected spatial location is then correlated with the feature data in the set of abnormal features to form a three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and the spatial association result is output.

5. The method according to claim 4, characterized in that, The spatial coordinates are transformed according to the transformation parameters, and the transformed spatial coordinates are corrected for errors using a spatial registration algorithm to obtain the accurate spatial position in the robot coordinate system, including: Calculate the rotation matrix and translation vector based on the transformation parameters, substitute the spatial coordinates into the rotation matrix and translation vector to perform coordinate system transformation, and obtain the initial transformed coordinates; A spatial point cloud is established based on the initial transformed coordinates. An error correction is performed on the spatial point cloud using a spatial registration algorithm to generate a corrected spatial position. The corrected spatial position is then mapped to the robot coordinate system to obtain the accurate spatial position in the robot coordinate system.

6. The method according to claim 1, characterized in that, Based on the spatial correlation results and preset equipment safety constraints, a robot motion trajectory and shooting posture sequence that balances diagnostic accuracy and inspection efficiency is generated through multi-objective optimization, including: Based on the spatial correlation results, the detection point sequence of the abnormal area is extracted, the spatial distance between adjacent detection points in the detection point sequence is calculated, and the reachable path between detection points is determined in combination with the equipment safety constraint rules. Construct an inspection path graph based on the reachable path, calculate the node connectivity and path length in the inspection path graph, and generate an initial inspection path; The initial inspection path is optimized through multi-objective optimization, with detection coverage and total inspection time as optimization indicators, and a set of candidate paths is generated through cross-mutation operation. The fitness of the candidate path set is evaluated, the Pareto optimal solution is selected as the optimization result, and the robot motion trajectory and shooting posture sequence are output.

7. A power equipment infrared thermal imaging diagnostic and intelligent inspection robot control system, used to implement the method of any one of claims 1-6, characterized in that, include: The first module is used to acquire infrared thermal image data of the target power equipment and robot pose information, analyze the temperature field distribution of the infrared thermal image data, extract the spatial coordinates and temperature gradient features of the temperature anomaly area, and obtain an anomaly feature set. The second module is used to perform coordinate system transformation and spatial registration between the spatial coordinates in the abnormal feature set and the robot pose information, establish the three-dimensional mapping relationship of the abnormal region in the robot coordinate system, and obtain the spatial association result. The third module is used to generate a robot motion trajectory and shooting posture sequence that takes into account both diagnostic accuracy and inspection efficiency based on the spatial correlation results and the preset equipment safety constraint rules through multi-objective optimization. The multi-objective optimization uses the coverage integrity of the abnormal area as a hard constraint and the smoothness of robot joint movement and energy consumption as soft constraints to solve together to obtain control commands. The fourth module is used to drive the robot to perform inspection actions according to the control instructions, collect supplementary infrared thermal image data, calculate the actual coverage and temperature change trend of abnormal areas, and generate feedback information. The fifth module is used to adaptively adjust the anomaly identification threshold in the temperature field distribution analysis and the constraint weights in the multi-objective optimization using the feedback information.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images