An intelligent inspection method of a robot for power equipment inspection

Abstract

The present application relates to the field of power equipment inspection, and particularly relates to an intelligent inspection method of a robot for power equipment inspection, comprising: performing feature coding on a temperature image, a corona discharge intensity image, a dangerous gas concentration value and an obstacle grid map of the robot through a multi-branch convolutional neural network to generate temperature image coding features, corona discharge coding features, dangerous gas features and space features; performing feature fusion convolution modeling based on an attention mechanism and deformable convolution on the temperature image coding features, the corona discharge coding features, the dangerous gas features and the space features to generate an inspection dangerous thermal point grid map; and performing robot inspection path planning based on the inspection dangerous thermal point grid map. The present application performs accurate assessment on the hidden risk condition of power equipment based on the multi-sensor data of the power inspection robot, so as to ensure the safety of the inspection path of the robot.

Description

Technical Field

[0001] This invention relates to the field of power equipment inspection, and more particularly to an intelligent inspection method using a robot for power equipment inspection. Background Technology

[0002] Power equipment is the cornerstone of the safe and stable operation of the energy system. In particular, key scenarios such as substations, power distribution areas, and high-voltage transmission corridors are widely equipped with core power equipment such as transformers, GIS switchgear, and circuit breakers. These scenarios often have safety hazards such as flammable and explosive gases, high-voltage corona discharge, and high-temperature heating of equipment, and are typical high-risk explosion-proof operating environments. Therefore, explosion-proof power equipment inspection robots are being gradually applied to replace manual inspections.

[0003] With the development of robotics and intelligent sensing technologies, power inspection robots have gradually replaced manual inspections, becoming core equipment for the operation and maintenance of equipment in high-risk power scenarios. Power inspection robots can be equipped with various types of sensing devices such as infrared thermal imagers, ultraviolet imagers, gas sensors, and lidar, enabling automated inspections without human contact and effectively avoiding the safety risks of manual inspections.

[0004] Power inspection robots only inspect and identify low-risk items such as local overheating, corona discharge, insulation deterioration, and dielectric leakage. For complex high-risk items, they must be reported in a timely manner and the robot must be detoured to ensure its inspection safety.

[0005] However, the multi-sensor inspection solution for power grid inspection robots relies on independent analysis of multi-source data and simple superposition of results, failing to achieve deep feature-level fusion of heterogeneous data. Infrared and ultraviolet image data, gas concentration numerical data, and lidar spatial grid data are heterogeneous multimodal data. Existing technologies cannot effectively correlate and complement the features of different modal data, cannot accurately bind fault characteristics with spatial location information, and struggle to conduct comprehensive risk assessments of coupled composite faults such as overheating, discharge, and gas leakage. This can easily lead to high-risk projects still being inspected, compromising inspection safety.

[0006] Therefore, how to accurately assess the hidden risks of power equipment based on multi-sensor data from power inspection robots in order to ensure the safety of the robot's inspection path is a technical problem that needs to be solved. Summary of the Invention

[0007] To address this, the present invention provides an intelligent inspection method for robots used in power equipment inspection, which overcomes the problem in the prior art that it is impossible to accurately assess the hidden risks of power equipment based on the multi-sensor data of power inspection robots, and that it is impossible to guarantee the safety of the robot's inspection path.

[0008] To achieve the above objectives, this invention proposes an intelligent inspection method for a robot used in the inspection of power equipment, comprising:

[0009] The system acquires temperature images generated by the infrared thermal imager, corona discharge intensity images generated by the ultraviolet imager, hazardous gas concentration values ​​collected by the gas sensor, and obstacle grid maps generated by the lidar of the explosion-proof power equipment inspection robot.

[0010] The temperature image, corona discharge intensity image, hazardous gas concentration value, and obstacle grid image are encoded using a multi-branch convolutional neural network to generate temperature image encoding features, corona discharge encoding features, hazardous gas features, and spatial features.

[0011] The temperature image coding features, corona discharge coding features, hazardous gas features, and spatial features are used to generate a grid map of hazardous thermal points for inspection through a feature fusion convolution model based on attention mechanism and deformable convolution.

[0012] The inspection path of the robot is planned based on the grid map of the inspection hazards.

[0013] Furthermore, the process of generating temperature image coding features, corona discharge coding features, hazardous gas features, and spatial features includes:

[0014] The temperature image is passed through a first convolution branch to generate temperature image encoded features;

[0015] The corona discharge intensity image is passed through a second convolution branch to generate corona discharge encoded features;

[0016] The concentration values ​​of the hazardous gas are passed through a multilayer sensor to generate hazardous gas characteristics;

[0017] The obstacle grid is passed through a third convolutional branch to generate spatial features;

[0018] The multi-branch convolutional neural network includes a first convolutional branch, a second convolutional branch, a multilayer perceptron, and a third convolutional branch.

[0019] Furthermore, the process of generating a grid map of inspection hazard heat points through a feature fusion convolution model includes:

[0020] The temperature image encoding features and hazardous gas features are used through an attention mechanism to generate space gas features;

[0021] The temperature image coding features, corona discharge coding features, spatial gas features, and the first concatenated vector of spatial features are fused through a convolutional block to generate the inspection hazard thermal point raster map.

[0022] The feature fusion convolution model includes an attention mechanism and a fusion convolution block, wherein the fusion convolution block includes deformable convolutional layers.

[0023] Furthermore, the process of generating a grid map of hazardous heat points by fusing convolutional blocks includes:

[0024] The first concatenated vector is passed sequentially through a first convolutional layer, a first normalization layer, a second convolutional layer, and a second normalization layer to generate initial fused features;

[0025] The second concatenated vector of the initial fused features and spatial features is passed through a deformable convolutional layer to generate the final fused features;

[0026] The final fused features are passed through the output convolutional layer to generate the inspection hazard heat map.

[0027] The fused convolutional block further includes a first convolutional layer, a first normalization layer, a second convolutional layer, a second normalization layer, and an output convolutional layer.

[0028] Furthermore, the process of generating space gas features through attention mechanisms includes:

[0029] The third concatenated vector of the temperature image encoding features and hazardous gas features is processed through an attention mechanism to generate guiding attention weights.

[0030] The hazardous gas features are weighted based on the guided attention weights to generate space gas features.

[0031] Furthermore, intelligent inspection methods also include:

[0032] A combined loss function is constructed based on MSE, SmoothL1Loss, and multi-scale gradient consistency loss term, and the combined loss function is used to train and optimize multi-branch convolutional neural networks and feature fusion convolutional models.

[0033] Furthermore, the process of constructing the combined loss function includes:

[0034] The hazardous heat map and the actual hazardous point grid labels are respectively downsampled by the first average pooling to generate the first predicted gradient value and the first actual gradient value.

[0035] The first predicted gradient value and the first true gradient value are respectively downsampled by the second average pooling to generate the second predicted gradient value and the second true gradient value.

[0036] The first predicted gradient value and the first true gradient value are passed through the first gradient operator, and the first difference is obtained. The second predicted gradient value and the second true gradient value are passed through the second gradient operator, and the second difference is obtained.

[0037] The first difference and the second difference are averaged to calculate the multi-scale gradient consistency loss term;

[0038] The hazardous heat map and the actual hazardous point grid labels are respectively processed using the root mean square error function and the SmoothL1Loss function to calculate MSE and SmoothL1Loss.

[0039] The combined loss function is constructed by weighting and summing the MSE, SmoothL1Loss, and multi-scale gradient consistency loss term.

[0040] Furthermore, the process of planning the robot's inspection path based on the aforementioned inspection hazard heat map includes:

[0041] Based on the obstacle grid, the node-based risk cost and task-inspired cost are calculated; based on the inspection hazard heat map, the generated risk-inspired cost is calculated.

[0042] Based on the node-based risk cost, task-inspired cost, and risk-inspired cost, a hazard heuristic function is calculated, and the hazard heuristic function is used through the AStar path planning model to generate the inspection path for the explosion-proof power equipment inspection robot.

[0043] Furthermore, the process of calculating the risk heuristic cost based on the inspected hazard heat map grid includes:

[0044] The grid map of the inspection hazard heat points and the candidate moving points are used to generate the node hazard direction gradient value through the Sobel operator;

[0045] Calculate the maximum value of the product of the node's dangerous direction gradient value and the candidate movement point and zero, and then sum the maximum value with the node's dangerous direction gradient value using weighted summation to calculate the risk velocity function in each direction;

[0046] The risk heuristic cost is generated by passing the anisotropic risk velocity function through a fast travel method.

[0047] Furthermore, the process of generating risk heuristic costs from the anisotropic risk velocity function using the fast travel method includes:

[0048] Calculate the minimum estimated cumulative risk cost of neighboring points in the x-direction and the minimum estimated cumulative risk cost of neighboring points in the y-direction based on the current grid points;

[0049] Calculate the initial reference velocity based on the gradient value of the dangerous direction of the node;

[0050] The initial iterative risk estimate is calculated based on the minimum cumulative risk cost estimate of neighboring points in the y-direction, the minimum cumulative risk cost estimate of neighboring points in the y-direction, the initial reference velocity, and the grid side length.

[0051] The initial iterative risk estimate is passed through a Newton-style iterative loop based on the anisotropic risk rate function to generate the final iterative risk estimate.

[0052] The final iterative risk estimate of all points in the inspection hazard heat map is used as the risk heuristic cost.

[0053] Compared with the prior art, the beneficial effects of the present invention are that, through the synergy of multi-branch convolutional neural networks and feature fusion convolutional models, the present invention achieves accurate perception and quantification of multi-dimensional fault risks, realizes semantic association and feature complementarity of multi-source data, and achieves a significant improvement in the representation capability of multi-modal fusion features. It can accurately identify early composite fault hazards of the same origin and solve the defects of missed detection and misjudgment of composite faults in the independent detection mode of the prior art.

[0054] In particular, this invention effectively solves the problem of insufficient targeting of path planning in existing inspection methods by deeply fusing multi-sensor data and accurately adapting the path planning model. Through the synergy of a multi-branch convolutional neural network model and an improved AStar path planning model, the path planning algorithm can be combined with the explosion-proof environment detection data of the multi-sensor explosion-proof power equipment inspection robot to ensure the safety of the robot's inspection path. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating an intelligent inspection method using a robot for inspecting power equipment, according to an embodiment of the present invention.

[0056] Figure 2 This is a flowchart illustrating the multi-branch convolutional neural network of the intelligent inspection method for a robot used for power equipment inspection according to an embodiment of the present invention.

[0057] Figure 3 This is a flowchart illustrating the feature fusion convolutional model of the intelligent inspection method for a robot used for power equipment inspection according to an embodiment of the present invention.

[0058] Figure 4 This is a flowchart illustrating the inspection path planning process of an intelligent inspection method for power equipment inspection using a robot, according to an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0060] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0061] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0062] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0063] like Figures 1 to 4 As shown, the present invention provides an intelligent inspection method for robots used in power equipment inspection, which overcomes the problem in the prior art that it is impossible to accurately assess the hidden risks of power equipment based on the multi-sensor data of power inspection robots, and that it is impossible to ensure the safety of the robot's inspection path.

[0064] like Figure 1 As shown, this embodiment proposes an intelligent inspection method for robots used in the inspection of power equipment, including:

[0065] The system acquires temperature images generated by the infrared thermal imager, corona discharge intensity images generated by the ultraviolet imager, hazardous gas concentration values ​​collected by the gas sensor, and obstacle grid maps generated by the lidar of the explosion-proof power equipment inspection robot.

[0066] The temperature image, corona discharge intensity image, hazardous gas concentration value, and obstacle grid image are encoded using a multi-branch convolutional neural network to generate temperature image encoding features, corona discharge encoding features, hazardous gas features, and spatial features.

[0067] The temperature image coding features, corona discharge coding features, hazardous gas features, and spatial features are used to generate a grid map of hazardous thermal points for inspection through a feature fusion convolution model based on attention mechanism and deformable convolution.

[0068] The inspection path of the robot is planned based on the grid map of the inspection hazards.

[0069] Specifically, the explosion-proof power equipment inspection robot is a Guodian Ruiyuan laser navigation explosion-proof inspection robot, which is equipped with an infrared thermal imager, a laser radar for generating point clouds, and a gas sensor as standard, and an optional ultraviolet imager. The infrared thermal imager is used for infrared temperature measurement of power equipment to avoid thermal faults in the equipment along the robot's inspection path. The ultraviolet imager eliminates background interference under full sunlight to avoid partial discharge phenomena such as corona and arcing along the robot's inspection path. The gas sensor is used to detect flammable and explosive gases in the substation. The laser radar is used for environmental modeling and obstacle avoidance for laser SLAM autonomous navigation. By cooperating with the infrared thermal imager, gas sensor, and ultraviolet imager, the robot avoids the aforementioned hazards along the inspection path. That is, the inspection hazard heat map is a grid map that marks the aforementioned hazards and is aligned with the obstacle grid map.

[0070] like Figure 2 As shown, the process of generating temperature image coding features, corona discharge coding features, hazardous gas features, and spatial features further includes:

[0071] The temperature image is passed through a first convolution branch to generate temperature image encoded features;

[0072] The corona discharge intensity image is passed through a second convolution branch to generate corona discharge encoded features;

[0073] The concentration values ​​of the hazardous gas are passed through a multilayer sensor to generate hazardous gas characteristics;

[0074] The obstacle grid is passed through a third convolutional branch to generate spatial features;

[0075] The multi-branch convolutional neural network includes a first convolutional branch, a second convolutional branch, a multilayer perceptron, and a third convolutional branch.

[0076] like Figure 3 As shown, the process of generating a grid map of inspection hazard heat points through a feature fusion convolution model further includes:

[0077] The temperature image encoding features and hazardous gas features are used through an attention mechanism to generate space gas features;

[0078] The temperature image coding features, corona discharge coding features, spatial gas features, and the first concatenated vector of spatial features are fused through a convolutional block to generate the inspection hazard thermal point raster map.

[0079] The feature fusion convolution model includes an attention mechanism and a fusion convolution block, wherein the fusion convolution block includes deformable convolutional layers.

[0080] Furthermore, the process of generating a grid map of hazardous heat points by fusing convolutional blocks includes:

[0081] The first concatenated vector is passed sequentially through a first convolutional layer, a first normalization layer, a second convolutional layer, and a second normalization layer to generate initial fused features;

[0082] The second concatenated vector of the initial fused features and spatial features is passed through a deformable convolutional layer to generate the final fused features;

[0083] The final fused features are passed through the output convolutional layer to generate the inspection hazard heat map.

[0084] The fused convolutional block further includes a first convolutional layer, a first normalization layer, a second convolutional layer, a second normalization layer, and an output convolutional layer.

[0085] like Figure 3 As shown, the process of generating space gas features through the attention mechanism further includes:

[0086] The third concatenated vector of the temperature image encoding features and hazardous gas features is processed through an attention mechanism to generate guiding attention weights.

[0087] The hazardous gas features are weighted based on the guided attention weights to generate space gas features.

[0088] Furthermore, intelligent inspection methods also include:

[0089] A combined loss function is constructed based on MSE, SmoothL1Loss, and multi-scale gradient consistency loss term, and the combined loss function is used to train and optimize multi-branch convolutional neural networks and feature fusion convolutional models.

[0090] Furthermore, the process of constructing the combined loss function includes:

[0091] The hazardous heat map and the actual hazardous point grid labels are respectively downsampled by the first average pooling to generate the first predicted gradient value and the first actual gradient value.

[0092] The first predicted gradient value and the first true gradient value are respectively downsampled by the second average pooling to generate the second predicted gradient value and the second true gradient value.

[0093] The first predicted gradient value and the first true gradient value are passed through the first gradient operator, and the first difference is obtained. The second predicted gradient value and the second true gradient value are passed through the second gradient operator, and the second difference is obtained.

[0094] The first difference and the second difference are averaged to calculate the multi-scale gradient consistency loss term;

[0095] The hazardous heat map and the actual hazardous point grid labels are respectively processed using the root mean square error function and the SmoothL1Loss function to calculate MSE and SmoothL1Loss.

[0096] The combined loss function is constructed by weighting and summing the MSE, SmoothL1Loss, and multi-scale gradient consistency loss term.

[0097] The process of generating a grid map of hazardous heat points for inspection using a multi-branch convolutional neural network model includes:

[0098] The temperature image is passed through a first convolution branch to generate temperature image encoded features;

[0099] The corona discharge intensity image is passed through a second convolution branch to generate corona discharge encoded features;

[0100] The concentration values ​​of the hazardous gas are passed through a multilayer sensor to generate hazardous gas characteristics;

[0101] The obstacle grid is passed through a third convolutional branch to generate spatial features;

[0102] The temperature image encoding features and hazardous gas features are used through an attention mechanism to generate space gas features;

[0103] The temperature image coding features, corona discharge coding features, spatial gas features, and the first concatenated vector of spatial features are fused through a convolutional block to generate a raster map of hazardous thermal points for inspection.

[0104] The multi-branch convolutional neural network model includes a first convolutional branch, a second convolutional branch, a multilayer perceptron, a third convolutional branch, an attention mechanism, and a fusion convolutional block.

[0105] Specifically, the first, second, and third convolutional branches adopt the U-Net architecture. The encoder of the U-Net architecture uses two convolutional blocks and one downsampling layer, the bottleneck layer uses two convolutional blocks, and the decoder uses transposed convolution for upsampling and then sets two convolutional blocks.

[0106] Specifically, the hazardous gas concentration values ​​are normalized by the maximum and minimum values, and then passed through a multilayer perceptron with two fully connected layers and one output mapping layer to generate corona discharge coding features.

[0107] Furthermore, the process of generating a grid map of inspection hazards by fusing the first concatenated vector through a convolutional block includes:

[0108] The first concatenated vector is passed sequentially through a first convolutional layer, a first normalization layer, a second convolutional layer, and a second normalization layer to generate initial fused features;

[0109] The initial fused features and the spatial features are passed through an offset prediction network and deformable convolutional layers to generate the final fused features;

[0110] The final fused features are passed through the output convolutional layer to generate the inspection hazard heat map.

[0111] The fused convolutional block includes a first convolutional layer, a first normalization layer, a second convolutional layer, a second normalization layer, an offset prediction network, a deformable convolutional layer, and an output convolutional layer.

[0112] Specifically, the multi-branch convolutional neural network and the feature fusion convolutional model can be represented as:

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120] In the formula, Indicates the weight of guiding attention. The third concatenated vector represents the temperature image coding features and hazardous gas features. This indicates element-wise multiplication. Indicates the characteristics of space gases. The first concatenated vector is represented by . This represents a vector concatenation operation. These represent temperature image coding features, corona discharge coding features, spatial gas features, and spatial features, respectively. This indicates that the approval is assigned to one level. Indicates a convolutional layer. express Activation function Indicates the second fusion feature, Indicates the initial fusion features. This represents an offset prediction network, consisting of one convolutional layer and a normalization layer, which predicts the offset field required for deformable convolution. This represents a deformable convolutional layer to align the obstacle grid map with the inspection hazard heatmap grid map. This represents a grid map showing hazardous hotspots during inspections.

[0121] Therefore, considering the inherent heterogeneity of the four types of inspection data, parallel differentiated coding branches are designed to avoid the inability of traditional single-network coding to adapt to different modal data and insufficient extraction of key features. The first and second convolutional branches both adopt the U-Net architecture. The encoder extracts multi-scale fault semantic features through double convolutional blocks and downsampling layers, and the bottleneck layer enhances the extraction of deep fault features. The decoder restores the spatial resolution of features through transposed convolution upsampling. Finally, the output is temperature image coding features and corona discharge coding features with the same size as the original image and carrying multi-scale fault semantics, which adapts to the spatial domain pixel feature extraction requirements of image data, while retaining the accurate spatial location information of the fault.

[0122] To address the one-dimensional global scalar characteristics of hazardous gas concentration values, a multilayer perceptron coding branch is designed: first, the gas concentration values ​​are normalized to eliminate the influence of dimensions by performing maximum and minimum value normalization; then, deep semantic features of gas risk are extracted through two fully connected layers; finally, fixed-dimensional hazardous gas features are generated through the output mapping layer. This solves the problem of dimensional mismatch between one-dimensional scalar data and two-dimensional image data, and fully preserves the global risk semantics of flammable and explosive gases.

[0123] The third convolutional branch adopts the U-Net architecture, which is of the same origin as the image branch. Based on the spatial topological characteristics of the obstacle grid map, it extracts core spatial topological features such as passable areas, equipment location coordinates, and obstacle distribution. Finally, it outputs spatial features that are completely aligned with the size of infrared and ultraviolet encoded features, thus achieving spatial dimension unification between spatial topological features and fault semantic features.

[0124] By using attention weighting, the limitations of global gas concentration detection and the inability to locate the spatial location of the leak source are avoided. By using infrared temperature characteristics as a guide, spatial location of gas risks is achieved, laying the foundation for the comprehensive assessment of subsequent coupled faults.

[0125] To address the limitations of traditional fixed convolutional kernels in terms of receptive field, inability to adapt to fault features at different scales, and misalignment between faults and spatial locations, a deformable convolutional structure is introduced. Guided by spatial features, an offset prediction network generates adaptive convolutional kernel sampling point offsets, allowing the sampling area of ​​the convolutional kernel to adaptively focus on the fault area of ​​the power equipment and the passable space. At the same time, it achieves precise alignment between fault semantic features and spatial topological features, realizing accurate binding between fault features and spatial grids, and improving the positioning accuracy of dangerous areas.

[0126] Furthermore, intelligent inspection methods also include:

[0127] A combined loss function is constructed based on MSE, SmoothL1Loss, and multi-scale gradient consistency loss term, and the combined loss function is used to train and optimize multi-branch convolutional neural networks and feature fusion convolutional models.

[0128] Furthermore, the process of constructing the combined loss function includes:

[0129] The hazardous heat map and the actual hazardous point grid labels are respectively downsampled by the first average pooling to generate the first predicted gradient value and the first actual gradient value.

[0130] The first predicted gradient value and the first true gradient value are respectively downsampled by the second average pooling to generate the second predicted gradient value and the second true gradient value.

[0131] The first predicted gradient value and the first true gradient value are passed through the first gradient operator, and the first difference is obtained. The second predicted gradient value and the second true gradient value are passed through the second gradient operator, and the second difference is obtained.

[0132] The first difference and the second difference are averaged to calculate the multi-scale gradient consistency loss term;

[0133] The hazardous heat map and the actual hazardous point grid labels are respectively processed using the root mean square error function and the SmoothL1Loss function to calculate MSE and SmoothL1Loss.

[0134] The combined loss function is constructed by weighting and summing the MSE, SmoothL1Loss, and multi-scale gradient consistency loss term.

[0135] Specifically, the gradient operator is the Scharr operator. Gradient calculation is performed in both horizontal and vertical directions to obtain the gradient magnitude of each pixel. The gradient map then reflects the region with the most dramatic changes in thermal values ​​at that scale, corresponding to the edges or risk transition zones of hazard sources. On the same gradient, the first and second differences are calculated separately, and the sum of the differences of all pixels is divided by the total number of pixels to obtain the multi-scale gradient consistency loss term. This encourages the predicted map and the ground truth map to have consistent risk change rates and edge positions under the same gradient. Therefore, the multi-scale gradient consistency loss term avoids the tendency of single-scale gradient loss to cause the model to focus only on edge features at a certain resolution. Multi-scale constraints can simultaneously capture the hierarchical structure of the hazard heatmap from coarse outlines to fine edges, making the generated heatmap closer to the real distribution in terms of visual perception and physical plausibility.

[0136] Specifically, the multi-scale gradient consistency loss term can be expressed as:

[0137]

[0138] In the formula, The term represents the multi-scale gradient consistency loss, and s represents the number of average pooling layers, which is 2 in total. This corresponds to the first and second average pooling downsampling, both with a downsampling factor of 2. s-1 , This represents the gradient operator, specifically the Scharr operator. This indicates average pooling downsampling. Represents the total number of samples. This represents the hazardous heat map of the i-th sample. Represents the raster label of the true danger point of the i-th sample.

[0139] Specifically, the combined loss function is calculated by weighting the SmoothL1 Loss, MSE (mean squared error), and multi-scale gradient consistency loss terms with weighting coefficients of 0.3, 0.7, and 0.2 respectively. This ensures accurate and robust pixel-level supervision through MSE and SmoothL1, and structural constraints through multi-scale gradient consistency loss. The weighting is primarily based on SmoothL1, with MSE and gradient loss as secondary factors. This design enables the generated hazardous heat map to achieve excellent performance in terms of numerical accuracy, clear boundaries, and physical plausibility.

[0140] Specifically, on-site measured inspection data of explosion-proof power scenarios were obtained, and a dataset including single faults and coupled compound faults such as overheating, discharge, and gas leakage of power equipment was constructed. The grid labels of real danger points are completely consistent with the size of the obstacle grid map. Each grid cell is labeled with a comprehensive risk ground value from 0 to 1, where 0 is no risk and 1 is the highest level of risk, realizing a one-to-one correspondence between fault areas and spatial grids. The dataset is randomly divided into training set, validation set, and test set in a ratio of 7:2:1. Control group 1 uses multi-branch ordinary CNN encoding, where all features are directly concatenated through channels and then fused through ordinary convolutional layers. It has no attention mechanism, no deformable convolution, and uses the MSE single loss function. Control group 2 uses a general multimodal fusion network MMF, which uses a shared encoder and cross-modal attention mechanism, but has no adaptation design for power inspection scenarios. The test group uses the multi-branch convolutional neural network and feature fusion convolutional model of this embodiment. All groups use the same training hyperparameters, the optimizer AdamW, the weight decay of 1e-5, the initial learning rate of 1e-4, the batch size of 8, and the total number of training rounds of 100. On the test set, the fault identification mAP@0.5 of control group 1, control group 2, and test group were 81.56%, 85.29%, and 90.72%, respectively. It can be seen that the multi-branch convolutional neural network and feature fusion convolutional model significantly improve the ability to identify early faults and small-scale faults with strong concealment, and completely solve the problems of missed detection and false detection of complex faults.

[0141] Furthermore, the process of planning the robot's inspection path based on the aforementioned inspection hazard heat map includes:

[0142] Based on the obstacle grid, the node-based risk cost and task-inspired cost are calculated; based on the inspection hazard heat map, the generated risk-inspired cost is calculated.

[0143] Based on the node-based risk cost, task-inspired cost, and risk-inspired cost, a hazard heuristic function is calculated, and the hazard heuristic function is used through the AStar path planning model to generate the inspection path for the explosion-proof power equipment inspection robot.

[0144] Furthermore, the process of calculating the risk heuristic cost based on the inspected hazard heat map grid includes:

[0145] The grid map of the inspection hazard heat points and the candidate moving points are used to generate the node hazard direction gradient value through the Sobel operator;

[0146] Calculate the maximum value of the product of the node's dangerous direction gradient value and the candidate movement point and zero, and then sum the maximum value with the node's dangerous direction gradient value using weighted summation to calculate the risk velocity function in each direction;

[0147] The risk heuristic cost is generated by passing the anisotropic risk velocity function through a fast travel method.

[0148] Furthermore, the process of generating risk heuristic costs from the anisotropic risk velocity function using the fast travel method includes:

[0149] Calculate the minimum estimated cumulative risk cost of neighboring points in the x-direction and the minimum estimated cumulative risk cost of neighboring points in the y-direction based on the current grid points;

[0150] Calculate the initial reference velocity based on the gradient value of the dangerous direction of the node;

[0151] The initial iterative risk estimate is calculated based on the minimum cumulative risk cost estimate of neighboring points in the y-direction, the minimum cumulative risk cost estimate of neighboring points in the y-direction, the initial reference velocity, and the grid side length.

[0152] The initial iterative risk estimate is passed through a Newton-style iterative loop based on the anisotropic risk rate function to generate the final iterative risk estimate.

[0153] The final iterative risk estimate of all points in the inspection hazard heat map is used as the risk heuristic cost.

[0154] Specifically, the process of calculating the initial iterative risk estimate can be expressed as:

[0155]

[0156]

[0157]

[0158] In the formula, , Let represent the minimum estimated cumulative risk cost of neighboring points in the x-direction and the minimum estimated cumulative risk cost of neighboring points in the y-direction, respectively. , All represent the current grid point Estimation of the cumulative risk cost of neighboring points in the x-direction. , All represent the current grid point The cumulative risk cost estimate of neighboring points in the y-direction can be understood as follows: the inspection hazard heat map records the current cumulative risk cost estimate of each grid cell. , This indicates that the Sobel operator will be used to inspect the current grid point p in the hazardous heat map. The partial derivatives in the x and y directions obtained from the convolution are used to calculate the current grid point p. The H value of the 3×3 neighborhood grid points, This represents the gradient value in the dangerous direction of the node. This represents the initial iterative risk estimate. This represents the grid side length, preferably 1. The weighting coefficient is 0.3. Therefore, the above process of generating the initial iterative risk estimate is the algorithm's initialization process, executed only once and not repeatedly in a loop, in order to generate an accurate initial iterative risk estimate during the initialization process.

[0159] Therefore, Newton iteration performs nonlinear optimization on the risk velocity function in all directions, exhibiting extremely strong convergence and avoiding the oscillation and non-convergence problems of traditional iterative methods.

[0160] Furthermore, the process of generating the final iterative risk estimate from the initial iterative risk estimate through a Newton-style iterative loop based on the risk rate function includes:

[0161] The propagation direction is estimated by calculating the initial iterative risk estimate and the grid side length;

[0162] Substitute the estimated propagation direction into the risk heuristic cost, and calculate the residual function based on the risk heuristic cost;

[0163] The first derivative is calculated based on the residual function and the initial iterative risk estimate, and the initial iterative risk estimate is updated based on the residual function and the first derivative to generate the iterative risk estimate.

[0164] If the difference between the iterative risk estimate and the initial iterative risk estimate is less than a threshold, the iteration ends; otherwise, the iterative risk estimate is used as the initial iterative risk estimate to calculate the estimated propagation direction, residual function, and first derivative.

[0165] Specifically, the process of generating the final iterative risk estimate through a Newton-style iterative loop based on the risk rate function can be expressed as:

[0166]

[0167]

[0168]

[0169]

[0170]

[0171]

[0172]

[0173]

[0174]

[0175] In the formula, , Let represent the gradients of the iterative risk estimates in the x and y directions, respectively, which characterize the gradient values ​​of risk changes in the two directions. This represents the initial iterative risk estimate for the current iteration. , Both represent the current grid point p Estimation of the cumulative risk cost of neighboring points in the x-direction. , Both represent the current grid point p Estimation of the cumulative risk cost of neighboring points in the y-direction. This represents the grid side length, preferably 1. This indicates an estimated direction of propagation, in the event of a change in risk. Then, divide the gradients of the iterative risk estimates in the x and y directions by the magnitudes of the gradients of the iterative risk estimates, respectively. To obtain the fastest estimated propagation direction of gradient descent. ,For example If (60 / 100, 80 / 100) = (0.6, 0.8), it means that the explosion-proof power equipment inspection robot moves in the combined direction of 0.6 in the x-direction and 0.8 in the y-direction, where the risk decreases the fastest, thus achieving optimal risk avoidance. These represent the grid points p in the inspected hazardous heat map grid map, obtained by using the Sobel operator in the Newton iteration loop. The partial derivatives in the x and y directions obtained from the convolution calculation are...

[0176] therefore, This represents the gradient value in the dangerous direction of the node. This represents the magnitude of the gradient value in the dangerous direction of the node. This represents the risk-heuristic cost calculated based on the estimated propagation direction and the gradient values ​​of the node's dangerous direction. , Indicates the weighting coefficient. This is the dot product of the gradient vector and the direction of movement. If the direction of movement is the same as the gradient direction, the dot product is positive, meaning the movement is in the direction of increasing risk. This term will increase the denominator, reduce speed, and increase cost. Otherwise, it is negative. To ensure that only actions entering high-risk areas are penalized, and actions leaving high-risk areas are not rewarded, thus avoiding unsafe shortcuts, a risk-heuristic cost is achieved by weighted summation of the risk cost with the basic nodal risk direction gradient values. This risk-heuristic cost is defined as the cost of moving from point p along a unit direction vector. The slower the movement, the higher the cost, moving towards the risk gradient. When moving, the speed decreases significantly and the cost increases. Avoiding going against the current into high-risk areas is crucial. When moving along a direction perpendicular to the gradient, the speed decreases moderately. When moving away from the risk gradient direction, the speed decreases slightly or remains the same. , Let these represent the residual function and the first derivative, respectively. The essence of Newton's iterative loop is to iteratively move step by step along the tangent direction of the residual function towards... The point approximation of 0 is determined by calculating the residual function of the tangent slope at Tk, which determines the correction direction. make Correct in the direction of decreasing residual. , Let represent the minimum estimated cumulative risk cost of neighboring points in the x-direction and the minimum estimated cumulative risk cost of neighboring points in the y-direction, respectively. The iterative risk estimate is represented by the value of the initial iterative risk estimate. If the difference between the initial iterative risk estimate and the value of the initial iterative risk estimate is less than the threshold 1e-5, the iteration ends; otherwise, the initial iterative risk estimate is used as the new initial iterative risk estimate.

[0177] Specifically, the Fast March Method (FMM) uses the above formula to perform the following main loop:

[0178] Step S1: Extract the point p with the minimum current cost T from the Trial, and calculate the initial iteration risk estimate. .

[0179] Step S2: Move point p from the Trial set to the Known set.

[0180] Step S3: Traverse every non-obstacle neighbor q (4-neighborhood) of p: Using the local solver composed of the above formula, recalculate the iterative risk estimate based on all the determined neighbors of point p. .if If it is less than the current T[q], then update T[q] = If q is in Far, move it into Trial; if q is already in Trial, update its position in the priority queue. When the main loop ends, Trial is empty or all reachable points have been processed, and the cost graph T[q] is the final result. Its matrix T[q] is used as the final iterative risk estimate.

[0181] Furthermore, the process of calculating the basic risk cost of nodes based on the obstacle grid map includes:

[0182] The basic risk value of adjacent points is calculated based on the obstacle grid map, as well as the distance between adjacent points. The basic risk cost of the node is calculated based on the product of the basic risk value of adjacent points and the distance between adjacent points.

[0183] The process of calculating the heuristic cost of a task based on an obstacle grid map includes:

[0184] The task value is calculated based on the obstacle grid map, and the task values ​​of reachable points in the obstacle grid map are summed to calculate the task heuristic cost.

[0185] Furthermore, the process of calculating the hazard heuristic function based on the node-based risk cost, task heuristic cost, and risk heuristic cost includes:

[0186] The risk heuristic function is calculated by weighted summation of the node-based risk cost, task heuristic cost, and risk heuristic cost.

[0187] Specifically, the danger heuristic function can be expressed as:

[0188]

[0189]

[0190]

[0191] In the formula, Indicates a dangerous heuristic function. , , This represents the weighting coefficient, preferably 1.0, 10, or 3, which ensures that the robot's inspection path is far from obstacles, wide and safe, and may detour to the task point. , , Let $n$ represent the node-based risk cost, task heuristic cost, and risk heuristic cost from the current node $n$ to the target node $G$, respectively. This represents the set of reachable points in an obstacle grid. This represents the task value, including no-value zones (-100), and tasks severely obscured by obstacles (0). 'p' represents reachable and visible task points, which is a set value. These represent the base risks for points m and n, respectively, with the restricted area at 100, the adjacent area at a fixed high-risk value of 5, and the safe area at a fixed low-risk value of 0.1. This represents the distance between points m and n.

[0192] Specifically, in the AStar path planning model, the total cost f(n) = g(n) + h(n) is calculated, where f(n) is the total cost of node n, g(n) is the cumulative cost of node n, and h(n) represents the danger heuristic function. Node n is removed from the queue, and all its reachable neighbor nodes m are examined. The new actual cost g_temp from the starting point through n to m is calculated, which is the temporary cumulative cost. If a better path to m is found, and g_temp is smaller, then m's g(m) and parent node are updated, and its f(m) = g(m) + h(m) is recalculated. Then, m is reordered according to its new f(m) value and put into the queue.

[0193] In this embodiment, the synergy between a multi-branch convolutional neural network and a feature fusion convolutional model enables accurate perception and quantification of multi-dimensional fault risks. It achieves semantic association and feature complementarity of multi-source data, significantly enhancing the representational capabilities of multi-modal fusion features. This allows for accurate identification of early-stage composite fault hazards originating from the same source, overcoming the shortcomings of missed detection and misjudgment of composite faults in existing independent detection modes. Through deep fusion of multi-sensor data and precise adaptation to the path planning model, the lack of specificity in path planning in existing inspection methods is effectively addressed. The synergy between the multi-branch convolutional neural network model and the improved AStar path planning model allows the path planning algorithm to incorporate multi-sensor explosion-proof environment detection data from the explosion-proof power equipment inspection robot, ensuring the safety of the robot's inspection path. The risk heuristic cost is calculated in a refined manner using a multi-sensor fusion-generated inspection hazard heat map. First, the inspection hazard heat map and candidate movement points are processed using the Sobel operator to generate node hazard direction gradient values ​​that accurately reflect the hazard direction. This yields an anisotropic risk velocity function that quantifies the risk differences in different movement directions. Based on the current grid point, the minimum cumulative risk cost estimates of neighboring points in the x and y directions are calculated. Combined with the initial baseline velocity calculated from the node hazard direction gradient values ​​and the grid side length, an initial iterative risk estimate is obtained. After iterative optimization, the final iterative risk estimate of all points in the inspection hazard heat map is used as the risk heuristic cost, making risk assessment more targeted and ensuring the safety of the robot's inspection path. Finally, the node basic risk cost, task heuristic cost, and the refined risk heuristic cost are organically integrated to calculate a hazard heuristic function, which is then input into the AStar path planning model to generate the optimal inspection path adapted to the inspection scenario of explosion-proof power equipment. This path planning method breaks through the limitations of the traditional AStar algorithm, which only focuses on obstacle avoidance. It can guide the robot to effectively avoid obstacles in the environment by using the basic risk cost of nodes, avoiding collisions and ensuring the robot's operational safety in high-risk scenarios. It can also accurately avoid areas with high risk levels by using finely calculated risk-inspired costs combined with high-precision hazard state representation data. At the same time, it adapts to the risk differences of different movement directions, achieving more intelligent detour and risk avoidance, reducing safety hazards during the inspection process, and ensuring the safety of the robot's inspection path.

[0194] Those skilled in the art will recognize that the modules and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0196] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligent inspection method for a robot used in the inspection of power equipment, characterized in that, include: The system acquires temperature images generated by the infrared thermal imager, corona discharge intensity images generated by the ultraviolet imager, hazardous gas concentration values ​​collected by the gas sensor, and obstacle grid maps generated by the lidar of the explosion-proof power equipment inspection robot. The temperature image, corona discharge intensity image, hazardous gas concentration value, and obstacle grid image are encoded using a multi-branch convolutional neural network to generate temperature image encoding features, corona discharge encoding features, hazardous gas features, and spatial features. The temperature image coding features, corona discharge coding features, hazardous gas features, and spatial features are used to generate a grid map of hazardous thermal points for inspection through a feature fusion convolution model based on attention mechanism and deformable convolution. The inspection path of the robot is planned based on the grid map of the inspection hazards.

2. The intelligent inspection method for a robot used for power equipment inspection according to claim 1, characterized in that, The process of generating temperature image coding features, corona discharge coding features, hazardous gas features, and spatial features includes: The temperature image is passed through a first convolution branch to generate temperature image encoded features; The corona discharge intensity image is passed through a second convolution branch to generate corona discharge encoded features; The concentration values ​​of the hazardous gas are passed through a multilayer sensor to generate hazardous gas characteristics; The obstacle grid is passed through a third convolutional branch to generate spatial features; The multi-branch convolutional neural network includes a first convolutional branch, a second convolutional branch, a multilayer perceptron, and a third convolutional branch.

3. The intelligent inspection method for a robot used for inspecting power equipment according to claim 1, characterized in that, The process of generating a grid map of hazardous heat points for inspection using a feature fusion convolutional model includes: The temperature image encoding features and hazardous gas features are used through an attention mechanism to generate space gas features; The temperature image coding features, corona discharge coding features, spatial gas features, and the first concatenated vector of spatial features are fused through a convolutional block to generate the inspection hazard thermal point raster map. The feature fusion convolution model includes an attention mechanism and a fusion convolution block, wherein the fusion convolution block includes deformable convolutional layers.

4. The intelligent inspection method for a robot used for inspecting power equipment according to claim 3, characterized in that, The process of generating a grid map of hazardous heat points for inspection by fusing convolutional blocks includes: The first concatenated vector is passed sequentially through a first convolutional layer, a first normalization layer, a second convolutional layer, and a second normalization layer to generate initial fused features; The second concatenated vector of the initial fused features and spatial features is passed through a deformable convolutional layer to generate the final fused features; The final fused features are passed through the output convolutional layer to generate the inspection hazard heat map. The fused convolutional block further includes a first convolutional layer, a first normalization layer, a second convolutional layer, a second normalization layer, and an output convolutional layer.

5. The intelligent inspection method for a robot used for inspecting power equipment according to claim 3, characterized in that, The process of generating space gas features through attention mechanisms includes: The third concatenated vector of the temperature image encoding features and hazardous gas features is processed through an attention mechanism to generate guiding attention weights. The hazardous gas features are weighted based on the guided attention weights to generate space gas features.

6. The intelligent inspection method for a robot used for inspecting power equipment according to claim 1, characterized in that, Also includes: A combined loss function is constructed based on MSE, SmoothL1Loss, and multi-scale gradient consistency loss term, and the combined loss function is used to train and optimize multi-branch convolutional neural networks and feature fusion convolutional models.

7. The intelligent inspection method for a robot used for inspecting power equipment according to claim 6, characterized in that, The process of constructing the combined loss function includes: The hazardous heat map and the actual hazardous point grid labels are respectively downsampled by the first average pooling to generate the first predicted gradient value and the first actual gradient value. The first predicted gradient value and the first true gradient value are respectively downsampled by the second average pooling to generate the second predicted gradient value and the second true gradient value. The first predicted gradient value and the first true gradient value are passed through the first gradient operator, and the first difference is obtained. The second predicted gradient value and the second true gradient value are passed through the second gradient operator, and the second difference is obtained. The first difference and the second difference are averaged to calculate the multi-scale gradient consistency loss term; The hazardous heat map and the actual hazardous point grid labels are respectively processed using the root mean square error function and the SmoothL1Loss function to calculate MSE and SmoothL1Loss. The combined loss function is constructed by weighting and summing the MSE, SmoothL1Loss, and multi-scale gradient consistency loss term.

8. The intelligent inspection method for a robot used for inspecting power equipment according to any one of claims 1 to 7, characterized in that, The process of planning the robot's inspection path based on the aforementioned inspection hazard heat map includes: Based on the obstacle grid, the node-based risk cost and task-inspired cost are calculated; based on the inspection hazard heat map, the generated risk-inspired cost is calculated. Based on the node-based risk cost, task-inspired cost, and risk-inspired cost, a hazard heuristic function is calculated, and the hazard heuristic function is used through the AStar path planning model to generate the inspection path for the explosion-proof power equipment inspection robot.

9. The intelligent inspection method for a robot used for inspecting power equipment according to claim 8, characterized in that, The process of calculating the risk heuristic cost based on the inspected hazardous heat map grid includes: The grid map of the inspection hazard heat points and the candidate moving points are used to generate the node hazard direction gradient value through the Sobel operator; Calculate the maximum value of the product of the node's dangerous direction gradient value and the candidate movement point and zero, and then sum the maximum value with the node's dangerous direction gradient value using weighted summation to calculate the risk velocity function in each direction; The risk heuristic cost is generated by passing the anisotropic risk velocity function through a fast travel method.

10. The intelligent inspection method for a robot used for inspecting power equipment according to claim 9, characterized in that, The process of generating risk heuristic costs from the anisotropic risk velocity function using the fast travel method includes: Calculate the minimum estimated cumulative risk cost of neighboring points in the x-direction and the minimum estimated cumulative risk cost of neighboring points in the y-direction based on the current grid points; Calculate the initial reference velocity based on the gradient value of the dangerous direction of the node; The initial iterative risk estimate is calculated based on the minimum cumulative risk cost estimate of neighboring points in the y-direction, the minimum cumulative risk cost estimate of neighboring points in the y-direction, the initial reference velocity, and the grid side length. The initial iterative risk estimate is passed through a Newton-style iterative loop based on the anisotropic risk rate function to generate the final iterative risk estimate. The final iterative risk estimate of all points in the inspection hazard heat map is used as the risk heuristic cost.

Images