A machine vision-based power inspection equipment fault identification and prediction method and system

Abstract

The application relates to the technical field of electric power inspection, and discloses a power inspection equipment fault identification and prediction method and system based on machine vision, which comprises the following steps: collecting images of an electric power inspection area, performing local feature extraction and candidate area screening processing, dynamically generating a fault candidate area graph in a local area, identifying the fault types of each candidate area, predicting the risk levels, outputting fault identification and prediction results, constructing a state-decision model with an inspection task target as a reward function, and mapping the optimal treatment scheme and an inspection path adjustment strategy into action control instructions of the foot-type robot. The method has the beneficial effects that the multi-modal acquisition capability of equipment surface state, connection structure and thermal imaging is enhanced, noise interference is reduced, the accuracy and positioning precision of fault detection are improved, the robustness of fault type identification and the reliability of risk early warning are effectively improved, the intelligent level and operation automation capability of the system are significantly improved, and the cost of human intervention is reduced.

Description

Technical Field

[0001] This invention relates to the field of power inspection technology, and in particular to a method and system for fault identification and prediction of power inspection equipment based on machine vision. Background Technology

[0002] As the scale of power systems continues to expand and operational complexity increases, the reliability of power equipment and the ability to provide early warnings of faults have become core elements in ensuring the safe and stable operation of the power grid. Traditional power inspections mostly rely on manual on-site checks or use wheeled or tracked inspection robots equipped with infrared and thermal imaging sensors to detect equipment status. These methods generally suffer from drawbacks such as high labor intensity, significant constraints from terrain and environment, limited perspective on status perception, and long inspection cycles, making it difficult to achieve efficient and precise fault identification and risk prediction. These problems are particularly prominent in scenarios such as substations, densely intersecting sections of power lines, and complex terrain.

[0003] In recent years, artificial intelligence and machine vision technologies have evolved rapidly, and intelligent inspection of power equipment based on visual perception has gradually become a key area of ​​research and application in the industry. However, existing technical solutions still have significant shortcomings, as detailed below: Most existing visual recognition algorithms analyze the entire image and lack dynamic feature modeling and precise localization of local abnormal regions, which can easily lead to missed detections and false detections. In diverse inspection environments and with different types of equipment, the training model has weak adaptability and cannot stably identify multiple types and multiple locations of faults. Existing systems mostly only identify fault types, without combining power grid operation and maintenance business logic with on-site inspection scenarios to classify the severity of faults. Consequently, assessment and intelligent handling strategy generation cannot provide direct support for inspection decisions. Summary of the Invention

[0004] The purpose of this invention is to provide a machine vision-based fault identification and prediction method and system for power inspection equipment in order to solve the above-mentioned problems.

[0005] To achieve the above objectives, this invention provides a fault identification and prediction method for power inspection equipment based on machine vision, the specific steps of which are as follows: S1. Acquire images of the power inspection area using a high-resolution camera deployed on a legged robot; S2. Based on the power inspection area image, after local feature extraction and candidate area screening, a fault candidate area map is dynamically generated in the local area. S3. Based on the fault candidate region map, identify the fault type and predict the risk level of each candidate region, and output the fault identification and prediction results. S4. Based on the fault identification and prediction results, construct a state-decision model with the inspection task objective as the reward function, perform adaptive optimization training on the fault handling strategy, and dynamically generate the optimal handling plan and inspection path adjustment strategy. S5. Map the optimal handling plan and inspection path adjustment strategy into motion control commands for the legged robot, and drive the legged robot to autonomously complete fault verification and subsequent operation tasks according to the optimized path.

[0006] Furthermore, the images of the power inspection area include images of the power equipment itself, images of the connection and installation parts, images of the operating status indication, images of the surface condition, images of the environmental background, and thermal imaging images.

[0007] Further, step S2 includes the following steps: S21. Divide the power inspection area image into several local image sub-blocks according to a fixed window sliding method, and extract the texture gradient features, edge density index, thermal imaging temperature change rate and structural self-similarity of each local image sub-block to construct local features; S22. Based on the local features, a local discriminative embedding algorithm based on sparse representation is used to construct a sparse dictionary containing normal working condition samples and historical fault samples, and sparse coefficient decomposition is performed on the texture detail vector and heat distribution curve. S23. Introduce a local discriminant embedding criterion in the sparse coefficient space, calculate the sparse residual distribution of normal operating condition samples and historical fault samples, and obtain a discriminant embedding vector with fault sensitivity by maximizing inter-class boundaries and minimizing intra-class compactness, and generate preliminary candidate region responses by combining neighborhood topological relationships. S24. Perform cross-regional connectivity clustering and residual intensity screening on the initial candidate region response to generate the final fault candidate region map of potential defect locations.

[0008] Furthermore, step S3 includes the following steps: S31. Extract multimodal fusion features from each candidate region in the fault candidate region map, including texture gradient of optical image, edge direction histogram, temperature time-series curve features of thermal imaging image, and structural topology and background association features. S32. Input the multimodal fusion features into the multimodal migration forest algorithm, and achieve preliminary classification of fault types across scenarios by aligning the feature distribution between the source domain training set and the target domain inspection scenario; S33. Based on the classification results of the multimodal migration forest algorithm, a conditional random field model is introduced. The spatial neighborhood consistency and contextual dependency between candidate regions are used as constraints to globally optimize the classification probability map output by the forest, resulting in a more stable distribution of region labels for category discrimination. S34. Based on the region label distribution optimized by the conditional random field, and combined with the classification confidence, residual energy function and historical fault modes, the risk level of the candidate region is predicted, and the fault identification prediction result containing fault category label, confidence and risk score is output.

[0009] Furthermore, the formula for the conditional random field model is as follows: E(X) = ∑ i (-logP(x i |F i )+λ·||r i ||2)+∑ (i,j) η·I(x i ≠x j )·exp(-||F i -F j || 2 / σ 2 ), Where E(X) represents the global energy function of the label distribution of all candidate regions in the entire power line inspection image; X represents the set of predicted labels for all candidate regions; x i F represents the fault category label for fault candidate region i; i Represents the multimodal fusion feature vector of fault candidate region i; -logP(x i |F i ) indicates that the fault candidate region i belongs to the label Classification confidence loss; r i The residual vector representing region i is generated based on sparse representation and a local discriminative embedding algorithm; ||r i ||2 represents the sparse residual intensity of region i; λ represents the penalty weight coefficient of residual energy; (i,j) represents a pair of candidate regions with spatial adjacency in the graph structure; I(x i ≠x j ) represents an indicator function; ||F i -F j || 2 σ represents the degree of difference between adjacent regions in the multimodal feature space; σ represents the scaling coefficient of the Gaussian kernel function; and η represents the global weight of the spatial neighborhood consistency constraint.

[0010] Further, step S4 includes the following steps: S41. Based on the fault identification and prediction results, a high-dimensional state space integrating multi-dimensional information of power inspection is constructed. The multi-dimensional information of power inspection includes power inspection state variables, equipment fault status, robot system status, and inspection environment characteristics. S42. Based on the high-dimensional state space, a composite reward function is set with the objectives of minimizing the efficiency of inspection task completion, the accuracy of fault handling, and the resource consumption. A state transition function is constructed by combining the kinematic constraints of the legged robot and the task priority. S43. Based on the state transition function, a reinforcement learning algorithm based on deep Q-network and policy gradient joint optimization is adopted to perform policy iterative optimization in a simulated interactive environment, and dynamically generate multi-dimensional action output including detour path, inspection order and handling method. S44. Based on the converged policy network, the current state is mapped to the optimal handling plan and inspection path adjustment strategy, and the task execution sequence is constructed in a graph structure.

[0011] Furthermore, the formula for the reinforcement learning algorithm is as follows: J(p) = E_{(s,a,s')~π p}[ w(s)( (s,a) +gV(s') V(s))logπ p (a|s)]+cE[C(s,a)], Where J(p) represents the reinforcement learning optimization objective function; E_{(s,a,s')~π p} represents the strategy π p The empirical expected value of the sampled state-action-transition triplet; s represents the fault identification prediction result; a represents the inspection behavior that the robot can perform; s' represents the next state reached after performing the action; π p This represents the policy distribution determined by parameter p; (s,a) represents the composite immediate reward function; V(s) and V(s') represent the value estimates of the value network in state s and the next state s'; g represents the discount factor; w(s) represents the scenario risk-information weight; π p (a|s) represents the policy probability of taking action a in state s; c represents the cost penalty coefficient; C(s,a) represents the action cost function.

[0012] Furthermore, in the process of generating the motion control command in step S5, a Bayesian-optimized motion fuzzy correction mechanism is introduced to perform real-time dynamic correction of path nodes with terrain undulation interference or obstacle interference.

[0013] A machine vision-based fault identification and prediction system for power inspection equipment includes an image acquisition module, a fault candidate region extraction module, a fault identification and prediction module, a strategy optimization module, and an execution control module. Each module is connected in sequence to communicate with each other and work together to complete the entire process of power equipment image acquisition, fault region extraction, fault identification risk assessment, inspection strategy optimization, and robot autonomous control. The image acquisition module is used to acquire images of the power inspection area through a high-resolution camera deployed on the legged robot; The fault candidate region extraction module is communicatively connected to the image acquisition module and is used to receive the power inspection area image and dynamically generate a fault candidate region map in the local area using a local discriminant embedding algorithm based on sparse representation. The fault identification and prediction module is communicatively connected to the fault candidate region extraction module. It is used to receive the fault candidate region map, use a multimodal migration forest algorithm based on conditional random field optimization to identify the fault type and predict the risk level of each candidate region, and output the fault identification and prediction results. The strategy optimization module is communicatively connected to the fault identification and prediction module. It is used to receive the fault identification and prediction results, and to construct a state-decision model with the inspection task objective as the reward function using a reinforcement learning algorithm. It then performs adaptive optimization training on the fault handling strategy and dynamically generates the optimal handling plan and inspection path adjustment strategy. The execution control module is communicatively connected to the strategy optimization module and is used to map the optimal handling plan and inspection path adjustment strategy into motion control commands for the legged robot, and drive the legged robot to autonomously complete fault verification and subsequent operation tasks according to the optimized path.

[0014] The beneficial effects of the machine vision-based fault identification and prediction method and system for power inspection equipment provided by this invention are as follows: By deploying high-resolution cameras on legged robots, it is possible to stably acquire multi-view images of power equipment in complex terrain, effectively enhancing the ability to collect multi-modal information such as equipment surface condition, connection structure and thermal imaging, and providing high-quality raw data support for subsequent fault identification; By employing a local discriminative embedding algorithm based on sparse representation, discriminative feature representations can be extracted from local regions of an image, dynamically generating a fault candidate region map, reducing noise interference, and improving the accuracy of fault detection and localization. It is particularly suitable for small-scale fault region extraction in complex backgrounds. The multimodal migration forest algorithm based on conditional random field optimization can comprehensively process image modalities, structural features and contextual relationships, perform joint reasoning and risk level classification of candidate regions, effectively improve the robustness of fault type identification and the reliability of risk warning, and is particularly suitable for cross-scene migration identification tasks in new environments. By introducing reinforcement learning to construct a state-decision model with the inspection task objective as the reward function, it can automatically generate adaptive optimal handling solutions and path adjustment strategies based on the fault type and inspection history, realizing the transformation from passive response to active avoidance and optimization, and improving fault handling efficiency and inspection task completion quality. By mapping the optimal handling plan and path strategy into control commands for a legged robot, the robot is equipped with the ability to dynamically adjust its path and execute operational tasks. It can autonomously complete fault verification and subsequent response operations during actual inspections, significantly improving the system's intelligence level and operational automation capabilities, and reducing the cost of human intervention. Attached Figure Description

[0015] Figure 1 This is a flowchart of the fault identification and prediction method for power inspection equipment based on machine vision as described in this invention. Figure 2 This is a detailed flowchart of step S3 in the machine vision-based fault identification and prediction method for power inspection equipment according to the present invention. Figure 3 This is a schematic diagram of the module of the machine vision-based power inspection equipment fault identification and prediction system described in this invention.

[0016] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings.

[0017] The attached figures are labeled as follows: 1. Image acquisition module; 2. Fault candidate region extraction module; 3. Fault identification and prediction module; 4. Strategy optimization module; 5. Execution control module. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without creative effort are within the scope of protection of the invention.

[0019] This invention provides a machine vision-based method and system for fault identification and prediction of power inspection equipment. The following is a description of the method in conjunction with the appendix. Figure 1-3 The following describes some embodiments of the present invention in detail. Unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0020] Specific reference Figure 1-2 A machine vision-based fault identification and prediction method for power inspection equipment includes the following steps: S1. Acquire images of the power inspection area using a high-resolution camera deployed on a legged robot; wherein, the images of the power inspection area include images of the power equipment itself, images of the connection and installation parts, images of the operating status indication, images of the surface status, images of the environmental background, and thermal imaging images.

[0021] Specifically, the front end of the legged robot is equipped with a multi-sensor composite visual acquisition module, including but not limited to at least one of binocular visible light camera module, visible light high-definition camera, low-light camera, structured light / depth camera, thermal imaging infrared camera, and auxiliary TOF / structured light depth camera; Among them, the binocular visible light camera module (resolution ≥ 4K) is used to acquire RGB images with rich structural details, supports stereo imaging and depth estimation, has a horizontal field of view ≥ 120° to ensure wide coverage, and is equipped with automatic exposure and wide dynamic range compensation mechanism to adapt to strong light / backlight environments. Visible light high-definition cameras (resolution ≥ 8 million pixels) are used to capture images of the power equipment itself and the appearance features of its connection and installation parts. The acquisition frequency is set to 30fps to ensure dynamic capture of the equipment's operating status. Low-light cameras are used at night or in low-light conditions to capture operating status indicator signals (such as red and green lights) and the status displayed on the instrument panel LCD. Structured light / depth cameras obtain three-dimensional point cloud information of the equipment casing and surrounding environment by actively projecting coded light spots, which can be used to detect surface cracks of insulators, deformation of equipment casings and surrounding obstacles; The thermal imaging infrared camera has a detection wavelength range of 8~14μm and a thermal sensitivity of ≤0.05°C. It is used to capture abnormally hot areas on the surface of equipment and identify early signs of thermal failure. An auxiliary TOF / structured light depth camera (optional) is used to acquire close-range structural contours to assist in local 3D modeling and navigation obstacle avoidance.

[0022] During the aforementioned visual acquisition process, the robot's posture changes are monitored in real time through the IMU (Inertial Measurement Unit), supporting image posture correction and odometer estimation.

[0023] The image acquisition task for the power inspection area is autonomously planned and dynamically adjusted, covering the following key areas and scenarios: Images of the power equipment itself, such as transformer casings, circuit breaker bodies, and conductor connections, are used to ensure the integrity of key structures can be identified. Images of connection and installation points, including bolts, brackets, and insulator connections, are suitable for identifying minor defects such as loosening, detachment, and corrosion. The system displays operational status images and collects intuitive information from ammeters, voltmeters, status indicator lights, and mechanical switch positions. Surface condition images, including abnormal features on the equipment casing surface such as cracks, corrosion, foreign object obstruction, and coating peeling; Environmental background images are used to record vegetation obstructions, water accumulation, and obstacles around the equipment, which helps in global risk modeling. Thermal imaging images display the surface temperature distribution of equipment, identifying hidden faults such as overheating, short circuits, and abnormal loads.

[0024] A task-based inspection and scheduling system is used to control the acquisition nodes and set fixed-point / timed shooting strategies. Each target device acquires 3 to 5 angle images to ensure 3D information coverage. The image frame number is synchronously bound to the pose data to form a timestamp sequence for subsequent mapping and analysis.

[0025] Spatial registration of RGB images and thermal images was performed, and scale-invariant feature transform (SIFT / SURF) and depth-assisted reprojection were used. Camera trajectory estimation was performed using IMU and visual odometry (VIO) to eliminate image blur and distortion. Image dehazing, noise filtering and exposure compensation were performed on all image data before uploading.

[0026] S2. Based on the power inspection area image, after local feature extraction and candidate area screening, a fault candidate area map is dynamically generated in the local area.

[0027] The specific implementation process for this step is as follows: S21. Divide the power inspection area image into several local image sub-blocks according to a fixed window sliding method, extract the texture gradient features, edge density index, thermal imaging temperature change rate and structural self-similarity of each local image sub-block, and construct local features.

[0028] Specifically, the power inspection area image is divided into sliding windows of a fixed size (e.g., 64×64 pixels), with the step size controlled at 10% to 25% of the image resolution to ensure overlapping area coverage and enhance continuity. Simultaneously, windowing is performed on the RGB image and the thermal imaging image to ensure spatial alignment. For each local image sub-block, four types of multi-dimensional features—texture gradient, edge density, thermal imaging temperature change rate, and structural self-similarity—are extracted and fused to construct a feature vector. The specific definitions and representational functions of each type of feature are as follows: Texture gradient features: By calculating features such as histogram of oriented gradients (HOG) and local binary pattern (LBP), the surface texture and structural complexity of local regions are quantitatively characterized. Edge density index: The edge pixel density is calculated using the Canny or Sobel operator to determine the degree of change in the structural contour; Thermal imaging temperature change rate: obtained by extracting the temperature distribution curve of the region in the infrared image and calculating the temperature change gradient, used to detect abnormal heat accumulation; Structural self-similarity: obtained by calculating the structural similarity between autocorrelation image blocks within a region, used to identify device connection anomalies or surface structural mutations.

[0029] S22. Based on the local features, a local discriminative embedding algorithm based on sparse representation is used to construct a sparse dictionary containing normal operating condition samples and historical fault samples, and sparse coefficient decomposition is performed on the texture detail vector and heat distribution curve.

[0030] Specifically, before system deployment, a training set is constructed using offline labeled equipment image samples, and the samples are divided into "normal working condition images" and "known fault images". The sliding sub-block features of each training image are extracted, and feature dictionary libraries representing "normal class" and "fault class" are constructed accordingly. Each dictionary atom corresponds to a typical feature pattern (such as normal insulation surface texture, common overheating edge, connection point offset, etc.).

[0031] The feature vectors of the newly acquired local image sub-blocks are projected onto the aforementioned feature dictionary. Through sparse reconstruction operations, the reconstruction capability of the local image sub-blocks on different category dictionaries is calculated to obtain the sparse coefficient vector corresponding to each sub-block, so as to characterize the distribution tendency of its features in normal mode and fault mode.

[0032] S23. Introduce a local discriminant embedding criterion in the sparse coefficient space, calculate the sparse residual distribution of normal operating condition samples and historical fault samples, and obtain a discriminant embedding vector with fault sensitivity by maximizing inter-class boundaries and minimizing intra-class compactness, and generate preliminary candidate region responses by combining neighborhood topological relationships.

[0033] Specifically, a local graph structure is constructed in the sparse coefficient space, where nodes are sub-blocks and edges represent spatial proximity and feature similarity; the distribution of sparse reconstruction residuals is analyzed to identify regions with strong reconstruction capabilities for fault classes but weak reconstruction capabilities for normal classes; a discriminative embedding strategy is used to remap the sparse residuals, emphasizing inter-class differences and compressing intra-class volatility.

[0034] The fault sensitivity of each sub-block's embedding vector is mapped to a thermal response value; a preliminary candidate response map is drawn in the image space, and the high response area is the potential abnormal area; the preliminary candidate map is presented as a grayscale heat map or a color heat map to visually highlight the abnormal sub-blocks.

[0035] S24. Perform cross-regional connectivity clustering and residual intensity screening on the initial candidate region response to generate the final fault candidate region map of potential defect locations.

[0036] S3. Based on the fault candidate region map, identify the fault type and predict the risk level of each candidate region, and output the fault identification and prediction results.

[0037] The specific implementation process for this step is as follows: S31. Extract multimodal fusion features from each candidate region in the fault candidate region map, including texture gradient of optical image, edge direction histogram, temperature time-series curve features of thermal imaging image, and structural topology and background association features. S32. Input the multimodal fusion features into the multimodal migration forest algorithm, and achieve preliminary classification of fault types across scenarios by aligning the feature distribution between the source domain training set and the target domain inspection scenario; Specifically, a source domain training set is constructed based on historical fault image samples (such as labeled images from different sites and under different shooting conditions). Each sample includes multimodal features and corresponding fault category labels (such as insulation aging, surface corrosion, local overheating, structural deformation, etc.). Candidate region features from current on-site inspection images are used as input to the target domain. During training, a feature alignment mechanism (such as distribution reweighting, node gain adjustment, etc.) is introduced to ensure that each tree in the forest can learn cross-domain feature mappings. A set of fault category prediction probability values ​​is output for each candidate region, forming a preliminary classification probability map. Simultaneously, the classification confidence of each prediction result is recorded for subsequent optimization and risk modeling.

[0038] S33. Based on the classification results of the multimodal migration forest algorithm, a conditional random field model is introduced. The spatial neighborhood consistency and contextual dependency between candidate regions are used as constraints to globally optimize the classification probability map output by the forest, resulting in a more stable distribution of region labels for category discrimination. Specifically, all candidate fault regions are constructed into an undirected graph structure G=(V,E), where each node in the node set V corresponds to a candidate region in the image, and the edge set E is used to connect spatially adjacent regions, for example, establishing edges between regions physically less than 50 pixels apart. Each edge carries weight information, reflecting the structural / texture / heat distribution similarity between two regions. Each node is assigned a preliminary class probability distribution vector (derived from migration forest), and the edge weights reflect the "probability that two nodes should have the same label".

[0039] If two regions share a common edge contour, it indicates that they may belong to the same fault characteristic region; if the heat distribution between regions transitions smoothly without obvious temperature gradient, they may belong to the same fault hot spot; identify the structural constraint relationship of power equipment (such as multiple connection points belonging to the same busbar) and add it as a spatial constraint condition; introduce prior associations through the relative position, shape direction, gradient direction, etc. between candidate regions.

[0040] On the preliminary classification probability map, the Conditional Random Field (CRF) inference process is performed. The core objectives are: to enhance the consistency of labels between regions, especially in regions with low confidence, by referring to the category tendency of their neighboring high-confidence regions; to retain isolated strong confidence outliers, prevent complete smoothing, and avoid losing local true faults.

[0041] The output includes the updated category labels, label confidence redistribution values, and possible corrections to edge region states (e.g., blurred boundary regions are forced to fit).

[0042] S34. Based on the region label distribution optimized by the conditional random field, and combined with the classification confidence, residual energy function and historical fault modes, the risk level of the candidate region is predicted, and the fault identification prediction result containing fault category label, confidence and risk score is output.

[0043] Furthermore, the formula for the conditional random field model is as follows: E(X) = ∑ i (-logP(x i |F i )+λ·||r i ||2)+∑ (i,j) η·I(x i ≠x j )·exp(-||F i -F j || 2 / σ 2 ), Where E(X) represents the global energy function of the label distribution of all candidate regions in the entire power line inspection image. The smaller its value, the better the label distribution is in terms of classification confidence, anomaly residuals, and spatial consistency. It is the core function minimized by the entire conditional random field optimization objective; X represents the set of predicted labels for all candidate regions; x i F represents the fault category label for fault candidate region i; i Represents the multimodal fusion feature vector of fault candidate region i; -logP(x i |F i ) indicates that the fault candidate region i belongs to the label Classification confidence loss; r i The residual vector representing region i, generated based on sparse representation and local discriminative embedding algorithm, reflects the reconstruction difference between the current region and normal samples and historical fault samples in the sparse dictionary space; ||r i||2 represents the sparse residual intensity of region i. A larger value indicates a greater deviation from the normal pattern in structure or thermal imaging, suggesting a higher anomaly potential. λ represents the penalty weighting coefficient of the residual energy, used to balance the influence of classification confidence and anomaly significance. (i,j) represents a pair of candidate regions with spatial adjacency in the graph structure; I(x i ≠x j ) represents the indicator function, which is 1 if the predicted labels of adjacent regions i and j are inconsistent, and 0 otherwise; ||F i -F j || 2 This indicates the degree of difference between adjacent regions in the multimodal feature space. The more similar the features, the smaller this value, indicating that the two regions should have the same label. σ represents the scaling coefficient of the Gaussian kernel function, which is used to adjust the influence range of feature distance in the similarity weight calculation. η represents the global weight of the spatial neighborhood consistency constraint.

[0044] In a preferred embodiment of this solution, firstly, an undirected graph structure is constructed for all candidate regions within the fault candidate region graph, G=(V,E), where the node set V corresponds to all fault candidate regions extracted from the image, and each node maps to a local abnormal region of the device; the edge set E is used to characterize the spatial adjacency relationship between each candidate region. Taking multiple adjacent high-temperature areas detected in the image of power equipment as an example, for multiple adjacent high-temperature abnormal areas detected in the image, the connection relationship can be established by combining the pixel distance between the areas, texture similarity and heat distribution continuity, so that multiple areas belonging to the same equipment fault source can be associated.

[0045] Then, a multimodal fusion feature vector F is extracted for each fault candidate region. i This fusion feature comprehensively integrates multi-dimensional device status information, specifically including visible light image texture gradient features, edge direction histogram features, thermal imaging temperature time-series change curve features, and device structure topology association features, which can comprehensively characterize the abnormal characteristics of candidate regions.

[0046] By inputting the above features into a multimodal migration forest model, the initial probability distribution of various equipment faults corresponding to each candidate region can be output, thus completing the preliminary determination of the fault category. This process is illustrated below: For example, after model inference, the failure probability of a candidate region for a certain device is as follows: insulation aging failure probability 0.25, surface corrosion failure probability 0.15, and local overheating failure probability 0.60. The model uses the local overheating fault with the highest probability as the initial predicted label for that candidate region, and constructs a label set from the initial labels of all candidate regions. The input conditional random field model is used to iteratively optimize the global label distribution by minimizing the global energy function E(X). According to the formula of the conditional random field model, the global energy function consists of three constraint terms, as follows: The first term is -logP(x) i |F i This term represents the classification confidence loss of a single candidate region based on its own multimodal fusion features. Its core function is to constrain the direction of label optimization, ensure that the global optimization result fits the preliminary classification conclusion of the multimodal migration forest model, and avoid the optimization process from deviating from the basic feature judgment result. The second term is λ||r i ||2, where r i The residual vector is the output of the pre-sparse representation and local discriminative embedding algorithm. It is used to quantify the degree of feature deviation between the current abnormal region and the normal operating state of the equipment. In practical applications, although the initial classification probability of some candidate regions of equipment connection points is low, the thermal imaging temperature change features are significant and the sparse reconstruction residuals of normal equipment samples are significantly different. This term can effectively increase the matching probability of its abnormal fault label.

[0047] The third term is ηI(x) i ≠x j )exp(-||F i -F j || 2 / σ 2 This term is used to constrain the consistency of labels in adjacent candidate regions. If two adjacent regions have similar texture structures, continuous heat distribution, and adjacent spatial locations, the model will reduce the probability of assigning different fault labels to them, so that adjacent regions tend to maintain a consistent fault category. For obvious boundary regions, different labels are allowed to be maintained to avoid excessive smoothing that leads to the loss of fault boundaries.

[0048] After iterative optimization of the aforementioned conditional random field model, a more stable regional label distribution is obtained. The risk score is calculated by combining classification confidence, residual energy, and historical fault modes. The final output includes fault identification prediction results including fault category labels, classification confidence, and risk level scores, providing state input for subsequent reinforcement learning strategy optimization.

[0049] S4. Based on the fault identification and prediction results, construct a state-decision model with the inspection task objective as the reward function, adaptively optimize and train the fault handling strategy, and dynamically generate the optimal handling plan and inspection path adjustment strategy.

[0050] The specific implementation process for this step is as follows: S41. Based on the fault identification and prediction results, a high-dimensional state space integrating multi-dimensional information of power inspection is constructed. The multi-dimensional information of power inspection includes power inspection state variables, equipment fault status, robot system status, inspection environment characteristics and other variables, specifically covering fault category labels, risk scores, confidence levels, pose information, task progress and environmental constraints, etc., which are used to provide complete decision input for reinforcement learning policy networks. Specifically, based on the fault identification and prediction results, the equipment fault identification results, robot operating status, and inspection scenario information are fused and uniformly encoded into a reinforcement learning input state vector. The definitions of various information types are as follows: Equipment fault status information (static discrimination information) includes the relative position coordinates of the fault candidate area (such as the relative position on the equipment), fault type labels (such as thermal anomaly, insulation damage, rust flaking, etc.), identification confidence level and risk level score, which are used to characterize the abnormal attributes and degree of harm of the equipment. The robot system status information (dynamic execution information) includes the robot's current pose (position and orientation), task progress (number of processed / pending faults, inspection sequence), remaining battery power, and system resource load, which are used to characterize the execution status of the inspection task; The inspection environment feature information (scene constraint information) includes local terrain categories (flat, slope, dense obstacles, etc.), path access risk (based on image and map analysis) and task time constraints (such as task deadline and cycle requirements), which are used to characterize the accessibility of the inspection scene and the task time limit requirements.

[0051] The structured information described above is encoded into a unified vector form to form the high-dimensional state space, which serves as the input state for the reinforcement learning policy network, ensuring that each state vector fully reflects the current inspection context. The state vector simultaneously contains static information (such as recognition results) and dynamic information (such as task progress and consumed resources), reflecting the temporal and evolutionary nature of the inspection task.

[0052] S42. Based on the high-dimensional state space, a composite reward function is set with the objectives of minimizing the efficiency of inspection task completion, the accuracy of fault handling, and the resource consumption. A state transition function is constructed by combining the kinematic constraints of the legged robot and the task priority. Specifically, to ensure that reinforcement learning behavior meets multiple engineering objectives, a reward structure is designed for the following three main objectives: The task efficiency target encourages the rapid completion of inspection tasks and improves the timeliness of fault response; The goal of accurate handling is to assess whether the handling is correct based on the S3 predicted fault confidence level and subsequent verification feedback. Resource consumption control targets, including electricity consumption, time, and detour distance, incentivize the system to save energy and optimize routes.

[0053] While setting the above three types of objectives, task priority weights (such as prioritizing response to high-risk faults) are introduced as weighting factors in the reward function to support differentiated management of task scheduling.

[0054] In the reinforcement learning model, the state transition function describes how the system enters the next state after "current state + action taken". The transition function is constructed based on the kinematic model of the legged robot, the inspection task logic, and takes into account environmental feedback factors. The kinematic model of the legged robot incorporates gait switching, turning radius, and obstacle crossing restrictions to ensure that the generated actions are physically executable. The inspection task logic is as follows: if the action is "detour" or "skip", the task status flag is updated in the task sequence; if the action is "handle the current fault", the current fault is removed from the pending list and converted into history.

[0055] The environmental feedback factors refer to the real-time updates of environmental status information through sensor feedback mechanisms when encountering sudden obstacles or terrain disturbances during path execution, thereby affecting the state transition logic.

[0056] S43. Based on the state transition function, a reinforcement learning algorithm based on deep Q-network and policy gradient joint optimization is adopted to perform policy iterative optimization in a simulated interactive environment, and dynamically generate multi-dimensional action output including detour path, inspection order and handling method. Specifically, a joint training mode of Deep Q-Network (DQN) and Policy Gradient Algorithm (such as A3C or PPO) is adopted. The Deep Q-Network is used to handle discrete action decisions (such as selecting the next fault point or executing the path selection), and the Policy Gradient Algorithm optimizes the continuous action dimension (such as detour angle, span, and avoidance path length). The joint training of the two enables the algorithm to achieve a balance between policy continuity and action selection discreteness.

[0057] To provide a reliable environment for the above training, a multi-task inspection simulation system needs to be constructed. The multi-task inspection simulation system includes high-fidelity equipment layout simulation, editable terrain obstacle distribution, injectable multi-type fault scenarios (supporting risk level labels and simulated handling response feedback), and system simulation supporting robot dynamics and perception. Furthermore, randomness (such as random fault location, path obstacle changes, etc.) is introduced during the training process to improve the strategy generalization ability.

[0058] Each policy iteration outputs a multi-dimensional action vector, typically including: Path selection, i.e., the selection of the next target point (based on the current state and the remaining tasks); Avoidance action, i.e., whether to execute avoidance behavior and its parameters; Task decision-making actions, such as whether to handle it immediately, postpone handling, or transfer it to manual review; Gait mode switching command, used to adjust posture to adapt to different terrain conditions.

[0059] S44. Based on the converged policy network, the current state is mapped to the optimal handling plan and inspection path adjustment strategy, and the task execution sequence is constructed in a graph structure.

[0060] Specifically, after training converges, the learned policy is deployed in the actual inspection task. Whenever the system receives the current state vector, the policy network automatically outputs the corresponding optimal action decision. Multiple rounds of policy execution form a complete task execution path.

[0061] The final inspection path and handling plan are represented in the form of a graph structure. Nodes represent each candidate fault area and its handling status; edges represent the actual path connections of the robot, including physical connection paths and task sequence paths; weights represent the resource consumption or risk level jump values ​​on the edges, used for subsequent system backtracking analysis and path optimization reuse.

[0062] This graph structure can be used not only for real-time robot navigation, but also uploaded to the backend system for task recording, operation and maintenance analysis, and human-machine collaboration.

[0063] Furthermore, the formula for the reinforcement learning algorithm is as follows: J(p) = E_{(s,a,s')~π p}[ w(s)( (s,a) +gV(s') V(s))logπ p (a|s)]+cE[C(s,a)], Where J(p) represents the reinforcement learning optimization objective function, used to evaluate the merits of the current handling plan or inspection path; E_{(s,a,s')~πp} represents the optimization objective function for policy π. p The empirical expected value of the sampled state-action-transition triplet; s represents the fault identification prediction result; a represents the inspection behaviors that the robot can perform, including detour path selection, inspection sequence adjustment, and handling methods; s' represents the next state reached after performing the action, including updated image features, risk score, and robot position; π p This represents the policy distribution determined by parameter p; (s,a) represents the composite immediate reward function; V(s) and V(s') represent the value estimates of the value network in state s and the next state s'; g represents the discount factor, used to balance immediate and future rewards, preventing the robot from focusing only on the shortest path in the short term while ignoring long-term risks and inspection efficiency; w(s) represents the scenario risk-information weight; πp (a|s) represents the policy probability of taking action a in state s; c represents the cost penalty coefficient; C(s,a) represents the action cost function.

[0064] The formula for calculating (s,a) is as follows: (s,a)=α1·E task (s,a) +α2·A acc (s,a) α3·R cons (s,a), Among them, E task (s,a) represents a task efficiency indicator, such as the percentage of inspection nodes completed per unit of time; A acc (s,a) represents the accuracy index of the handling, such as the improved stability of fault category labels after re-inspection and the reduction of false alarms / false negatives; R cons (s,a) represents resource consumption indicators, such as walking time, steps, and energy consumption; α1, α2, and α3 represent weighting coefficients, with values ​​greater than 0.

[0065] The formula for calculating C(s,a) is as follows: C(s,a)=r1L(s,a)+r2E(s,a)+r3S(s,a), Wherein, L(s,a) represents the path cost, which is calculated including path length, number of obstacle crossings (especially crossing of obstacles within 40cm), and number of detour steps; E(s,a) represents the energy consumption estimate, which is calculated based on the robot's dynamics model and joint actuator feedback (current, torque) to determine the energy consumption during the action execution process; S(s,a) represents the safety penalty, which measures whether the robot's action enters a high-temperature / electrical area or violates the minimum safe distance; r1, r2, and r3 represent the cost weights, which are determined by the requirements of energy efficiency, safety, and path efficiency.

[0066] In a preferred embodiment of this solution, a reinforcement learning state input is constructed based on the fault identification prediction result output in step S3. This reinforcement learning state input integrates multi-dimensional perception and operational information, specifically including the fault candidate region location coordinates, fault category label, classification confidence score, risk level score, as well as the current pose of the legged robot, remaining battery power, task completion progress, and surrounding environmental obstacle information. This multi-dimensional information is encoded into a state vector s, as illustrated below: If the fault identification result determines that there is a local overheating fault in the transformer connection, the system will automatically extract the spatial coordinates of the fault candidate area, the fault category of "local overheating", the risk score of 0.85, and the real-time spatial distance and pose information of the legged robot relative to the fault point, and integrate them to form the reinforcement learning state input at the current moment for subsequent action strategy solution.

[0067] After the model obtains the current global state s, the policy network determines the policy distribution π. p (a|s) Inference outputs the executable action 'a' of the legged robot. This action includes approaching the fault area, bypassing the obstacle area, adjusting the inspection sequence, re-acquiring images, and performing fault verification. After the robot completes the current action, the environment and equipment status are updated synchronously, and it jumps to the next state 's', which includes updated equipment fault information, robot pose information, and environmental change information.

[0068] During the above-mentioned action interaction process, an immediate reward function is calculated based on the execution effect of the action. (s,a) enables a quantitative evaluation of the quality of actions. The reward mechanism employs a design logic that combines positive incentives and negative penalties: When a robot quickly completes high-risk fault verification, reduces path consumption, and improves fault confirmation accuracy, it is given a high positive reward; if the action causes redundancy in the inspection path, increases equipment energy consumption, or causes the robot to mistakenly enter a high-risk operation area, the basic reward is discounted through a cost function to achieve a negative constraint.

[0069] Each time a complete experience sample s,a,s' of "current state-action-next state" is generated, the system randomly samples the sample and inputs it into the reinforcement learning network through the experience replay mechanism to complete the iterative update of the policy parameter p, continuously maximize the policy optimization objective function J(p), and gradually iteratively optimize the robot's intelligent inspection decision strategy.

[0070] During parameter optimization, the value network fits the current state value V(s) and the next state value V(s') after iteration, respectively. A discount factor g is introduced to quantify the impact on long-term future returns. Simultaneously, the strategy focus under different fault scenarios is adjusted by combining the scenario risk-information weight w(s). For high-risk fault areas, the risk weight is automatically increased, constraining the robot to prioritize safety-assured actions such as close-range fault review and precise verification. For low-risk routine inspection areas, the risk constraint weight is reduced, prioritizing overall inspection efficiency and task progress speed, achieving a dynamic balance between safety and efficiency.

[0071] After multiple rounds of simulated environment training and interactive optimization, when the policy optimization objective function tends to converge, the policy network has end-to-end optimal decision-making capability. It can adaptively output the optimal action policy of the robot based on the real-time input global state, generate the optimal handling scheme including dynamic adjustment of inspection path, priority ranking of fault verification, and equipment fault handling method, and map the policy instructions into low-level control instructions that the legged robot can recognize, so as to complete the automated and intelligent substation equipment inspection operation.

[0072] S5. Map the optimal handling scheme and inspection path adjustment strategy into motion control commands for the legged robot, and drive the legged robot to autonomously complete fault verification and subsequent operation tasks according to the optimized path. In the process of generating the motion control commands, a Bayesian-optimized motion fuzzy correction mechanism is introduced to perform real-time dynamic correction of path nodes with terrain undulation interference or obstacle interference.

[0073] Specifically, the system receives the optimal inspection path and fault handling plan output by the reinforcement learning module, including the target equipment number, priority, path node sequence, and operation action type (such as approach / retake / identify / mark, etc.). The motion scheduling module parses the scheme and transforms it into a sequence of underlying control instructions. The sequence of underlying control instructions mainly includes path tracking instructions, task execution instructions, and environmental interaction instructions. The path tracking instructions are used to control the robot to move gradually to the target area. The task execution instructions include image re-acquisition, near-field fault identification, robotic arm pointing, and sending prompt signals. The environmental interaction instructions include adjusting posture, lifting legs to cross obstacles, and adapting to changes in posture due to terrain.

[0074] It should be noted that when the legged robot performs its tasks, unpredictable terrain disturbances at the inspection site, such as rocks, sloping ramps, and puddles, may cause its standard actions to fail or pose a risk of failure. The specific processing mechanism is as follows: First, risk perception of path nodes is performed, which is based on the robot's foot force sensors, inertial navigation system and forward depth vision module to monitor terrain features in the path in real time. Secondly, risk nodes are identified. When a high-risk characteristic such as a large height difference, excessive slope, or potential slippage is identified at the current path node, an action correction mechanism is triggered. Finally, by using a Bayesian optimization scheduler, without relying on a large number of field samples, the robot's accumulated terrain-motion history experience is used to fuzzily evaluate the reliability of the current action execution under different adjustment amounts, and output a set of action parameter adjustment combinations with the most stable expectations (such as increasing stride length, increasing leg lift height, and switching gait to climbing mode).

[0075] The controller receives the optimized output correction scheme and injects it into the existing motion trajectory in real time; it realizes sliding adjustment, flexible obstacle avoidance and stable recovery during continuous path tracking; if a node cannot be safely passed, the system can actively switch to the backup path or issue a manual intervention prompt.

[0076] Once the robot reaches the designated fault location, it automatically switches to stable observation mode and controls the head camera to take multiple re-images of the target area according to the aforementioned task instructions. If there are task feedback conditions (such as re-inspection confirmation / infrared temperature measurement), the corresponding instructions are executed. The image / thermal imaging data is uploaded to the edge server or cloud platform in real time for rapid secondary identification and verification. After the verification is completed, the robot continues to execute the subsequent inspection path according to the scheduling instructions, or sends back new path adjustment suggestions based on the actual execution status.

[0077] Reference Figure 3 A fault identification and prediction system for power inspection equipment based on machine vision includes an image acquisition module 1, a fault candidate region extraction module 2, a fault identification and prediction module 3, a strategy optimization module 4, and an execution control module 5. Image acquisition module 1 is used to acquire images of the power inspection area through a high-resolution camera deployed on the legged robot; The fault candidate region extraction module 2 is communicatively connected to the image acquisition module 1 and is used to receive the power inspection area image and dynamically generate a fault candidate region map in the local area using a local discriminant embedding algorithm based on sparse representation. The fault identification and prediction module 3 is communicatively connected to the fault candidate region extraction module 2. It is used to receive the fault candidate region map, use a multimodal migration forest algorithm based on conditional random field optimization to identify the fault type and predict the risk level of each candidate region, and output the fault identification and prediction results. The strategy optimization module 4 is communicatively connected to the fault identification and prediction module 3. It is used to receive the fault identification and prediction results, construct a state-decision model with the inspection task objective as the reward function using a reinforcement learning algorithm, adaptively optimize and train the fault handling strategy, and dynamically generate the optimal handling plan and inspection path adjustment strategy. The execution control module 5 is communicatively connected to the strategy optimization module 4, and is used to map the optimal handling plan and inspection path adjustment strategy into motion control commands for the legged robot, and drive the robot to autonomously complete fault verification and subsequent operation tasks according to the optimized path.

[0078] A machine vision-based fault identification and prediction system for power inspection equipment is used to execute the machine vision-based fault identification and prediction method for power inspection equipment described in this embodiment, which will not be elaborated here.

[0079] The machine vision-based fault identification and prediction method for power inspection equipment described in this embodiment can stably acquire multi-view images of power equipment in complex terrain by deploying a high-resolution camera on a legged robot. This effectively enhances the ability to collect multi-modal information such as equipment surface condition, connection structure, and thermal imaging, providing high-quality raw data support for subsequent fault identification.

[0080] Furthermore, by employing a local discriminative embedding algorithm based on sparse representation, it can extract discriminative feature representations in local regions of images, dynamically generate fault candidate region maps, reduce noise interference, and improve the accuracy of fault detection and positioning. It is particularly suitable for small-scale fault region extraction in complex backgrounds.

[0081] The multimodal migration forest algorithm based on conditional random field optimization in step S3 can comprehensively process image modalities, structural features and contextual relationships, perform joint reasoning and risk level classification of candidate regions, effectively improve the robustness of fault type identification and the reliability of risk warning, and is particularly suitable for cross-scene migration identification tasks in new environments.

[0082] By introducing reinforcement learning to construct a state-decision model with the inspection task objective as the reward function, it can automatically generate adaptive optimal handling solutions and path adjustment strategies based on the fault type and inspection history, realizing the transformation from passive response to proactive avoidance and optimization, and improving fault handling efficiency and inspection task completion quality.

[0083] By mapping the optimal handling plan and path strategy into control commands for a legged robot, the robot is equipped with the ability to dynamically adjust its path and execute operational tasks. It can autonomously complete fault verification and subsequent response operations during actual inspections, significantly improving the system's intelligence level and operational automation capabilities, and reducing the cost of human intervention.

[0084] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A fault identification and prediction method for power inspection equipment based on machine vision, characterized in that, The specific steps are as follows: S1. Acquire images of the power inspection area using a high-resolution camera deployed on a legged robot; S2. Based on the power inspection area image, after local feature extraction and candidate area screening, a fault candidate area map is dynamically generated in the local area. S3. Based on the fault candidate region map, identify the fault type and predict the risk level of each candidate region, and output the fault identification and prediction results. S4. Based on the fault identification and prediction results, construct a state-decision model with the inspection task objective as the reward function, perform adaptive optimization training on the fault handling strategy, and dynamically generate the optimal handling plan and inspection path adjustment strategy. S5. Map the optimal handling plan and inspection path adjustment strategy into motion control commands for the legged robot, and drive the legged robot to autonomously complete fault verification and subsequent operation tasks according to the optimized path.

2. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 1, characterized in that: The images of the power inspection area include images of the power equipment itself, images of the connection and installation parts, images of the operating status indication, images of the surface condition, images of the environmental background, and thermal imaging images.

3. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 1, characterized in that, Step S2 includes the following steps: S21. Divide the power inspection area image into several local image sub-blocks according to a fixed window sliding method, and extract the texture gradient features, edge density index, thermal imaging temperature change rate and structural self-similarity of each local image sub-block to construct local features; S22. Based on the local features, a local discriminative embedding algorithm based on sparse representation is used to construct a sparse dictionary containing normal working condition samples and historical fault samples, and sparse coefficient decomposition is performed on the texture detail vector and heat distribution curve. S23. Introduce a local discriminant embedding criterion in the sparse coefficient space, calculate the sparse residual distribution of normal operating condition samples and historical fault samples, and obtain a discriminant embedding vector with fault sensitivity by maximizing inter-class boundaries and minimizing intra-class compactness, and generate preliminary candidate region responses by combining neighborhood topological relationships. S24. Perform cross-regional connectivity clustering and residual intensity screening on the initial candidate region response to generate the final fault candidate region map of potential defect locations.

4. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 1, characterized in that, Step S3 includes the following steps: S31. Extract multimodal fusion features from each candidate region in the fault candidate region map, including texture gradient of optical image, edge direction histogram, temperature time-series curve features of thermal imaging image, and structural topology and background association features. S32. Input the multimodal fusion features into the multimodal migration forest algorithm, and achieve preliminary classification of fault types across scenarios by aligning the feature distribution between the source domain training set and the target domain inspection scenario; S33. Based on the classification results of the multimodal migration forest algorithm, a conditional random field model is introduced. The spatial neighborhood consistency and contextual dependency between candidate regions are used as constraints to globally optimize the classification probability map output by the forest, resulting in a more stable distribution of region labels for category discrimination. S34. Based on the region label distribution optimized by the conditional random field, and combined with the classification confidence, residual energy function and historical fault modes, the risk level of the candidate region is predicted, and the fault identification prediction result containing fault category label, confidence and risk score is output.

5. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 4, characterized in that, The formula for the conditional random field model is as follows: E(X)=∑ i (-logP(x i |F i )+λ·||r i ||2)+∑ (i,j) η·I(x i ≠x j )·exp(-||F i -F j || 2 / s 2 ), Where E(X) represents the global energy function of the label distribution of all candidate regions in the entire power line inspection image; X represents the set of predicted labels for all candidate regions; x i F represents the fault category label for fault candidate region i; i Represents the multimodal fusion feature vector of fault candidate region i; -logP(x i |F i ) indicates that the fault candidate region i belongs to the label Classification confidence loss; r i The residual vector representing region i is generated based on sparse representation and a local discriminative embedding algorithm; ||r i ||2 represents the sparse residual intensity of region i; λ represents the penalty weight coefficient of residual energy; (i,j) represents a pair of candidate regions with spatial adjacency in the graph structure; I(x i ≠x j ) represents an indicator function; ||F i -F j || 2 σ represents the degree of difference between adjacent regions in the multimodal feature space; σ represents the scaling coefficient of the Gaussian kernel function; and η represents the global weight of the spatial neighborhood consistency constraint.

6. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 1, characterized in that, Step S4 includes the following steps: S41. Based on the fault identification and prediction results, a high-dimensional state space integrating multi-dimensional information of power inspection is constructed. The multi-dimensional information of power inspection includes power inspection state variables, equipment fault status, robot system status, and inspection environment characteristics. S42. Based on the high-dimensional state space, a composite reward function is set with the objectives of minimizing the efficiency of inspection task completion, the accuracy of fault handling, and the resource consumption. A state transition function is constructed by combining the kinematic constraints of the legged robot and the task priority. S43. Based on the state transition function, a reinforcement learning algorithm based on deep Q-network and policy gradient joint optimization is adopted to perform policy iterative optimization in a simulated interactive environment, and dynamically generate multi-dimensional action output including detour path, inspection order and handling method. S44. Based on the converged policy network, the current state is mapped to the optimal handling plan and inspection path adjustment strategy, and the task execution sequence is constructed in a graph structure.

7. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 6, characterized in that, The formula for the reinforcement learning algorithm is as follows: J(p)=E_{(s,a,s')~π p }[ w(s)( (s,a) +gV(s') V(s))logπ p (a|s)]+cE[C(s,a)], Where J(p) represents the reinforcement learning optimization objective function; E_{(s,a,s')~π p } represents the strategy π p The empirical expected value of the sampled state-action-transition triplet; s represents the fault identification prediction result; a represents the inspection behavior that the robot can perform; s' represents the next state reached after performing the action; π p This represents the policy distribution determined by parameter p; (s,a) represents the composite immediate reward function; V(s) and V(s') represent the value estimates of the value network in state s and the next state s'; g represents the discount factor; w(s) represents the scenario risk-information weight; π p (a|s) represents the policy probability of taking action a in state s; c represents the cost penalty coefficient; C(s,a) represents the action cost function.

8. The fault identification and prediction method for power inspection equipment based on machine vision according to claim 1, characterized in that: In step S5, a Bayesian-optimized motion fuzzy correction mechanism is introduced during the generation of motion control commands to dynamically correct path nodes that are subject to terrain undulations or obstacles in real time.

9. A machine vision-based fault identification and prediction system for power inspection equipment, used to execute the machine vision-based fault identification and prediction method for power inspection equipment as described in any one of claims 1-8, characterized in that: It includes an image acquisition module, a fault candidate region extraction module, a fault identification and prediction module, a strategy optimization module, and an execution control module; The image acquisition module is used to acquire images of the power inspection area through a high-resolution camera deployed on the legged robot; The fault candidate region extraction module is communicatively connected to the image acquisition module and is used to receive the power inspection area image and dynamically generate a fault candidate region map in the local area using a local discriminant embedding algorithm based on sparse representation. The fault identification and prediction module is communicatively connected to the fault candidate region extraction module. It is used to receive the fault candidate region map, use a multimodal migration forest algorithm based on conditional random field optimization to identify the fault type and predict the risk level of each candidate region, and output the fault identification and prediction results. The strategy optimization module is communicatively connected to the fault identification and prediction module. It is used to receive the fault identification and prediction results, and to construct a state-decision model with the inspection task objective as the reward function using a reinforcement learning algorithm. It then performs adaptive optimization training on the fault handling strategy and dynamically generates the optimal handling plan and inspection path adjustment strategy. The execution control module is communicatively connected to the strategy optimization module and is used to map the optimal handling plan and inspection path adjustment strategy into motion control commands for the legged robot, and drive the legged robot to autonomously complete fault verification and subsequent operation tasks according to the optimized path.

Images