Power transformation equipment multi-target inspection method and system based on deep reinforcement learning

Abstract

The invention relates to the technical field of power transformation equipment, in particular to a power transformation equipment multi-target inspection method and system based on deep reinforcement learning, and the method comprises the steps: obtaining equipment temperature, vibration, ultrasonic wave and image data in real time through a multi-source sensor network, and constructing state space representation; generating a multi-target inspection strategy by using a deep reinforcement learning model, and guiding the execution of an equipment inspection task; and finally generating an inspection report containing equipment state evaluation and fault diagnosis suggestions. According to the method, multiple innovative technologies are fused, intelligent, self-adaptive and efficient inspection is realized, the inspection efficiency and accuracy are remarkably improved, a new scheme is provided for intelligent operation and maintenance of a transformer substation, and the method has important significance for improving the safety and reliability of a power system.

Description

Technical Field

[0001] This invention relates to the field of power equipment technology, and in particular to a multi-target inspection method and system for power equipment based on deep reinforcement learning. Background Art

[0002] With the continuous development and increasing complexity of power systems, the safe and stable operation of substation equipment plays a crucial role in the reliability of the entire power grid. Traditional substation equipment inspection methods mainly rely on regular manual inspections. This approach is not only time-consuming and labor-intensive but also easily affected by human factors, making it difficult to guarantee the comprehensiveness and timeliness of inspections. In recent years, with the rapid development of artificial intelligence technology, intelligent inspection robots have begun to be applied in the field of substation inspection.

[0003] Currently, intelligent inspection robots mainly employ a preset path and fixed frequency inspection method. While this method represents an improvement over manual inspection, it still presents several problems. First, preset paths struggle to adapt to the complex and ever-changing environment of substations, failing to flexibly adjust inspection strategies based on equipment status and environmental changes. Second, the fixed-frequency inspection mode cannot differentiate inspections based on the importance and fault risk of different equipment, easily leading to resource waste or insufficient inspection of critical equipment. Furthermore, existing inspection systems often focus only on a single objective, such as equipment status monitoring or fault diagnosis, making it difficult to achieve comprehensive optimization across multiple objectives.

[0004] Furthermore, existing intelligent inspection systems have shortcomings in data processing and decision-making. Most systems rely on simple threshold judgment or rule matching methods for fault diagnosis, making it difficult to handle complex equipment states and fault modes. At the same time, these systems lack the ability to learn autonomously and continuously optimize, failing to improve performance as inspection experience accumulates.

[0005] Existing systems also have limited obstacle avoidance capabilities when facing dynamic obstacles. They typically employ simple stop or detour strategies, making it difficult to maintain high inspection efficiency while ensuring safety. Furthermore, existing systems face challenges in model deployment. Due to the differences between the real-world environment and the training environment, directly applying a trained model to a real-world scenario often leads to performance degradation.

[0006] In summary, existing substation equipment inspection methods and systems have significant shortcomings in terms of intelligence, adaptability, multi-objective optimization, continuous learning, and practical deployment, making it difficult to meet the growing demand for intelligent operation and maintenance in modern substations. Summary of the Invention

[0007] This invention aims to solve the aforementioned technical problems and provides a method and system for multi-target inspection of substation equipment based on deep reinforcement learning. This method achieves intelligent and adaptive multi-target inspection of substation equipment by fusing multi-source sensor data, applying deep reinforcement learning technology, introducing multi-target optimization strategies, employing dynamic obstacle avoidance algorithms, and realizing model transfer and incremental learning.

[0008] This invention proposes a multi-target inspection method and system for substation equipment based on deep reinforcement learning, including: The acquisition steps include: Real-time status data of substation equipment is acquired using a multi-source sensor network, including temperature data, vibration data, ultrasonic data, and image data. The processing steps include: Based on the real-time status data, a state-space representation of the substation equipment is constructed; Based on the state space representation, a multi-target inspection strategy is generated using a deep reinforcement learning model. Based on the multi-target inspection strategy, the inspection task of the substation equipment is executed. Output steps, including: Generate an inspection report, which includes equipment status assessment results and fault diagnosis suggestions.

[0009] Preferably, the processing steps specifically include: Feature extraction is performed on the real-time state data using a convolutional neural network; An attention mechanism is introduced to weightedly fuse features from different modalities; The fused high-dimensional features are mapped to a low-dimensional state representation space.

[0010] As a preferred embodiment, the deep reinforcement learning model is trained using the following objective function: , Representing the state space, Represents the action space. Representative strategy, Indicates the attenuation factor. This represents the reward function.

[0011] The reward function is preferred. Includes the following sub-objectives: , in, Indicates a reward for the completion of the inspection. This indicates a reward for equipment status monitoring. This indicates a reward for fault diagnosis. Indicates energy consumption optimization reward, This indicates an obstacle avoidance reward. to These are the corresponding weighting coefficients.

[0012] As an alternative, dynamic obstacle perception and obstacle avoidance steps are also included: Information about the surrounding environment is obtained using inertial sensors and ultrasonic sensors; A dynamic obstacle model is constructed based on the environmental information; Based on the dynamic obstacle model, the inspection path and frequency are dynamically adjusted.

[0013] The preferred method for dynamically adjusting the inspection frequency is as follows: , in, This indicates the adjusted inspection frequency. Indicates the initial inspection frequency. Indicates the distance to the obstacle. Indicates the safe threshold distance. This is for adjusting the coefficient.

[0014] As an alternative, a hierarchical reinforcement learning step is also included: Build a high-level strategy network to formulate global inspection strategies; Construct a low-level execution network to perform specific inspection actions; The high-level policy network and the low-level execution network are optimized using a hierarchical reinforcement learning algorithm.

[0015] As a preferred option, a model transfer step is also included: Pre-train deep reinforcement learning models in a simulation environment; Transfer learning methods are used to transfer pre-trained models to actual inspection robots; Model fine-tuning and continuous learning are performed in real-world environments.

[0016] As a preferred option, incremental model learning steps are also included: Utilize the experience replay buffer to store historical inspection experience; The model parameters are updated incrementally using the stochastic gradient descent algorithm. The ADAM optimizer is used to adaptively adjust the learning rate.

[0017] A deep reinforcement learning-based multi-target inspection system for substation equipment, used to execute the method, includes: The data acquisition module is used to acquire real-time status data of substation equipment using a multi-source sensor network; The state representation module is used to construct a state-space representation of the substation equipment based on the real-time state data; A deep reinforcement learning module is used to generate a multi-target inspection strategy based on the state space representation. The inspection execution module is used to execute the inspection tasks of the substation equipment based on the multi-target inspection strategy. The dynamic obstacle avoidance module is used to sense the surrounding environment and dynamically adjust the inspection path and frequency; The hierarchical decision-making module, consisting of a high-level policy network and a low-level execution network, is used to implement hierarchical reinforcement learning. The model transfer module is used to transfer pre-trained models to the actual inspection robot and perform fine-tuning. The incremental learning module is used to continuously optimize the deep reinforcement learning model based on new inspection experience. The results output module is used to generate inspection reports that include equipment status assessment results and fault diagnosis suggestions.

[0018] The beneficial effects of this invention: 1. Adaptive Multi-Objective Inspection Strategy: This invention utilizes deep reinforcement learning technology to automatically generate inspection strategies based on real-time status data, achieving dynamic optimization of inspection paths and frequencies. This method not only adapts to the complex and ever-changing substation environment but also simultaneously addresses multiple objectives such as equipment status monitoring, fault diagnosis, and energy consumption optimization, significantly improving the comprehensiveness and efficiency of inspections.

[0019] 2. Comprehensive Status Awareness: By fusing multi-source data such as temperature, vibration, ultrasound, and images, this invention achieves comprehensive status awareness of substation equipment. Combining deep learning feature extraction and fusion techniques, the system can capture more subtle and complex equipment status changes, enabling early fault diagnosis and predictive maintenance.

[0020] 3. Intelligent Dynamic Obstacle Avoidance: The dynamic obstacle perception and avoidance algorithm introduced in this invention enables the inspection robot to dynamically adjust its inspection path and frequency while ensuring safety. This not only improves the safety of inspections but also enhances inspection efficiency to some extent.

[0021] 4. Continuous Learning and Optimization: Through model transfer and incremental learning techniques, the system of this invention can continuously learn from new inspection experiences and continuously optimize inspection strategies. This capability enables the system to adapt to long-term changes in equipment status and emerging failure modes, maintaining long-term high efficiency.

[0022] 5. Hierarchical Decision Architecture: Employing a hierarchical reinforcement learning architecture with a high-level policy network and a low-level execution network, the system can simultaneously handle problems of different scales: long-term planning and short-term execution. This architecture improves the system's decision-making ability and execution efficiency in complex environments.

[0023] 6. Improved Deployment Efficiency: By pre-training the model in a simulation environment and applying it to the real-world environment using transfer learning techniques, this invention significantly reduces the difficulty and cost of model deployment. This method not only accelerates model convergence in the real-world environment but also reduces potential risks to actual equipment.

[0024] 7. Intelligent Report Generation: The system of this invention can automatically generate inspection reports that include equipment status assessment results and fault diagnosis suggestions. This intelligent report generation method provides maintenance personnel with more intuitive and accurate decision support, helping to improve the overall operation and maintenance level of substations.

[0025] In summary, the deep reinforcement learning-based multi-target inspection method and system for substation equipment provided by this invention achieves intelligent, adaptive, and highly efficient substation equipment inspection through the organic combination of multiple innovative technologies. This not only significantly improves the efficiency and accuracy of inspections but also provides a new solution for the intelligent operation and maintenance of substations, which is of great significance for enhancing the safety and reliability of power systems. Attached Figure Description

[0026] Figure 1 This is a logic block diagram of the present invention. DETAILED DESCRIPTION

[0027] Please refer to the attached document. Figure 1 This invention provides a multi-target inspection method and system for substation equipment based on deep reinforcement learning. This method utilizes a multi-source sensor network to acquire real-time status data of substation equipment and combines this with deep reinforcement learning technology to generate multi-target inspection strategies, thus achieving intelligent and adaptive substation equipment inspection. The specific implementation methods of this invention will be described in detail below.

[0028] First, the method of this invention includes an acquisition step, a processing step, and an output step. In the acquisition step, this invention utilizes a multi-source sensor network to acquire real-time status data of the substation equipment. Specifically, this real-time status data includes temperature data, vibration data, ultrasonic data, and image data. Preferably, temperature data can be acquired using an infrared thermal imager with an accuracy of ±0.1°C; vibration data can be acquired using an accelerometer with a sampling frequency typically between 1kHz and 10kHz; ultrasonic data can be acquired using an ultrasonic probe with a frequency range typically between 20kHz and 100kHz; and image data can be acquired using a high-definition camera with a resolution of at least 1080p. This multi-source data acquisition method can comprehensively reflect the operating status of the substation equipment, providing a rich information foundation for subsequent intelligent inspections.

[0029] In the processing steps, this invention first constructs a state-space representation of the substation equipment based on the acquired real-time state data. The purpose of this step is to transform multi-source heterogeneous data into a unified data format that is easy for machine learning algorithms to process. Specifically, this invention employs a feature extraction and fusion method based on deep learning. This method uses a convolutional neural network (CNN) to extract features from different types of data, and then uses an attention mechanism to weightedly fuse these features. Finally, the fused high-dimensional features are mapped to a low-dimensional state representation space. This method not only effectively captures the features of different types of data but also adaptively adjusts the importance of different features, thereby obtaining a comprehensive and compact state representation.

[0030] This invention generates multi-objective inspection strategies based on a constructed state-space representation using a deep reinforcement learning model. The deep reinforcement learning model used here is a Q-learning algorithm based on value function approximation. The objective function of this algorithm can be expressed as: , in, Represents the state space. Represents the action space, Representative strategy, It is the attenuation factor. This is the reward function. The physical meaning of this objective function is to maximize the cumulative reward of the current action and possible future actions. In this invention, the state space... This includes real-time status data of the power equipment and its operating space. This includes operations such as inspection route selection, inspection frequency adjustment, and equipment testing. Strategy It is a mapping function that selects the optimal action based on the current state.

[0031] It is worth noting that this invention employs a multi-objective reward function design, enabling the generated inspection strategy to simultaneously consider multiple objectives. The specific reward function form is as follows: , in, Indicates a reward for the completion of the inspection. This indicates a reward for equipment status monitoring. This indicates a reward for fault diagnosis. Indicates energy consumption optimization reward, This indicates an obstacle avoidance reward. to These are the corresponding weighting coefficients. This multi-objective reward function design can guide deep reinforcement learning models to learn an inspection strategy that balances multiple objectives. For example, It can be set to 0.3. and It can be set to 0.25. and It can be set to 0.1. This weighting reflects the importance of inspection completion and equipment status detection in the inspection task, while also taking into account the needs of energy consumption optimization and safety obstacle avoidance.

[0032] Based on the generated multi-target inspection strategy, the method of this invention performs inspection tasks on substation equipment. During execution, the inspection robot autonomously determines the inspection path, inspection frequency, and key equipment parts to focus on based on real-time status and learned strategies. This intelligent inspection method significantly improves inspection efficiency while enhancing the targeting and effectiveness of the inspection.

[0033] Finally, in the output step, the method of this invention generates an inspection report. This report not only includes routine equipment status data, but also equipment status assessment results and fault diagnosis suggestions derived from deep learning model analysis. This intelligent report generation method can provide maintenance personnel with more intuitive and accurate decision support.

[0034] Next, this invention further details the specific implementation of the processing steps. First, this invention utilizes a convolutional neural network (CNN) to extract features from real-time state data. Different CNN structures are employed for different types of data. For example, a ResNet50 network can be used for image data; a one-dimensional CNN structure can be used for time-series data (such as vibration data). This targeted network design can better capture the features of different types of data.

[0035] After feature extraction, this invention introduces an attention mechanism to perform weighted fusion of features from different modalities. Specifically, a multi-head self-attention mechanism is employed. The mathematical expression of this mechanism is: , in, , , These represent the query, key, and value matrices, respectively. This refers to the dimension of the key vector. Through this mechanism, the model can adaptively adjust the importance of different features, thereby achieving more effective feature fusion.

[0036] Finally, this invention maps the fused high-dimensional features to a low-dimensional state representation space. This step is typically implemented using a fully connected layer and can be represented as: , in, It is a high-dimensional feature after fusion. and These are learnable parameters. It is an activation function (such as ReLU). In this way, we obtain a compact and information-rich state representation. This provides a good input for subsequent reinforcement learning algorithms.

[0037] The deep reinforcement learning model used in this invention is based on the Q-learning algorithm, but with improvements to adapt to the needs of multi-target inspection tasks. Specifically, we use a double Q-network structure to reduce the bias in Q-value estimation and introduce a prioritized experience replay mechanism to improve learning efficiency.

[0038] The model update process can be represented as: , and These are the parameters of the current Q-network and the target Q-network, respectively. It's an instant reward. It is a discount factor. This improved Q-learning algorithm can better handle complex decision-making problems such as substation equipment inspection.

[0039] As can be seen from the detailed description above, the multi-target inspection method for substation equipment based on deep reinforcement learning provided by this invention has the following advantages: First, it can comprehensively perceive the operating status of substation equipment; second, it can adaptively generate multi-target inspection strategies; and finally, it can continuously learn and optimize, constantly improving inspection efficiency and accuracy. These advantages make this invention highly valuable in practical applications, significantly improving the operation and maintenance level of substation equipment and ensuring the safe and stable operation of the power system.

[0040] In a preferred embodiment of the present invention, the reward function It is designed as a weighted sum of multiple sub-objectives. Specifically, the reward function... It can be represented as: , in, Indicates a reward for the completion of the inspection. This indicates a reward for equipment status monitoring. This indicates a reward for fault diagnosis. Indicates energy consumption optimization reward, This indicates an obstacle avoidance reward. to These are the corresponding weighting coefficients.

[0041] This multi-objective reward function is designed to guide deep reinforcement learning models to learn an inspection strategy that balances multiple objectives. In practical applications, the choice of weight coefficients is crucial to model performance. Preferably, It can be set to 0.3. and It can be set to 0.25. and It can be set to 0.1. This weighting reflects the importance of inspection completion and equipment status detection in the inspection task, while also taking into account the needs of energy consumption optimization and safety obstacle avoidance.

[0042] Specifically, rewards for inspection completion This can be defined as the ratio of the number of inspected devices to the total number of devices. Device status monitoring reward. It can be calculated based on the number and severity of detected anomalies. Fault diagnosis reward. The energy consumption optimization reward can be determined based on the number and accuracy of successfully diagnosed faults. This can be calculated by comparing the ratio of the total length of the inspection path to the theoretical shortest path. Finally, obstacle avoidance bonus. It can be defined based on the number of successful obstacle avoidances and the minimum distance to the obstacle.

[0043] The method of this invention also includes a dynamic obstacle perception and avoidance step. In a substation environment, there may be dynamic obstacles such as moving personnel and temporarily placed equipment. To ensure the safe execution of inspection tasks, this method utilizes inertial sensors and ultrasonic sensors to acquire information about the surrounding environment.

[0044] Preferably, the inertial sensor can be a six-axis IMU (Inertial Measurement Unit), comprising a three-axis accelerometer and a three-axis gyroscope, with a sampling frequency typically between 100Hz and 200Hz. The ultrasonic sensor's detection range can be set to 0.2m-4m, with an accuracy of ±1cm. The data from these sensors are processed using data fusion algorithms (such as Kalman filtering) to obtain more accurate environmental information.

[0045] Based on the acquired environmental information, this method constructs a dynamic obstacle model. This model can take the form of a probabilistic occupancy grid map, where each grid cell stores a value representing the probability of being occupied. The map is updated in real time based on new sensor data, thus reflecting the dynamic changes in the environment.

[0046] Based on the constructed dynamic obstacle model, this method can dynamically adjust the inspection path and frequency. Specifically, when an obstacle is detected, the system replans the path to avoid it. Simultaneously, the system adjusts the inspection frequency according to the obstacle's position and speed.

[0047] In one embodiment of the present invention, the method for dynamically adjusting the inspection frequency can be expressed as follows: , in, This indicates the adjusted inspection frequency. Indicates the initial inspection frequency. Indicates the distance to the obstacle. Indicates the safe threshold distance. This is an adjustment coefficient. Preferably, It can be set to 0.5-0.8. It can be set to 2m-3m. This dynamic adjustment method can maintain inspection efficiency as much as possible while ensuring safety.

[0048] To further improve the decision-making capabilities of the inspection system, this invention adopts a hierarchical reinforcement learning architecture. This architecture consists of two parts: a high-level policy network and a low-level execution network.

[0049] The high-level policy network is responsible for formulating the global inspection strategy. It receives the status information of the entire substation as input and outputs a high-level action, such as "inspect a certain area" or "focus on inspecting a certain type of equipment". This network can adopt an Actor-Critic structure, where the Actor network outputs the action probability distribution and the Critic network estimates the state value function.

[0050] The lower-level execution network is responsible for carrying out specific inspection actions. It receives local environmental information and instructions from the higher-level network as input, and outputs specific movement commands, detection operations, etc. This network can adopt a DQN (Deep Q-Network) structure to directly output the Q-values ​​of different actions.

[0051] The high-level policy network and the low-level execution network are optimized using a hierarchical reinforcement learning algorithm. Specifically, the hierarchical Actor-Critic (HAC) algorithm can be employed. In this algorithm, the reward for the high-level network is determined by the performance of the low-level network in completing the task, while the low-level network receives a reward based on its performance in fulfilling the instructions of the high-level network. This hierarchical learning approach effectively addresses the different scales of long-term planning and short-term execution, thereby improving overall inspection efficiency.

[0052] As can be seen from the detailed description above, the multi-objective inspection method for substation equipment based on deep reinforcement learning provided by this invention can not only comprehensively perceive equipment status and environmental information, but also adaptively generate and execute complex inspection strategies. This method improves inspection efficiency while ensuring the safety and comprehensiveness of the inspection, which is of great significance for enhancing the intelligent operation and maintenance level of substations.

[0053] A key feature of this invention is the use of a model transfer method to address the problem of sample scarcity in practical applications. Specifically, this method includes pre-training a deep reinforcement learning model in a simulation environment, transferring the pre-trained model to an actual inspection robot using transfer learning methods, and fine-tuning and continuously learning the model in the real-world environment.

[0054] In the pre-training phase of the simulation environment, this method constructs a highly realistic digital twin model of a substation. This model not only includes the geometric information of the substation equipment but also simulates the equipment's operating status and potential fault conditions. Preferably, the simulation environment can be built based on a game engine such as Unity3D or Unreal Engine to provide highly realistic visual effects and physical simulation. In this environment, the deep reinforcement learning model can quickly accumulate a large amount of "experience" without worrying about damaging the actual equipment.

[0055] After pre-training, this method employs transfer learning techniques to transfer the model to the actual inspection robot. An improved domain adaptation algorithm is used here. Specifically, this algorithm first introduces a domain adversarial training mechanism at the feature extraction layer to reduce the distributional differences between the source domain (simulation environment) and the target domain (real environment). Second, at the policy network layer, a progressive transfer strategy is adopted, i.e., most network parameters are fixed initially, only the last few layers are fine-tuned, and then more layers are gradually unfrozen for fine-tuning. This method effectively prevents catastrophic forgetting during the transfer process.

[0056] In a real-world environment, this method also employs a continuous learning strategy. Specifically, the inspection robot continuously collects new data during task execution and uses this data to update the model. To address the catastrophic forgetting problem in continuous learning, this method uses the Elastic Weight Consolidation (EWC) algorithm. The core idea of ​​the EWC algorithm is to apply stronger regularization constraints to important parameters when updating model parameters, thereby maintaining the model's performance on older tasks.

[0057] To further improve the model's adaptability and learning efficiency, this invention also includes an incremental learning step. First, this method utilizes an experience replay buffer to store historical inspection experiences. This buffer employs a prioritized experience replay mechanism, which determines the sampling probability based on the importance of the experience (typically measured by TD error). This mechanism enables the model to learn more from experiences with high information content, thereby improving learning efficiency.

[0058] Secondly, this method uses stochastic gradient descent to incrementally update the model parameters. Specifically, a mini-batch of data is sampled from the empirical replay buffer each time, the gradient of the loss function is calculated, and this gradient is used to update the model parameters. The loss function can be expressed as: , in, Indicates the current network parameters. Indicates the target network parameters. This represents the experience replay buffer. It is a discount factor.

[0059] Finally, this method uses the ADAM optimizer to adaptively adjust the learning rate. The update rule of the ADAM optimizer can be expressed as: , in, It is the initial learning rate. and These are the first-order moment estimate and the second-order moment estimate, respectively. It is a small constant used for numerical stability. Preferably, the initial learning rate... It can be set to 0.001. It can be set to 10 -8 .

[0060] By comprehensively applying the above technologies, the method of the present invention can continuously learn and adapt to new situations in practical applications, thereby continuously improving inspection performance.

[0061] Finally, this invention also provides a multi-target inspection system for substation equipment based on deep reinforcement learning. This system includes multiple functional modules, each responsible for a specific task, which work together to complete intelligent inspection.

[0062] This invention also provides a multi-target inspection system for substation equipment based on deep reinforcement learning. The system consists of multiple functional modules, each responsible for a specific task, which work together to complete the intelligent inspection process.

[0063] Data acquisition module 1 is the foundation of the system, responsible for acquiring real-time status data of the substation equipment using a multi-source sensor network. This module integrates various sensors, including temperature, vibration, ultrasonic, and image sensors, enabling comprehensive acquisition of equipment operating status information. Preferably, the temperature sensor has an accuracy of ±0.1°C, the vibration sensor has a sampling frequency of 1kHz-10kHz, the ultrasonic sensor has a detection range of 0.2m-4m, and the image acquisition device has a resolution of at least 1080p. This multi-source data acquisition method provides a rich information foundation for subsequent intelligent analysis.

[0064] State representation module 2 receives the output from data acquisition module 1 and is responsible for constructing the state space representation of the substation equipment. This module employs deep learning technology to process and fuse multi-source heterogeneous data. Specifically, it first uses a convolutional neural network to extract features from different types of data, then uses an attention mechanism to weight and fuse these features, and finally maps the fused high-dimensional features to a low-dimensional state representation space. This method not only effectively captures the features of different types of data but also adaptively adjusts the importance of different features.

[0065] The deep reinforcement learning module 3 is the core of the system, responsible for generating multi-objective inspection strategies based on the state-space representation. This module employs an improved Q-learning algorithm, including a Q-network, an objective network, and an experience replay buffer. The Q-network is used to estimate the state-action value function, the objective network is used to stabilize the learning process, and the experience replay buffer is used to store and sample historical experience. This structure can effectively handle complex decision-making problems such as substation equipment inspection.

[0066] The inspection execution module 4 is responsible for executing specific inspection tasks based on the strategy generated by the deep reinforcement learning module 3. This module includes two main functions: path planning and action execution. The path planning function generates the optimal inspection path according to the inspection strategy, while the action execution function controls the robot to perform specific inspection actions, such as moving, taking pictures, and measuring.

[0067] The dynamic obstacle avoidance module 5 is used to perceive the surrounding environment and dynamically adjust the inspection path and frequency. This module acquires environmental information in real time through integrated sensors, constructs a dynamic obstacle model, and adjusts the inspection path and frequency in real time based on the model. This dynamic obstacle avoidance capability enables the system to maintain high inspection efficiency while ensuring safety.

[0068] The hierarchical decision-making module 6 implements a hierarchical reinforcement learning architecture, consisting of a high-level policy network and a low-level execution network. The high-level policy network is responsible for formulating global inspection strategies, such as deciding which area or type of equipment to inspect; the low-level execution network is responsible for executing specific inspection actions. This hierarchical architecture can effectively handle the problems of long-term planning and short-term execution at different scales.

[0069] Model transfer module 7 is used to transfer the pre-trained model to the actual inspection robot and perform fine-tuning. This module uses an improved domain adaptation algorithm, which can effectively reduce the distribution difference between the simulation environment and the actual environment, thereby accelerating the convergence of the model in the actual environment.

[0070] Incremental learning module 8 is used to continuously optimize the deep reinforcement learning model based on new inspection experience. This module is implemented through two steps: experience collection and model update. The experience collection step uses a priority experience replay mechanism to store new inspection experience, while the model update step uses the stochastic gradient descent algorithm and the ADAM optimizer to incrementally update the model.

[0071] The output module 9 is responsible for generating inspection reports that include equipment status assessment results and fault diagnosis suggestions. This module first performs in-depth analysis of the inspection data, and then organizes the analysis results into an easy-to-understand report format. This intelligent report generation method provides maintenance personnel with intuitive and accurate decision support.

[0072] Through the coordinated operation of these modules, the system of this invention enables intelligent and adaptive multi-target inspection of substation equipment, significantly improving inspection efficiency and accuracy, and providing strong support for the safe and stable operation of substations. This system not only adapts to complex and ever-changing substation environments but also dynamically adjusts inspection strategies according to actual conditions, achieving intelligent and refined management of inspection tasks.

[0073] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-target inspection method for substation equipment based on deep reinforcement learning, characterized in that, include: The acquisition steps include: Real-time status data of substation equipment is acquired using a multi-source sensor network, including temperature data, vibration data, ultrasonic data, and image data. The processing steps include: Based on the real-time status data, a state-space representation of the substation equipment is constructed; Based on the state space representation, a multi-target inspection strategy is generated using a deep reinforcement learning model. Based on the multi-target inspection strategy, the inspection task of the substation equipment is executed. Output steps, including: Generate an inspection report, which includes equipment status assessment results and fault diagnosis suggestions.

2. The method according to claim 1, characterized in that, The processing steps specifically include: Feature extraction is performed on the real-time state data using a convolutional neural network; An attention mechanism is introduced to weightedly fuse features from different modalities; The fused high-dimensional features are mapped to a low-dimensional state representation space.

3. The method according to claim 1, characterized in that, The deep reinforcement learning model is trained using the following objective function: , Representing the state space, Represents the action space, Representative strategy, Indicates the attenuation factor. This represents the reward function.

4. The method according to claim 3, characterized in that, The reward function Includes the following sub-objectives: , in, Indicates a reward for the completion of the inspection. This indicates a reward for equipment status monitoring. This indicates a reward for fault diagnosis. Indicates energy consumption optimization reward, This indicates an obstacle avoidance reward. to These are the corresponding weighting coefficients.

5. The method according to claim 1, characterized in that, It also includes dynamic obstacle perception and avoidance steps: Information about the surrounding environment is obtained using inertial sensors and ultrasonic sensors; A dynamic obstacle model is constructed based on the environmental information; Based on the dynamic obstacle model, the inspection path and frequency are dynamically adjusted.

6. The method according to claim 5, characterized in that, The method for dynamically adjusting the inspection frequency is as follows: , in, This indicates the adjusted inspection frequency. Indicates the initial inspection frequency. Indicates the distance to the obstacle. Indicates the safe threshold distance. This is for adjusting the coefficient.

7. The method according to claim 1, characterized in that, It also includes hierarchical reinforcement learning steps: Build a high-level strategy network to formulate global inspection strategies; Construct a low-level execution network to perform specific inspection actions; The high-level policy network and the low-level execution network are optimized using a hierarchical reinforcement learning algorithm.

8. The method according to claim 1, characterized in that, It also includes the model transfer step: Pre-train deep reinforcement learning models in a simulation environment; Transfer learning methods are used to transfer pre-trained models to actual inspection robots; Model fine-tuning and continuous learning are performed in real-world environments.

9. The method according to claim 1, characterized in that, It also includes the incremental learning step of the model: Utilize the experience replay buffer to store historical inspection experience; The model parameters are updated incrementally using the stochastic gradient descent algorithm. The ADAM optimizer is used to adaptively adjust the learning rate.

10. A multi-target inspection system for substation equipment based on deep reinforcement learning for performing the method according to any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire real-time status data of substation equipment using a multi-source sensor network; The state representation module is used to construct a state-space representation of the substation equipment based on the real-time state data; A deep reinforcement learning module is used to generate a multi-target inspection strategy based on the state space representation. The inspection execution module is used to execute the inspection tasks of the substation equipment based on the multi-target inspection strategy. The dynamic obstacle avoidance module is used to sense the surrounding environment and dynamically adjust the inspection path and frequency; The hierarchical decision-making module, consisting of a high-level policy network and a low-level execution network, is used to implement hierarchical reinforcement learning. The model transfer module is used to transfer pre-trained models to the actual inspection robot and perform fine-tuning. The incremental learning module is used to continuously optimize the deep reinforcement learning model based on new inspection experience. The results output module is used to generate inspection reports that include equipment status assessment results and fault diagnosis suggestions.

Images