Power transformation equipment thermal defect detection method based on Q-learning

Abstract

The invention relates to a Q-learning-based power transformation equipment thermal defect detection method, and aims to achieve the technical purpose of efficient and accurate detection of power transformation equipment thermal defects, and the thermal defect detection method is carried out after a CPU of a server receives a heating area of power transformation equipment and infrared temperature data corresponding to the heating area. By processing and analyzing the environmental data, whether the infrared temperature data of the heating area has the thermal defect or not is finally judged, through automatic data acquisition and analysis, the burden of manual operation is reduced, the tedious process of depending on handheld equipment and manual recording in a traditional detection method is avoided, and the detection efficiency is greatly improved. Meanwhile, the method does not need to depend on experience judgment of high-technician personnel, the technical requirement for detection personnel is lowered, and the problem that manpower resources are insufficient is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power equipment inspection, and relates to a method for detecting thermal defects of substation equipment, in particular to a method for detecting thermal defects of substation equipment based on Q-learning. Background Art

[0002] In the substation operation and maintenance of the power system, the traditional equipment thermal defect detection method relies heavily on manual temperature measurement and experience judgment. It usually requires two operators to work together, one to operate a handheld infrared thermal imager to detect the substation equipment, and the other to record the temperature measurement results. This method is not only cumbersome and time-consuming, but also has lower work efficiency when there are a large number of equipment. In addition, the operator must have rich experience and technical capabilities, and master the structure of the equipment, the heating principle and the defect judgment rules, but due to the shortage of skilled talents, the quality of detection is difficult to guarantee. This manual-dependent detection method has many limitations, including that the temperature measurement results are easily affected by subjectivity, the risk of misjudgment is high, and the lack of comprehensive analysis capabilities of multi-dimensional data leads to an incomplete and inaccurate assessment of the equipment status, which in turn increases the risk of failures and power outages.

[0003] In addition, in the prior art, there are a variety of traditional judgment methods to judge the thermal defects of substation equipment, such as surface temperature judgment method, relative temperature difference judgment method, similar comparison judgment method, image feature judgment method, archive analysis judgment method, and real-time analysis judgment method. The following is an analysis of the advantages and disadvantages of these six judgment methods: The advantages of the surface temperature judgment method are direct and simple. It measures the temperature on the surface of the equipment and compares it with the set safety threshold to judge whether there is a thermal defect. It is suitable for real-time monitoring and can quickly obtain results; the disadvantage is that it only relies on the surface temperature, which may ignore potential problems inside the equipment and cannot deeply reflect the overall status of the equipment. When the ambient temperature fluctuates greatly, false alarms or missed alarms may occur. The advantage of the relative temperature difference judgment method is that by comparing the temperature difference of different parts of the same equipment or the same part at different times, it can more accurately judge whether the equipment is abnormal, reduce the impact of ambient temperature fluctuations, and is suitable for dynamically changing scenarios; the disadvantage is that it requires long-term data accumulation and historical temperature records, and the real-time performance is relatively poor. In the case of insufficient data or low quality, it may be misjudged. The advantage of the similar comparison judgment method is that it can effectively identify abnormal phenomena by comparing the temperature data of the equipment with the temperature of the same type of equipment, enhance the accuracy and flexibility of judgment, and is suitable for large-scale equipment monitoring; the disadvantage is that it requires sufficient data of similar equipment as a reference, and these data must be reliable and consistent, and it is not suitable for single or specific equipment. The advantage of the image feature judgment method is that it can detect subtle changes in temperature distribution through infrared image feature extraction, which is suitable for complex equipment or large-scale monitoring, is sensitive to changes in the surface state of the equipment, and can detect potential thermal defects early; the disadvantage is that it relies on the accuracy of image processing and feature extraction algorithms, the algorithm complexity is relatively high, and may be affected by the quality of infrared cameras, ambient light or noise. The advantage of the archive analysis judgment method is that it is based on the historical data and maintenance archives of the equipment, can provide comprehensive background and long-term trend analysis, judge the health status of the equipment through long-term data trends, and predict potential failures; the disadvantage is that it requires a large amount of historical data support, the data collection and maintenance costs are high, and it is not applicable to new equipment or equipment lacking historical data. The advantage of the real-time analysis and judgment method is that it can quickly respond to abnormal equipment conditions through real-time data monitoring, achieve efficient early warning, and is suitable for important scenarios such as substation equipment that require real-time monitoring, reducing the time delay of faults. The disadvantage is that it requires efficient data acquisition and analysis system support, and has high requirements for hardware and software. If the real-time data is not accurate or delayed, it may lead to unstable judgment results. These judgment methods have their own advantages and disadvantages. In specific applications, the most appropriate solution can be selected according to the equipment type, environmental conditions, and the convenience of data acquisition.

[0004] It can be seen that the above six judgment methods have their own characteristics. If only one method is selected based on experience, it is not scientific enough, and the subjective factor of human experience is too large, which is likely to cause major mistakes, thereby bringing safety hazards to the entire substation system. At the same time, the selection of different judgment methods is likely to result in different judgment results. Therefore, while realizing the automatic detection technology, it is also necessary to accurately use the optimal judgment method, which is a difficult problem that technicians in this field need to overcome. Summary of the invention

[0005] In order to solve the technical problem of efficient and accurate detection of thermal defects of substation equipment, the present invention designs a thermal defect detection method for substation equipment based on Q-learning. The method collects real-time data such as environment and thermal image temperature, dynamically adjusts feature weights through the adaptive gradient algorithm AdaGrad, combines multiple defect judgment methods for comprehensive analysis, and optimizes the decision-making strategy in combination with the improved Q-learning reinforcement learning algorithm. Finally, the optimal judgment method is selected for judgment, and a detailed defect report is generated, providing the defect type, location, severity and recommended maintenance measures, thereby reducing manual operations and improving detection accuracy and automation level.

[0006] The technical solution adopted by the present invention is a Q-learning-based method for detecting thermal defects of substation equipment. The thermal defect detection method is performed after the CPU of the server receives the heating area of ​​the substation equipment and the infrared temperature data corresponding to the heating area. By processing and analyzing the environmental data, it is finally determined whether the infrared temperature data of the heating area has thermal defects. The thermal defect detection method includes a training step and a detection step. The specific steps are as follows:

[0007] In the training step,

[0008] a. The CPU obtains the historical heating area and the infrared temperature data corresponding to the heating area, as well as the environmental data of the substation equipment at the corresponding time. The environmental data includes temperature and humidity, operating voltage and current, wind direction and light intensity. The environmental data is pre-processed to eliminate noise and abnormal values, and normalized;

[0009] b. The CPU extracts features from the temperature data and environmental data, including the time gradient change of temperature, humidity fluctuation, abnormal fluctuation of current and voltage, wind direction and light intensity;

[0010] c. Calculate the cumulative gradient of each feature through the AdaGrad adaptive gradient algorithm and adjust the weights of all features;

[0011] d. All the features after weight adjustment and the temperature data corresponding to the heating area of ​​the substation equipment and the current heating area are taken as input, and are respectively input into the six judgment methods for processing and the results of each judgment method are obtained, and all the results are digitally quantized and normalized to obtain the size of the results, among which the maximum value represents the optimal judgment method, and the result corresponding to the optimal judgment method is fed back to the AdaGrad adaptive gradient algorithm with the help of the Q-learning reinforcement learning algorithm to optimize and adjust the weight allocation strategy of the next AdaGrad adaptive gradient algorithm. The six judgment methods are surface temperature judgment method, relative temperature difference judgment method, similar comparison judgment method, image feature judgment method, archive analysis judgment method, and real-time analysis judgment method;

[0012] e. Repeat the processing of all historical heating areas, infrared temperature data corresponding to the heating areas, and corresponding environmental data step by step according to steps ad, and repeatedly use the Q-learning reinforcement learning algorithm to feed back the results of the last optimal judgment method to the AdaGrad adaptive gradient algorithm;

[0013] When performing thermal defect detection on substation equipment, in the detection step,

[0014] f. After the CPU receives a group of heating areas of substation equipment and the infrared temperature data corresponding to the heating areas within a period of time, it collects the environmental data of the substation equipment at the current moment, repeats steps ad, and uses the optimal judgment method obtained at all time points within this period of time to detect the thermal defects of the equipment and generate a defect report.

[0015] In the step a, the specific method of pre-processing the environmental data is to convert the environmental data into a distribution with a mean value of 0 and a standard deviation of 1 through a standardized formula, and the formula is: Where x' is the standardized value, μ is the mean of feature x, and σ is the standard deviation of feature x.

[0016] In the step b, the time gradient change of temperature is used to analyze whether the equipment has abnormal heating, the humidity fluctuation is used to analyze the potential impact of the external environment on the equipment, and the abnormal fluctuation of current and voltage is used to detect signs of equipment overload or failure.

[0017] In step c, the AdaGrad adaptive gradient algorithm accumulates the historical gradient of the feature of each environmental data. The larger the accumulated gradient, the smaller the learning rate of the feature will become. The smaller the accumulated gradient, the larger the learning rate of the feature will be. The accumulated gradient sum of any feature i is expressed as: Where t represents the number of iterations, represents the gradient of feature i at the tth iteration,

[0018] In each iteration, the learning rate is adjusted according to the formula: where η is the global learning rate, ∈ is a small value that avoids division by zero, Represents the learning rate of feature i at the tth iteration.

[0019] The beneficial effects of the present invention are:

[0020] 1. Improve detection efficiency and reduce skill dependence: Through automated data collection and analysis, the burden of manual operation is reduced, and the cumbersome process of relying on handheld devices and manual records in traditional detection methods is avoided, which greatly improves detection efficiency. At the same time, this method does not need to rely on the experience and judgment of highly skilled personnel, reduces the technical requirements for detection personnel, and solves the problem of insufficient human resources.

[0021] 2. Improve detection accuracy: Using the adaptive gradient algorithm and Q-learning reinforcement learning algorithm to optimize data analysis and decision-making strategies can more comprehensively and accurately evaluate the status of substation equipment and reduce the risk of subjective judgment errors.

[0022] 3. Comprehensive analysis of multi-dimensional data: This method realizes all-round detection and analysis of substation equipment by integrating multiple data such as temperature, environmental conditions such as humidity, wind direction, and lighting, and equipment operating status, ensuring the comprehensiveness of the detection results.

[0023] 4. Intelligent decision support: Through the improved Q-learning algorithm for reinforcement learning, automatic optimization and real-time feedback adjustment are achieved. It can dynamically adapt to changes in the equipment operating environment and generate accurate maintenance suggestions, thereby improving the intelligence level of the system. DETAILED DESCRIPTION

[0024] The core technical principle of the present invention is to train historical data through a machine, and use the trained model to process new data. For the six traditional defect judgment methods, each method has its own advantages and disadvantages. Therefore, the present invention introduces the collection and processing of environmental data, and automatically determines which defect judgment method is most appropriate by analyzing the environmental data, and then uses the current optimal judgment method to perform thermal defect detection on the substation equipment.

[0025] The specific steps of historical data training are:

[0026] a. The CPU obtains the historical heating area and the infrared temperature data corresponding to the heating area, as well as the environmental data of the substation at the corresponding time. The environmental data includes temperature and humidity, operating voltage and current, wind direction and light intensity. The environmental data is pre-processed to eliminate noise and abnormal values, and normalized. The specific method of pre-processing the environmental data is to convert the environmental data into a distribution with an average value of 0 and a standard deviation of 1 through a standardized formula. The formula is: Where x' is the standardized value, μ is the mean of feature x, and σ is the standard deviation of feature x. The preprocessing process can effectively eliminate the dimensional differences in the substation environment data, making each feature more adaptable and accurate when analyzed in a unified manner. This method is applicable to a variety of machine learning models and improves the learning effect of the model.

[0027] b. The CPU extracts features from temperature data and environmental data, including the time gradient change of temperature, humidity fluctuation, abnormal current and voltage fluctuation, wind direction and light intensity. The time gradient change of temperature is used to analyze whether the equipment has abnormal heating. The humidity fluctuation analyzes the potential impact of the external environment on the equipment. The abnormal current and voltage fluctuation is used to detect signs of equipment overload or failure.

[0028] Here we need to distinguish the feature x mentioned in step a from the characteristics involved in step b. The feature in step a refers to the characteristics of environmental data, such as temperature, humidity, current, voltage, etc. These data will be standardized during preprocessing to maintain consistent scale and stability in subsequent analysis and calculation. Therefore, the "feature" here refers to the original environmental data indicators.

[0029] The characteristics mentioned in step b are higher-level analyses based on these standardized environmental data. For example, the time gradient change of temperature is used to determine whether the device is abnormally heated by analyzing the rate of temperature change over time; humidity fluctuations are used to assess the potential impact of the environment on the device; abnormal fluctuations in current and voltage are used to detect whether the device has signs of overload or failure.

[0030] In other words, the features in step a are the basic indicators of environmental data, while the characteristics in step b are high-level characteristics that are further analyzed and extracted from these basic indicators. Therefore, the two are concepts at different levels.

[0031] c. The cumulative gradient of each feature is calculated through the AdaGrad adaptive gradient algorithm, and the weights of all features are adjusted. By calculating the cumulative gradient of each feature, higher weights are given to features that change dramatically or have a greater impact, ensuring that features that have a significant impact on the device status are given focus in subsequent analysis. For example, in infrared temperature data, if the temperature continues to rise, AdaGrad will increase the weight of the temperature feature and reduce the weight of the more stable voltage feature.

[0032] The AdaGrad adaptive gradient algorithm accumulates the historical gradient of the features of each environmental data. The larger the accumulated gradient, the smaller the learning rate of the feature will become. The smaller the accumulated gradient, the larger the learning rate of the feature will be. The accumulated gradient sum of any feature i is expressed as: Where t represents the number of iterations, Represents the gradient of feature i at the tth iteration. In each iteration, the adjustment of the learning rate follows the formula: where η is the global learning rate, ∈ is a small value that avoids division by zero, Represents the learning rate of feature i at the tth iteration. Through this dynamic adjustment, the model can balance the influence of different features. Frequently changing features will have a smaller learning rate, while slowly changing or sparse features will retain a larger learning rate. After processing, AdaGrad outputs weighted and adjusted environmental feature data. These standardized and cumulative gradient-weighted data will be input as state information into subsequent reinforcement learning, helping the reinforcement learning model make more accurate decisions and optimize strategies.

[0033] d. All the features after weight adjustment and the temperature data corresponding to the heating area of ​​the substation equipment and the current heating area are taken as input, and are respectively input into the six judgment methods for processing to obtain the results of each judgment method, and all the results are digitally quantized and normalized to obtain the size of the results, among which the maximum value represents the optimal judgment method, and the result corresponding to the optimal judgment method is fed back to the AdaGrad adaptive gradient algorithm with the help of the Q-learning reinforcement learning algorithm to optimize and adjust the weight allocation strategy of the next AdaGrad adaptive gradient algorithm. The six judgment methods are surface temperature judgment method, relative temperature difference judgment method, similar comparison judgment method, and image feature judgment method.

[0034] In the process of decision optimization, the Q-learning reinforcement learning algorithm achieves optimized judgment of equipment defects by comprehensively analyzing the weighted feature input with the results of the surface temperature judgment method, relative temperature difference judgment method, similar equipment comparison method and image feature judgment method. The results of each judgment method, such as "general", "serious" and "crisis", are quantified into numerical values, such as 1, 2, and 3, and input into the system as components of the feature vector. In order to ensure the comparability of the results of different methods, these features are normalized and mapped to a unified range between 0 and 1, thereby eliminating the magnitude difference between different feature values. The processed feature vector can more accurately reflect the impact of various judgment results on the final decision. Q-learning is based on the state-action-reward mechanism to select an optimal decision from several judgment results. The system balances exploration and exploitation through the ε-greedy strategy, that is, in most cases, the currently known optimal strategy is selected, while occasionally exploring new possibilities to avoid local optimal traps. After each decision, the system adjusts the Q value of the decision based on actual feedback, such as whether the equipment status has improved or the problem has worsened, to ensure that effective decisions will receive higher weights in the future, so that the decision will be preferred in similar situations. Through this real-time learning and feedback mechanism, Q-learning continuously optimizes its strategy selection, allowing the results of multiple judgment methods to be flexibly combined and weighted for comprehensive analysis, improving the accuracy and robustness of system decisions. Ultimately, this process ensures that when dealing with complex equipment defect detection tasks, the system can quickly adapt to changes and provide reliable judgment results.

[0035] e. Repeat the processing of all historical heating areas, the infrared temperature data corresponding to the heating areas, and the corresponding environmental data according to steps ad, and repeatedly use the Q-learning reinforcement learning algorithm to feed back the results of the last optimal judgment method to the AdaGrad adaptive gradient algorithm.

[0036] After training the historical data multiple times through the above steps, the model training process is completed. After the model calculation reaches a certain accuracy, it can be used for thermal defect detection during daily inspections. For example, when the server receives a set of video files of the heating of substation equipment, the system will capture images according to the number of video frames and generate infrared thermal imaging pictures of the substation equipment. The system identifies the type of substation equipment and infrared temperature data through software, but the equipment failure situation may not be the same under different environmental conditions. For example, when the weather is hot, it is normal for the surface temperature of the substation equipment to reach 40℃, but in winter, the surface temperature of the substation equipment may reach 30℃, which may cause a failure. Therefore, environmental factors are important factors in judging thermal defects of equipment. However, due to different environmental factors, the method of selecting thermal defects is also different. Therefore, it is necessary to analyze and judge the current environmental factors and select an optimal judgment method according to the current environmental conditions, so that the results of this judgment method are more accurate and reliable. Therefore, during the inspection, the above steps a to d must be carried out, and data processing must be performed one by one according to the several frames of images divided into the above video. That is, all images and the environmental data at the corresponding moment must be judged and processed multiple times, and then the thermal defect situation of the current equipment can be comprehensively obtained, and a report can be generated to inform possible risks and solutions.

[0037] When performing thermal defect detection on substation equipment, in the detection step,

[0038] f. After the CPU receives a group of heating areas of substation equipment and the infrared temperature data corresponding to the heating areas within a period of time, it collects the environmental data of the substation equipment at the current moment, repeats steps ad, and uses the optimal judgment method obtained at all time points within this period of time to detect the thermal defects of the equipment and generate a defect report.

[0039] The optimal judgment method obtained at all time points during this period is to calculate an optimal judgment method at each time node of the video and weight it according to the importance or weight of the time node. For example, a higher weight can be given to the time point in the video where the device temperature is obviously abnormal. Calculate the weighted average judgment method and select the closest one as the final optimal judgment method.

[0040] To specify the optimal judgment method of weighted calculation, you can refer to the following steps and details:

[0041] Determination of the importance of time nodes:

[0042] For each time node, a weight value is assigned according to the changes in key indicators such as temperature, humidity, current and voltage of the substation equipment. For example: if the temperature of a certain time node exceeds a set threshold (such as 80% of the rated temperature upper limit of the equipment), a higher weight (such as a weight of 2.0) is assigned to the node. If the temperature change at a certain time node is small and within a safe range, a lower weight (such as a weight of 0.5) is assigned to the node. Similar weight rules can also be set for fluctuations in humidity, current and voltage. For example, if the humidity changes drastically or the current fluctuation exceeds the set range, the corresponding weights can be assigned respectively.

[0043] Calculation of weighted average judgment method:

[0044] The judgment method calculated at each time node (e.g., the strategy selected by the Q-learning algorithm) is represented by a numerical value or vector, which is then multiplied by the corresponding weight to obtain the weighted value. The weighted judgment methods of all time nodes in the video are summed and divided by the sum of the weights of all time nodes to obtain the weighted average judgment method.

[0045] Select the final optimal judgment method:

[0046] According to the result of the weighted average judgment method, the one closest to the weighted average is selected from the calculated judgment method set as the final optimal judgment method. This ensures that the final result is more representative because it comprehensively considers the importance of all time nodes while avoiding the influence of extreme values ​​or noise on the overall judgment.

[0047] The above weighted allocation process helps provide clearer guidance during implementation.

[0048] Another core technical principle of the present invention is to combine the adaptive gradient algorithm and the reinforcement learning algorithm in the feedback loop of dynamic adjustment of feature weights and strategy optimization. The adaptive gradient algorithm adjusts the feature weights by accumulating gradients to ensure that the features can be flexibly adjusted according to the real-time changes of the equipment and environment, and the reinforcement learning algorithm selects the optimal action through a dynamic decision-making strategy. After each judgment, Q-learning adjusts the strategy according to the detection feedback, and these feedbacks will also affect the next round of AdaGrad feature weight allocation. For example, when Q-learning finds that the surface temperature of the device is abnormal, it will focus on analyzing the area, and the system will then adjust the relevant feature weights according to the temperature change trend. If the temperature returns to normal, AdaGrad will reduce the weight of the temperature feature, otherwise it will be further improved.

[0049] The adaptive gradient algorithm introduces a dynamic weighting factor, which is calculated based on the real-time volatility of the feature data and used to adjust the learning rate. At each gradient update, the system dynamically determines the size of the factor by calculating the change in the feature gradient. Specifically, the real-time calculation is based on the feature's gradient accumulation and the square value of the current gradient, that is, the volatility is determined by tracking the sum of the squares of the gradients of each feature. If the gradient of the feature changes greatly (indicating high volatility), the dynamic weighting factor will increase, allowing a larger learning rate; if the gradient changes less (low volatility), the weighting factor will be reduced, reducing the learning rate and ensuring a smoother adjustment of the weight. Through this dynamic weighting factor calculation method based on gradient changes, the model can adaptively optimize the learning rate in real-time data changes, thereby improving the efficiency and stability of feature weight adjustment. At the same time, by tracking and evaluating the performance of different features in a specific time window, the learning rate for each feature can be dynamically adjusted. In this strategy, the historical gradients of the features are recorded and divided into a time window for cumulative analysis. The model calculates the average gradient change or variance over a period of time for each feature as a measure of the volatility of the feature. If the gradient change of a feature in this time window is large, it indicates that its impact on the model is more important, so a higher learning rate is assigned to the feature, allowing faster weight adjustment. On the contrary, if the gradient change of a feature is small, the system reduces its learning rate to avoid excessive updates and noise interference.

[0050] Q-learning in reinforcement learning algorithms is essentially about selecting the optimal action by learning a state-action value function, namely the Q function, so that the agent can maximize the cumulative reward in its interaction with the environment. Its core idea is to gradually update the value estimate of the state-action pair based on the Bellman equation. The mathematical logic of Q-learning can be expressed as follows: given a state s and an action a, the Q value Q(s, a) represents the expected total reward that can be obtained by selecting action a in this state and continuing to execute the optimal strategy. Q-learning approximates the optimal Q function by iteratively updating the Q value. The specific update formula is: Q(s t ,a t )←Q(s t ,a t )+α[r t+1 +γmax a Q(s t+1 , a t )-Q(s t ,a t )], where α is the learning rate, which controls the step size of Q value update; r t+1 Is to perform action a t The immediate reward obtained after γ is the discount factor used to measure the weight of future rewards; maxa Q(s t+1 , a t ) means in the next state s t+1 Through this updating method, Q-learning can gradually correct the Q-value estimate so that the action selected in each state can maximize the future reward.

[0051] The Q-learning reinforcement learning algorithm can implement online learning strategies combined with adaptive adjustment algorithms. The model dynamically adjusts the update frequency based on real-time feedback to cope with the rate of change of data. A weighting strategy based on time decay is used to give higher weights to recent data to more accurately capture data change trends. During the model update process, real-time error feedback is used for optimization. Each time the model parameters are updated, the learning rate and weights are automatically adjusted to ensure that the model can be effectively adjusted according to the latest data features, enhancing the model's ability to respond to sudden changes.

[0052] Introduce an adaptive learning rate adjustment strategy based on historical errors. The Q value represents the total reward that may be obtained in the future after taking a certain action in a certain state. It is the core indicator of strategy optimization. Each time the Q value is updated, the learning rate is dynamically adjusted according to the prediction error, that is, the difference between the Q value and the actual reward, so that the learning process can adapt to changes in the environment more quickly.

[0053] Use a higher learning rate in the early stages of training so that the model can quickly explore and learn global information; as training progresses, gradually reduce the learning rate and focus on fine-tuning the strategy to avoid instability caused by over-adjustment, ensure that the model gradually converges and improve the stability of the strategy.

[0054] In the specific software algorithm, an ε-greedy strategy adjustment mechanism based on environmental feedback is introduced. The ε value represents the balance between exploration and utilization, and determines whether the model chooses random exploration or uses the existing optimal strategy in the current state. This adjustment mechanism dynamically modifies the ε value by evaluating the performance of the model in the environment in real time. When the model performs well and obtains high returns or low prediction errors for many consecutive times, the system gradually reduces the ε value, reduces the proportion of random exploration, and relies more on the current optimal strategy to improve decision-making efficiency; conversely, if the model performs poorly, the system increases the ε value, increases the proportion of exploration, and encourages the model to try more unexplored actions to find a better strategy. In this way, the model can flexibly adjust the trade-off between exploration and utilization at different stages, so as to adapt to the dynamic environment more quickly and improve the optimization effect of the overall strategy.

[0055] Combined with the principle of information gain, we prioritize exploring states that the model does not fully understand, that is, the current environment, and the corresponding optional actions, that is, the decisions that the model can take in this state. This exploration method prioritizes the parts with insufficient information, improves the comprehensiveness and accuracy of the strategy, enables the model to better adapt to changes in the environment, and thus optimizes the decision-making process.

[0056] Optimized Q-value adjustment method. In order to process various information in complex environments, Q-values ​​are updated at different levels. This hierarchical processing method allows the model to adjust Q-values ​​at different levels of abstraction. At a high level, Q-value updates focus on global strategies and long-term rewards, helping the model identify overall trends and patterns; at a low level, Q-value updates focus on local details and immediate feedback, optimizing short-term reward responses to specific actions. This approach enables the model to more comprehensively understand and process information at different levels, improving the accuracy of decision-making.

[0057] When updating the Q value, a reward accumulation mechanism that considers the next few time steps is introduced, rather than just relying on single-step rewards. This method helps the model develop more effective long-term strategies by evaluating the total rewards that may be obtained in the future, thereby enhancing the stability and long-term effectiveness of decision-making.

[0058] Real-time feedback mechanism, real-time feedback adjustment: After each decision, the system adjusts the Q value and strategy based on the actual environment feedback. By timely correcting the Q value and strategy, the feedback mechanism improves the adaptability of decision-making in actual applications, enabling the model to respond quickly to environmental changes. Use real-time feedback data to adjust the balance between exploration and utilization strategies. The system dynamically adjusts the exploration ratio to ensure that the model can continuously adapt to changes in the environment, and continuously optimizes the decision-making strategy in a changing environment to improve the overall learning effect.

[0059] Through the combination of the above methods, it is possible to automatically detect the thermal defects of the current substation equipment through the machine, and select the best traditional method for judgment. The work efficiency is extremely high, and the accuracy is stable and reliable.

Claims

1. A Q-learning-based method for detecting thermal defects in substation equipment, characterized in that: The thermal defect detection method is performed after the CPU of the server receives the heating area of ​​the substation equipment and the infrared temperature data corresponding to the heating area. By processing and analyzing the environmental data, it is finally determined whether the infrared temperature data of the heating area has a thermal defect. The thermal defect detection method includes a training step and a detection step. The specific steps are as follows: In the training step, a. The CPU obtains the historical heating area and the infrared temperature data corresponding to the heating area, as well as the environmental data of the substation equipment at the corresponding time. The environmental data includes temperature and humidity, operating voltage and current, wind direction and light intensity. The environmental data is pre-processed to eliminate noise and abnormal values, and normalized; b. The CPU extracts features from temperature data and environmental data, including time gradient changes in temperature, humidity fluctuations, abnormal current and voltage fluctuations, wind direction, and light intensity; c. Calculate the cumulative gradient of each feature through the AdaGrad adaptive gradient algorithm and adjust the weights of all features; d. All the features after weight adjustment and the temperature data corresponding to the heating area of ​​the substation equipment and the current heating area are taken as input, and are respectively input into the six judgment methods for processing and the results of each judgment method are obtained, and all the results are digitally quantized and normalized to obtain the size of the results, among which the maximum value represents the optimal judgment method, and the result corresponding to the optimal judgment method is fed back to the AdaGrad adaptive gradient algorithm with the help of the Q-learning reinforcement learning algorithm to optimize and adjust the weight allocation strategy of the next AdaGrad adaptive gradient algorithm. The six judgment methods are surface temperature judgment method, relative temperature difference judgment method, similar comparison judgment method, image feature judgment method, archive analysis judgment method, and real-time analysis judgment method; e. Repeat the processing of all historical heating areas, infrared temperature data corresponding to the heating areas, and corresponding environmental data step by step according to steps ad, and repeatedly use the Q-learning reinforcement learning algorithm to feed back the results of the last optimal judgment method to the AdaGrad adaptive gradient algorithm; When performing thermal defect detection on substation equipment, in the detection step, f. After the CPU receives a group of heating areas of substation equipment and the infrared temperature data corresponding to the heating areas within a period of time, it collects the environmental data of the substation equipment at the current moment, repeats steps ad, and uses the optimal judgment method obtained at all time points within this period of time to detect the thermal defects of the equipment and generate a defect report.

2. According to claim 1, a Q-learning-based thermal defect detection method for substation equipment is characterized in that: In the step a, the specific method of pre-processing the environmental data is to convert the environmental data into a distribution with a mean value of 0 and a standard deviation of 1 through a standardized formula, and the formula is: Where x' is the standardized value, μ is the mean of feature x, and σ is the standard deviation of feature x.

3. According to claim 1, a Q-learning-based thermal defect detection method for substation equipment is characterized in that: In the step b, the time gradient change of temperature is used to analyze whether the equipment has abnormal heating, the humidity fluctuation is used to analyze the potential impact of the external environment on the equipment, and the abnormal fluctuation of current and voltage is used to detect signs of equipment overload or failure.

4. The Q-learning-based thermal defect detection method for substation equipment according to claim 1 is characterized in that: In step c, the AdaGrad adaptive gradient algorithm accumulates the historical gradient of the feature of each environmental data. The larger the accumulated gradient, the smaller the learning rate of the feature will become. The smaller the accumulated gradient, the larger the learning rate of the feature will be. The accumulated gradient sum of any feature i is expressed as: Where t represents the number of iterations, represents the gradient of feature i at the tth iteration, In each iteration, the learning rate is adjusted according to the formula: where η is the global learning rate, ∈ is a small value that avoids division by zero, Represents the learning rate of feature i at the tth iteration.

Images