Power inspection embodied intelligent path optimization method and system based on reinforcement learning

Abstract

The present invention provides an embodied intelligent path optimization method and system for electric power inspection based on reinforcement learning, which relates to the field of power supply systems. The method comprises: establishing an environmental model of an area to be inspected, including a plurality of nodes to be inspected; determining the risk association relationship between the plurality of nodes to be inspected according to historical operation information of the plurality of nodes to be inspected, and constructing an inspection path optimization model based on a deep Q network, wherein a reward function thereof is related to the risk association relationship between the plurality of nodes to be inspected, the real-time risk value of each node to be inspected, and the inspection energy consumption; determining the real-time risk value of each node to be inspected according to the real-time operation information and the risk association relationship of the plurality of nodes to be inspected; generating a real-time electric power inspection optimization path according to the real-time risk value of each node to be inspected through the inspection path optimization model, and controlling an electric power inspection embodied robot to perform electric power inspection, which has the advantages of improving the efficiency of electric power inspection and the real-time performance of responding to the risk of electric power failure, and reducing energy consumption.

Description

Technical Field

[0001] The present invention relates to the field of power supply systems, and in particular to a method and system for optimizing an embodied intelligent path for power inspection based on reinforcement learning. Background Art

[0002] With the acceleration of urbanization and the growth of electricity demand, the scale of power systems continues to expand, including substations, transmission lines and distributed energy equipment. This makes inspection work more complicated and requires more resources and time. At the same time, the rise of energy efficiency and renewable energy has put higher demands on power systems. Power systems need more effective maintenance to ensure the sustainability and stability of energy supply.

[0003] The inspection of power systems is basically based on manual arrangements, with limited intelligence. Inspections based on fixed rules cannot flexibly respond to changes in power system problems, resulting in waste of resources and inefficiency. The inspection of power systems requires engineers to enter dangerous areas, such as high-voltage towers or substations, which poses safety risks. In response to the above problems, inspection robots have replaced traditional manual inspection methods and are widely used in online inspections of power systems, effectively improving the quality and efficiency of inspection work and playing an important role. In the prior art, inspection robots are used to inspect power equipment. It is necessary to pre-plan a fixed inspection path based on the environment inside the substation, input the fixed inspection path into the inspection robot, and then conduct inspections. The flexibility is poor, and it is impossible to detect power equipment that may be at risk in a timely manner. In addition, it is impossible to reduce the robot inspection time, save the robot's power consumption, and increase the robot's inspection efficiency.

[0004] Therefore, it is necessary to provide an embodied intelligent path optimization method and system for power inspection based on reinforcement learning, so as to improve the efficiency of power inspection and the real-time response to power failure risks and reduce energy consumption. Summary of the invention

[0005] The present invention provides an embodied intelligent path optimization method for power inspection based on reinforcement learning, comprising: obtaining regional information of an area to be inspected, establishing an environmental model of the area to be inspected based on the regional information of the area to be inspected, wherein the environmental model of the area to be inspected includes a plurality of nodes to be inspected; obtaining historical operation information of the plurality of nodes to be inspected; determining the risk association relationship between the plurality of nodes to be inspected based on the historical operation information of the plurality of nodes to be inspected; constructing an inspection path optimization model based on a deep Q network based on the environmental model of the area to be inspected and the risk association relationship between the plurality of nodes to be inspected, wherein the inspection path optimization The reward function of the model is related to the risk association relationship between the multiple nodes to be inspected, the real-time risk value of each node to be inspected, and the inspection energy consumption; the inspection path optimization model is trained; the real-time operation information of the multiple nodes to be inspected is obtained; the real-time risk value of each node to be inspected is determined according to the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected; a real-time power inspection optimization path is generated according to the real-time risk value of each node to be inspected through the trained inspection path optimization model; based on the real-time power inspection optimization path, the power inspection embodied robot is controlled to perform power inspection.

[0006] Furthermore, based on the historical operation information of the multiple nodes to be inspected, the risk association relationship between the multiple nodes to be inspected is determined, including: based on the historical operation information of the multiple nodes to be inspected, determining the state value of each node to be inspected at multiple historical time points, and determining the first fault association parameters of any two nodes to be inspected based on the state value of each node to be inspected at multiple historical time points; through a clustering algorithm, determining multiple clusters of nodes to be inspected based on the first fault association parameters of any two nodes to be inspected; for each of the clusters of nodes to be inspected, based on the historical operation information of the multiple nodes to be inspected, screening the target meteorological factors corresponding to the cluster of nodes to be inspected, and based on the target meteorological factors corresponding to the cluster of nodes to be inspected, determining the second fault association parameters of any two nodes to be inspected included in the cluster of nodes to be inspected under multiple meteorological conditions, wherein the risk association relationship between the multiple nodes to be inspected includes the second fault association parameters of any two nodes to be inspected included in each of the clusters of nodes to be inspected under multiple meteorological conditions.

[0007] Furthermore, through a clustering algorithm, a plurality of node clusters to be inspected are determined according to the first fault association parameters of any two nodes to be inspected, including: S11, calculating the discreteness of the first fault association parameters according to the first fault association parameters of any two nodes to be inspected, and determining the initial number of clusters K according to the discreteness of the first fault association parameters; S12, randomly selecting K nodes to be inspected as cluster centers according to the number of clusters K; S13, for each node to be inspected that is not a cluster center, based on the first fault association parameters of the node to be inspected and each cluster center, allocating the node to be inspected to the node cluster to be inspected in the cluster center with the largest first fault association parameter; S14, for each of the node clusters to be inspected, calculating the intra-cluster discrete parameters of the node cluster to be inspected according to the first fault association parameters of any two nodes to be inspected included in the node cluster to be inspected; S15, calculating the intra-cluster discrete parameters of the node cluster to be inspected according to the first fault association parameters of each node to be inspected. The intra-cluster discrete parameters of the node cluster to be inspected are used to determine whether the first clustering condition is met. If not, execute S16, and if so, execute S17; S16, for each of the node clusters to be inspected, according to the first fault association parameters of any two nodes to be inspected included in the node cluster to be inspected, calculate the single-point discrete parameters corresponding to each node to be inspected included in the node cluster to be inspected, and update the cluster center of the node cluster to be inspected according to the single-point discrete parameters corresponding to each node to be inspected, and execute S13; S17, according to the first fault association parameters between any two cluster centers, calculate the out-of-cluster discrete parameters; S18, according to the out-of-cluster discrete parameters, determine whether the second clustering condition is met, if so, determine the multiple node clusters to be inspected, if not, execute S19; S19, according to the out-of-cluster discrete parameters, determine the iteration step size, update the number of clusters K according to the iteration step size, and execute S12.

[0008] Furthermore, based on the historical operation information of the multiple nodes to be inspected, the target meteorological factors corresponding to the cluster of nodes to be inspected are screened, including: for each node to be inspected included in the cluster of nodes to be inspected, the meteorological association parameters between each meteorological factor and the node to be inspected are calculated based on the historical operation information of the node to be inspected and the historical meteorological information of the area to be inspected; for each meteorological factor, the target meteorological factor corresponding to the cluster of nodes to be inspected is screened based on the meteorological association parameters between the meteorological factor and each node to be inspected included in the cluster of nodes to be inspected.

[0009] Furthermore, according to the target meteorological factor corresponding to the node cluster to be inspected, the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions are determined, including: for each target meteorological factor corresponding to the node cluster to be inspected, a plurality of sample meteorological conditions are sampled according to the historical meteorological information of the area to be inspected; for each sample meteorological condition, the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological condition is determined according to the historical meteorological information of the area to be inspected, and the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under the sample meteorological conditions are calculated according to the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological condition; through an interpolation algorithm, according to the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple sample meteorological conditions, the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions are determined.

[0010] Furthermore, based on the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected, the real-time risk value of each of the nodes to be inspected is determined, including: obtaining the meteorological information of the area to be inspected; for each cluster of nodes to be inspected, according to the target meteorological factor corresponding to the cluster of nodes to be inspected, the meteorological information of the area to be inspected and the second fault association parameters of any two nodes to be inspected included in the cluster of nodes to be inspected under multiple meteorological conditions, calculate the current second fault association parameters of any two nodes to be inspected included in the cluster of nodes to be inspected; for each of the nodes to be inspected, according to the real-time operation information of the nodes to be inspected, determine the initial risk value of the node to be inspected; for each cluster of nodes to be inspected, according to the current second fault association parameters of any two nodes to be inspected included in the cluster of nodes to be inspected and the initial risk value of each node to be inspected included in the cluster of nodes to be inspected, determine the real-time risk value of each node to be inspected included in the cluster of nodes to be inspected.

[0011] Further, according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the initial risk value of each node to be inspected included in the node cluster to be inspected, the real-time risk value of each node to be inspected included in the node cluster to be inspected is determined, including: S21, according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the initial risk value of each node to be inspected included in the node cluster to be inspected, updating the risk value of each node to be inspected included in the node cluster to be inspected; S22. Determine whether convergence has occurred based on the risk value before and after the update of each node to be inspected included in the node cluster to be inspected. If so, use the updated risk value of each node to be inspected included in the node cluster to be inspected as the real-time risk value. If not, execute S23. S23. Update the risk value of each node to be inspected included in the node cluster to be inspected again based on the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the updated risk value of each node to be inspected, and execute S22.

[0012] Furthermore, the reward function of the inspection path optimization model is: ,in, For the current state Actions taken Then transfer to the next state The total reward received when For priority rewards, For association rewards, For energy consumption rewards, , and is the weight coefficient, , and greater than 0, .

[0013] Furthermore, the priority reward, association reward and energy consumption reward are calculated based on the following formula: , , ,in, For the next state The corresponding real-time risk value of the node to be inspected, is the first attenuation coefficient, greater than 0, is an exponential function, From the start node to the next state The number of steps of the corresponding node to be inspected, For the current state The corresponding node to be inspected and the next state The current second fault association parameters between the corresponding nodes to be inspected, is the second attenuation coefficient, greater than 0, From the starting node to the current state The number of steps of the corresponding node to be inspected, For unit energy consumption.

[0014] The present invention provides an embodied intelligent path optimization system for electric power inspection based on reinforcement learning, which is used to execute the above-mentioned embodied intelligent path optimization method for electric power inspection based on reinforcement learning, including: a model building module, which is used to obtain regional information of an area to be inspected, and to establish an environmental model of the area to be inspected based on the regional information of the area to be inspected, wherein the environmental model of the area to be inspected includes multiple nodes to be inspected, and is also used to obtain historical operation information of the multiple nodes to be inspected, and determine the risk association relationship between the multiple nodes to be inspected according to the historical operation information of the multiple nodes to be inspected, and based on the environmental model of the area to be inspected and the risk association relationship between the multiple nodes to be inspected, construct an inspection path optimization model based on a deep Q network, wherein the inspection path The reward function of the path optimization model is related to the risk association relationship between the multiple nodes to be inspected, the real-time risk value of each node to be inspected, and the inspection energy consumption; a model training module is used to train the inspection path optimization model; a risk assessment module is used to obtain the real-time operation information of the multiple nodes to be inspected, and determine the real-time risk value of each node to be inspected according to the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected; a path generation module is used to generate a real-time power inspection optimization path according to the real-time risk value of each node to be inspected through the trained inspection path optimization model; an automatic inspection module is used to control the power inspection embodied robot to perform power inspection based on the real-time power inspection optimization path.

[0015] Compared with the prior art, the power inspection embodied intelligent path optimization method and system based on reinforcement learning provided by the present invention have at least the following beneficial effects:

[0016] 1. By constructing an environmental model of the area to be inspected and the risk association relationship, this method can more accurately assess the risk of each node, thereby optimizing the inspection path. The introduction of real-time risk values ​​enables the inspection robot to dynamically adjust its path and prioritize the inspection of areas with higher risks, thereby improving the overall inspection efficiency. The optimization of the inspection path reduces unnecessary movement, thereby reducing energy consumption and labor costs during the inspection process. Automated and intelligent inspections reduce reliance on manual inspections, further reducing costs. Combined with advanced artificial intelligence technologies such as deep Q networks, it can handle complex inspection scenarios and variable risk factors. It can adapt to areas to be inspected of different sizes and complexities, and provide personalized inspection solutions. The optimization of the inspection path can be dynamically adjusted according to actual conditions to ensure the flexibility and adaptability of the inspection. It improves the efficiency of power inspections and the real-time response to power failure risks, reducing energy consumption.

[0017] 2. Through detailed historical operation information analysis, this method can more accurately determine the risk association relationship between the nodes to be inspected, including the fault correlation under different meteorological conditions. This helps to more accurately evaluate the real-time risk value of each node, thereby guiding the optimization of the inspection path. The nodes to be inspected are divided into different node clusters using a clustering algorithm, and can be grouped according to the fault correlation between the nodes, making the analysis more efficient and targeted. It helps to reduce redundant data, improve the accuracy of analysis, and provide strong support for subsequent risk assessment and path optimization. It can dynamically adjust the risk value and inspection path according to real-time meteorological information and node operation information, making the inspection more flexible and efficient.

[0018] 3. The reward function comprehensively considers the priority reward, association reward and energy consumption reward, which makes the optimization of the inspection path not only consider the risk priority of the node, but also the fault correlation between nodes and the energy consumption during the inspection process. This comprehensive reward mechanism helps guide the inspection robot to complete the inspection task more efficiently and safely. The priority reward is calculated based on the real-time risk value of the node to be inspected corresponding to the next state, and decays with the increase of the number of steps. This means that the inspection robot will give priority to visiting nodes with higher risk values. The introduction of priority rewards helps to improve the efficiency and pertinence of inspections. The association reward considers the current second fault association parameter between the node to be inspected corresponding to the current state and the node to be inspected corresponding to the next state. By considering the fault correlation between nodes, the inspection robot can better understand the overall risk status of the inspection area, give priority to inspecting multiple nodes to be inspected that have a correlation and a higher real-time risk value, and make a more sensible path choice. The consideration of association rewards helps to improve the accuracy and comprehensiveness of inspections. The energy consumption reward is calculated based on the unit energy consumption E_step and the step difference, encouraging the inspection robot to choose a path with lower energy consumption. This helps to reduce energy consumption costs during the inspection process and improve the economic efficiency of the inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0020] Figure 1 It is a flowchart of a method for optimizing an embodied intelligent path for power inspection based on reinforcement learning according to some embodiments of this specification;

[0021] Figure 2 It is a schematic diagram of a process of determining multiple node clusters to be inspected according to some embodiments of this specification;

[0022] Figure 3 It is a module diagram of an embodied intelligent path optimization system for power inspection based on reinforcement learning as shown in some embodiments of this specification. DETAILED DESCRIPTION

[0023] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0024] Figure 1 is a flow chart of a method for optimizing an intelligent path for power inspection based on reinforcement learning according to some embodiments of this specification, such as Figure 1 As shown, the power inspection embodied intelligent path optimization method based on reinforcement learning can include the following steps.

[0025] Step 110: Acquire regional information of the area to be inspected, and establish an environmental model of the area to be inspected based on the regional information of the area to be inspected.

[0026] The environment model of the area to be inspected includes multiple nodes to be inspected, wherein each node to be inspected may include power equipment (such as transformers, electric poles, etc.). Specifically, the area to be inspected is modeled as a graph or a grid, wherein nodes represent nodes to be inspected and edges represent paths between nodes to be inspected.

[0027] Step 120: Obtain historical operation information of multiple nodes to be inspected.

[0028] Specifically, the historical operation information of the node to be inspected may include a collection of operation status, performance parameters, fault records, and other relevant data of the power equipment of the node to be inspected at multiple historical time points.

[0029] Step 130: determining the risk association relationship between the multiple nodes to be inspected based on the historical operation information of the multiple nodes to be inspected.

[0030] Specifically include:

[0031] According to the historical operation information of multiple nodes to be inspected, the state value of each node to be inspected at multiple historical time points is determined; when the state of the node to be inspected at the historical time point is a fault state, the state value of the node to be inspected at the historical time point is 1; when the state of the node to be inspected at the historical time point is a normal state, the state value of the node to be inspected at the historical time point is 0; according to the state value of each node to be inspected at multiple historical time points, the first fault association parameters of any two nodes to be inspected are determined;

[0032] Determine multiple clusters of nodes to be inspected according to first fault association parameters of any two nodes to be inspected by a clustering algorithm;

[0033] For each node cluster to be inspected, the target meteorological factor corresponding to the node cluster to be inspected is screened according to the historical operation information of the multiple nodes to be inspected, and the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions are determined according to the target meteorological factor corresponding to the node cluster to be inspected, wherein the risk association relationship between the multiple nodes to be inspected includes the second fault association parameters of any two nodes to be inspected included in each node cluster to be inspected under multiple meteorological conditions.

[0034] Specifically, the first fault correlation parameters of the two nodes to be inspected can be calculated according to the following formula:

[0035]

[0036] in, is the first fault association parameter between the i-th node to be inspected and the j-th node to be inspected, is the state value of the i-th node to be inspected at the m-th historical time point, is the state value of the jth node to be inspected at the mth historical time point, is the total number of historical time points sampled.

[0037] Figure 2 is a schematic diagram of a process for determining multiple node clusters to be inspected according to some embodiments of this specification, such as Figure 2 As shown, through a clustering algorithm, according to the first fault association parameters of any two nodes to be inspected, multiple clusters of nodes to be inspected are determined, including:

[0038] S11, calculating the discreteness of the first fault association parameters according to the first fault association parameters of any two nodes to be inspected, and determining the initial number of clusters K according to the discreteness of the first fault association parameters;

[0039] S12. According to the number of clusters K, randomly select K nodes to be inspected as cluster centers;

[0040] S13, for each node to be inspected that is not a cluster center, based on the first fault association parameter between the node to be inspected and each cluster center, assign the node to be inspected to the cluster of nodes to be inspected in the cluster center with the largest first fault association parameter;

[0041] S14, for each node cluster to be inspected, calculating the intra-cluster discrete parameter of the node cluster to be inspected according to the first fault association parameters of any two nodes to be inspected included in the node cluster to be inspected;

[0042] S15, judging whether the first clustering condition is met according to the intra-cluster discrete parameters of each node cluster to be inspected, if not, executing S16, if yes, executing S17, for example, the first clustering condition can be that the intra-cluster discrete parameters of each node cluster to be inspected are all less than the intra-cluster discrete parameter threshold;

[0043] S16, for each node cluster to be inspected, according to the first fault association parameters of any two nodes to be inspected included in the node cluster to be inspected, calculate the single-point discrete parameter corresponding to each node to be inspected included in the node cluster to be inspected, according to the single-point discrete parameter corresponding to each node to be inspected, update the cluster center of the node cluster to be inspected, and execute S13, for example, the node to be inspected with the smallest single-point discrete parameter can be used as the new cluster center of the node cluster to be inspected;

[0044] S17, calculating the out-of-cluster discrete parameter according to the first fault association parameter between any two cluster centers;

[0045] S18, judging whether the second clustering condition is met according to the out-cluster discrete parameter, if so, determining multiple node clusters to be inspected, if not, executing S19, for example, the second clustering condition may be that the out-cluster discrete parameter is less than the out-cluster discrete parameter threshold;

[0046] S19. Determine the iteration step length according to the out-cluster discrete parameters, update the number of clusters K according to the iteration step length, and execute S12.

[0047] Specifically, the discreteness of the first fault association parameter may be calculated according to the following formula:

[0048]

[0049]

[0050] in, is the discreteness of the first fault associated parameter, is the mean value of the first fault-related parameter of the nth node to be inspected, is the total number of nodes to be inspected, is the first fault association parameter between the nth node to be inspected and the gth node to be inspected.

[0051] The initial number of clusters K can be calculated according to the following formula:

[0052]

[0053] in, is the preset number of clusters, is the preset parameter, Greater than 0.

[0054] The intra-cluster dispersion parameter can be calculated according to the following formula:

[0055]

[0056] in, is the intra-cluster discrete parameter of the kth node cluster to be inspected, is the first fault association parameter of the ith node to be inspected and the jth node to be inspected included in the kth node cluster to be inspected, is the total number of nodes to be inspected included in the kth node cluster to be inspected.

[0057] The single-point discrete parameters corresponding to the node to be inspected can be calculated according to the following formula:

[0058]

[0059] in, is the single-point discrete parameter of the ith node to be inspected included in the kth node cluster to be inspected.

[0060] The out-of-cluster dispersion parameter can be calculated according to the following formula:

[0061]

[0062] in, is the out-of-cluster discrete parameter, is the first fault association parameter between the e-th cluster center and the f-th cluster center.

[0063] The iteration step size can be calculated according to the following formula:

[0064]

[0065] in, is the iteration step length, is the preset iteration step size, is the preset parameter, Greater than 0.

[0066] The number of clusters K can be updated according to the following formula:

[0067]

[0068] in, is the updated number of clusters, is the number of clusters before updating.

[0069] In some embodiments, based on the historical operation information of multiple nodes to be inspected, the target meteorological factor corresponding to the cluster of nodes to be inspected is screened, including:

[0070] For each node to be inspected included in the node cluster to be inspected, calculate the meteorological correlation parameters of each meteorological factor (for example, temperature, humidity, wind speed, etc.) and the node to be inspected according to the historical operation information of the node to be inspected and the historical meteorological information of the area to be inspected;

[0071] For each meteorological factor, a target meteorological factor corresponding to the node cluster to be inspected is screened according to the meteorological factor and the meteorological association parameter of each node to be inspected included in the node cluster to be inspected.

[0072] Specifically, the state value of each node to be inspected at multiple historical time points can be determined based on the historical operation information of the node to be inspected, and the factor value of each meteorological factor at multiple corresponding historical time points can be determined based on the historical meteorological information of the area to be inspected. The meteorological correlation parameters of the meteorological factors and the nodes to be inspected are calculated based on the state values ​​of the nodes to be inspected at multiple historical time points and the factor values ​​of each meteorological factor at multiple corresponding historical time points, wherein the method of calculating the meteorological correlation parameters of the meteorological factors and the nodes to be inspected is consistent with the method of calculating the first fault correlation parameters of the two nodes to be inspected, which will not be repeated here.

[0073] For each meteorological factor, the meteorological correlation parameters of the meteorological factor and each node to be inspected included in the node cluster to be inspected can be averaged to obtain the mean value of the meteorological correlation parameters corresponding to the meteorological factor, and the meteorological factor whose meteorological correlation parameter mean value is greater than the meteorological correlation parameter mean threshold value is used as the target meteorological factor corresponding to the node cluster to be inspected.

[0074] In some embodiments, according to the target meteorological factor corresponding to the node cluster to be inspected, determining the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under various meteorological conditions includes:

[0075] For each target meteorological factor corresponding to the node cluster to be inspected, a plurality of sample meteorological conditions are sampled according to the historical meteorological information of the area to be inspected. For example, the meteorological conditions at the historical time point A1 can be sampled as sample meteorological conditions according to the historical meteorological information of the area to be inspected. After multiple rounds of sampling, the meteorological conditions at multiple historical time points are obtained as sample meteorological conditions. According to each target meteorological factor corresponding to the node cluster to be inspected and the historical meteorological information of the area to be inspected, the meteorological feature vector corresponding to each sample meteorological condition is determined, wherein the meteorological feature vector may include the factor value of each target meteorological factor corresponding to the node cluster to be inspected, the cosine similarity of the meteorological feature vectors of any two meteorological conditions is calculated, and the plurality of sample meteorological conditions are deduplicated according to the cosine similarity of the meteorological feature vectors of any two meteorological conditions to obtain the plurality of deduplicated sample meteorological conditions.

[0076] For each sample meteorological condition, the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological condition is determined according to the historical meteorological information of the area to be inspected. The second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under the sample meteorological condition are calculated according to the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological condition. For example, for a certain historical time point, the meteorological feature vector of the historical time point is obtained according to the historical meteorological information of the historical time point. The cosine similarity between the meteorological feature vector of the historical time point and the meteorological feature vector of each sample meteorological condition is calculated. The sample meteorological condition with the maximum cosine similarity and greater than the cosine similarity threshold is taken as the belonging sample meteorological condition. Then, the historical operation information of each node to be inspected included in the node cluster to be inspected at the historical time point is the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the belonging sample meteorological condition, thereby realizing grouping of the historical operation information of multiple historical time points according to multiple sample meteorological conditions.

[0077] Through the interpolation algorithm, the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under various meteorological conditions are determined according to the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under various sample meteorological conditions.

[0078] Specifically, based on the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological conditions, the method of calculating the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under the sample meteorological conditions is consistent with the method of calculating the first fault association parameters of the two nodes to be inspected, which will not be repeated here.

[0079] Interpolation is a mathematical method used to estimate or predict the value of unknown data points between known data points. It estimates the value of unknown points by constructing a mathematical model (such as polynomial, exponential function, etc.) based on the distribution and trend of known data points.

[0080] Determine the second fault association parameters of the node cluster to be inspected measured under multiple sample meteorological conditions (such as different temperatures, humidity, wind speeds, etc.). These sample meteorological conditions may represent various environments that the device or system may encounter in actual operation. Through interpolation algorithms (such as linear interpolation, polynomial interpolation, Lagrange interpolation, Newton interpolation, etc.), determine the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions under other (unmeasured) meteorological conditions.

[0081] Using the selected interpolation method, a mathematical model is constructed based on the known data points. This model will be able to predict the second fault associated parameter under unknown meteorological conditions. Predict unknown values: Input the meteorological conditions to be predicted into the model to obtain the corresponding predicted values ​​of the second fault associated parameter.

[0082] Step 140: construct an inspection path optimization model based on a deep Q network based on the environmental model of the area to be inspected and the risk association relationship between the multiple nodes to be inspected.

[0083] Specifically, the inspection path optimization model based on the deep Q network can be constructed according to the following process:

[0084] 1. Problem definition and modeling

[0085] Environmental modeling: Model the power network as a graph or grid, where nodes represent power facilities (such as transformers, poles, etc.) and edges represent paths or connections between facilities.

[0086] State space: Define the state space, including the current position, visited nodes, priorities and correlation coefficients of remaining nodes, etc.

[0087] Action Space: Define the action space, including actions to move to adjacent nodes.

[0088] Reward function: Design the reward function to reflect the combined effects of priority, correlation coefficient, and energy consumption.

[0089] 2. Reward function design

[0090] Among them, the reward function of the inspection path optimization model is related to the risk association relationship between multiple nodes to be inspected, the real-time risk value of each node to be inspected, and the inspection energy consumption.

[0091] Specifically, the reward function of the inspection path optimization model is:

[0092] in, For the current state Actions taken Then transfer to the next state The total reward received when For priority rewards, For association rewards, For energy consumption rewards, , and is the weight coefficient, , and greater than 0, .

[0093] Priority rewards, association rewards, and energy consumption rewards are calculated based on the following formula:

[0094]

[0095]

[0096]

[0097] in, For the next state The corresponding real-time risk value of the node to be inspected, is the first attenuation coefficient, greater than 0, is an exponential function, From the start node to the next state The number of steps of the corresponding node to be inspected, For the current state The corresponding node to be inspected and the next state The current second fault association parameters between the corresponding nodes to be inspected, is the second attenuation coefficient, greater than 0, From the starting node to the current state The number of steps of the corresponding node to be inspected, For unit energy consumption.

[0098] 3. Inspection path optimization model based on deep Q network

[0099] Step 150: training the inspection path optimization model.

[0100] Specifically include:

[0101] Initialization: Initialize weights and set hyperparameters such as learning rate and discount factor.

[0102] State sampling: Start from the initial state, select an action based on the current policy, execute the action and observe the new state and reward.

[0103] Experience storage: Store state transitions (state, action, reward, new state) in the experience replay buffer.

[0104] Network update: Randomly sample a batch of experiences from the experience replay buffer to update the weights.

[0105] Strategy improvement: Improve the current strategy based on the updated network.

[0106] Iteration: Repeat the above process until the predetermined number of training rounds or convergence condition is reached.

[0107] Step 160: Acquire real-time operation information of multiple nodes to be inspected.

[0108] Specifically, the real-time operation information of the node to be inspected may include the operation status, performance parameters, fault records and other relevant data of the power equipment of the node to be inspected at the current time point.

[0109] Step 170 , determining a real-time risk value of each node to be inspected according to the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected.

[0110] Specifically include:

[0111] Obtain weather information of the area to be inspected;

[0112] For each node cluster to be inspected, according to the target meteorological factor corresponding to the node cluster to be inspected, the meteorological information of the area to be inspected, and the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under various meteorological conditions, calculate the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected;

[0113] For each node to be inspected, determine the initial risk value of the node to be inspected based on the real-time operation information of the node to be inspected;

[0114] For each node cluster to be inspected, the real-time risk value of each node to be inspected included in the node cluster is determined according to the current second fault association parameters of any two nodes to be inspected included in the node cluster and the initial risk value of each node to be inspected included in the node cluster.

[0115] Specifically, for each node cluster to be inspected, the current meteorological feature vector is determined according to the target meteorological factor corresponding to the node cluster to be inspected and the meteorological information of the area to be inspected, the cosine similarity between the current meteorological feature vector and the meteorological feature vector of each meteorological condition is calculated, and the meteorological condition with the maximum cosine similarity and greater than the cosine similarity threshold is taken as the target meteorological condition. The second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under the target meteorological condition are obtained as the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected.

[0116] In some embodiments, determining the real-time risk value of each node to be inspected included in the node cluster to be inspected according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the initial risk value of each node to be inspected included in the node cluster to be inspected includes:

[0117] S21, updating the risk value of each node to be inspected included in the node cluster to be inspected according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the initial risk value of each node to be inspected included in the node cluster to be inspected;

[0118] S22, judging whether convergence has occurred according to the risk value before and after the update of each node to be inspected included in the node cluster to be inspected, if so, taking the updated risk value of each node to be inspected included in the node cluster to be inspected as the real-time risk value, if not, executing S23;

[0119] S23, updating the risk value of each node to be inspected included in the node cluster to be inspected again according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the updated risk value of each node to be inspected, and executing S22.

[0120] Just as an example, for the node A to be inspected, the impact value of each node to be inspected (for example, node B to be inspected and node C to be inspected, etc.) included in the node cluster to be inspected, whose current second fault associated parameter with the node A to be inspected is greater than the current second fault associated parameter threshold, on it is calculated. This impact value can be the risk value of the node B to be inspected and the node C to be inspected multiplied by their current second fault associated parameter with the node A to be inspected, and then summed. Similarly, for the node B to be inspected and the node C to be inspected, we also need to calculate the impact value of the node cluster to be inspected, including other nodes to be inspected, whose current second fault associated parameter with the node to be inspected is greater than the current second fault associated parameter threshold.

[0121] For each node to be inspected included in the node cluster to be inspected, the risk value of the node to be inspected can be updated according to the impact value of the node to be inspected whose current second fault association parameter with the node to be inspected is greater than the current second fault association parameter threshold.

[0122] For example, the risk value of the node to be inspected can be updated according to the following formula:

[0123]

[0124] in, is the risk value of the i-th node to be inspected in the t+1-th iteration, is the risk value of the i-th node to be inspected in the t-th iteration, is the current second fault association parameter of the jth node to be inspected included in the i-th node to be inspected cluster and the current second fault association parameter of the j-th node to be inspected whose current second fault association parameter is greater than the current second fault association parameter threshold, is the risk value of the tth iteration of the jth node to be inspected whose current second fault association parameter of the i-th node to be inspected is greater than the current second fault association parameter threshold, is the risk value of the t-1th iteration of the jth node to be inspected whose current second fault association parameter of the i-th node to be inspected is greater than the current second fault association parameter threshold, The total number of nodes to be inspected in the node cluster to be inspected whose current second fault association parameter with the i-th node to be inspected is greater than the current second fault association parameter threshold.

[0125] Check whether the risk value of each node to be inspected has converged. This can be achieved by comparing whether the difference in risk value before and after the update is less than a certain threshold. If the risk values ​​of all nodes to be inspected have converged, the iteration process ends and the real-time risk value of each node to be inspected is obtained. If the risk values ​​of nodes to be inspected that are greater than the number threshold have not converged, the next iteration is required. In the next iteration, use the updated risk value obtained in the previous iteration as a new starting point, repeat the above steps of calculating the impact value and updating the risk value, and check the convergence again.

[0126] Step 180 , generating a real-time power inspection optimization path according to the real-time risk value of each node to be inspected through the trained inspection path optimization model.

[0127] Step 190, based on the real-time power inspection optimization path, control the power inspection embodied robot to perform power inspection.

[0128] Figure 3 is a module diagram of an embodied intelligent path optimization system for power inspection based on reinforcement learning according to some embodiments of this specification, such as Figure 3 As shown, the power inspection embodied intelligent path optimization system based on reinforcement learning can include a model building module, a model training module, a risk assessment module, a path generation module and an automatic inspection module.

[0129] The model building module can be used to obtain regional information of the area to be inspected, and to establish an environmental model of the area to be inspected based on the regional information of the area to be inspected, wherein the environmental model of the area to be inspected includes multiple nodes to be inspected, and is also used to obtain historical operation information of the multiple nodes to be inspected, and determine the risk association relationship between the multiple nodes to be inspected based on the historical operation information of the multiple nodes to be inspected, and based on the environmental model of the area to be inspected and the risk association relationship between the multiple nodes to be inspected, construct an inspection path optimization model based on a deep Q network, wherein the reward function of the inspection path optimization model is related to the risk association relationship between the multiple nodes to be inspected, the real-time risk value of each node to be inspected, and the inspection energy consumption.

[0130] The model training module can be used to train the inspection path optimization model.

[0131] The risk assessment module can be used to obtain real-time operation information of multiple nodes to be inspected, and determine the real-time risk value of each node to be inspected based on the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected.

[0132] The path generation module can be used to generate a real-time power inspection optimization path according to the real-time risk value of each node to be inspected through the trained inspection path optimization model.

[0133] The automatic inspection module can be used to optimize the path based on real-time power inspection and control the power inspection embodied robot to perform power inspection.

[0134] The power inspection embodied intelligent path optimization system based on reinforcement learning can be used to execute the power inspection embodied intelligent path optimization method based on reinforcement learning. For more descriptions of the power inspection embodied intelligent path optimization system based on reinforcement learning, please refer to the relevant descriptions of the power inspection embodied intelligent path optimization method based on reinforcement learning, which will not be repeated here.

[0135] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. The embodied intelligent path optimization method for power inspection based on reinforcement learning is characterized by: include: Acquire regional information of the area to be inspected, and establish an environmental model of the area to be inspected based on the regional information of the area to be inspected, wherein the environmental model of the area to be inspected includes a plurality of nodes to be inspected; Obtaining historical operation information of the plurality of nodes to be inspected; According to the historical operation information of the multiple nodes to be inspected, the risk association relationship between the multiple nodes to be inspected is determined, specifically including: according to the historical operation information of the multiple nodes to be inspected, the state value of each node to be inspected at multiple historical time points is determined, and the first fault association parameters of any two nodes to be inspected are determined according to the state value of each node to be inspected at multiple historical time points; through a clustering algorithm, according to the first fault association parameters of any two nodes to be inspected, multiple clusters of nodes to be inspected are determined; for each of the clusters of nodes to be inspected, according to the historical operation information of the multiple nodes to be inspected, the target meteorological factors corresponding to the cluster of nodes to be inspected are screened, and according to the target meteorological factors corresponding to the cluster of nodes to be inspected, the second fault association parameters of any two nodes to be inspected included in the cluster of nodes to be inspected under multiple meteorological conditions are determined, wherein the risk association relationship between the multiple nodes to be inspected includes the second fault association parameters of any two nodes to be inspected included in each cluster of nodes to be inspected under multiple meteorological conditions; Based on the environmental model of the area to be inspected and the risk association relationship between the multiple nodes to be inspected, a patrol path optimization model based on a deep Q network is constructed, wherein the reward function of the patrol path optimization model is related to the risk association relationship between the multiple nodes to be inspected, the real-time risk value of each node to be inspected, and the patrol energy consumption; Training the inspection path optimization model; Obtain real-time operation information of multiple nodes to be inspected; Determine a real-time risk value of each of the nodes to be inspected according to the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected; Generate a real-time power inspection optimization path according to the real-time risk value of each node to be inspected through the trained inspection path optimization model; Based on the real-time power inspection optimization path, the power inspection embodied robot is controlled to perform power inspection.

2. The method for embodied intelligent path optimization for power inspection based on reinforcement learning according to claim 1 is characterized in that: By using a clustering algorithm, a plurality of clusters of nodes to be inspected are determined according to first fault association parameters of any two nodes to be inspected, including: S11, calculating the discreteness of the first fault association parameters according to the first fault association parameters of any two nodes to be inspected, and determining the initial number of clusters K according to the discreteness of the first fault association parameters; S12. According to the number of clusters K, randomly select K nodes to be inspected as cluster centers; S13, for each node to be inspected that is not a cluster center, based on the first fault association parameter between the node to be inspected and each cluster center, assign the node to be inspected to the cluster of nodes to be inspected in the cluster center with the largest first fault association parameter; S14, for each of the node clusters to be inspected, calculating the intra-cluster discrete parameters of the node cluster to be inspected according to the first fault association parameters of any two nodes to be inspected included in the node cluster to be inspected; S15, judging whether the first clustering condition is met according to the intra-cluster discrete parameters of each of the node clusters to be inspected, if not, executing S16, if yes, executing S17; S16, for each of the node clusters to be inspected, according to the first fault association parameters of any two nodes to be inspected included in the node cluster to be inspected, calculate the single point discrete parameter corresponding to each node to be inspected included in the node cluster to be inspected, according to the single point discrete parameter corresponding to each node to be inspected, update the cluster center of the node cluster to be inspected, and execute S13; S17, calculating the out-of-cluster discrete parameter according to the first fault association parameter between any two cluster centers; S18, judging whether the second clustering condition is met according to the out-of-cluster discrete parameter, if so, determining the multiple node clusters to be inspected, if not, executing S19; S19, determining an iteration step length according to the out-of-cluster discrete parameter, updating the number of clusters K according to the iteration step length, and executing S12.

3. The method for embodied intelligent path optimization for power inspection based on reinforcement learning according to claim 1 is characterized in that: Screening target meteorological factors corresponding to the cluster of nodes to be inspected according to historical operation information of the plurality of nodes to be inspected includes: For each node to be inspected included in the node cluster to be inspected, calculating the meteorological correlation parameter between each meteorological factor and the node to be inspected according to the historical operation information of the node to be inspected and the historical meteorological information of the area to be inspected; For each meteorological factor, the target meteorological factor corresponding to the node cluster to be inspected is screened according to the meteorological factor and the meteorological association parameter of each node to be inspected included in the node cluster to be inspected.

4. The method for optimizing the power inspection embodied intelligent path based on reinforcement learning according to claim 2 is characterized in that: According to the target meteorological factor corresponding to the node cluster to be inspected, second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions are determined, including: For each target meteorological factor corresponding to the node cluster to be inspected, a plurality of sample meteorological conditions are sampled according to the historical meteorological information of the area to be inspected; For each sample meteorological condition, according to the historical meteorological information of the area to be inspected, determine the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological condition, and according to the historical operation information of each node to be inspected included in the node cluster to be inspected corresponding to the sample meteorological condition, calculate the second fault association parameter of any two nodes to be inspected included in the node cluster to be inspected under the sample meteorological condition; Through an interpolation algorithm, the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions are determined according to the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple sample meteorological conditions.

5. The method for embodied intelligent path optimization for power inspection based on reinforcement learning according to any one of claims 1 to 4, characterized in that: Determining a real-time risk value of each of the nodes to be inspected according to the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected includes: Obtain weather information of the area to be inspected; For each of the node clusters to be inspected, according to the target meteorological factor corresponding to the node cluster to be inspected, the meteorological information of the area to be inspected, and the second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected under multiple meteorological conditions, calculate the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected; For each of the nodes to be inspected, determining an initial risk value of the node to be inspected according to the real-time operation information of the node to be inspected; For each of the node clusters to be inspected, the real-time risk value of each node to be inspected included in the node cluster is determined based on the current second fault association parameters of any two nodes to be inspected included in the node cluster and the initial risk value of each node to be inspected included in the node cluster.

6. The method for embodied intelligent path optimization for power inspection based on reinforcement learning according to claim 5 is characterized in that: Determining the real-time risk value of each node to be inspected included in the node cluster to be inspected according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the initial risk value of each node to be inspected included in the node cluster to be inspected, including: S21, updating the risk value of each node to be inspected included in the node cluster to be inspected according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the initial risk value of each node to be inspected included in the node cluster to be inspected; S22, judging whether convergence has occurred according to the risk value before and after the update of each node to be inspected included in the node cluster to be inspected, if so, taking the updated risk value of each node to be inspected included in the node cluster to be inspected as the real-time risk value, if not, executing S23; S23, updating the risk value of each node to be inspected included in the node cluster to be inspected again according to the current second fault association parameters of any two nodes to be inspected included in the node cluster to be inspected and the updated risk value of each node to be inspected, and executing S22.

7. The method for embodied intelligent path optimization for power inspection based on reinforcement learning according to claim 5 is characterized in that: The reward function of the inspection path optimization model is: ;in, For the current state Actions taken Then transfer to the next state The total reward received when For priority rewards, For association rewards, For energy consumption rewards, , and is the weight coefficient, , and greater than 0, .

8. The method for embodied intelligent path optimization for power inspection based on reinforcement learning according to claim 7 is characterized in that: Priority rewards, association rewards, and energy consumption rewards are calculated based on the following formula: ; ; ;in, For the next state The corresponding real-time risk value of the node to be inspected, is the first attenuation coefficient, greater than 0, is an exponential function, From the start node to the next state The number of steps of the corresponding node to be inspected, For the current state The corresponding node to be inspected and the next state The current second fault association parameters between the corresponding nodes to be inspected, is the second attenuation coefficient, greater than 0, From the starting node to the current state The number of steps of the corresponding node to be inspected, For unit energy consumption.

9. The power inspection embodied intelligent path optimization system based on reinforcement learning is characterized by: The method for implementing the power inspection embodied intelligent path optimization method based on reinforcement learning according to any one of claims 1 to 8 comprises: A model building module, used to obtain regional information of the area to be inspected, and to establish an environmental model of the area to be inspected based on the regional information of the area to be inspected, wherein the environmental model of the area to be inspected includes multiple nodes to be inspected, and is also used to obtain historical operation information of the multiple nodes to be inspected, and to determine the risk association relationship between the multiple nodes to be inspected according to the historical operation information of the multiple nodes to be inspected, and to construct an inspection path optimization model based on a deep Q network based on the environmental model of the area to be inspected and the risk association relationship between the multiple nodes to be inspected, wherein the reward function of the inspection path optimization model is related to the risk association relationship between the multiple nodes to be inspected, the real-time risk value of each node to be inspected, and the inspection energy consumption; A model training module, used for training the inspection path optimization model; A risk assessment module, used to obtain real-time operation information of multiple nodes to be inspected, and determine the real-time risk value of each node to be inspected according to the real-time operation information of the multiple nodes to be inspected and the risk association relationship between the multiple nodes to be inspected; A path generation module, used to generate a real-time power inspection optimization path according to the real-time risk value of each node to be inspected through a trained inspection path optimization model; The automatic inspection module is used to control the power inspection embodied robot to perform power inspection based on the real-time power inspection optimization path.

Images