Power agent data collection and processing method, and system, device and medium

Abstract

A power agent data collection and processing method, and a system, a device and a medium, which relate to the technical field of power agents. The method comprises: on the basis of spatial information and point cloud data in a relatively dense three-dimensional map of a power scenario, constructing a graph representation of a dense point cloud model (S2); on the basis of pose information of a power agent and environmental features, constructing a decision condition tree for each path decision point (S3); traversing each condition branch of the decision condition tree, using a graph theory algorithm to solve in the graph representation of the dense point cloud model the shortest path from a starting point of the power agent to each inspection target, and generating a potential preselected path set (S4); constructing a path dependency network on the basis of the potential preselected path set, and determining a conditional dependency relationship between paths (S5); and on the basis of the conditional dependency relationship between the paths, using a dynamic path optimization algorithm based on environmental perception to perform iterative optimization, so as to acquire an optimal inspection path (S6). The method not only improves the efficiency and adaptability of path planning, but also enhances the decision-making capability of a robot when faced with environmental changes, thereby ensuring that a power agent can efficiently and accurately complete an inspection task.

Description

Power agent data acquisition processing method, system, device and medium TECHNICAL FIELD

[0001] The present application relates to the technical field of power agents, and in particular to a power agent data acquisition processing method, system, device and medium. BACKGROUND

[0002] With the rapid development of power systems and the increasing demand for intelligentization, the inspection work of power equipment has become more important. As an important tool for automated inspection, the technical research and application of power agents have gradually become the focus of the power industry. When a power agent performs a task in a complex unknown environment, it needs to have a high degree of autonomy and intelligence to cope with changing working environments and task requirements.

[0003] In recent years, with the rapid development of machine vision and deep learning technologies, power agents have initially realized the function of automated inspection. However, current power agent inspection systems often rely only on a single sensor or technology, such as laser radar, ultrasonic waves, etc. These sensors cannot provide sufficient visual perception in some complex unknown environments, resulting in poor inspection results. Moreover, existing power agent inspection systems are mostly limited to basic positioning and navigation functions and cannot meet the needs of power agents for rapid response and efficient inspection in dynamic environments. It is difficult for them to comprehensively perceive and adapt to the environment. Therefore, how to improve the efficient and accurate path planning capabilities of power agents in complex and variable power environments to meet their real-time, autonomous and intelligent application requirements is a technical problem that needs to be solved. SUMMARY

[0004] The present application aims to provide a power agent data acquisition processing method, system, device and medium that can consider environmental perception information and path relationship to quickly solve the optimal inspection path and meet the needs of power agents for rapid response and efficient inspection.

[0005] In a first aspect, the present application provides a power agent data acquisition processing method, which comprises the following steps:

[0006] Capturing mixed visual perception information of the environment around the power agent, and constructing a relatively dense three-dimensional map of the power scene and power agent pose information based on an event-driven visual SLAM algorithm according to the mixed visual perception information;

[0007] Constructing a graph representation of the dense point cloud model according to the spatial information and point cloud data in the relatively dense three-dimensional map of the power scene;

[0008] Determine the location of the inspection target in the graph representation of the dense point cloud model, and construct a decision condition tree for each path decision point according to the power agent pose information and environmental characteristics;

[0009] Traverse each condition branch of the decision condition tree, and use a graph algorithm to solve the shortest path from the power agent starting point to each inspection target in the graph representation of the dense point cloud model to generate a set of potential pre-selected paths;

[0010] Based on the set of potential pre-selected paths, a path-dependent network is constructed, and the conditional dependence relationship between paths in the path-dependent network is analyzed;

[0011] According to the conditional dependence relationship between paths, a dynamic path optimization algorithm based on environmental perception is used to iteratively optimize the set of potential pre-selected paths to obtain the optimal inspection path.

[0012] In further embodiments, the step of constructing a relatively dense three-dimensional map of the power scene and power agent pose information based on the mixed visual perception information through an event-driven visual SLAM algorithm comprises:

[0013] Feature extraction is performed on the mixed visual perception information to obtain key feature points of the power scene, and the neighborhood region around each key feature point of the power scene is quantitatively described to obtain a feature point descriptor;

[0014] According to the feature point descriptor, key feature point matching of the power scene is performed between consecutive frames or mixed visual perception information of different perspectives to obtain matching feature points associated with power equipment;

[0015] The matching feature points are taken as nodes, and the spatiotemporal relationship between the matching feature points is taken as edges to construct a power scene event graph, and the matching feature points in the power scene event graph are taken as event-driven tracking targets of the visual SLAM algorithm;

[0016] When a change in the event-driven tracking target in the power scene event graph is detected, the topological structure and geometric information of the event triggering region in the power scene event graph are converted into state estimation graph constraints according to an event triggering mechanism;

[0017] The state estimation graph constraints are fused into the state estimation of the visual SLAM algorithm, and the relatively dense three-dimensional map of the power scene and the power agent pose information are obtained using the visual SLAM algorithm according to the spatial positioning information of the matching feature points.

[0018] In further embodiments, the state estimation graph constraint is specifically:

[0019] In the formula, F represents the state estimation graph constraint. represents the relative pose between node A and node B calculated according to the current estimated pose x; T AB represents the relative pose between node A and node B in the power scene event graph; represents the estimated position of node i; D i represents the actual position of node i; G represents the edge set of the power scene event graph.

[0020] In further embodiments, the step of constructing a graph representation of a dense point cloud model according to spatial information in a relatively dense three-dimensional map of a power scene and point cloud data comprises:

[0021] spatially align the spatial information of the relatively dense three-dimensional map of the power scene with the panoramic video stream using geographic information system technology, and perform frame-by-frame semantic segmentation on the spatially aligned panoramic video stream to obtain a semantic segmentation result of the video frames with semantic labels;

[0022] map the semantic segmentation result of the video frames with the spatial information of the relatively dense three-dimensional map of the power scene to generate a semantic map fused with spatial information, and construct a slice attribute graph based on the semantic map;

[0023] divide the nodes in the slice attribute graph into different community subgraphs using a label propagation algorithm, and calculate the node importance of each node in the slice attribute graph and the community subgraph importance of each community subgraph using a graph attention network;

[0024] obtain point cloud data of the power work environment according to the relatively dense three-dimensional map of the power scene, and identify key semantic regions and key community subgraphs with high correlation with the point cloud data from the slice attribute graph according to the node importance and the community subgraph importance;

[0025] map the key community subgraphs and key semantic regions back to the corresponding video frames and regions in the panoramic video stream to form a panoramic video slice corresponding to the point cloud data;

[0026] spatially register the point cloud data using power agent pose information to obtain registered point cloud data, project the registered point cloud data onto the corresponding position of the panoramic video slice, and form a graph representation of a dense point cloud model.

[0027] In further embodiments, the step of constructing a slice attribute graph based on a semantic map comprises:

[0028] In the semantic map, according to the semantic labels and the spatial information in the relatively dense three-dimensional map of the power scene, identify semantic regions;

[0029] The attribute of each semantic region is defined as a node of the slice attribute graph, and the spatial relationship and the semantic relationship between the semantic regions are defined as edges of the slice attribute graph to construct the slice attribute graph; wherein the attribute of the semantic region includes a region type and a region spatial geometry attribute.

[0030] In further embodiments, the step of constructing a decision condition tree for each path decision point according to the power agent pose information and the environment feature includes:

[0031] Obtaining environment feature information of an environment in which a power agent is located, determining a path decision point according to the power agent pose information and the environment feature information;

[0032] According to the task requirement of the power agent and the environment feature information, identifying a path selection decision condition considering a safe operation constraint for each path decision point;

[0033] Creating a decision condition node of the decision condition tree according to the path selection decision condition, and analyzing a logical relationship between each path selection decision condition to determine a conversion logical relationship between the decision condition nodes;

[0034] Obtaining all behavior states of the power agent, and determining a state conversion condition between different behavior states;

[0035] Connecting each decision condition node according to the conversion logical relationship between the decision condition nodes, and mapping the state conversion condition to the decision condition node to form the decision condition tree.

[0036] In further embodiments, the step of constructing a path-dependent network based on the set of potential preselected paths and analyzing a condition-dependent relationship between paths in the path-dependent network includes:

[0037] Defining all inspection targets and path decision points passed through by each potential preselected path in the set of potential preselected paths as network nodes, and defining the potential preselected paths as edges to construct the path-dependent network;

[0038] Analyzing a topological structure of the path-dependent network by a depth-first search algorithm, and dividing paths in the set of potential preselected paths into circular dependent paths and non-circular dependent paths;

[0039] According to a path transmission chain and a path decision point between each circular dependent path, determining a path transmission decision condition of the circular dependent path, and according to the path transmission decision condition, obtaining a condition transmission relationship between the circular dependent paths;

[0040] The execution precondition and the execution postcondition of the non-cyclic dependency path are identified by traversing each network node on the non-cyclic dependency path, and the correlation between the execution preconditions and the execution postconditions of different non-cyclic dependency paths is analyzed to obtain the conditional correlation of the non-cyclic dependency path.

[0041] According to the conditional transmission relationship and the conditional correlation, a conditional dependency relationship between paths in the path-dependent network is formed.

[0042] In further embodiments, the step of obtaining the optimal inspection path according to the conditional dependency relationship between paths and using a dynamic path optimization algorithm based on environmental perception to iteratively optimize the set of potential preselected paths comprises:

[0043] According to the mixed visual perception information, the current environmental state is obtained, and in the iteration process, a potential preselected path located at the starting point of the power agent is randomly selected from the set of potential preselected paths as an initial inspection path;

[0044] According to the conditional dependency relationship between paths, the next optimal connected path for inspection starting from the initial inspection path is selected and connected from the remaining potential preselected paths, and the initial inspection path is updated, and the step is repeated to gradually select the next optimal connected path until the target inspection path containing all inspection targets is constructed;

[0045] Based on the conditional dependency relationship between paths and the current environmental state, the path fitness value of the target inspection path constructed in the current iteration is evaluated using a fitness function;

[0046] It is judged whether the path fitness value reaches the fitness value convergence condition, if yes, the current target inspection path is output as the optimal inspection path, otherwise, the initial inspection path is selected from the set of potential preselected paths for iteration.

[0047] In further embodiments, the mathematical expression of the fitness function is:

[0048] wherein,

[0049] In the formula, f l represents the path fitness value; L l represents the length of the target inspection path l; S l (c n ) represents the degree of satisfaction of the target inspection path l to the conditional dependency relationship c n ; c n represents the nth conditional dependency relationship; N l represents the number of conditional dependency relationships on the target inspection path l; E nrepresents the influence degree of the current environment state on the nth conditional dependency relationship; a represents a path length weight factor; and β represents a conditional weight factor.

[0050] In a second aspect, the present application provides an electric power agent data acquisition and processing system, which comprises:

[0051] a three-dimensional map construction module, configured to capture mixed visual perception information of an environment around the electric power agent, and to construct a relatively dense three-dimensional map of the electric power scene and pose information of the electric power agent based on an event-driven visual SLAM algorithm according to the mixed visual perception information;

[0052] a point cloud model construction module, configured to construct a graph representation of a dense point cloud model according to spatial information and point cloud data in the relatively dense three-dimensional map of the electric power scene;

[0053] a conditional tree establishment module, configured to determine a location of an inspection target in the graph representation of the dense point cloud model, and to construct a decision conditional tree for each path decision point according to the pose information of the electric power agent and environmental characteristics;

[0054] a preselected path generation module, configured to traverse each conditional branch of the decision conditional tree, to solve a shortest path from a starting point of the electric power agent to each inspection target in the graph representation of the dense point cloud model using a graph theory algorithm, and to generate a set of potential preselected paths;

[0055] an optimal path planning module, configured to construct a path-dependent network based on the set of potential preselected paths, to analyze conditional dependency relationships between paths in the path-dependent network, and to iteratively optimize the set of potential preselected paths using a dynamic path optimization algorithm based on environmental perception according to the conditional dependency relationships between the paths, so as to obtain an optimal inspection path.

[0056] In further embodiments, the construction of the relatively dense three-dimensional map of the electric power scene and the pose information of the electric power agent based on the mixed visual perception information using the event-driven visual SLAM algorithm specifically comprises:

[0057] feature extraction is performed on the mixed visual perception information to obtain key feature points of the electric power scene, and a neighborhood region around each key feature point of the electric power scene is quantitatively described to obtain a feature point descriptor;

[0058] key feature points of the electric power scene are matched between continuous frames or mixed visual perception information of different perspectives according to the feature point descriptor, so as to obtain matching feature points associated with the electric power equipment;

[0059] Match the feature points as nodes, and match the spatio-temporal relationship between the feature points as edges, construct a power scene event graph, and take the matching feature points in the power scene event graph as the event-driven tracking target of the visual SLAM algorithm;

[0060] When the event-driven tracking target in the power scene event graph is detected to change, the topological structure and geometric information of the event triggering area in the power scene event graph are converted into state estimation graph constraints according to the event triggering mechanism;

[0061] Fuse the state estimation graph constraints into the state estimation of the visual SLAM algorithm, and according to the spatial positioning information of the matching feature points, use the visual SLAM algorithm to obtain a relatively dense three-dimensional map of the power scene and power agent pose information.

[0062] In further embodiments, the state estimation graph constraint is specifically:

[0063] In the formula, F represents the state estimation graph constraint; represents the relative pose between node A and node B calculated according to the current estimated pose x; T AB represents the relative pose between node A and node B in the power scene event graph; represents the estimated position of node i; D i represents the actual position of node i; G represents the edge set of the power scene event graph.

[0064] In further embodiments, the point cloud model construction module is specifically configured to:

[0065] Align the spatial information of the relatively dense three-dimensional map of the power scene with the panoramic video stream using geographic information system technology, and perform frame-by-frame semantic segmentation on the spatially aligned panoramic video stream to obtain a video frame semantic segmentation result with semantic labels;

[0066] Map the video frame semantic segmentation result and the spatial information of the relatively dense three-dimensional map of the power scene to generate a semantic map fused with spatial information, and construct a slice attribute graph based on the semantic map;

[0067] Divide the nodes in the slice attribute graph into different community subgraphs using a label propagation algorithm, and calculate the node importance of each node and the community subgraph importance of each community subgraph in the slice attribute graph using a graph attention network;

[0068] Obtain point cloud data of the power operation environment according to the relatively dense three-dimensional map of the power scene, and identify key semantic regions and key community subgraphs with high correlation with the point cloud data from the slice attribute graph according to the node importance and the community subgraph importance.

[0069] mapping the key community subgraph and the key semantic region back to the corresponding video frames and regions in the panoramic video stream, to form a panoramic video slice corresponding to the point cloud data;

[0070] spatially registering the point cloud data using the power agent pose information to obtain registered point cloud data, and projecting the registered point cloud data onto the corresponding positions of the panoramic video slice to form a graph representation of the dense point cloud model.

[0071] In a further embodiment, the slice attribute graph is constructed based on semantic mapping, specifically comprising:

[0072] In the semantic mapping, a semantic region is identified according to the semantic label and the spatial information in the relatively dense three-dimensional map of the power scene;

[0073] defining the attributes of each semantic region as nodes of the slice attribute graph, defining the spatial and semantic relationships between the semantic regions as edges of the slice attribute graph, and constructing the slice attribute graph; wherein the attributes of the semantic region include the region type and the region spatial geometry attribute.

[0074] In a further embodiment, the decision condition tree is constructed for each path decision point according to the power agent pose information and the environmental characteristics, specifically comprising:

[0075] obtaining environmental characteristic information of the environment in which the power agent is located, determining a path decision point according to the power agent pose information and the environmental characteristic information;

[0076] identifying path selection decision conditions considering safety operation constraints for each path decision point according to the task requirements of the power agent and the environmental characteristic information;

[0077] creating decision condition nodes of the decision condition tree according to the path selection decision conditions, and analyzing the logical relationships between each of the path selection decision conditions to determine the conversion logic relationships between the decision condition nodes;

[0078] obtaining all behavior states of the power agent, and determining state transition conditions between different behavior states;

[0079] connecting each decision condition node according to the conversion logic relationships between the decision condition nodes, and mapping the state transition conditions to the decision condition nodes to form the decision condition tree.

[0080] In a further embodiment, the path-dependent network is constructed based on the set of potential preselected paths, and the conditional dependence relationships between paths in the path-dependent network are analyzed, specifically comprising:

[0081] all the inspection targets and path decision points through which each potential pre-selected path in the potential pre-selected path set passes are defined as network nodes, and the potential pre-selected paths are defined as edges, to construct a path-dependent network;

[0082] The topology of the path-dependent network is analyzed by a depth-first search algorithm, and the paths in the potential pre-selected path set are divided into cyclically dependent paths and non-cyclically dependent paths;

[0083] According to the path transmission chain and the path decision points between each cyclically dependent path, the path transmission decision condition of the cyclically dependent path is determined, and the conditional transmission relationship between the cyclically dependent paths is obtained according to the path transmission decision condition;

[0084] Each network node on each non-cyclically dependent path is traversed, the execution precondition and the execution postcondition of the non-cyclically dependent path are identified, and the association relationship between the execution preconditions and the execution postconditions of different non-cyclically dependent paths is analyzed, to obtain the conditional association relationship of the non-cyclically dependent paths;

[0085] According to the conditional transmission relationship and the conditional association relationship, a conditional dependence relationship between the paths in the path-dependent network is formed.

[0086] In further embodiments, the potential pre-selected path set is iteratively optimized using a dynamic path optimization algorithm based on environmental perception according to the conditional dependence relationship between the paths, to obtain an optimal inspection path, specifically including:

[0087] The current environmental state is obtained according to the mixed visual perception information, and in the iteration process, a potential pre-selected path located at the starting point of the power agent is randomly selected from the potential pre-selected path set as an initial inspection path;

[0088] According to the conditional dependence relationship between the paths, the next optimal connected path for inspection starting from the initial inspection path is selected and connected from the remaining potential pre-selected paths, and the initial inspection path is updated, the step is repeated, the next optimal connected path is gradually selected, and the target inspection path containing all the inspection targets is constructed;

[0089] Based on the conditional dependence relationship between the paths and the current environmental state, the path fitness value of the target inspection path constructed in the current iteration is evaluated using a fitness function;

[0090] It is judged whether the path fitness value reaches a fitness value convergence condition, if yes, the current target inspection path is output as the optimal inspection path, otherwise, the initial inspection path is selected from the potential pre-selected path set for iteration.

[0091] In further embodiments, the mathematical expression of the fitness function is:

[0092] wherein,

[0093] wherein, f l represents the path fitness value; L l represents the length of the target inspection path l; S l (c n ) represents the degree of satisfaction of the target inspection path l to the condition dependency relationship c n ; c n represents the nth condition dependency relationship; N l represents the number of condition dependency relationships on the target inspection path l; E n represents the influence degree of the current environment state on the nth condition dependency relationship; a represents a path length weight factor; and b represents a condition weight factor.

[0094] In a third aspect, the present application further provides a computer device, comprising a processor and a memory, wherein the processor is connected with the memory, the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, so that the computer device executes the steps of the above method.

[0095] In a fourth aspect, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0096] The present application provides an electric power agent data acquisition and processing method, system, device and medium. The method analyzes mixed visual perception information based on an event-driven visual SLAM algorithm to obtain a graph representation of a dense point cloud model. A decision condition tree is constructed for each path decision point according to electric power agent pose information and environment characteristics. Each condition branch of the decision condition tree is traversed, and a graph theory algorithm is used to solve the shortest path from the starting point of the electric power agent to each inspection target in the graph representation of the dense point cloud model to generate a set of potential preselected paths. A path dependency network is constructed based on the set of potential preselected paths, and the condition dependency relationships between paths in the path dependency network are analyzed. According to the condition dependency relationships between paths, a dynamic path optimization algorithm based on environment perception is used to iteratively optimize the set of potential preselected paths to obtain an optimal inspection path. Compared with traditional path planning methods, the method plans paths in a visual dense point cloud model, analyzes the condition dependency relationships between paths, and uses a dynamic path optimization algorithm to obtain an optimal inspection path, so that the planning result is more consistent with the actual environment, significantly improves the efficiency of electric power agent inspection, ensures the accuracy of inspection work, and reduces energy consumption and inspection time. BRIEF DESCRIPTION OF DRAWINGS

[0097] Fig. 1 is a flowchart of a power intelligent agent data acquisition and processing method according to an embodiment of the present application;

[0098] Fig. 2 is a block diagram of a power intelligent agent data acquisition and processing system according to an embodiment of the present application;

[0099] Fig. 3 is a structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0100] The embodiments of the present application will be described in detail below with reference to the drawings, and the embodiments are given only for illustrative purposes and cannot be understood as limiting the present application. The accompanying drawings are only for reference and illustration, and do not constitute a limitation on the scope of patent protection of the present application, because many changes can be made to the present application without departing from the spirit and scope of the present application.

[0101] Referring to Fig. 1, the present embodiment provides a power intelligent agent data acquisition and processing method, as shown in Fig. 1, the method comprises the following steps:

[0102] S1. Capture the mixed visual perception information of the environment around the power intelligent agent, and construct a relatively dense three-dimensional map of the power scene and the pose information of the power intelligent agent according to the mixed visual perception information by using an event-driven visual SLAM algorithm.

[0103] In the present embodiment, a binocular camera and a panoramic camera are configured to capture visual information of the environment, wherein the binocular camera is used to capture binocular visual information around the power intelligent agent to provide depth information for target detection and positioning; the panoramic camera can be used to capture panoramic image information of the environment around the power intelligent agent, and then the binocular visual information and the panoramic image information are fused to form the mixed visual perception information of the environment around the power intelligent agent, and a relatively dense three-dimensional map of the power scene is constructed by using an event-driven visual SLAM algorithm, and the pose information of the power intelligent agent in the three-dimensional map is obtained by using the SLAM algorithm. In the present embodiment, the step of constructing a relatively dense three-dimensional map of the power scene and the pose information of the power intelligent agent according to the mixed visual perception information by using an event-driven visual SLAM algorithm comprises:

[0104] Feature extraction is performed on the mixed visual perception information to obtain key feature points of the power scene, and a neighborhood region around each key feature point of the power scene is quantitatively described to obtain a feature point descriptor;

[0105] According to the feature point descriptor, key feature point matching of the power scene is performed between continuous frames or mixed visual perception information of different perspectives to obtain matching feature points associated with power equipment;

[0106] Matched feature points are taken as nodes, and the spatio-temporal relationship between the matched feature points is taken as edges to construct a power scene event graph, and the matched feature points in the power scene event graph are taken as event-driven tracking targets of a visual SLAM algorithm;

[0107] When it is detected that the event-driven tracking target in the power scene event graph changes, the topology and geometric information of the event triggering region in the power scene event graph are converted into state estimation graph constraints according to an event triggering mechanism; wherein the state estimation graph constraints are specifically:

[0108] In the formula, F represents the state estimation graph constraint; represents the relative pose between node A and node B calculated according to the current estimated pose x; T AB represents the relative pose between node A and node B in the power scene event graph; represents the estimated position of node i; D i represents the actual position of node i; G represents the edge set of the power scene event graph;

[0109] The state estimation graph constraints are fused into the state estimation of the visual SLAM algorithm, and according to the spatial positioning information of the matched feature points, the visual SLAM algorithm is used to obtain a relatively dense three-dimensional map of the power scene and power agent pose information.

[0110] Specifically, the embodiment needs to first perform a preprocessing operation such as noise removal on the mixed visual perception information to improve data quality, and then perform feature extraction on the preprocessed mixed visual perception information, find out power scene key feature points, and calculate feature point descriptors for each power scene key feature point. The feature point descriptor is a quantitative description of the neighborhood region around the power scene key feature point, so as to match the descriptors of the power scene key feature points between continuous frames or image information of different perspectives to find out the same feature points as matching feature points. The neighborhood region around the power scene key feature point can be a fixed window or a local region around the key feature point, which can be adaptively determined by a person skilled in the art according to the scale of the key point. The feature point descriptor can be obtained by an algorithm such as SIFT descriptor or SURF descriptor, but the feature point descriptor needs to have invariance to the rotation, scaling and partial brightness change of the image. Then the embodiment regards the matching feature points as nodes, and the space-time relationship (such as spatial distance, time sequence) between the nodes as edges, constructs an event graph for tracking the changes of the matching feature points in different images, and provides a data basis for the SLAM algorithm. The event graph contains the topological structure (connection relationship between nodes) and geometric information (such as node position, direction, etc.) in the power scene. The topological information describes the spatial and temporal relationship between different events or objects, and the geometric information provides the specific position and direction of these events or objects in space.

[0111] The embodiment monitors the changes of the matching feature points in the event graph, such as the addition or disappearance of the feature points related to the power equipment, and the changes can trigger specific events. When the matching feature points in the event graph are monitored, the topological structure and geometric information of the event triggering region in the event graph are extracted, the constraint condition containing the event graph information is constructed in the SLAM system, the state estimation graph constraint is generated and added to the pose optimization problem of the SLAM, in the state estimation of the SLAM, the state vector containing the pose (position and direction) of the power agent and the spatial position of all feature points is constructed, in the optimization process of the pose estimation, the state estimation graph constraint of the event graph can be added as an additional term to the objective function in a weighted manner, and a nonlinear optimization algorithm is used to minimize the optimization objective function, so as to solve the state vector. The state vector is updated through the iteration process until a stable solution is converged. In each iteration, the calculation of the state estimation graph constraint is updated according to the current pose and the position of the matching feature points to obtain the latest pose estimation, so as to realize real-time and high-precision pose estimation. The key event topological structure and geometric information in the power scene event graph are directly integrated into the visual SLAM framework as the state estimation graph constraint, so that the visual SLAM algorithm can not only realize the pose estimation of the power agent, but also construct a relatively dense three-dimensional map fused with the event graph, so that the autonomous navigation of the power agent is more efficient and reliable, and the robustness and accuracy of the system are improved.

[0112] S2. Construct a graph representation of the dense point cloud model according to the spatial information in the relatively dense three-dimensional map of the power scene and the point cloud data.

[0113] In the embodiment, the step of constructing a graph representation of the dense point cloud model according to the spatial information in the relatively dense three-dimensional map of the power scene and the point cloud data comprises:

[0114] The spatial information of the relatively dense three-dimensional map of the power scene is spatially aligned with the panoramic video stream by using geographic information system technology, and the spatially aligned panoramic video stream is frame-by-frame semantic segmented to obtain a video frame semantic segmentation result with semantic labels;

[0115] The video frame semantic segmentation result is mapped with the spatial information of the relatively dense three-dimensional map of the power scene to generate a semantic mapping fused with spatial information, and a slice attribute graph is constructed based on the semantic mapping;

[0116] The nodes in the slice attribute graph are divided into different community subgraphs by using a label propagation algorithm, and the node importance of each node in the slice attribute graph and the community subgraph importance of each community subgraph are calculated by using a graph attention network;

[0117] Point cloud data of a power operation environment is acquired according to a relatively dense three-dimensional map of a power scene, and key semantic regions and key community subgraphs with high correlation degrees with the point cloud data are identified from a slice attribute graph according to node importance and community subgraph importance.

[0118] The key community subgraphs and the key semantic regions are mapped back to corresponding video frames and regions in a panoramic video stream to form panoramic video slices corresponding to the point cloud data.

[0119] The point cloud data is spatially registered using power agent pose information to obtain registered point cloud data, and the registered point cloud data is projected onto corresponding positions of the panoramic video slices to form a graph representation of the dense point cloud model.

[0120] Specifically, the embodiment acquires a panoramic video stream from mixed visual perception information, matches spatial information of a relatively dense three-dimensional map of a power scene with location information in the panoramic video stream using geographic information system (GIS) technology, and simultaneously ensures that the relatively dense three-dimensional map of the power scene and the panoramic video stream are time-stamped synchronously to align the spatial information of the relatively dense three-dimensional map of the power scene with the panoramic video stream. Then, a deep learning model is used to perform frame-by-frame semantic segmentation on the spatially aligned panoramic video stream to obtain semantic segmentation results of video frames with semantic labels, and a semantic map containing spatial information and semantic information is generated by mapping the semantic segmentation results of the video frames to corresponding positions on the relatively dense three-dimensional map of the power scene. A slice attribute graph is constructed based on the semantic map, for example, when the semantic segmentation results of the video frames are video frames with semantic labels, the embodiment maps pixel positions in the video frames to corresponding positions on the relatively dense three-dimensional map of the power scene. In the embodiment, the step of constructing the slice attribute graph based on the semantic map includes:

[0121] In the semantic map, semantic regions are identified according to the semantic labels and the spatial information in the relatively dense three-dimensional map of the power scene. The attributes of each semantic region are defined as nodes of the slice attribute graph, and the spatial and semantic relationships between the semantic regions are defined as edges of the slice attribute graph to construct the slice attribute graph. The attributes of the semantic regions include region types and region spatial geometric attributes.

[0122] Then, the embodiment identifies the community structure in the slice attribute graph by using a community detection algorithm such as a label propagation algorithm, divides the nodes in the slice attribute graph into different community subgraphs, the nodes in each community subgraph are closely connected, and the connection between the community subgraphs is relatively sparse, so as to calculate the node importance of each node in the slice attribute graph by using a graph attention network (GAT), and calculate the community subgraph importance of each community subgraph by using the graph attention network. The embodiment combines the importance values of the nodes and the subgraphs, and screens out the key semantic regions and the key community subgraphs with high correlation in the point cloud data by setting a proper threshold value. These subgraphs will be used as the basis for subsequent video slice extraction. The identified key subgraphs and key regions are mapped back to the corresponding video frames and regions in the panoramic video stream, so as to form a panoramic video slice corresponding to the point cloud data. For example, the corresponding pixel position in the video frame can be found to realize the projection of the point cloud data from the three-dimensional space to the two-dimensional panoramic video slice plane on the basis of registration, so as to ensure the accurate position of the point cloud data on the panoramic video slice. The projected point cloud data is represented as a graph structure in the embodiment, wherein each projected point cloud data is a node in the graph structure, each node contains its position on the two-dimensional image and the attribute extracted from the point cloud data, and the connection relationship between the nodes is established to form a graph representation of the dense point cloud model containing spatial and semantic information, thereby providing strong support for subsequent analysis and application.

[0123] S3. Determine the position of the inspection target in the graph representation of the dense point cloud model, and construct a decision condition tree for each path decision point according to the power agent pose information and the environment characteristics.

[0124] In the embodiment, the step of constructing a decision condition tree for each path decision point according to the power agent pose information and the environment characteristics includes:

[0125] Obtain the environment characteristic information of the environment where the power agent is located, and determine the path decision point according to the power agent pose information and the environment characteristic information;

[0126] According to the task demand of the power agent and the environment characteristic information, identify the path selection decision condition considering the safety operation constraint of each path decision point;

[0127] According to the path selection decision condition, create a decision condition node of the decision condition tree, and analyze the logical relationship between each path selection decision condition to determine the conversion logical relationship between the decision condition nodes;

[0128] Obtain all behavior states of the power agent, and determine the state conversion condition between different behavior states;

[0129] The decision condition nodes are connected according to the transition logic relationship between the decision condition nodes, and the state transition conditions are mapped with the decision condition nodes, to form a decision condition tree.

[0130] Specifically, the embodiment identifies a path decision point, such as a turning point, an intersection, and a vicinity of an obstacle, according to the current pose information of the power agent and the feature information of the environment in which the power agent is located, and identifies a decision condition affecting path selection, such as path width, power level, and task priority, based on the task demand of the power agent and the environmental feature information. In order to ensure the safe operation of the power agent, the embodiment determines a safe operation constraint condition, such as an obstacle avoidance distance, a maximum speed, and a maximum acceleration, according to the safety requirement of the power agent and the environmental feature. Then, the embodiment uses the path selection decision condition as a decision condition node of the decision condition tree, analyzes the logic relationship between the decision condition nodes, determines the transition logic relationship between them, such as if an obstacle is detected, the robot needs to choose to detour or stop, and if the power level is low, it needs to return to the charging station. These transition logic relationships can be used to determine the connection mode and transition conditions between the decision condition nodes, to obtain all behavior states (such as moving, stopping, obstacle avoidance, and charging) of the power agent, determine the state transition conditions between different behavior states, connect each data node according to the determined transition logic relationship, and map the state transition conditions with the nodes of the decision condition tree, to trigger the corresponding state transition under different conditions, form a decision condition tree, and construct a decision condition tree for each path decision point, which contains a safety constraint condition and behaviors and state transitions of the robot in different states, to ensure that the constructed decision condition tree can effectively guide the behavior of the power agent, so that the power agent can more intelligently cope with complex environments and task demands.

[0131] S4. Traverse each condition branch of the decision condition tree, and use a graph theory algorithm to solve the shortest path from the starting point of the power agent to each inspection target in the graph representation of the dense point cloud model, to generate a set of potential preselected paths.

[0132] The embodiment constructs a decision condition tree by considering environmental features and robot states, so that the decision condition tree not only considers all possible decision conditions, but also can adapt to environmental changes, thereby generating potential preselected paths with strong environmental adaptability. In order to generate potential preselected paths, the embodiment traverses the decision condition tree, uses a graph theory algorithm (such as Dijkstra algorithm or A* algorithm) to explore all potential preselected paths, and solves the shortest path between the starting point of the power agent and each inspection target in the graph representation of the dense point cloud model, thereby generating a set of potential preselected paths in a complex environment.

[0133] S5. Construct a path-dependent network based on the set of potential pre-selected paths, and analyze the conditional dependency relationship between paths in the path-dependent network.

[0134] In the embodiment, the step of constructing a path-dependent network based on the set of potential pre-selected paths, and analyzing the conditional dependency relationship between paths in the path-dependent network comprises:

[0135] Defining all the inspection targets and the path decision points passed by each potential pre-selected path in the set of potential pre-selected paths as network nodes, and defining the potential pre-selected paths as edges, to construct a path-dependent network;

[0136] Analyzing the topological structure of the path-dependent network by a depth-first search algorithm, and dividing the paths in the set of potential pre-selected paths into cyclically dependent paths and non-cyclically dependent paths;

[0137] Determining the path transmission decision conditions of the cyclically dependent paths according to the path transmission chains and the path decision points between each cyclically dependent path, and obtaining the conditional transmission relationship between the cyclically dependent paths according to the path transmission decision conditions;

[0138] Traversing the network nodes on each non-cyclically dependent path, identifying the execution preconditions and the execution postconditions of the non-cyclically dependent paths, and analyzing the association relationship between the execution preconditions and the execution postconditions of different non-cyclically dependent paths, to obtain the conditional association relationship of the non-cyclically dependent paths;

[0139] Forming the conditional dependency relationship between paths in the path-dependent network according to the conditional transmission relationship and the conditional association relationship.

[0140] Specifically, the embodiment traverses all the inspection targets, defines the inspection targets and path decision points passed by each potential pre-selected path in the potential pre-selected path set as network nodes, and defines the potential pre-selected paths as edges to construct a path-dependent network. The path-dependent network is analyzed by using a depth-first search algorithm. Starting from any network node, all unvisited adjacent network nodes of the network node are recursively visited. When a network node is visited, it is marked as visited. If a network node is found to have been visited during the search process, there is a circular dependent path. Otherwise, it is marked as a non-circular dependent path. Thus, the paths in the potential pre-selected path set are divided into circular dependent paths and non-circular dependent paths. For each circular dependent path, the path is traversed from the starting node to the terminating node along the edges to obtain and record all the nodes on the corresponding path, determine the path transmission chain between each circular dependent path, identify the key decision points that affect the selection of the circular dependent path from the path decision points of the path transmission chain, determine the path transmission decision condition of the circular dependent path based on the decision logic between the key decision points, analyze the transmission correlation of the path transmission decision condition in the circular path according to the path transmission chain and the path transmission decision condition, and form a condition transmission relationship.

[0141] For each non-circular dependent path, all network nodes thereon are traversed. For each network node on the path, the preconditions required for execution and the postconditions triggered after execution are determined, and the association relationship between the execution preconditions and the execution postconditions of different non-circular dependent paths is analyzed to form a condition association relationship between different non-circular dependent paths. The condition transmission relationship and the condition association relationship are integrated to form a condition-dependent relationship between paths in the path-dependent network.

[0142] S6. According to the condition-dependent relationship between paths, the potential pre-selected path set is iteratively optimized by using a dynamic path optimization algorithm based on environmental perception to obtain an optimal inspection path.

[0143] In the embodiment, the step of iteratively optimizing the potential pre-selected path set by using a dynamic path optimization algorithm based on environmental perception according to the condition-dependent relationship between paths to obtain an optimal inspection path includes:

[0144] According to the mixed visual perception information, the current environmental state is obtained, and in the iteration process, a potential pre-selected path located at the starting point of the power agent is randomly selected from the potential pre-selected path set as an initial inspection path;

[0145] According to the conditional dependency relationship between paths, a next optimal connection path is selected and connected from the initial inspection path, and the initial inspection path is updated, and the step is repeated to select the next optimal connection path step by step until a target inspection path containing all inspection targets is constructed;

[0146] Based on the conditional dependency relationship between paths and the current environment state, the path fitness value of the target inspection path constructed in the current iteration is evaluated by using a fitness function;

[0147] It is judged whether the path fitness value reaches a fitness value convergence condition, if yes, the current target inspection path is output as an optimal inspection path, otherwise, an initial inspection path is selected from the set of potential preselected paths for iteration; wherein the mathematical expression of the fitness function is:

[0148] Wherein,

[0149] In the formula, f l represents the path fitness value; L l represents the length of the target inspection path l; S l (c n ) represents the degree of satisfaction of the target inspection path l to the conditional dependency relationship c n ; c n represents the nth conditional dependency relationship; N l represents the number of conditional dependency relationships on the target inspection path l; E n represents the influence degree of the current environment state on the nth conditional dependency relationship; a represents the path length weight factor; and b represents the condition weight factor.

[0150] Specifically, the embodiment initializes the inspection path, randomly selects a potential preselected path located at the starting point of the power intelligent agent from the set of potential preselected paths as an initial inspection path, for the last inspection target of the initial inspection path, considering the influence of the conditional dependency relationship between paths and the current environment state, the connection quality between it and the remaining potential preselected paths is evaluated, the path with the optimal connection quality is selected as the next optimal connection path, and the step is repeated to gradually construct a complete inspection path until a target inspection path containing all inspection targets is constructed. In the embodiment, the mathematical expression of the next optimal connection path is:

[0151] Wherein,

[0152] In the formula, H optrepresents the next optimal connection path; CD(P) represents a set of potential pre-selected paths connected with the initial inspection path P and existing conditional dependency relationship; λ represents a weight coefficient; Cost(p', P) represents the time cost of connecting the potential pre-selected path p' to the current target inspection path P; Score(p', E, C) represents the score of the potential pre-selected path p' according to the current environment state E and the inter-path conditional dependency relationship C; δ represents the entropy value of the potential pre-selected path p'; P E (e m |p') represents the probability distribution of each current environment state e m under the potential pre-selected path p'. C (c n |p') represents the probability distribution of each conditional dependency relationship c n under the potential pre-selected path p'.

[0153] Then, the embodiment evaluates the path fitness value of the target inspection path constructed in the current iteration by using the fitness function, and outputs the current target inspection path as the optimal inspection path if the path fitness value reaches the preset fitness value convergence condition; if the path fitness value does not reach the fitness value convergence condition, the initial inspection path is selected from the set of potential pre-selected paths for iteration. In the embodiment, the fitness value convergence condition can be: if the fitness values f(t) and f(t-1) for the current iteration t and the previous iteration (t-1) satisfy (|f(t)-f(t-1)|<ε) and continuously for n times, the iteration is terminated, where ε is a preset fitness threshold. It should be noted that the fitness threshold used in the fitness value convergence condition can be set by the person skilled in the art according to the specific implementation. The embodiment fully utilizes the conditional dependency relationship and environment perception to guide the path optimization, which not only improves the efficiency and adaptability of the path planning, but also enables real-time response to environmental changes, enhances the decision-making ability of the robot when facing environmental changes, and thus realizes the automatic planning of the global path of the power intelligent agent, ensuring that the robot can complete the task efficiently and safely in a complex power environment.

[0154] The embodiment of the present application provides a power intelligent agent data acquisition processing method, the method analyzes mixed visual perception information based on an event-driven visual SLAM algorithm to obtain a graph representation of a dense point cloud model; a decision condition tree is constructed for each path decision point according to power intelligent agent pose information and environmental characteristics; each condition branch of the decision condition tree is traversed, and a graph theory algorithm is used to solve the shortest path from the starting point of the power intelligent agent to each inspection target in the graph representation of the dense point cloud model, to generate a potential preselected path set; a path-dependent network is constructed based on the potential preselected path set, and the conditional dependence relationship between paths in the path-dependent network is analyzed; according to the conditional dependence relationship between paths, a dynamic path optimization algorithm based on environmental perception is used to iteratively optimize the potential preselected path set, to obtain an optimal inspection path. Compared with a traditional path planning method, the method provided in the embodiment combines a decision condition tree and a dynamic path optimization algorithm based on environmental perception, can respond to environmental changes in real time and optimize the inspection path of the power intelligent agent, not only improves the efficiency and adaptability of path planning, but also enhances the decision-making ability of the robot when facing environmental changes, and ensures that the power intelligent agent can efficiently and accurately complete an inspection task.

[0155] It should be noted that the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0156] In one embodiment, as shown in FIG. 2, the embodiment of the present application provides a power intelligent agent data acquisition processing system, the system comprises:

[0157] The three-dimensional map construction module 101 is configured to capture mixed visual perception information of the environment around the power intelligent agent, and construct a relatively dense three-dimensional map of the power scene and power intelligent agent pose information through an event-driven visual SLAM algorithm based on the mixed visual perception information;

[0158] The point cloud model construction module 102 is configured to construct a graph representation of a dense point cloud model according to spatial information and point cloud data in the relatively dense three-dimensional map of the power scene;

[0159] The condition tree establishment module 103 is configured to determine the positions of the inspection targets in the graph representation of the dense point cloud model, and construct a decision condition tree for each path decision point according to the power intelligent agent pose information and environmental characteristics;

[0160] The preselected path generation module 104 is configured to traverse each condition branch of the decision condition tree, use a graph theory algorithm to solve the shortest path from the starting point of the power intelligent agent to each inspection target in the graph representation of the dense point cloud model, and generate a potential preselected path set;

[0161] The optimal path planning module 105 is configured to construct a path-dependent network based on the set of potential preselected paths, analyze conditional dependency relationships between paths in the path-dependent network, and iteratively optimize the set of potential preselected paths by using a dynamic path optimization algorithm based on environmental perception according to the conditional dependency relationships between paths to obtain an optimal inspection path.

[0162] In this embodiment, the relative dense three-dimensional map of the power scene and the power agent pose information are constructed by using the event-driven visual SLAM algorithm based on the mixed visual perception information, and specifically include:

[0163] The mixed visual perception information is subjected to feature extraction to obtain key feature points of the power scene, and a neighborhood region around each key feature point of the power scene is quantitatively described to obtain a feature point descriptor.

[0164] The key feature points of the power scene are matched between the mixed visual perception information of continuous frames or different perspectives according to the feature point descriptor to obtain matching feature points associated with power equipment.

[0165] The matching feature points are taken as nodes, and the spatiotemporal relationship between the matching feature points is taken as edges to construct a power scene event graph, and the matching feature points in the power scene event graph are taken as event-driven tracking targets of the visual SLAM algorithm.

[0166] When a change in the event-driven tracking target in the power scene event graph is detected, the topological structure and geometric information of an event triggering region in the power scene event graph are converted into state estimation graph constraints according to an event triggering mechanism, and the state estimation graph constraints are specifically:

[0167] In the formula, F represents the state estimation graph constraint. represents the relative pose between node A and node B calculated according to the current estimated pose x. AB represents the relative pose between node A and node B in the power scene event graph. represents the estimated position of node i. i represents the actual position of node i. G represents an edge set of the power scene event graph.

[0168] The state estimation graph constraints are fused into state estimation of the visual SLAM algorithm, and the relative dense three-dimensional map of the power scene and the power agent pose information are obtained by using the visual SLAM algorithm according to the spatial positioning information of the matching feature points.

[0169] In this embodiment, the point cloud model construction module is specifically configured to:

[0170] The spatial information of the power scene relatively dense three-dimensional map is spatially aligned with the panoramic video stream by using a geographic information system technology, and the spatially aligned panoramic video stream is subjected to frame-by-frame semantic segmentation to obtain a semantic segmentation result of a video frame with a semantic label.

[0171] The video frame semantic segmentation result is mapped with the spatial information of the power scene relatively dense three-dimensional map to generate a semantic mapping fused with spatial information, and a slice attribute graph is constructed based on the semantic mapping.

[0172] The nodes in the slice attribute graph are divided into different community subgraphs by using a label propagation algorithm, and the node importance of each node in the slice attribute graph and the community subgraph importance of each community subgraph are calculated by using a graph attention network.

[0173] Point cloud data of a power operation environment is obtained according to the power scene relatively dense three-dimensional map, and key semantic regions and key community subgraphs with high correlation degrees with the point cloud data are identified from the slice attribute graph according to the node importance and the community subgraph importance.

[0174] The key community subgraphs and the key semantic regions are mapped back to corresponding video frames and regions in the panoramic video stream to form a panoramic video slice corresponding to the point cloud data.

[0175] The point cloud data is spatially registered by using power agent pose information to obtain registered point cloud data, and the registered point cloud data is projected onto the corresponding position of the panoramic video slice to form a graph representation of a dense point cloud model.

[0176] The slice attribute graph is constructed based on the semantic mapping, specifically including:

[0177] In the semantic mapping, a semantic region is identified according to the semantic label and the spatial information in the power scene relatively dense three-dimensional map.

[0178] The attributes of each semantic region are defined as nodes of the slice attribute graph, and the spatial and semantic relationships between the semantic regions are defined as edges of the slice attribute graph to construct the slice attribute graph; wherein the attributes of the semantic region include the region type and the region spatial geometry attribute.

[0179] In this embodiment, the decision condition tree is constructed for each path decision point according to the power agent pose information and the environment features, specifically including:

[0180] The environment feature information of the environment in which the power agent is located is obtained, and the path decision point is determined according to the power agent pose information and the environment feature information.

[0181] According to the task requirement and the environment characteristic information of the power agent, a path selection decision condition considering a safe operation constraint at each path decision point is identified;

[0182] A decision condition node of a decision condition tree is created according to the path selection decision condition, and a logical relationship between each path selection decision condition is analyzed to determine a conversion logical relationship between the decision condition nodes;

[0183] All behavior states of the power agent are acquired, and a state conversion condition between different behavior states is determined;

[0184] Each decision condition node is connected according to the conversion logical relationship between the decision condition nodes, and the state conversion condition is mapped to the decision condition node to form the decision condition tree.

[0185] In this embodiment, the path-dependent network is constructed based on the set of potential preselected paths, and a condition-dependent relationship between paths in the path-dependent network is analyzed, specifically including:

[0186] All inspection targets and path decision points passed through by each potential preselected path in the set of potential preselected paths are defined as network nodes, and the potential preselected paths are defined as edges to construct the path-dependent network;

[0187] The topology structure of the path-dependent network is analyzed by using a depth-first search algorithm, and the paths in the set of potential preselected paths are divided into circular dependent paths and non-circular dependent paths;

[0188] A path transmission decision condition of the circular dependent path is determined according to a path transmission chain and a path decision point between each circular dependent path, and a condition transmission relationship between the circular dependent paths is acquired according to the path transmission decision condition;

[0189] Network nodes on each non-circular dependent path are traversed, an execution precondition and an execution postcondition of the non-circular dependent path are identified, and an association relationship between the execution preconditions and the execution postconditions of different non-circular dependent paths is analyzed to acquire a condition association relationship of the non-circular dependent path;

[0190] According to the condition transmission relationship and the condition association relationship, a condition-dependent relationship between paths in the path-dependent network is formed.

[0191] In this embodiment, the set of potential preselected paths is iteratively optimized by using a dynamic path optimization algorithm based on environment perception according to the condition-dependent relationship between paths to acquire an optimal inspection path, specifically including:

[0192] According to the mixed visual perception information, the current environment state is acquired, and in an iteration process, a potential pre-selected path located at the starting point of the power agent is randomly selected from a set of potential pre-selected paths as an initial inspection path;

[0193] According to the conditional dependency relationship between the paths, a next optimal connected path for inspection starting from the initial inspection path is selected and connected from the remaining potential pre-selected paths, and the initial inspection path is updated, and the step is repeated to gradually select the next optimal connected path until a target inspection path containing all the inspection targets is constructed;

[0194] Based on the conditional dependency relationship between the paths and the current environment state, a fitness function is used to evaluate a path fitness value of the target inspection path constructed in the current iteration;

[0195] It is judged whether the path fitness value reaches a fitness value convergence condition, if yes, the current target inspection path is output as an optimal inspection path, otherwise, the initial inspection path is selected again from the set of potential pre-selected paths for iteration; wherein, a mathematical expression of the fitness function is:

[0196] wherein,

[0197] In the formula, f l represents the path fitness value; L l represents the length of the target inspection path l; S l (c n ) represents the degree of satisfaction of the target inspection path l to the conditional dependency relationship c n ; c n represents the nth conditional dependency relationship; N l represents the number of conditional dependency relationships on the target inspection path l; E n represents the influence degree of the current environment state to the nth conditional dependency relationship; a represents a path length weight factor; and b represents a condition weight factor.

[0198] The specific limitations of the power agent data acquisition and processing system can be referred to the limitations of the power agent data acquisition and processing method described above, which will not be repeated here. Those skilled in the art can realize that the various modules and steps described in combination with the embodiments disclosed in the present application can be realized in hardware, software or combination of both. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0199] The embodiment of the present application provides a power intelligent agent data acquisition and processing system, the system analyzes mixed visual perception information through a three-dimensional map construction module to obtain a graph representation of a dense point cloud model; a point cloud model construction module and a conditional tree establishment module are used to construct a decision condition tree for each path decision point; each conditional branch of the decision condition tree is traversed through a preselected path generation module, a graph theory algorithm is used to solve the shortest path from a power intelligent agent starting point to each inspection target in the graph representation of the dense point cloud model, and a potential preselected path set is generated; an optimal path planning module is used to construct a path dependent network based on the potential preselected path set, and analyze the conditional dependency relationship between paths in the path dependent network; according to the conditional dependency relationship between paths, a dynamic path optimization algorithm based on environment perception is used to iteratively optimize the potential preselected path set, and an optimal inspection path is obtained. Compared with the traditional path planning technology, the system provided in the embodiment combines the decision condition tree and the dynamic path optimization algorithm based on environment perception, can respond to environmental changes in real time and optimize the inspection path of the power intelligent agent, not only improves the efficiency and adaptability of path planning, but also enhances the decision-making ability of the robot when facing environmental changes, and ensures that the power intelligent agent can efficiently and accurately complete the inspection task.

[0200] Fig. 3 is a computer device provided by the embodiment of the present application, which includes a memory, a processor and a transceiver connected through a bus; the memory is used to store a set of computer program instructions and data, and can transmit the stored data to the processor; the processor can execute the program instructions stored in the memory to execute the steps of the above method.

[0201] The memory can include a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories; the processor can be a central processing unit, a microprocessor, an application specific integrated circuit, a programmable logic device or a combination thereof. By way of example but not limitation, the programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a general array logic or any combination thereof.

[0202] In addition, the memory can be a physically independent unit, or can be integrated with the processor.

[0203] Those skilled in the art can understand that the structure shown in Fig. 3 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied; the specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0204] In one embodiment, the computer readable storage medium having stored thereon computer program is provided, and the computer program is executed by a processor to implement the steps of the above method.

[0205] The power intelligent agent data acquisition and processing method, system, device and medium provided by the embodiment of the application combine the decision condition tree with the dynamic path optimization algorithm based on environment perception, can respond to environmental changes in real time and optimize the inspection path of the power intelligent agent, not only improves the efficiency and adaptability of path planning, but also enhances the decision-making ability of the robot when facing environmental changes, and ensures that the power intelligent agent can efficiently and accurately complete the inspection task.

[0206] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, SSD) and the like.

[0207] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of the method can be included.

[0208] The above-mentioned embodiments only express several preferred embodiments of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent. It should be noted that for the improvements and replacements of the present application, these improvements and replacements should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the protection scope of the claims.

Claims

1. A power intelligent entity data acquisition processing method, characterized in that, The method comprises the following steps: capturing mixed visual perception information of the environment around the power agent, and constructing a relatively dense three-dimensional map of the power scene and the pose information of the power agent through an event-driven visual SLAM algorithm based on the mixed visual perception information; constructing a graph representation of the dense point cloud model according to the spatial information and the point cloud data in the relatively dense three-dimensional map of the power scene; determining the positions of the inspection targets in the graph representation of the dense point cloud model, and constructing a decision condition tree for each path decision point according to the pose information of the power agent and the environmental characteristics; traversing each condition branch of the decision condition tree, and solving the shortest paths from the starting point of the power agent to each inspection target in the graph representation of the dense point cloud model by using a graph theory algorithm to generate a set of potential preselected paths; constructing a path-dependent network based on the set of potential preselected paths, and analyzing the conditional dependence relationships between the paths in the path-dependent network; iteratively optimizing the set of potential preselected paths by using a dynamic path optimization algorithm based on environmental perception according to the conditional dependence relationships between the paths, and obtaining an optimal inspection path.

2. The power agent data collection and processing method of claim 1, wherein, The step of constructing a relatively dense three-dimensional map of the power scene and the pose information of the power agent through an event-driven visual SLAM algorithm based on the mixed visual perception information comprises the following steps: extracting features from the mixed visual perception information to obtain key feature points of the power scene, and quantitatively describing the neighborhood regions around each key feature point to obtain feature point descriptors; matching the key feature points of the power scene between continuous frames or mixed visual perception information of different perspectives according to the feature point descriptors, and obtaining matching feature points associated with power equipment; constructing a power scene event graph by taking the matching feature points as nodes and the spatiotemporal relationships between the matching feature points as edges, and taking the matching feature points in the power scene event graph as event-driven tracking targets of the visual SLAM algorithm; when a change in the event-driven tracking targets in the power scene event graph is detected, converting the topological structure and geometric information of the event triggering region in the power scene event graph into state estimation graph constraints according to an event triggering mechanism; fusing the state estimation graph constraints into the state estimation of the visual SLAM algorithm, and obtaining the relatively dense three-dimensional map of the power scene and the pose information of the power agent by using the visual SLAM algorithm according to the spatial positioning information of the matching feature points.

3. The power agent data collection and processing method of claim 2, wherein, The state estimation graph constraints are specifically: In the formula, F represents a state estimation map constraint. represents the relative pose between node A and node B calculated from the current estimated pose x; T AB represents the relative pose between node A and node B in the power scenario event graph; represents the estimated position of node i; D i represents the actual position of node i; G represents the edge set of the power scenario event graph.

4. The power agent data collection and processing method of claim 1, wherein, The step of constructing a graph representation of the dense point cloud model according to the spatial information and the point cloud data in the relatively dense three-dimensional map of the power scene comprises the following steps: spatially aligning the spatial information of the relatively dense three-dimensional map of the power scene with a panoramic video stream by using a geographic information system technology, and performing frame-by-frame semantic segmentation on the spatially aligned panoramic video stream to obtain a video frame semantic segmentation result with semantic labels; mapping the video frame semantic segmentation result and the spatial information of the relatively dense three-dimensional map of the power scene to generate a semantic mapping fused with spatial information, and constructing a slice attribute graph based on the semantic mapping; The nodes in the slice attribute graph are divided into different community subgraphs by using a label propagation algorithm, and the node importance of each node in the slice attribute graph and the community subgraph importance of each community subgraph are calculated by using a graph attention network; According to the point cloud data of the power operation environment obtained from the relatively dense three-dimensional map of the power scene, and according to the node importance and the community subgraph importance, the key semantic regions and the key community subgraphs with high correlation degrees with the point cloud data are identified from the slice attribute graph; The key community subgraphs and the key semantic regions are mapped back to the corresponding video frames and regions in the panoramic video stream to form panoramic video slices corresponding to the point cloud data; The spatial registration of the point cloud data is performed by using the pose information of the power agent, the registered point cloud data is obtained, and the registered point cloud data is projected onto the corresponding positions of the panoramic video slices to form a graph representation of the dense point cloud model.

5. The power agent data collection and processing method of claim 4, wherein, The step of constructing the slice attribute graph based on the semantic mapping comprises: In the semantic mapping, the semantic regions are identified according to the semantic labels and the spatial information in the relatively dense three-dimensional map of the power scene; The attributes of each semantic region are defined as the nodes of the slice attribute graph, and the spatial and semantic relationships between the semantic regions are defined as the edges of the slice attribute graph, thereby constructing the slice attribute graph; wherein the attributes of the semantic regions include the region type and the region spatial geometry attribute.

6. The power agent data collection and processing method of claim 1, wherein, The step of constructing a decision condition tree for each path decision point according to the pose information of the power agent and the environmental characteristics comprises: The environmental characteristic information of the environment in which the power agent is located is obtained, and the path decision points are determined according to the pose information of the power agent and the environmental characteristic information; According to the task requirements of the power agent and the environmental characteristic information, the path selection decision conditions considering the safety operation constraints of each path decision point are identified; The decision condition nodes of the decision condition tree are created according to the path selection decision conditions, and the logical relationships between the path selection decision conditions are analyzed to determine the conversion logical relationships between the decision condition nodes; All behavior states of the power agent are obtained, and the state conversion conditions between different behavior states are determined; The decision condition nodes are connected according to the conversion logical relationships between the decision condition nodes, and the state conversion conditions are mapped to the decision condition nodes to form the decision condition tree.

7. The power agent data collection and processing method of claim 1, wherein, The step of constructing a path-dependent network based on the set of potential preselected paths and analyzing the conditional dependence relationships between paths in the path-dependent network comprises: All the inspection targets and the path decision points passed through by each potential preselected path in the set of potential preselected paths are defined as network nodes, and the potential preselected paths are defined as edges, thereby constructing the path-dependent network; The set of potential preselected paths is divided into cyclically dependent paths and non-cyclically dependent paths by analyzing the topological structure of the path-dependent network through a depth-first search algorithm; The path transmission decision conditions of the cyclically dependent paths are determined according to the path transmission chains and the path decision points between each cyclically dependent path, and the conditional transmission relationships between the cyclically dependent paths are obtained according to the path transmission decision conditions. The network nodes on each non-cyclic dependency path are traversed to identify execution preconditions and execution postconditions of the non-cyclic dependency paths, and a correlation between the execution preconditions and the execution postconditions of different non-cyclic dependency paths is analyzed to obtain a conditional correlation of the non-cyclic dependency paths; According to the conditional transmission relationship and the conditional correlation, a conditional dependency relationship between paths in the path-dependent network is formed.

8. The power agent data collection and processing method of claim 1, wherein, The step of iteratively optimizing the set of potential preselected paths according to the conditional dependency relationship between paths includes: According to the mixed visual perception information, the current environment state is obtained, and in the iteration process, a potential preselected path located at the starting point of the power agent is randomly selected from the set of potential preselected paths as an initial inspection path; According to the conditional dependency relationship between paths, the next optimal connected path for inspection starting from the initial inspection path is selected and connected from the remaining potential preselected paths, and the initial inspection path is updated, and the step is repeated to gradually select the next optimal connected path until the target inspection path containing all inspection targets is constructed; Based on the conditional dependency relationship between paths and the current environment state, the path fitness value of the target inspection path constructed in the current iteration is evaluated by using a fitness function; If the path fitness value meets the fitness value convergence condition, the current target inspection path is output as the optimal inspection path, otherwise, the initial inspection path is selected from the set of potential preselected paths for iteration.

9. The power agent data collection and processing method of claim 8, wherein, The mathematical expression of the fitness function is: wherein In the formula, f l represents the path fitness value; L l represents the length of the target inspection path l; S l (c n ) represents the degree of satisfaction of the target inspection path l to the condition-dependent relationship c n ; c n represents the nth condition-dependent relationship; N l represents the number of condition-dependent relationships on the target inspection path l; E n represents the influence degree of the current environment state on the nth condition-dependent relationship; α represents the path length weight factor; and β represents the condition weight factor.

10. An electric power agent data acquisition and processing system, characterized by, The system includes: A three-dimensional map construction module is configured to capture mixed visual perception information of an environment around the power agent, and construct a relatively dense three-dimensional map of the power scene and pose information of the power agent based on the mixed visual perception information by using an event-driven visual SLAM algorithm; A point cloud model construction module is configured to construct a graph representation of a dense point cloud model based on spatial information and point cloud data in the relatively dense three-dimensional map of the power scene; A conditional tree establishment module is configured to determine the positions of the inspection targets in the graph representation of the dense point cloud model, and construct a decision condition tree for each path decision point based on the pose information of the power agent and the environmental features; A preselected path generation module is configured to traverse each conditional branch of the decision condition tree, and solve the shortest paths from the starting point of the power agent to each inspection target in the graph representation of the dense point cloud model by using a graph theory algorithm to generate a set of potential preselected paths; An optimal path planning module is configured to construct a path-dependent network based on the set of potential preselected paths, and analyze the conditional dependency relationship between paths in the path-dependent network; And iteratively optimize the set of potential preselected paths according to the conditional dependency relationship between paths by using a dynamic path optimization algorithm based on environmental perception to obtain an optimal inspection path.

11. An electric power agent data acquisition and processing system as in claim 10, wherein, The relatively dense three-dimensional map of the power scene and the pose information of the power agent are constructed based on the mixed visual perception information by using an event-driven visual SLAM algorithm, and specifically include: feature extraction is performed on the mixed visual perception information to obtain power scene key feature points, and a neighborhood region around each power scene key feature point is quantitatively described to obtain a feature point descriptor; According to the feature point descriptor, the power scene key feature points are matched between the continuous frames or the mixed visual perception information of different perspectives, and the matching feature points associated with the power equipment are obtained; The matching feature points are taken as nodes, and the space-time relationship between the matching feature points is taken as edges, to construct a power scene event graph, and the matching feature points in the power scene event graph are taken as event-driven tracking targets of a visual SLAM algorithm; When it is detected that the event-driven tracking target in the power scene event graph changes, the topological structure and geometric information of an event triggering region in the power scene event graph are converted into state estimation graph constraints according to an event triggering mechanism; The state estimation graph constraints are fused into state estimation of the visual SLAM algorithm, and the power scene relative dense three-dimensional map and the power agent pose information are obtained by using the visual SLAM algorithm according to the spatial positioning information of the matching feature points.

12. An electrically intelligent agent data acquisition and processing system as in claim 11, wherein, The state estimation graph constraints are specifically: In the formula, F represents a state estimation map constraint; represents the relative pose between node A and node B calculated from the current estimated pose x; T AB represents the relative pose between node A and node B in the power scenario event graph; represents the estimated position of node i; D i represents the actual position of node i; G represents the edge set of the power scenario event graph.

13. The electrically intelligent agent data collection and processing system of claim 10, wherein, The point cloud model construction module is specifically configured to: spatial information of the power scene relative dense three-dimensional map is spatially aligned with the panoramic video stream by using a geographic information system technology, and frame-by-frame semantic segmentation of the spatially aligned panoramic video stream is performed to obtain a video frame semantic segmentation result with semantic labels; The video frame semantic segmentation result is mapped with the spatial information of the power scene relative dense three-dimensional map to generate a semantic mapping fused with spatial information, and a slice attribute graph is constructed based on the semantic mapping; The nodes in the slice attribute graph are divided into different community subgraphs by using a label propagation algorithm, and the node importance of each node and the community subgraph importance of each community subgraph in the slice attribute graph are calculated by using a graph attention network; According to the power scene relative dense three-dimensional map, point cloud data of the power work environment is obtained, and according to the node importance and the community subgraph importance, key semantic regions and key community subgraphs with high correlation degree to the point cloud data are identified from the slice attribute graph; The key community subgraphs and the key semantic regions are mapped back to corresponding video frames and regions in the panoramic video stream to form a panoramic video slice corresponding to the point cloud data; The point cloud data is spatially registered by using the power agent pose information to obtain registered point cloud data, and the registered point cloud data is projected onto the corresponding position of the panoramic video slice to form a graph representation of the dense point cloud model.

14. The electric power agent data collection and processing system of claim 13 wherein, The slice attribute graph is constructed based on the semantic mapping, specifically including: In the semantic mapping, according to the semantic labels and the spatial information in the power scene relative dense three-dimensional map, semantic regions are identified; The attributes of each semantic region are defined as nodes of the slice attribute graph, and the spatial relationship and semantic relationship between the semantic regions are defined as edges of the slice attribute graph to construct the slice attribute graph; wherein the attributes of the semantic regions include region type and region spatial geometry attribute.

15. The electric power agent data collection and processing system of claim 10 wherein, The decision condition tree is constructed for each path decision point according to the power agent pose information and the environment features, specifically including: Acquire environment characteristic information of an environment where a power agent is located, and determine a path decision point according to the position information of the power agent and the environment characteristic information; According to the task demand of the power agent and the environment characteristic information, identify the path selection decision condition of each path decision point considering the safety operation constraint; According to the path selection decision condition, create a decision condition node of a decision condition tree, and analyze the logical relationship between each path selection decision condition to determine the conversion logical relationship between the decision condition nodes; Acquire all behavior states of the power agent, and determine the state conversion condition between different behavior states; According to the conversion logical relationship between the decision condition nodes, connect each decision condition node, and map the state conversion condition to the decision condition node to form a decision condition tree.

16. The electric power agent data collection and processing system of claim 10 wherein, The path-dependent network is constructed based on the set of potential preselected paths, and the conditional dependency relationship between paths in the path-dependent network is analyzed, specifically including: All inspection targets and path decision points passed through by each potential preselected path in the set of potential preselected paths are defined as network nodes, and the potential preselected paths are defined as edges to construct a path-dependent network; The topology structure of the path-dependent network is analyzed by using a depth-first search algorithm, and the paths in the set of potential preselected paths are divided into circular dependent paths and non-circular dependent paths; According to the path transmission chain and the path decision point between each circular dependent path, the path transmission decision condition of the circular dependent path is determined, and the conditional transmission relationship between the circular dependent paths is acquired according to the path transmission decision condition; Each network node on each non-circular dependent path is traversed to identify the execution precondition and the execution postcondition of the non-circular dependent path, and the association relationship between the execution preconditions and the execution postconditions of different non-circular dependent paths is analyzed to acquire the conditional association relationship of the non-circular dependent path; According to the conditional transmission relationship and the conditional association relationship, the conditional dependency relationship between paths in the path-dependent network is formed.

17. The electric power agent data collection and processing system of claim 10 wherein, According to the conditional dependency relationship between paths, the set of potential preselected paths is iteratively optimized by using a dynamic path optimization algorithm based on environment perception to acquire an optimal inspection path, specifically including: The current environment state is acquired according to mixed visual perception information, and in the iteration process, a potential preselected path located at the starting point of the power agent is randomly selected from the set of potential preselected paths as an initial inspection path; According to the conditional dependency relationship between paths, the next optimal connection path for inspection starting from the initial inspection path is selected and connected from the remaining potential preselected paths, and the initial inspection path is updated, and the step is repeated to gradually select the next optimal connection path until the target inspection path containing all inspection targets is constructed; Based on the conditional dependency relationship between paths and the current environment state, the path fitness value of the target inspection path constructed in the current iteration is evaluated by using a fitness function; It is judged whether the path fitness value reaches the fitness value convergence condition, if yes, the current target inspection path is output as the optimal inspection path, otherwise, the initial inspection path is selected from the set of potential preselected paths for iteration.

18. An electrically intelligent agent data acquisition and processing system as in claim 17, wherein, The mathematical expression of the fitness function is: wherein, wherein f l represents the path fitness value; L l represents the length of the target inspection path l; S l (c n ) represents the degree of satisfaction of the target inspection path l to the condition dependency c n ; c n represents the nth condition dependency; N l represents the number of condition dependencies on the target inspection path l; E n represents the influence degree of the current environment state to the nth condition dependency; a represents the path length weight factor; and b represents the condition weight factor.

19. A computer device, comprising: The computer device comprises a processor and a memory, the processor is connected with the memory, the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, so that the computer device executes the method in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, and the computer program is executed to implement the method in any one of claims 1 to 9.

Images