Indoor mobile robot path planning method and system for substation equipment inspection

Abstract

The invention relates to the field of electric power automation, in particular to an indoor mobile robot path planning method and system for transformer substation equipment inspection, and the method comprises the steps: building a grid map according to a working scene which actually needs to be inspected, determining the positions of a starting node and a target node, sorting nodes in an open list, and storing the sorted nodes in the open list; setting the node with the minimum node cost as the current node, adding the current node into a closing list, and then expanding the current node by using a child node expansion strategy; when the node existing in the open list is searched, calculating the node cost again; outputting a path according to the closing list and carrying out repeated broken line optimization; the trained improved DQN algorithm is used for carrying out obstacle avoidance processing on unknown obstacles in the global path until a collision-free path from the starting point to the terminal point is planned, and the problems that a traditional inspection robot path planning algorithm is low in efficiency, unsmooth in path and poor in obstacle avoidance effect during path planning are solved.

Description

Technical Field

[0001] The present invention relates to the field of electric power automation, and in particular to a path planning method and system for an indoor mobile robot used for inspecting substation equipment. Background Art

[0002] Currently, most substations use manual inspections during equipment inspections. This is labor-intensive, risky, and cumbersome, and is subject to limitations imposed by factors such as the inspector's professional expertise, workload, work experience, and sense of responsibility. This can lead to missed inspections and false positives, which can lead to substation equipment failures and become a major safety hazard for power system operations. Direct economic losses from substation equipment failures due to missed and false positives reach hundreds of millions of yuan annually. Therefore, traditional manual inspections are no longer sufficient to meet the operational needs of modern substations and power systems. Introducing robots into the field of power equipment inspection, utilizing intelligent algorithms and advanced equipment to replace manual inspections, and developing low- or unmanned substation inspection technologies are of critical theoretical and practical significance for the safe operation and effective inspection of substation equipment.

[0003] The traditional A-star algorithm tends to generate jagged paths in narrow cabinet aisles, leading to frequent starts and stops for the robot's end effector, high energy consumption, and collisions. While the traditional fast traversal random tree (RRT) algorithm can generate smooth paths, it converges slowly in high-dimensional spaces and lacks real-time performance. Therefore, path planning for indoor mobile robots used for substation equipment inspection faces multiple challenges. It requires a comprehensive consideration of the algorithm's convergence speed, local optimality, and a balance between performance optimization objectives such as minimizing the robot's path, avoiding obstacles, and minimizing energy consumption, in order to converge on the optimal overall path.

[0004] For example, the invention patent with publication number CN119984326A discloses a path planning method and system based on Beidou satellite positioning, which uses an electronic map matching method to match electronic map data with location information to obtain electronic map data corresponding to the current location; a traffic flow prediction model is constructed based on location information and a machine learning algorithm, and the location information is input into the traffic flow prediction model to obtain a traffic condition prediction result; based on the traffic condition prediction result, an A-star path search algorithm is used to plan the path between the current location and the destination to obtain a preliminary path; the preliminary path is personalized according to user preference settings to obtain a recommended path; a real-time monitoring and feedback mechanism is used to monitor the final recommended path, evaluate the position changes and traffic conditions in the path, and obtain the final path planning strategy.

[0005] For example, the invention patent with publication number CN111707266A discloses a path planning method for a substation intelligent operation and maintenance robot. On the basis of the standard A-star algorithm, it adds a heuristic function weight obtained by fuzzy calculation. The weight integrates the two factors that affect the heuristic cost, and uses the fuzzy algorithm to fuzzify and defuzzify the input quantity to obtain the weight value of the heuristic function. It avoids the disadvantages of long and tortuous path caused by the vector cross multiplication factor that forcibly pulls the path to the straight line from the starting point to the target point, making the calculated heuristic cost closer to the actual cost. The evaluation function of the improved A-star algorithm is more intelligent, so that the improved A-star algorithm achieves a good balance between time and pathfinding quality, but it still cannot solve problems such as local optimality. Summary of the Invention

[0006] Purpose of the invention: In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a path planning method for an indoor mobile robot for substation equipment inspection. This method solves the problems of poor path planning ability in global planning and the problem that local planning easily falls into local optimum and has slow convergence speed. The present invention also provides a path planning system for an indoor mobile robot for substation equipment inspection.

[0007] Technical solution: According to a first aspect of the present invention, a path planning method for an indoor mobile robot for substation equipment inspection is provided, the method comprising:

[0008] A grid map is created based on the actual work scenario that needs to be inspected. The coordinates of the center point of each grid are defined as the location of the grid. 0 represents an unoccupied grid, meaning there is no obstacle at that location, and 1 represents an occupied grid, meaning there is an obstacle at that location.

[0009] Determine the positions of the start node and the target node, add the start node to the open list, sort the nodes in the open list, set the node with the lowest cost as the current node and add it to the closed list, and then expand the current node using the child node expansion strategy; when the node already in the open list is found, recalculate the node cost and update the open list; repeat the search process until the current node is the target point, then output the path according to the closed list and perform repeated polyline optimization; finally, use the moving average method to smooth the path after repeated polyline optimization to generate the final global path;

[0010] The trained improved DQN algorithm avoids unknown obstacles in the global path until a collision-free path is planned from the starting point to the end point. DQN (Deep Q-Network) is an algorithm that combines deep learning and reinforcement learning. It approximates the Q function through a deep neural network, uses Q-learning for training, and introduces techniques such as experience replay and target networks to address training stability issues.

[0011] Further, including:

[0012] The method of using the trained improved DQN algorithm to avoid unknown obstacles in the global path includes the following steps:

[0013] Step 1: Initialize the maximum number of training rounds of the algorithm, the size of the experience pools D1 and D2, network parameters, initial sample priority and sampling probability, number of sample samples, network parameter update interval, and set the starting point and target point positions according to the global path;

[0014] Step 2: Initialize the map environment and get the current state s t ;

[0015] Step 3: Use the improved action selection function to select an action with probability ε, otherwise select the action with the largest Q value;

[0016] Step 4 The robot performs action a t , according to the optimized reward function, the reward value r for executing the current action is obtained t and the next state s t +1;

[0017] Step 5: According to the reward value r t Generate samples, if r t >0, then (s t ,a t ,r t ,s t +1) is stored in the positive reward sample experience pool D1; if r t ≤0, then (s t ,a t ,r t ,s t +1) stored in the negative reward and zero reward experience pool D2;

[0018] Step 6 uses an adaptive exploration strategy to extract samples, that is, extract samples from the experience pool D1 with a probability of ρ, and extract samples from the experience pool D2 with a probability of 1-ρ; sampling is performed according to the probability P of the sample being sampled;

[0019] Step 7: If the next state of the sampled sample is the target point state, the target Q value is the reward value of the target point state. Otherwise, the target Q value is the sum of the reward value of the target point state and the discounted maximum Q value of the next state.

[0020] Step 8: Repeat steps 6 to 7 until the number of samples is reached and the final target Q value is obtained;

[0021] Step 9 calculates the loss function and updates the network parameter w;

[0022] Step 10: When the number of network parameter update steps is reached, the target network parameters are obtained;

[0023] Step 11 determines whether the robot has reached the target point. If not, repeat steps 2 to 10. If it has reached the target point, further determine whether the network training has reached the maximum number of rounds. If not, repeat steps 1 to 10. If the maximum number of training rounds has been reached, the training ends.

[0024] Further, including:

[0025] The improved action selection function is expressed as:

[0026]

[0027] Among them, d m is the distance threshold, D m is the maximum distance in the grid map, and K is a coefficient. Therefore, when the distance to the target point is greater than the distance threshold, the greater the distance, the smaller the gravity; when it is less than the distance threshold, the gravity is a constant value.

[0028] Further, including:

[0029] The optimization reward function is expressed as:

[0030]

[0031] Among them, ξ is the discount factor of the reward value, and r is the immediate feedback of the environment to the agent's behavior.

[0032] Further, including:

[0033] The adaptive exploration strategy includes setting the exploration rate to be updated according to a set time step decay, expressed as:

[0034] ε t+1 =max(ε min ,ε t *ε decay );

[0035] Among them, ε t+1is the exploration rate at the next time step, ε min is the minimum value of the exploration rate, ε t is the exploration value of the current time step, ε decay It is the decay coefficient of the set exploration rate. The exploration rate will be updated at each set time step multiplied by the decay coefficient until the exploration rate is less than or equal to the minimum exploration rate.

[0036] Further, including:

[0037] The calculation formula of the node cost is expressed as:

[0038] F(n)=G(n)+H(n);

[0039] Where F(n) is the total cost from the starting point to the target node; G(n) represents the cost of moving from the starting point to the current node; H(n) represents the cost of moving from the current node to the target point;

[0040] The cost of moving from the current node to the target point corresponds to the heuristic function, which is expressed as:

[0041] H(n i )=max{H angle (n i ),H obstacle (n i )};

[0042] Among them, H angle (n i ) is considered the current node n i Heuristic function for travel angle constraint, H obstacle (n i ) is a heuristic function that takes obstacle constraints into account;

[0043] And the heuristic function considering the travel angle constraint is expressed as:

[0044]

[0045] Among them, H(n i ,n goal ) represents the current point n i and target point n goal The distance, α d is an adjustable weight parameter, d rs is the Chebyshev distance from the current node to the target end point;

[0046] The heuristic function considering obstacle constraints is expressed as:

[0047]

[0048] Among them, g(nj -n j-1 ) is the cost of moving a single grid, ω o In order to take into account the weight of the travel direction angle, it is defined as:

[0049]

[0050] Among them, α o is the penalty coefficient for the angular deviation of the traveling direction in the obstacle constraint calculation, Δd represents the current search direction, and D(n) represents the node and the two surrounding nodes of the adjacent child nodes that the line connecting the current node and the target end point passes through.

[0051] Further, including:

[0052] The expansion of the current node using the child node expansion strategy specifically includes:

[0053] The variable step size child node expansion strategy is used to expand the search for each child node. The corresponding search method is defined as:

[0054]

[0055] Among them, k is the number of iterations of the variable length radius, δ is the robot's front wheel angle, L is the wheelbase, and θ is the robot's current moving direction angle. is the minimum turning radius, Δr is the length of the variable turning radius that can be set, and n r is the discrete number of variable turning radius, x c 、y c is the current robot node coordinate, x i 、y i is the expanded robot node coordinate.

[0056] Further, including:

[0057] The method of expanding the current node using the child node expansion strategy further includes:

[0058] After searching for each child node using the variable step size child node expansion strategy, the child node safety expansion strategy is used to expand the specific node, including:

[0059] Put the current node into the open list and search all the child nodes around the current node in turn;

[0060] When searching for a child node in the neighborhood, first determine whether the child node is within the valid range of the map, and also determine whether the point is an obstacle, and save the expandable node;

[0061] Determine the positional relationship between the obstacle and the current node. If the obstacle is located at the horizontally or vertically adjacent child node of the current node, further process the expandable node. That is, when the obstacle is located in the vertical neighborhood of the current node, remove the two horizontally adjacent child nodes of the obstacle's location from the expandable nodes; when the obstacle is located in the horizontal neighborhood of the current node, remove the two vertically adjacent child nodes of the obstacle's location from the expandable nodes; and put the final expandable node into the open list.

[0062] Further, including:

[0063] Outputting the path according to the closed list and performing repeated broken line optimization, including the first broken line optimization, specifically includes:

[0064] Calculate the angles between the current node F and its parent node F1, and between the parent node F1 and the parent node F2. If the angle is 0, it means that the three points are on the same line. Therefore, the parent node of the parent node is directly used as the parent node of the current node.

[0065] If the angle is not 0, it means that the path needs to turn between the current node F and the parent node F1, and between the parent node F1 and the parent node F2. At this time, calculate the equation of the line between F and F2, and determine the range of points on the line;

[0066] Traverse the obstacle list and check whether there is an obstacle on the straight path and whether the distance is greater than the safe distance between the obstacle and the straight line. If there is an obstacle or the distance between the obstacle and the straight line is less than the safe distance, update the current node to its parent node and continue the next loop;

[0067] If there is no obstacle on the straight path and the distance from the obstacle to the straight line is greater than the safe distance, the parent node F2 of the parent node is used as the parent node F1 of the current node F, and the traversal continues forward;

[0068] The loop is completed and the optimized path is finally returned.

[0069] Further, including:

[0070] The outputting of the path according to the closed list and performing repeated broken line optimization also includes secondary broken line optimization, specifically including:

[0071] The path of the first polyline optimization is reversed and used as the input path of the second polyline optimization. A new path array A is defined, and the starting node of the original path is placed in array A.

[0072] Set the second node as the current node P, obtain its child node P1 and its child node P2, and calculate the intermediate node on the line connecting P1 and P2 as the candidate node for the new path;

[0073] For the intermediate nodes on the line connecting the current node P and P1 and P2, starting from the intermediate node closest to P2 on the line, check whether there are obstacles on the line connecting the current node P and the intermediate nodes, and calculate the vertical distance from the obstacle to the line;

[0074] If there is an obstacle or the distance to the obstacle is less than the safe distance, the intermediate node is discarded and the search continues until the first feasible intermediate node is found and added to the new path array A. It is used as the current node and the cycle continues;

[0075] If the child node of the current node is the last node in the original path, the loop ends; finally, the new path array A is returned as the path after quadratic polyline optimization.

[0076] On the other hand, the present invention also provides an indoor mobile robot path planning system for substation equipment inspection, the system comprising:

[0077] The environment modeling module is used to create a grid map based on the actual work scene that needs to be inspected. The coordinates of the center point of each grid are defined as the location of the grid, and 0 is set to represent an unoccupied grid, that is, there is no obstacle at this location, and 1 represents an occupied grid, indicating that there is an obstacle at this location;

[0078] The global path generation module is used to determine the positions of the start node and the target node, add the start node to the open list, sort the nodes in the open list, set the node with the lowest cost as the current node and add it to the closed list, and then expand the current node using the child node expansion strategy; when the node that already exists in the open list is found, the node cost is recalculated and the open list is updated; the search process is repeated until the current node is the target point, and then the path is output according to the closed list and repeatedly optimized; finally, the path after repeated optimization is smoothed using the moving average method to generate the final global path;

[0079] The local path generation module is used to avoid unknown obstacles in the global path using the trained improved DQN algorithm until a collision-free path from the starting point to the end point is planned.

[0080] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0081] (1) Taking into account the complexity of the working environment of the inspection robot, the present invention modifies the distance model in the A-star algorithm to a combination of Manhattan distance and Euclidean distance to be closer to the actual distance.

[0082] (2) The present invention considers the constraints of the robot's travel angle and obstacle constraints in the heuristic function of the traditional algorithm, thereby effectively accelerating the search efficiency and guiding the robot to leave the area where the current heading is inconsistent with the target point direction.

[0083] (3) The present invention proposes an expansion search method for variable-step-length child nodes based on a mixture of long and short steps. Based on this improvement, the inspection robot can not only conduct rapid searches in open areas, but also achieve high-precision turns in narrow areas or areas with many obstacles. In the process of child node expansion, when certain specific nodes are selected, the robot will pass through the vertices of the obstacles during movement and collide with the obstacles. This situation is relatively common in complex environments such as substations with many cabinets. Therefore, the present invention proposes a child node safe expansion strategy to ensure the safe expansion of child nodes.

[0084] (4) Based on the fact that when the A-star algorithm is searching, no matter where it is, the direction on the anti-diagonal line of its current movement direction does not need to be searched, the present invention changes the traditional 8 search node directions to 5, thereby improving the search efficiency by deleting the nodes that do not need to be searched. Based on this, a node angle selection rule table is established to avoid redundant node searches and improve search efficiency.

[0085] (5) The present invention adopts a repeated broken line optimization strategy to eliminate redundant nodes in the path and adopts a moving average method to smooth the path. The repeated broken line optimization strategy includes the first broken line optimization and the second broken line optimization. The validity of the nodes is guaranteed by the two broken line optimizations.

[0086] (6) This paper addresses the issues of the Q-network algorithm in substation path planning by making improvements: First, the traditional exploration strategy is improved to an adaptive exploration strategy to prevent the algorithm from falling into local optimality; second, the concept of gravitational field in the artificial potential field algorithm is introduced into the traditional action selection function to reduce the exploration of invalid actions; finally, the reward function is redesigned and the problem of sparse rewards is solved to a certain extent by adding an auxiliary reward function. The improved model can accelerate convergence and achieve better local planning.

[0087] (7) This paper combines the improved obstacle avoidance capability of the Deep Q Network algorithm in unknown environments with the global planning capability of the optimized A-star algorithm to design a hybrid path planning algorithm. First, a map model of the actual scene is built, and the established map model is used to plan a global path using the optimized A-star algorithm. Then, the trained improved DQN algorithm is used to avoid unknown obstacles in the global path until a collision-free path from the starting point to the end point is planned. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1is the search angle rule table according to the embodiment of the present invention;

[0089] Figure 2 is a global path planning flow chart of the optimized A-star algorithm according to an embodiment of the present invention;

[0090] Figure 3 This is an overall flow chart of the hybrid path planning algorithm described in an embodiment of the present invention. DETAILED DESCRIPTION

[0091] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention and not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0092] Embodiment 1: The embodiment of the present invention discloses a path planning method for an indoor mobile robot for substation equipment inspection, the method comprising the following steps:

[0093] S1 establishes a grid map based on the actual work scenario that needs to be inspected. The coordinates of the center point of each grid are defined as the position of the grid, and 0 is set to represent an unoccupied grid, that is, there is no obstacle at this position, and 1 represents an occupied grid, indicating that there is an obstacle at this position.

[0094] Specifically, the present invention uses a grid method to construct a map environment, dividing the actual physical space into a series of equally sized grid cells. Each grid cell corresponds to a specific location in the matrix, and the coordinates of the center point of each grid cell are defined as the location of that grid cell. This digital conversion allows for precise quantification of environmental information, facilitating subsequent path planning algorithms.

[0095] Set the value 0 to represent an unoccupied grid, meaning there are no obstacles at that location and the robot can pass freely; set the value 1 to represent an occupied grid, meaning there are obstacles at that location and the robot needs to avoid them. When the map environment changes, the numerical state of each grid can be updated independently, allowing the robot to adjust its state in real time according to these changes.

[0096] S2 determines the positions of the starting node and the target node, adds the starting node to the open list, sorts the nodes in the open list, sets the node with the smallest node cost as the current node and adds it to the closed list, and then expands the current node using the child node expansion strategy; when the node that already exists in the open list is searched, the node cost is recalculated and the open list is updated; the search process is repeated until the current node is the target point and the search is stopped, then the path is output according to the closed list and repeated broken line optimization is performed; finally, the path after repeated broken line optimization is smoothed using the moving average method to generate the final global path.

[0097] Preferably, this embodiment includes generating a global path based on an optimized A-star algorithm, including:

[0098] First, the traditional A-star algorithm searches in eight directions around the parent node. Under the premise that there are no obstacles, the inspection robot can move to the child nodes in the eight directions around it. When the A-star algorithm searches, no matter where it is, the direction on the anti-diagonal line of its current movement direction does not need to be searched, so the present invention improves the search efficiency by deleting the nodes that do not need to be searched. The traditional eight search node directions are changed to five. The present invention establishes a node angle selection rule table, such as Figure 1 As shown, redundant node searches are avoided and search efficiency is improved.

[0099] Secondly, the cost evaluation of the traditional A-star algorithm from the starting point to the target point is:

[0100] F(n)=G(n)+H(n) (1)

[0101] Among them, F(n) is the total cost from the starting point to the target node; G(n) represents the cost of moving from the starting point to the specified point; H(n) represents the cost of moving from the specified point to the target point.

[0102] Therefore, the traditional A-star path planning method has high search repetitiveness and low efficiency. To solve the above problems, the present invention optimizes the path planning ability of the algorithm through a series of improvements, including: improving the distance model, optimizing the heuristic function, proposing a variable step-size child node expansion strategy, proposing a child node security expansion strategy, formulating node selection rules and proposing an energy consumption optimization strategy.

[0103] Among them, the traditional A-star algorithm mostly uses Euclidean distance, Manhattan distance, or Chebyshev distance. Manhattan distance overestimates the actual running distance, while Euclidean distance underestimates the actual running distance. Chebyshev distance only considers the maximum difference between two points in each dimension, while ignoring other information in the actual path. In some cases, it may not accurately reflect the actual distance between points.

[0104] Taking into account the complexity of the inspection robot's working environment, the present invention combines Manhattan distance and Euclidean distance to get closer to the actual distance, and defines a new distance model as:

[0105]

[0106] Among them, β is the distance factor, β∈[0,1], and a specific value is formulated or trained according to the specific working environment so that the distance model reflects the actual distance.

[0107] On this basis, the heuristic function of the present invention takes into account the constraint of the robot's travel angle for the traditional algorithm, effectively accelerating the search efficiency and guiding the robot to leave the area where the current heading is inconsistent with the target point direction. The heuristic function taking into account the travel angle constraint is:

[0108]

[0109] Among them, H(n i ,n goal ) represents the distance between the current point and the target point, α d is an adjustable weight parameter, d rs is the Chebyshev distance from the current node to the target end point. When the robot is far away from the target point, the heuristic value of the distance is more meaningful, while when the robot is close to the target point, the heuristic value based on the travel direction angle can improve the search efficiency. Therefore, by adjusting α d , which can combine the advantages of the two types of heuristic values.

[0110] In addition, the heuristic function considering obstacle constraints is:

[0111]

[0112] Among them, g(n j -n j-1 ) is the cost of moving a single grid, ω o In order to take into account the weight of the travel direction angle, it is defined as:

[0113]

[0114] Among them, α ois the penalty coefficient for the angular deviation of the traveling direction in the obstacle constraint calculation, Δd represents the current search direction, and D(n) represents the node and the two surrounding nodes of the adjacent child nodes that the line connecting the current point and the target point passes through.

[0115] Combining the above two heuristic functions, the heuristic value is determined to be the maximum value of the two, which is defined as:

[0116] H(n i )=max{H angle (n i ),H obstacle (n i )} (6)

[0117] Finally, the traditional algorithm expands child nodes based on equal step sizes. If the step size is set too short, the search accuracy is improved, but the search efficiency is reduced; if the step size is set too long, although the search efficiency can be appropriately improved, the search accuracy is reduced. Therefore, this invention proposes a variable step size search method based on a mixture of long and short steps. Based on this improvement, the inspection robot can not only conduct rapid searches in open areas, but also achieve high-precision turns in narrow areas or areas with many obstacles. The variable step size child node expansion method is defined as:

[0118]

[0119] Among them, k is the number of iterations of the variable length radius, δ is the robot's front wheel angle, L is the wheelbase, and θ is the robot's current moving direction angle. is the minimum turning radius, Δr is the length of the variable turning radius that can be set, and n r is the discrete number of variable turning radius, x c 、y c is the current robot node coordinate, x i 、y i is the expanded robot node coordinate.

[0120] During the child node expansion process, selecting certain specific nodes can cause the robot to pass through the vertices of obstacles during movement, resulting in collisions. This situation is common in complex environments such as substations with many cabinets. Therefore, the present invention proposes a child node safe expansion strategy, which specifically involves the following steps:

[0121] Step 1: Put the current node into the open list OPEN table, and search all the child nodes around the current node in turn;

[0122] Step 2: When searching for child nodes in the neighborhood, it is necessary to determine whether the child node is within the valid range of the map, and also whether the point is an obstacle, and save the expandable nodes;

[0123] Step 3: Determine the positional relationship between the obstacle and the current node. If the obstacle is located at a horizontally or vertically adjacent child node of the current node, further process the expandable node: if the obstacle is located in the vertical neighborhood of the current node, remove the two horizontally adjacent child nodes of the obstacle's location from the expandable node; if the obstacle is located in the horizontal neighborhood of the current node, remove the two vertically adjacent child nodes of the obstacle's location from the expandable node;

[0124] Step 4: Finally, put the expandable node into the OPEN table.

[0125] In this embodiment, a path is output according to the closed list and repeated polyline optimization is performed. Finally, the path after repeated polyline optimization is smoothed using a moving average method to generate a final global path, specifically including:

[0126] Traditional algorithms generate a large number of global paths with numerous nodes. This leads to numerous corners and unnecessary turning points in complex maps. These turning points increase the robot's energy consumption and reduce its endurance and efficiency. Therefore, this paper uses a repeated broken line optimization strategy to remove redundant nodes from the path and a moving average method to smooth the path.

[0127] The first step of line optimization is:

[0128] Step 1: Calculate the angle between the current node F and its parent node F1, and between the parent node F1 and the parent node F2. If the angle is 0, it means that the three points are on the same line. Therefore, the parent node of the parent node can be directly used as the parent node of the current node.

[0129] Step 2: If the angle is not 0, it means that the path needs to turn between F and F1, or between F1 and F2. At this time, calculate the equation of the line between F and F2 and determine the range of points on the line;

[0130] Step 3: Traverse the obstacle list and check whether there is an obstacle on the straight path and whether the distance is greater than the safe distance between the obstacle and the straight line. If there is an obstacle or the distance between the obstacle and the straight line is less than the safe distance, update the current node to its parent node and continue the next loop;

[0131] Step 4: If there is no obstacle on the straight path and the distance from the obstacle to the straight line is greater than the safe distance, then the parent node F2 of the parent node is used as the parent node F1 of the current node F, and the traversal continues forward;

[0132] Step 5. Finally, return the optimized path.

[0133] The steps for repeated polyline optimization are:

[0134] Step 1: Reverse the path of the first polyline optimization and use it as the input path for the second polyline optimization. Define a new path array A and put the starting node of the original path into array A.

[0135] Step 2: Set the second node as the current node P, obtain its child node P1 (i.e., the next node of the current node) and its child node P2, and calculate the intermediate node on the line connecting P1 and P2 as the candidate node for the new path;

[0136] Step 3: For the intermediate nodes on the line connecting the current node P and P1 and P2, starting from the intermediate node closest to P2 on the line, check whether there are any obstacles on the line connecting the current node P and the intermediate nodes, and calculate the vertical distance from the obstacle to the line;

[0137] Step 4: If there is an obstacle or the distance to the obstacle is less than the safe distance, the intermediate node is discarded and the search continues until the first feasible intermediate node is found and added to the new path array A. It is used as the current node and the cycle continues.

[0138] Step 5: If the child node of the current node is the last node in the original path, the loop ends;

[0139] Step 6. Finally, return the new path array A as the path after quadratic polyline optimization.

[0140] Moving average can average the data points in the neighborhood to replace the center point value of the neighborhood, eliminating sharp turning points in the path, thereby achieving a smooth trajectory. The calculation formula of the moving average method is:

[0141]

[0142] Among them, Ft is the predicted value for the next period, n is the number of periods of moving average, At1 is the actual value of the previous period, At2, At3 and At n They represent the actual values ​​of the previous two periods, the previous three periods, and even the previous n periods. The specific steps of moving average path smoothing are:

[0143] Step 1: Take the optimized path points of the repeated polyline as input, traverse the adjacent path points, and determine the number of nodes that need to be inserted between two adjacent nodes by calculating the distance between the lines connecting the two adjacent points;

[0144] Step 2: Calculate the coordinates of the newly inserted node based on the equation of the line between the two adjacent points, and save the original nodes and the newly added nodes in order as the input path matrix of the moving average method;

[0145] Step 3: Use the moving average method to smooth the input path matrix;

[0146] Step 4: Output the smoothed path.

[0147] Therefore, the global path planning flowchart of the optimized A-star algorithm is as follows: Figure 2 As shown in the figure, the steps are as follows: First, a grid map is established according to the actual working scenario, various parameters of the algorithm are initialized, and the positions of the starting node and the target node are determined. When the algorithm is running, the starting node is added to the open list, the nodes in the open list are sorted, and the node with the smallest output node cost F(n)* is set as the current node and added to the closed list. Then, the current node is expanded using the child node expansion strategy. When a node that already exists in the open list is searched, the node cost F(n)* is recalculated and the open list is updated. The search process is repeated until the current node is the target point and the search is stopped. Then, the path is output according to the closed list and repeated broken line optimization is performed. Finally, the path after repeated broken line optimization is smoothed using the moving average method to generate the final global path.

[0148] S3 uses the trained improved DQN algorithm to avoid unknown obstacles in the global path until a collision-free path from the starting point to the end point is planned.

[0149] Specifically, in order to solve the problems of traditional deep Q-network algorithm exploration strategy, which is blind and easy to fall into local optimality, and excessive exploration of invalid actions by traditional action selection function, which makes the algorithm inefficient, this application proposes an improved algorithm. Improvements are made in three aspects: exploration strategy, action selection function, and reward function design. First, the traditional exploration strategy is improved to an adaptive exploration strategy to prevent the algorithm from falling into local optimality; second, the concept of gravitational field in the artificial potential field algorithm is introduced into the traditional action selection function to reduce the exploration of invalid actions; finally, the reward function is redesigned, and the problem of sparse rewards is solved to a certain extent by adding auxiliary reward functions. After improvement, the convergence speed can be accelerated and better local planning can be achieved.

[0150] First, in traditional strategies, the exploration rate is usually fixed, with exploration performed with a fixed probability ε and exploitation performed with a probability of 1-ε. This can lead to excessive random exploration during the exploration phase, while failing to fully utilize the learned experience during the exploitation phase. This fixed exploration rate cannot be flexibly adjusted according to the training phase, thus affecting the performance and efficiency of the algorithm. In addition, traditional exploration strategies suffer from the problem of local optimal solutions. Therefore, this paper proposes an adaptive ε strategy, in which the exploration rate is updated according to a set time step decay, defined as:

[0151] ε t+1 =max(ε min ,ε t *ε decay ) (9)

[0152] Among them, ε t+1 is the exploration rate at the next time step, ε min is the minimum value of the exploration rate, ε t is the exploration value of the current time step, ε decay Is the decay coefficient of the set exploration rate. The exploration rate will be updated at each set time step multiplied by the decay coefficient until the exploration rate is less than or equal to the minimum exploration rate.

[0153] In the initial stage of training, a higher exploration rate helps the robot fully explore the environment and obtain more information; as training progresses, the exploration rate gradually decreases, and the robot will make more use of existing experience to make decisions, thereby improving the accuracy and stability of decisions; and the exploration rate eventually equals the minimum exploration rate, and a certain amount of exploration ability is still retained in the later stages of training, which can avoid the algorithm from falling into the problem of local optimality.

[0154] Secondly, traditional deep Q-network algorithms spend too much time processing invalid states in the environment state space, resulting in a slow convergence rate. Therefore, this paper incorporates the concept of the gravity function in the artificial potential field method into the action selection process, providing guidance for the robot's actions towards the target when performing the task, reducing the robot's exploration of invalid states and improving the algorithm's convergence rate. The improved gravity function is:

[0155]

[0156] Among them, d m is the distance threshold, D m is the maximum distance in the map, and K is the gravitational constant, which is used to adjust the strength of gravity. When the distance to the target point is greater than the threshold, the greater the distance, the smaller the gravity; when it is less than the threshold, the gravity is a constant value.

[0157] Finally, for the reward function of the traditional deep Q network algorithm, the robot will only receive corresponding positive and negative reward values ​​when it reaches the target, collides, or goes out of bounds. The reward value in other states is 0, which will lead to the problem of sparse rewards. Due to the sparse reward signals in the environment, the robot will not be able to clearly explore the direction, and a large number of invalid states will appear in the exploration process, resulting in low efficiency of algorithm training. To solve this problem, the present invention sets an auxiliary reward function for the reward function, adds reward signals to other non-rewarded states, provides better guidance for the robot, and improves the efficiency of algorithm training. The optimized reward function is defined as:

[0158]

[0159] Where ξ is the discount factor for the reward value, and r is the immediate feedback from the environment to the agent's behavior. Guided by the auxiliary function, the robot will prioritize actions that will allow it to move closer to the target.

[0160] Therefore, in this embodiment, the specific steps of improving the deep Q network algorithm are:

[0161] Step 1: Initialize the maximum number of training rounds of the algorithm, the size of the experience pools D1 and D2, network parameters, initial sample priority and sampling probability, number of sample samples, network parameter update interval, and set the starting point and target point positions;

[0162] Step 2: Initialize the map environment and get the current state s t ;

[0163] Step 3: Use the improved action selection function to select an action with probability ε, otherwise select the action with the largest Q value;

[0164] Step 4: The robot performs action a t , according to the optimized reward function, the reward value r for executing the current action is obtained t and the next state s t+1 ;

[0165] Step 5: According to the reward value r t Generate samples, if r t > 0, then (s t ,a t ,r t ,s t+1 ) is stored in the positive reward sample experience pool D1; if r t ≤0, then (s t ,a t ,r t ,s t+1 ) is stored in the negative reward and zero reward experience pool D2;

[0166] Step 6: Use the adaptive exploration strategy to extract samples, that is, extract samples from the experience pool D1 with probability ρ, and extract samples from the experience pool D2 with probability 1-ρ; sample according to the probability P of the sample being sampled;

[0167] Step 7: If the next state of the sampled sample is the target point state, the target Q value is the reward value of the target point state. Otherwise, the target Q value is the sum of the reward value of the target point state and the discounted maximum Q value of the next state.

[0168] Step 8: Repeat steps 6 to 7 until the number of samples is reached and the final target Q value is obtained.

[0169] Step 9: Calculate the loss function and update the network parameter w;

[0170] Step 10: When the number of network parameter update steps is reached, the target network parameters are obtained;

[0171] Step 11: Determine whether the robot has reached the target point. If not, repeat steps 2 to 10. If it has reached the target point, further determine whether the network training has reached the maximum number of rounds. If not, repeat steps 1 to 10. If the maximum number of training rounds has been reached, the training ends.

[0172] like Figure 3 As shown in the figure, this paper combines the improved obstacle avoidance capabilities of the Deep Q Network algorithm in unknown environments with the global planning capabilities of the optimized A-star algorithm to design a hybrid path planning algorithm. It first builds a map model of the actual scene, then uses the optimized A-star algorithm to plan a global path. It then uses the trained improved DQN algorithm to avoid unknown obstacles in the global path until a collision-free path is planned from the starting point to the end point.

[0173] Embodiment 2: The present invention also provides an indoor mobile robot path planning system for substation equipment inspection, the system comprising:

[0174] The environment modeling module is used to create a grid map based on the actual work scene that needs to be inspected. The coordinates of the center point of each grid are defined as the location of the grid, and 0 is set to represent an unoccupied grid, that is, there is no obstacle at this location, and 1 represents an occupied grid, indicating that there is an obstacle at this location;

[0175] The global path generation module is used to determine the positions of the start node and the target node, add the start node to the open list, sort the nodes in the open list, set the node with the lowest cost as the current node and add it to the closed list, and then expand the current node using the child node expansion strategy; when the node that already exists in the open list is found, the node cost is recalculated and the open list is updated; the search process is repeated until the current node is the target point, and then the path is output according to the closed list and repeatedly optimized; finally, the path after repeated optimization is smoothed using the moving average method to generate the final global path;

[0176] The local path generation module is used to avoid unknown obstacles in the global path using the trained improved DQN algorithm until a collision-free path from the starting point to the end point is planned.

[0177] Other technical features of the indoor mobile robot path planning system for substation equipment inspection described in this embodiment are similar to the corresponding indoor mobile robot path planning method for substation equipment inspection, and will not be repeated here.

[0178] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. "Multiple" means two or more, unless otherwise specifically defined.

[0179] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0180] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0181] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0182] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0183] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0184] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0185] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0186] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0187] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A path planning method for an indoor mobile robot used for substation equipment inspection, characterized in that: The method includes: A grid map is created based on the actual work scenario that needs to be inspected. The coordinates of the center point of each grid are defined as the location of the grid. 0 represents an unoccupied grid, meaning there is no obstacle at that location, and 1 represents an occupied grid, meaning there is an obstacle at that location. Determine the positions of the start node and the target node, add the start node to the open list, sort the nodes in the open list, set the node with the lowest cost as the current node and add it to the closed list, and then expand the current node using the child node expansion strategy; when the node already in the open list is found, recalculate the node cost and update the open list; repeat the search process until the current node is the target point, then output the path according to the closed list and perform repeated polyline optimization; finally, use the moving average method to smooth the path after repeated polyline optimization to generate the final global path; The trained improved DQN algorithm is used to avoid unknown obstacles in the global path until a collision-free path from the starting point to the end point is planned.

2. The indoor mobile robot path planning method for substation equipment inspection according to claim 1 is characterized in that: The method of using the trained improved DQN algorithm to avoid unknown obstacles in the global path includes the following steps: Step 1: Initialize the maximum number of training rounds of the algorithm, the size of the experience pools D1 and D2, network parameters, initial sample priority and sampling probability, number of sample samples, network parameter update interval, and set the starting point and target point positions according to the global path; Step 2: Initialize the map environment and get the current state s t ; Step 3: Use the improved action selection function to select an action with probability ε, otherwise select the action with the largest Q value; Step 4 The robot performs action a t , according to the optimized reward function, the reward value r for executing the current action is obtained t and the next state s t +1; Step 5: According to the reward value r t Generate samples, if r t > 0, then (s t ,a t ,r t ,s t +1) is stored in the positive reward sample experience pool D1; if r t ≤0, then (s t ,a t ,r t ,s t +1) stored in the negative reward and zero reward experience pool D2; Step 6 uses an adaptive exploration strategy to extract samples, that is, extract samples from the experience pool D1 with a probability of ρ, and extract samples from the experience pool D2 with a probability of 1-ρ; sampling is performed according to the probability P of the sample being sampled; Step 7: If the next state of the sampled sample is the target point state, the target Q value is the reward value of the target point state. Otherwise, the target Q value is the sum of the reward value of the target point state and the discounted maximum Q value of the next state. Step 8: Repeat steps 6 to 7 until the number of samples is reached and the final target Q value is obtained; Step 9 calculates the loss function and updates the network parameter w; Step 10: When the number of network parameter update steps is reached, the target network parameters are obtained; Step 11 determines whether the robot has reached the target point. If not, repeat steps 2 to 10. If it has reached the target point, further determine whether the network training has reached the maximum number of rounds. If not, repeat steps 1 to 10. If the maximum number of training rounds has been reached, the training ends.

3. The indoor mobile robot path planning method for substation equipment inspection according to claim 2 is characterized in that: The improved action selection function is expressed as: Among them, d m is the distance threshold, D m is the maximum distance in the grid map, and K is the gravitational constant. Therefore, when the distance to the target point is greater than the distance threshold, the greater the distance, the smaller the gravitational force; when it is less than the distance threshold, the gravitational force is a constant value.

4. The indoor mobile robot path planning method for substation equipment inspection according to claim 2 is characterized in that: The optimization reward function is expressed as: Among them, ξ is the discount factor of the reward value, and r is the immediate feedback of the environment to the agent's behavior.

5. The indoor mobile robot path planning method for substation equipment inspection according to claim 2 is characterized in that: The adaptive exploration strategy includes setting the exploration rate to be updated according to a set time step decay, expressed as: e t+1 =max(e min ,he t *e decay ); Among them, ε t+1 is the exploration rate at the next time step, ε min is the minimum value of the exploration rate, ε t is the exploration value of the current time step, ε decay It is the decay coefficient of the set exploration rate. The exploration rate will be updated at each set time step multiplied by the decay coefficient until the exploration rate is less than or equal to the minimum exploration rate.

6. The indoor mobile robot path planning method for substation equipment inspection according to claim 1, characterized in that: The calculation formula of the node cost is expressed as: F(n)=G(n)+H(n); Among them, F(n) is the total cost from the starting point to the target node; G(n) represents the cost of moving from the starting point to the current node; H(n) represents the cost of moving from the current node to the target point; The cost of moving from the current node to the target point corresponds to the heuristic function, which is expressed as: H(n i )=max{H angle (n i ),H obstacle (n i )}; Among them, H angle (n i ) is considered the current node n i Heuristic function for travel angle constraint, H obstacle (n i ) is a heuristic function that takes obstacle constraints into account; And the heuristic function considering the travel angle constraint is expressed as: Among them, H(n i ,n goal ) represents the current point n i and target point n goal The distance, α d is an adjustable weight parameter, d rs is the Chebyshev distance from the current node to the target end point; The heuristic function considering obstacle constraints is expressed as: Among them, g(n j -n j-1 ) is the cost of moving a single grid, ω o In order to take into account the weight of the travel direction angle, it is defined as: Among them, α o is the penalty coefficient for the angular deviation of the traveling direction in the obstacle constraint calculation, Δd represents the current search direction, and D(n) represents the node and the two surrounding nodes of the adjacent child nodes that the line connecting the current node and the target end point passes through.

7. The indoor mobile robot path planning method for substation equipment inspection according to claim 6, characterized in that: The expansion of the current node using the child node expansion strategy specifically includes: The variable step size child node expansion strategy is used to expand the search for each child node. The corresponding search method is defined as: Among them, k is the number of iterations of the variable length radius, δ is the robot's front wheel angle, L is the wheelbase, and θ is the robot's current moving direction angle. is the minimum turning radius, Δr is the length of the variable turning radius that can be set, and n r is the discrete number of variable turning radius, x c 、y c is the current robot node coordinate, x i 、y i is the expanded robot node coordinate.

8. The indoor mobile robot path planning method for substation equipment inspection according to claim 7, characterized in that: The method of expanding the current node using the child node expansion strategy further includes: After searching for each child node using the variable step size child node expansion strategy, the child node safety expansion strategy is used to expand the specific node, including: Put the current node into the open list and search all the child nodes around the current node in turn; When searching for a child node in the neighborhood, first determine whether the child node is within the valid range of the map, and also determine whether the point is an obstacle, and save the expandable node; Determine the positional relationship between the obstacle and the current node. If the obstacle is located at the horizontally or vertically adjacent child node of the current node, further process the expandable node. That is, when the obstacle is located in the vertical neighborhood of the current node, remove the two horizontally adjacent child nodes of the obstacle's location from the expandable nodes; when the obstacle is located in the horizontal neighborhood of the current node, remove the two vertically adjacent child nodes of the obstacle's location from the expandable nodes; and put the final expandable node into the open list.

9. The indoor mobile robot path planning method for substation equipment inspection according to claim 1, characterized in that: Outputting the path according to the closed list and performing repeated broken line optimization, including the first broken line optimization, specifically includes: Calculate the angles between the current node F and its parent node F1, and between the parent node F1 and the parent node F2. If the angle is 0, it means that the three points are on the same line. Therefore, the parent node of the parent node is directly used as the parent node of the current node. If the angle is not 0, it means that the path needs to turn between the current node F and the parent node F1, and between the parent node F1 and the parent node F2. At this time, calculate the equation of the line between F and F2, and determine the range of points on the line; Traverse the obstacle list and check whether there is an obstacle on the straight path and whether the distance is greater than the safe distance between the obstacle and the straight line. If there is an obstacle or the distance between the obstacle and the straight line is less than the safe distance, update the current node to its parent node and continue the next loop; If there is no obstacle on the straight path and the distance from the obstacle to the straight line is greater than the safe distance, the parent node F2 of the parent node is used as the parent node F1 of the current node F, and the traversal continues forward; The loop is completed and the optimized path is finally returned.

10. The indoor mobile robot path planning method for substation equipment inspection according to claim 9, characterized in that: The outputting of the path according to the closed list and performing repeated broken line optimization also includes secondary broken line optimization, specifically including: The path of the first polyline optimization is reversed and used as the input path of the second polyline optimization. A new path array A is defined, and the starting node of the original path is placed in array A. Set the second node as the current node P, obtain its child node P1 and its child node P2, and calculate the intermediate node on the line connecting P1 and P2 as the candidate node for the new path; For the intermediate nodes on the line connecting the current node P and P1 and P2, starting from the intermediate node closest to P2 on the line, check whether there are obstacles on the line connecting the current node P and the intermediate nodes, and calculate the vertical distance from the obstacle to the line; If there is an obstacle or the distance to the obstacle is less than the safe distance, the intermediate node is discarded and the search continues until the first feasible intermediate node is found and added to the new path array A. It is used as the current node and the cycle continues; If the child node of the current node is the last node in the original path, the loop ends; finally, the new path array A is returned as the path after quadratic polyline optimization.

11. An indoor mobile robot path planning system for substation equipment inspection, characterized in that: The system includes: The environment modeling module is used to create a grid map based on the actual work scene that needs to be inspected. The coordinates of the center point of each grid are defined as the location of the grid, and 0 is set to represent an unoccupied grid, that is, there is no obstacle at this location, and 1 represents an occupied grid, indicating that there is an obstacle at this location; The global path generation module is used to determine the positions of the start node and the target node, add the start node to the open list, sort the nodes in the open list, set the node with the lowest cost as the current node and add it to the closed list, and then expand the current node using the child node expansion strategy; when the node that already exists in the open list is found, the node cost is recalculated and the open list is updated; the search process is repeated until the current node is the target point, and then the path is output according to the closed list and repeatedly optimized; finally, the path after repeated optimization is smoothed using the moving average method to generate the final global path; The local path generation module is used to avoid unknown obstacles in the global path using the trained improved DQN algorithm until a collision-free path from the starting point to the end point is planned.

Images