Robot path planning method and system and storage medium

Abstract

The invention discloses a robot path planning method and system and a storage medium, and the method comprises the steps: building an environment topology model based on sensor data and map information, building an adaptive map data structure, carrying out the global planning of a rough level through employing a low-resolution map, and carrying out the local optimization of a detail level through employing a high-resolution map. Adaptive dynamic switching is carried out between the two levels according to a path planning task; constructing a two-way alternate search strategy, and performing path search by combining an environment topology model and map information; environment change is monitored in real time, the connectivity probability of each edge in the probability graph is dynamically updated, and a path search strategy is adaptively adjusted according to the change of the probability graph; and establishing a backup path selection model based on reinforcement learning, and dynamically adjusting a backup path in combination with environment change and main path search. Through a bidirectional alternate search strategy, the path search efficiency is improved, the problem of path repetition caused by bidirectional search is solved, and the optimal path from the starting point to the end point can be found more quickly.

Description

Technical Field

[0001] The present invention belongs to the field of automation technology, and in particular relates to a robot path planning method, system and storage medium. Background Art

[0002] In recent years, with the rapid development of automation technology, robots have been deployed more and more widely in the construction industry, and the participation of robots in construction has become a global trend. In response to this development, researchers have proposed a robot system equipped with perception sensors and intelligent algorithms to assist construction supervisors in remotely identifying building materials, detecting component installation and defects. The application of construction robots combined with virtual reality technology has significantly improved workers' trust in robots, self-efficacy, and situational awareness of the construction site. In addition, researchers have proposed a hybrid localization algorithm with adaptive weights, which effectively improves the robust localization capability of robots in construction scenarios. In response to the initialization problem of robots when deployed in unknown locations, another study proposed an indoor robot initialization system based on building information modeling (BIM), which uses object detectors to achieve automatic initialization of robots in unknown locations. Despite many advances in the development of construction robot solutions, global path planning of mobile robots remains a key challenge in the process of realizing fully automatic robot solutions.

[0003] Global path planning is one of the fundamental problems in robotics and is the basis for robots to complete complex tasks. It is a static planning, usually described as follows: given a mobile robot and an environment model, the optimal path between the starting position and the final position must be found without colliding with obstacles, while meeting certain criteria such as distance, smoothness or safety. The results of path planning can be evaluated based on path length, turning angle, calculation time and risk level.

[0004] When planning a path in an uneven terrain environment at a construction site, obstacles and terrain slopes must be considered comprehensively. Compared with a two-dimensional plane, it is more restricted by the environment and has higher requirements for algorithms. The existing technology has problems such as long path planning calculation time, many redundant nodes, uneven paths, and insecurity. How to quickly plan a short, practical, smooth and safe path for the robot is a problem that needs to be further solved. Summary of the invention

[0005] In view of the defects existing in the above-mentioned prior art, the present invention provides a robot path planning method, system and storage medium, the method comprising the following steps:

[0006] Step 1, by building an environment topology model based on sensor data and map information, a probability graph is used to represent the structure of the environment topology model, where each node represents a location and an edge represents a path connecting two locations;

[0007] Step 2: Establish a multi-level adaptive map data structure. The coarse level uses a low-resolution map for global planning, and the detailed level uses a high-resolution map for local optimization. The two levels are adaptively and dynamically switched according to the path planning task.

[0008] Step 3, construct a bidirectional alternating search strategy, including defining a heuristic function and a cost function, and performing path search by combining the environment topology model and multi-level map information;

[0009] Step 4: monitor environmental changes in real time, dynamically update the connectivity probability of each edge in the probability graph, and adaptively adjust the path search strategy according to the changes in the probability graph;

[0010] Step 5: Establish a backup path selection model based on reinforcement learning, and dynamically adjust the backup path based on environmental changes and primary path search.

[0011] Wherein, step 1 further comprises:

[0012] The robot is equipped with sensors to obtain three-dimensional spatial information of the environment, collect environmental feature data including obstacle location, surface material, and possibility of passage, and use Kalman filtering to fuse multiple sensor data to eliminate noise and deviation.

[0013] And use the SLAM algorithm based on simultaneous positioning and map construction to correct and optimize the map information;

[0014] The environmental topology model is constructed by constructing a probability graph G = (V, E), where V represents location nodes, each node represents a reachable location, E represents the edges connecting the nodes, each edge represents a path between two locations, and for each edge (i, j) ∈ E, the connectivity probability p(i, j) of each edge (i, j) represents the possibility of travel from location i to location j.

[0015] Wherein, the method further comprises:

[0016] Based on the real-time data from the sensor, the value of p(i, j) is updated using the Bayesian formula

[0017] p(i,j|new_data)=p(new_data|i,j)*p(i,j) / p(new_data)

[0018] Among them, p(i, j|new_data) represents the connectivity probability from position i to position j after obtaining new sensor data, p(new_data|i, j) represents the probability of observing new sensor data in the connectivity state from i to j, p(i, j) represents the connectivity probability from i to j in the absence of new data, p(new_data) is a normalization factor, which represents the probability of the newly acquired sensor data, and new_data represents the latest environmental information acquired by the robot sensor.

[0019] Wherein, step 2 further comprises:

[0020] A global map is constructed based on a quadtree hierarchical spatial data structure, where each node includes the following information: location coordinates, traffic status (including passable / obstacles), arrival cost (including distance, environmental complexity) and references to child nodes;

[0021] Build a local map based on the grid map, where each node includes the following information: precise location coordinates, traffic status (including passable / obstacles), detailed environmental characteristics (including material, resistance, etc.) and references to neighboring nodes;

[0022] Furthermore, the adaptive dynamic switching between the two levels according to the path planning task includes:

[0023] Initialization: Read the rough information of the global map and build the initial global path plan;

[0024] Path planning: Perform path refinement search based on the global path plan, build local high-resolution maps, and use local map information to optimize local paths to obtain smoother and safer paths;

[0025] Adaptive switching: Continuously monitor the current local path planning. If the local path planning encounters a bottleneck (such as being unable to bypass an obstacle), switch back to the global map and replan the global path. If the local path planning progresses smoothly, continue to refine the local map along the current path.

[0026] Dynamic update: During the path planning process, the information of the global map and local map is updated in real time, and the newly acquired sensor data (such as obstacle detection and environmental perception) is applied to the map to optimize the map's topological structure and cost function.

[0027] Wherein, step 3 further comprises:

[0028] Define a heuristic function h(i, j) that represents the estimated cost from node i to target node j. Use the following formula to calculate the heuristic function:

[0029]

[0030] Where (x i ,y i ) and (x j ,y j ) are the coordinates of nodes i and j respectively, and p(i, j) is the connectivity probability from node i to node j, which is calculated based on the environment topology model;

[0031] Define the cost function g(i, j) that represents the actual cost from the starting point to node i, and use the following formula to calculate the cost function:

[0032]

[0033] Where g(i, k) is the actual cost from the starting point to node k, d(k, j) is the distance from node k to node j, and p(k, j) is the connectivity probability from node k to node j;

[0034] Define an evaluation function f(i)=g(i,j)+w*h(i,j) representing the weighted sum of the heuristic function h(i,j) and the cost function g(i,j), where w is the weight of the heuristic function;

[0035] During the path search process, the node with the smallest evaluation function value is selected for expansion.

[0036] The path planning based on the bidirectional alternating search strategy includes the following steps:

[0037] (1) Initialize the open list, closed list, and parent list of the forward and backward searches, add the starting point to the open list of the forward search, and then add the target point to the open list of the backward search;

[0038] (2) Determine whether the open list of the forward search or the reverse search is empty; if not, proceed to step (3); if yes, it means that after searching all searchable nodes, if no forward and reverse search encounter occurs, the path search fails;

[0039] (3) Move the node with the smallest total cost F in the forward open list to the forward closed list and set it as the forward current node NF; move the node with the smallest total cost F in the backward open list to the backward closed list and set it as the backward node NB;

[0040] (4) Determine whether the forward current node NF and the backward node NB are satisfied; if so, it means that the forward and backward searches have been satisfied, and then output the path nodes according to the forward and backward parent lists; if not, perform forward search;

[0041] (5) Start forward search: Taking the backward node NB as the target, determine in turn whether the adjacent node Mi (i=1-8) of NF is an obstacle or already in the closed table; if it is an obstacle, ignore the adjacent node; if it is not an obstacle, update the closed table and the parent node of Mi; then update the forward open table and the forward current node NF;

[0042] (6) Start reverse search: take the current node NF as the target, and the search steps are the same as the forward search;

[0043] (7) Return to step (2) until the forward search and reverse search paths meet.

[0044] Wherein, step 4 further comprises:

[0045] Each time a node is expanded through path planning based on a bidirectional alternating search strategy, the connectivity probability of the relevant edges needs to be updated first, and the search strategy needs to be adjusted based on the updated probability graph;

[0046] Define a function p(i, j, t) that updates the connectivity probability of edge (i, j) at time t in the probability graph in real time according to environmental changes. It is expressed using the following formula:

[0047] p(i,j,t)=p(i,j,t-1)*(1-λ*f(i,j,t))+λ*f(i,j,t)

[0048] Where p(i, j, t-1) represents the connectivity probability of edge (i, j) at time t-1, f(i, j, t) represents the connectivity state of edge (i, j) at time t, and takes the value of 0 or 1, indicating whether the edge is connected, and λ is the weight factor, indicating the degree of influence of environmental change information on the connectivity probability;

[0049] The path search strategy is adaptively adjusted according to the changes in the probability graph, and the heuristic function h(i, j, t) and cost function g(i, j, t) are updated using the following formula:

[0050]

[0051] Where (x i ,y i ) and (x j ,y j ) represent the coordinates of nodes i and j respectively, p(i, j, t) and p(k, j, t) represent the connectivity probabilities of edges (i, j) and (k, j) at time t, and d(k, j) represents the distance from node k to node j;

[0052] Recalculate the evaluation function f(i, t) based on the updated heuristic function and cost function:

[0053] f(i,t)=g(i,j,t)+w*h(i,j,t)

[0054] Among them, w is the weight coefficient.

[0055] Wherein, step 5 further comprises:

[0056] Collect relevant data to build a reinforcement learning model, including:

[0057] Build state space S: current location, main path search status, environment variables;

[0058] Action space A: candidate set of backup paths;

[0059] The reward function R(s, a, s') is expressed using the following formula:

[0060] R(s, a, s')=w1*q(a)-w2*d(s, s')-w3*p(s, s')

[0061] Where q(a) represents the quality score of backup path a, d(s, s') represents the distance from the current state s to the next state s', p(s, s') represents the connectivity probability from the current state s to the next state s', and w1, w2, and w3 are weight factors;

[0062] The state transition equation is expressed by the following formula:

[0063] s'=f(s,a,e)

[0064] Among them, s' represents the state at the next moment, s represents the current state, a represents the action taken (alternative path selection), and e represents the environment variable;

[0065] Based on Q-learning, the policy network π(a|s) is trained, where the policy network outputs the probability distribution of alternative path selection, and the Q-learning update rule is the Q function expressed by the following formula:

[0066] Q=Q(s,a)+α*[R(s,a,s')+γ*max_a'Q(s',a')-Q(s,a)]

[0067] Among them, Q(s, a) represents the Q value of taking action a in state s, α is the learning rate, R(s, a, s') represents the immediate reward obtained after taking action a from state s and transferring to state s', γ is the discount factor, and max_a'Q(s', a') represents selecting action a' with the maximum Q value in state s';

[0068] During the main path search process, the environment changes and the main path search status are monitored in real time. According to the current state s, the trained strategy network π(a|s) is used to select the backup path. If there is a problem with the main path (such as encountering an obstacle or a decrease in the connectivity probability), the backup path is immediately switched.

[0069] The situation of switching to the backup path is recorded, including the state s during switching, the selected backup path a, and the actual effect. The strategy network π(a|s) is continuously optimized based on the recorded data.

[0070] The present invention improves the efficiency of searching the path by introducing a two-way alternating search strategy. By filtering the path nodes, the path duplication problem caused by the two-way search is overcome, the number of key nodes is reduced, the difficulty of the search is reduced, and the optimal path from the starting point to the end point can be found more quickly. In addition, by establishing a probabilistic graph model, the impact of environmental changes on the path can be dynamically evaluated, and the search strategy can be adaptively adjusted to improve the robustness of path planning in a dynamic environment. By adopting an adaptive hierarchical data structure, the granularity of the data structure can be dynamically adjusted according to the complexity of the environment, the search efficiency can be improved in a complex environment, and the computational overhead can be reduced. By training the backup path selection strategy through reinforcement learning, a suitable backup path can be dynamically selected according to environmental changes and the main path search status. At the same time, the model can also continue to learn and optimize to improve the quality of backup path selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0072] Figure 1 is a flow chart showing a robot path planning method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0075] It should be understood that although the terms first, second, third, etc. may be used to describe ... in the embodiments of the present invention, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of the present invention, the first ... may also be referred to as the second ..., and similarly, the second ... may also be referred to as the first ....

[0076] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0077] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0078] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0079] The present invention adopts numerical simulation technology, machine learning methods and parallel computing technology to establish a high-precision condenser model of the ship power system, and realizes intelligent monitoring and optimization of the power system operation through a visual analysis platform, thereby improving the efficiency and accuracy of simulation analysis of the condenser of the ship power system.

[0080] like Figure 1 As shown, the present invention discloses a robot path planning method, the method comprising:

[0081] Step 1: By building an environment topology model based on sensor data and map information, a probabilistic graph is used to represent the structure of the environment topology model, where each node represents a location and an edge represents a path connecting two locations.

[0082] The environmental topology model based on probability graph provides a powerful modeling foundation for robot path planning. It uses various sensors carried by the robot, such as lidar and cameras, to collect information such as the geometry and obstacles of the environment, and integrates these data to build an environmental model. It also combines the existing map data with the sensor data to further improve and refine the environmental model. This environmental modeling method has good scalability and can adapt to more complex environments and more sensor data.

[0083] In one embodiment, the robot is equipped with sensors to obtain three-dimensional spatial information of the environment, collect environmental feature data including obstacle locations, surface materials, and passability, and use Kalman filtering to fuse multiple sensor data to eliminate noise and deviation.

[0084] The map information is corrected and optimized using the SLAM algorithm based on simultaneous positioning and map construction.

[0085] The environmental topology model is constructed by constructing a probability graph G = (V, E), where V represents location nodes, each node represents a reachable location, E represents the edges connecting the nodes, each edge represents a path between two locations, and for each edge (i, j) ∈ E, the connectivity probability p(i, j) of each edge (i, j) represents the possibility of travel from location i to location j.

[0086] In one embodiment, based on the real-time data obtained from the sensor, the value of p(i, j) is updated by the Bayesian formula.

[0087] p(i,j|new_data)=p(new_data|i,j)*p(i,j) / p(new_data)

[0088] Among them, p(i, j|new_data) represents the connectivity probability from position i to position j after obtaining new sensor data, p(new_data|i, j) represents the probability of observing new sensor data in the connectivity state from i to j, p(i, j) represents the connectivity probability from i to j in the absence of new data, p(new_data) is a normalization factor, which represents the probability of the newly acquired sensor data, and new_data represents the latest environmental information acquired by the robot sensor.

[0089] Step 2: Establish a multi-level adaptive map data structure. The coarse level uses a low-resolution map for global planning, and the detailed level uses a high-resolution map for local optimization. The two levels are adaptively and dynamically switched according to the path planning task.

[0090] Path planning based on a multi-level adaptive map data structure fully utilizes the advantages of maps with different resolutions, which can improve computational efficiency and path quality. Using low-resolution maps for global planning has low computational overhead and can quickly obtain a preliminary path. Only when necessary, switch to high-resolution maps for local optimization, avoiding the waste of computational over-refinement of the global map.

[0091] Detailed level maps can better capture environmental details, avoid obstacles, and find smoother and safer local paths. The combination of global planning and local optimization can obtain a better overall path. The adaptive switching mechanism can dynamically adjust according to task requirements, adapt to various complex environments and path planning requirements, better cope with dynamic environmental changes, and improve the robustness of path planning.

[0092] In one embodiment, a global map is constructed based on a quadtree hierarchical spatial data structure, wherein each node includes the following information: location coordinates, traffic status (including passable / obstacles), arrival cost (including distance, environment complexity) and references to child nodes;

[0093] Build a local map based on the grid map, where each node includes the following information: precise location coordinates, traffic status (including passable / obstacles), detailed environmental characteristics (including material, resistance, etc.) and references to neighboring nodes;

[0094] Furthermore, the adaptive dynamic switching between the two levels according to the path planning task includes:

[0095] Initialization: Read the rough information of the global map and build the initial global path plan;

[0096] Path planning: Perform path refinement search based on the global path plan, build local high-resolution maps, and use local map information to optimize local paths to obtain smoother and safer paths;

[0097] Adaptive switching: Continuously monitor the current local path planning. If the local path planning encounters a bottleneck (such as being unable to bypass an obstacle), switch back to the global map and replan the global path. If the local path planning progresses smoothly, continue to refine the local map along the current path.

[0098] Dynamic update: During the path planning process, the information of the global map and local map is updated in real time, and the newly acquired sensor data (such as obstacle detection and environmental perception) is applied to the map to optimize the map's topological structure and cost function.

[0099] Step 3: construct a bidirectional alternating search strategy, including defining a heuristic function and a cost function, and performing path search by combining the environment topology model and multi-level map information.

[0100] Traditional path planning strategies simply take the length of the geometric path as the actual cost, consume a lot of memory, take a long time to calculate, search for redundant nodes, and regard the mobile robot as a mass node, without considering the size of the robot itself, nor the actual situation of the mobile robot's movement and posture adjustment. The present invention solves the problem of long search time through a two-way search strategy.

[0101] In one embodiment, a heuristic function h(i, j) representing the estimated cost from node i to target node j is defined, and the heuristic function is calculated using the following formula:

[0102]

[0103] Where (x i ,y i ) and (x j ,y j ) are the coordinates of nodes i and j respectively, and p(i, j) is the connectivity probability from node i to node j, which is calculated based on the environment topology model;

[0104] Define the cost function g(i, j) that represents the actual cost from the starting point to node i, and use the following formula to calculate the cost function:

[0105]

[0106] Where g(i, k) is the actual cost from the starting point to node k, d(k, j) is the distance from node k to node j, and p(k, j) is the connectivity probability from node k to node j;

[0107] Define an evaluation function f(i)=g(i,j)+w*h(i,j) representing the weighted sum of the heuristic function h(i,j) and the cost function g(i,j), where w is the weight of the heuristic function;

[0108] During the path search process, the node with the smallest evaluation function value is selected for expansion.

[0109] In one embodiment, path planning is performed based on a bidirectional alternating search strategy, including the following steps:

[0110] (1) Initialize the open list, closed list, and parent list of the forward and backward searches, add the starting point to the open list of the forward search, and then add the target point to the open list of the backward search;

[0111] (2) Determine whether the open list of the forward search or the reverse search is empty; if not, proceed to step (3); if yes, it means that after searching all searchable nodes, if no forward and reverse search encounter occurs, the path search fails;

[0112] (3) Move the node with the smallest total cost F in the forward open list to the forward closed list and set it as the forward current node NF; move the node with the smallest total cost F in the backward open list to the backward closed list and set it as the backward node NB;

[0113] (4) Determine whether the forward current node NF and the backward node NB are satisfied; if so, it means that the forward and backward searches have been satisfied, and then output the path nodes according to the forward and backward parent lists; if not, perform forward search;

[0114] (5) Start forward search: Taking the backward node NB as the target, determine in turn whether the adjacent node Mi (i=1-8) of NF is an obstacle or already in the closed table; if it is an obstacle, ignore the adjacent node; if it is not an obstacle, update the closed table and the parent node of Mi; then update the forward open table and the forward current node NF;

[0115] (6) Start reverse search: take the current node NF as the target, and the search steps are the same as the forward search;

[0116] (7) Return to step (2) until the forward search and reverse search paths meet.

[0117] Step 4: Monitor environmental changes in real time, dynamically update the connectivity probability of each edge in the probability graph, and adaptively adjust the path search strategy according to the changes in the probability graph.

[0118] Based on the dynamic update of probability graph and adaptive path search strategy, it can effectively cope with complex dynamic environments, improve the real-time, robustness and computational efficiency of path planning, and provide strong support for various robot navigation and path planning scenarios.

[0119] In a certain embodiment, each time a node is expanded through path planning based on a bidirectional alternating search strategy, the connectivity probability of the relevant edges needs to be updated first, and the search strategy needs to be adjusted according to the updated probability graph;

[0120] Define a function p(i, j, t) that updates the connectivity probability of edge (i, j) at time t in the probability graph in real time according to environmental changes. It is expressed using the following formula:

[0121] p(i,j,t)=p(i,j,t-1)*(1-λ*f(i,j,t))+λ*f(i,j,t)

[0122] Where p(i, j, t-1) represents the connectivity probability of edge (i, j) at time t-1, f(i, j, t) represents the connectivity state of edge (i, j) at time t, and takes the value of 0 or 1, indicating whether the edge is connected, and λ is the weight factor, indicating the degree of influence of environmental change information on the connectivity probability;

[0123] The path search strategy is adaptively adjusted according to the changes in the probability graph, and the heuristic function h(i, j, t) and cost function g(i, j, t) are updated using the following formula:

[0124]

[0125] Where (x i ,y i ) and (x j ,y j ) represent the coordinates of nodes i and j respectively, p(i, j, t) and p(k, j, t) represent the connectivity probabilities of edges (i, j) and (k, j) at time t, and d(k, j) represents the distance from node k to node j;

[0126] Recalculate the evaluation function f(i, t) based on the updated heuristic function and cost function:

[0127] f(i,t)=g(i,j,t)+w*h(i,j,t)

[0128] Among them, w is the weight coefficient.

[0129] Step 5: Establish a backup path selection model based on reinforcement learning, and dynamically adjust the backup path based on environmental changes and primary path search.

[0130] The backup path selection model based on reinforcement learning can quickly respond to environmental changes, dynamically adjust the backup paths, improve the robustness of path planning and the interpretability of decisions, which is very important for robot navigation and path planning scenarios that need to cope with complex dynamic environments.

[0131] In one embodiment, collecting relevant data to establish a reinforcement learning model includes:

[0132] Build state space S: current location, main path search status, environment variables;

[0133] Action space A: candidate set of backup paths;

[0134] The reward function R(s, a, s') is expressed using the following formula:

[0135] R(s, a, s')=w1*q(a)-w2*d(s, s')-w3*p(s, s')

[0136] Where q(a) represents the quality score of backup path a, d(s, s') represents the distance from the current state s to the next state s', p(s, s') represents the connectivity probability from the current state s to the next state s', and w1, w2, and w3 are weight factors;

[0137] The state transition equation is expressed by the following formula:

[0138] s'=f(s,a,e)

[0139] Among them, s' represents the state at the next moment, s represents the current state, a represents the action taken (alternative path selection), and e represents the environment variable;

[0140] Based on Q-learning, the policy network π(a|s) is trained, where the policy network outputs the probability distribution of alternative path selection, and the Q-learning update rule is the Q function expressed by the following formula:

[0141] Q=Q(s,a)+α*[R(s,a,s')+γ*max_a'Q(s',a')-Q(s,a)]

[0142] Among them, Q(s, a) represents the Q value of taking action a in state s, α is the learning rate, R(s, a, s') represents the immediate reward obtained after taking action a from state s and transferring to state s', γ is the discount factor, and max_a'Q(s', a') represents selecting action a' with the maximum Q value in state s';

[0143] During the main path search process, the environment changes and the main path search status are monitored in real time. According to the current state s, the trained strategy network π(a|s) is used to select the backup path. If there is a problem with the main path (such as encountering an obstacle or a decrease in the connectivity probability), the backup path is immediately switched.

[0144] The situation of switching to the backup path is recorded, including the state s during switching, the selected backup path a, and the actual effect. The strategy network π(a|s) is continuously optimized based on the recorded data.

[0145] The present invention improves the efficiency of searching the path by introducing a two-way alternating search strategy. By filtering the path nodes, the path duplication problem caused by the two-way search is overcome, the number of key nodes is reduced, the difficulty of the search is reduced, and the optimal path from the starting point to the end point can be found more quickly. In addition, by establishing a probabilistic graph model, the impact of environmental changes on the path can be dynamically evaluated, and the search strategy can be adaptively adjusted to improve the robustness of path planning in a dynamic environment. By adopting an adaptive hierarchical data structure, the granularity of the data structure can be dynamically adjusted according to the complexity of the environment, the search efficiency can be improved in a complex environment, and the computational overhead can be reduced. By training the backup path selection strategy through reinforcement learning, a suitable backup path can be dynamically selected according to environmental changes and the main path search status. At the same time, the model can also continue to learn and optimize to improve the quality of backup path selection.

[0146] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0147] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0148] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0149] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0150] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0151] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, but is not intended to limit the present invention. All modifications, substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A robot path planning method, comprising: Step 1, by building an environment topology model based on sensor data and map information, a probability graph is used to represent the structure of the environment topology model, where each node represents a location and an edge represents a path connecting two locations; Step 2: Establish a multi-level adaptive map data structure. The coarse level uses a low-resolution map for global planning, and the detailed level uses a high-resolution map for local optimization. The two levels are adaptively and dynamically switched according to the path planning task. Step 3, construct a bidirectional alternating search strategy, including defining a heuristic function and a cost function, and performing path search by combining the environment topology model and multi-level map information; Step 4: monitor environmental changes in real time, dynamically update the connectivity probability of each edge in the probability graph, and adaptively adjust the path search strategy according to the changes in the probability graph; Step 5: Establish a backup path selection model based on reinforcement learning, and dynamically adjust the backup path based on environmental changes and primary path search.

2. A robot path planning method as claimed in claim 1, characterized in that: The step 1 specifically includes: The robot uses sensors to obtain three-dimensional spatial information of the environment, collect environmental feature data including obstacle location, surface material, and possibility of passage, and uses Kalman filtering to fuse multiple sensor data to eliminate noise and deviation; And use the SLAM algorithm based on simultaneous positioning and map construction to correct and optimize the map information; The environmental topology model is constructed by constructing a probability graph G = (V, E), where V represents location nodes, each node represents a reachable location, E represents the edges connecting the nodes, each edge represents a path between two locations, and for each edge (i, j) ∈ E, the connectivity probability p(i, j) of each edge (i, j) represents the possibility of travel from location i to location j.

3. A robot path planning method as claimed in claim 2, characterized in that: Also includes: Based on the real-time data from the sensor, the value of p(i, j) is updated using the Bayesian formula p(i,j|new_data)=p(new_data|i,j)*p(i,j) / p(new_data) Among them, p(i, j|new_data) represents the connectivity probability from position i to position j after obtaining new sensor data, p(new_data|i, j) represents the probability of observing new sensor data in the connectivity state from i to j, p(i, j) represents the connectivity probability from i to j in the absence of new data, p(new_data) is a normalization factor, which represents the probability of the newly acquired sensor data, and new_data represents the latest environmental information acquired by the robot sensor.

4. A robot path planning method as claimed in claim 1, characterized in that: Step 2 specifically includes: A global map is constructed based on a quadtree hierarchical spatial data structure, where each node includes the following information: location coordinates, traffic status, arrival cost, and references to child nodes; Construct a local map based on the grid map, where each node includes the following information: precise location coordinates, traffic status, detailed environmental characteristics and references to neighboring nodes; Furthermore, the adaptive dynamic switching between the two levels according to the path planning task includes: Initialization: Read the rough information of the global map and build the initial global path plan; Path planning: Perform path refinement search based on the global path plan, build local high-resolution maps, and use local map information to optimize local paths to obtain smoother and safer paths; Adaptive switching: Continuously monitor the status of the currently executed local path planning. If the local path planning encounters a bottleneck, switch back to the global map and replan the global path. If the local path planning progresses smoothly, continue to refine the local map along the current path. Dynamic update: During the path planning process, the global map and local map information are updated in real time, the newly acquired sensor data is applied to the map, and the map's topological structure and cost function are optimized.

5. A robot path planning method as claimed in claim 1, characterized in that: Step 3 specifically includes: Define a heuristic function h(i, j) that represents the estimated cost from node i to target node j. Use the following formula to calculate the heuristic function: Where (x i ,y i ) and (x j ,y j ) are the coordinates of nodes i and j respectively, and p(i, j) is the connectivity probability from node i to node j, which is calculated based on the environment topology model; Define the cost function g(i, j) that represents the actual cost from the starting point to node i, and use the following formula to calculate the cost function: Where g(i, k) is the actual cost from the starting point to node k, d(k, j) is the distance from node k to node j, and p(k, j) is the connectivity probability from node k to node j; Define an evaluation function f(i)=g(i,j)+w*h(i,j) representing the weighted sum of the heuristic function h(i,j) and the cost function g(i,j), where w is the weight of the heuristic function; During the path search process, the node with the minimum evaluation function value is selected for expansion.

6. A robot path planning method as claimed in claim 5, characterized in that: Path planning based on the bidirectional alternating search strategy includes the following steps: (1) Initialize the open list, closed list, and parent list of the forward and backward searches, add the starting point to the open list of the forward search, and then add the target point to the open list of the backward search; (2) Determine whether the open list of the forward search or the reverse search is empty; if not, proceed to step (3); if yes, it means that after searching all searchable nodes, if no forward and reverse search encounter occurs, the path search fails; (3) Move the node with the smallest total cost F in the forward open list to the forward closed list and set it as the forward current node NF; move the node with the smallest total cost F in the backward open list to the backward closed list and set it as the backward node NB; (4) Determine whether the forward current node NF and the backward node NB are satisfied; if so, it means that the forward and backward searches have been satisfied, and then output the path nodes according to the forward and backward parent lists; if not, perform forward search; (5) Start forward search: Taking the backward node NB as the target, determine in turn whether the adjacent node Mi (i=1-8) of NF is an obstacle or already in the closed table; if it is an obstacle, ignore the adjacent node; if it is not an obstacle, update the closed table and the parent node of Mi; then update the forward open table and the forward current node NF; (6) Start reverse search: take the current node NF as the target, and the search steps are the same as the forward search; (7) Return to step (2) until the forward search and reverse search paths meet.

7. A robot path planning method as claimed in claim 1, characterized in that: Step 4 specifically includes: Each time a node is expanded through path planning based on a bidirectional alternating search strategy, the connectivity probability of the relevant edges needs to be updated first, and the search strategy needs to be adjusted based on the updated probability graph; Define a function p(i, j, t) that updates the connectivity probability of edge (i, j) at time t in the probability graph in real time according to environmental changes. It is expressed using the following formula: p(i,j,t)=p(i,j,t-1)*(1-λ*f(i,j,t))+λ*f(i,j,t) Where p(i, j, t-1) represents the connectivity probability of edge (i, j) at time t-1, f(i, j, t) represents the connectivity state of edge (i, j) at time t, and takes the value of 0 or 1, indicating whether the edge is connected, and λ is the weight factor, indicating the degree of influence of environmental change information on the connectivity probability; The path search strategy is adaptively adjusted according to the changes in the probability graph, and the heuristic function h(i, j, t) and cost function g(i, j, t) are updated using the following formula: Where (x i ,y i ) and (x j ,y j ) represent the coordinates of nodes i and j respectively, p(i, j, t) and p(k, j, t) represent the connectivity probabilities of edges (i, j) and (k, j) at time t, and d(k, j) represents the distance from node k to node j; Recalculate the evaluation function f(i, t) based on the updated heuristic function and cost function: f(i,t)=g(i,j,t)+w*h(i,j,t) Among them, w is the weight coefficient.

8. A robot path planning method as claimed in claim 1, characterized in that: Step 5 specifically includes: Collect relevant data to build a reinforcement learning model, including: Build state space S: current location, main path search status, environment variables; Action space A: candidate set of backup paths; The reward function R(s, a, s') is expressed using the following formula: R(s, a, s')=w1*q(a)-w2*d(s, s')-w3*p(s, s') Where q(a) represents the quality score of backup path a, d(s, s') represents the distance from the current state s to the next state s', p(s, s') represents the connectivity probability from the current state s to the next state s', and w1, w2, and w3 are weight factors; The state transition equation is expressed by the following formula: s'=f(s,a,e) Among them, s' represents the state at the next moment, s represents the current state, a represents the action taken, and e represents the environment variable; Based on Q-learning, the policy network π(a|s) is trained, where the policy network outputs the probability distribution of alternative path selection, and the Q-learning update rule is the Q function expressed by the following formula: Q=Q(s,a)+α*[R(s,a,s')+γ*max_a'Q(s',a')-Q(s,a)] Among them, Q(s, a) represents the Q value of taking action a in state s, α is the learning rate, R(s, a, s') represents the immediate reward obtained after taking action a from state s and transferring to state s', γ is the discount factor, and max_a'Q(s', a') represents selecting action a' with the maximum Q value in state s'; During the main path search process, the environment changes and the main path search status are monitored in real time. According to the current state s, the trained strategy network π(a|s) is used to select the backup path. If there is a problem with the main path, it is immediately switched to the backup path. The situation of switching to the backup path is recorded, including the state s during switching, the selected backup path a, and the actual effect. The strategy network π(a|s) is continuously optimized based on the recorded data.

9. A robot path planning system, comprising: at least one processor; as well as at least one memory including computer program code, The at least one memory and the computer program code are configured to, together with the at least one processor, enable the apparatus to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 8 when executed by a processor.

Images