Robot automatic mapping method in unknown open environment and related equipment

Abstract

The invention discloses a robot automatic mapping method and related equipment in an unknown open environment, and the method comprises the steps: obtaining sensor data, and constructing a local map according to the sensor data; topologizing the local map to obtain a collision-free topological graph of the local map; performing sparse processing on the collision-free topological graph to obtain a sparse topological graph; and inputting the obtained sparse topological graph into a strategy network model based on deep reinforcement learning, and outputting a strategy for controlling the robot to move so as to move the robot. According to the method, the environment mapping problem is modeled into a graph structure, spatial dependency in the environment is modeled through the relation between the nodes, and effective learning of the robot on the relation between different areas in the complex environment is achieved. In addition, the attention mechanism helps the robot to preferentially select the area with the high potential return in the autonomous exploration process by dynamically adjusting the attention weight of each area, and therefore the exploration efficiency is improved. The robot can be widely applied to the technical field of robots.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot automatic mapping method in an unknown open environment and related equipment. BACKGROUND

[0002] In recent years, robots are required to perform more and more diverse tasks, and the types of robots are also increasingly diverse, such as cleaning robots, inspection robots, food delivery robots, search and rescue robots, etc. As a mainstream environment expression form, point cloud map provides basic positioning function for robots and is the basis for downstream planning, obstacle avoidance, etc. At present, the construction process of point cloud map is mainly based on manual remote control of robots to gradually traverse the environment, and the overall process is insufficient in automation, and subsequent updating of the point cloud map often requires secondary intervention of manual operation. Therefore, how to improve the autonomy and efficiency of robot mapping has gradually become an important research topic. Autonomous exploration technology refers to the process in which a robot models the local environment through sensor data in a completely unknown environment, and then plans and traverses the entire environment to complete the mapping task.

[0003] The existing traditional active SLAM technology, such as boundary-based method and sampling-based method, is greatly affected by its greedy strategy. When applied to large-scale, open and complex environments, the mapping efficiency is low, and even repeated back-and-forth phenomenon may occur due to unreasonable benefit function design, resulting in mapping failure. With the continuous development of the field of artificial intelligence in recent years, methods based on deep reinforcement learning have been proposed and applied to autonomous exploration tasks. Due to its ability to utilize experience obtained from historical interactions between agents and environments and its independence from human-designed rules, it can significantly improve the performance of agents in autonomous exploration tasks in unknown open environments. However, traditional network design mainly uses CNN architecture, which requires a fixed input to the network. Therefore, its generalization to different scales of environments is poor. This leads to instability in practical applications of this method, which may result in incomplete exploration or exploration failure. SUMMARY

[0004] To at least partially solve one of the technical problems existing in the prior art, the purpose of the present application is to provide a robot automatic mapping method in an unknown open environment and related equipment.

[0005] The first technical solution adopted by the present application is:

[0006] A robot automatic mapping method in an unknown open environment, comprising the following steps:

[0007] Obtaining sensor data and constructing a local map based on the sensor data;

[0008] topology of the local map;

[0009] performing sparse processing on the collision-free topology to obtain a sparse topology;

[0010] inputting the obtained sparse topology into a policy network model based on deep reinforcement learning to output a policy for controlling the motion of the robot, so as to move the robot;

[0011] The policy network model comprises an encoder and a decoder; the encoder is configured to extract features of the input sparse topology to obtain features of the topology, which are used in the learning process of the model on the spatial dependency between nodes in the topology; and the decoder is configured to output the policy according to the features of the topology.

[0012] Further, the obtaining of the sensor data and the construction of the local map according to the sensor data comprise:

[0013] The sensor data o is obtained by a laser sensor arranged on the robot, and the sensor data o is converted into the local map by SLAM technology.

[0014] Further, the topology of the local map comprises:

[0015] The local map is represented in the form of an x x y grid, the center coordinates (x i ,y i ) of each grid are obtained, and the local map is represented as a set of points P:

[0016]

[0017] According to different categories, the map is divided into known space M k and unknown space M u ; the known space M k is further divided into free space M f and occupied space M o ; the grid at the junction of the known space M k and the unknown space M u is defined as the boundary in the autonomous exploration task, and is represented by a set F:

[0018]

[0019] The collision-free topology is extracted from the local map to obtain a series of collision-free nodes and edges, and the node set is represented by a set V and the collision-free edge set is represented by a set E:

[0020]

[0021] Finally, a collision-free topology map G=(V, E) of the local map is obtained.

[0022] Further, the nodes v i in the collision-free topology map are divided into two categories: boundary nodes and non-boundary nodes. i i The boundary nodes are nodes that have been visited by the robot, and the non-boundary nodes are nodes that have not been visited by the robot. i i The boundary nodes have a value b i , and the non-boundary nodes have a value b i .

[0023] The value u i of the node v j is calculated as follows:

[0024]

[0025] where f k is the boundary observed by the robot at the current node.

[0026] Further, the collision-free topology map is sparsified to obtain a sparse topology map, including:

[0027] In the collision-free topology map, nodes with a value u i greater than a preset threshold are grouped according to adjacency position relationships to obtain a boundary cluster set.

[0028] An A* algorithm is used to search for paths from the current robot position to each boundary cluster, and nodes are sampled on these paths to obtain path nodes.

[0029] The boundary cluster nodes and the path nodes are retained, and the remaining nodes with a value u i of 0 are removed to obtain a sparse topology map.

[0030] Further, the encoder is composed of two layers of multi-head graph attention networks (GAT) and a cross-layer residual connection mechanism.

[0031] The GAT is used to realize the aggregation of node features and adjacent node features, so that the model can effectively learn the local topology relationship between nodes.

[0032] The cross-layer residual connection mechanism is used to enhance the gradient propagation and feature preservation ability.

[0033] For each node in the topology map, the feature update process of the GAT is as follows:

[0034]

[0035] where K is the number of attention heads in the multi-head graph attention mechanism, N(i) is the set of neighbor nodes of the node v i , h i is the feature vector of the neighbor nodes, and W i is the weight matrix.is the learnable linear transformation matrix for the kth head, is the node v in the kth head j is the attention weight of the node v i , and σ is the ReLU activation function.

[0036] Further, the multi-head graph attention mechanism is composed of six standard multi-head attention components, which receive a mask mask of the local topological graph relative to the global graph when performing feature transmission, so as to limit the attention mechanism to only focus on the part of the global graph that has been observed.

[0037] Further, the decoder is composed of a pointer network based on a multi-head attention layer, which outputs an action probability with the obtained attention weight, and the overall feature updating process is as follows:

[0038]

[0039] In the formula, the policy π θ is a learnable decoder output, which represents the probability of the agent making an action a t at time step t under the observation o t , where j represents a node in the neighbor node set of the robot, which will be the navigation point of the robot at the next moment; softmax j represents the probability of selecting the neighbor node j calculated by the softmax function; W Q is the learnable weight matrix of the query (Query); W K is the learnable weight matrix of the key (Key) vector; h c is the feature vector of the current node; is the feature vector of the neighbor node j; d is the dimension of the feature vector; and T is the transpose.

[0040] Further, the robot automatic mapping method further comprises an optimization step:

[0041] A soft actor-critic algorithm (SAC) is used to continuously optimize the policy π θ ; wherein the optimization objective of the SAC is:

[0042]

[0043] In the formula, π * represents the optimal policy; r t represents the immediate reward of the agent; α is a temperature parameter; represents the entropy of the policy; γ t represents the product of the discount factor γ at time t; and T is the maximum value of the decision time step.

[0044] The second technical solution adopted by the present application is:

[0045] An electronic device, comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the robot automatic mapping method in an unknown open environment as described above.

[0046] The third technical solution adopted by the present application is:

[0047] A computer-readable storage medium, the storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement the robot automatic mapping method in an unknown open environment as described above.

[0048] The fourth technical solution adopted by the present application is:

[0049] A computer program product or computer program, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to make the computer device execute the robot automatic mapping method in an unknown open environment as described above.

[0050] The beneficial effects of the present application are: the present application models the environment mapping problem as a graph structure, models the spatial dependence in the environment through the relationship between nodes, and realizes the effective learning of the relationship between different regions of the robot in the complex environment. In addition, the attention mechanism helps the robot to preferentially select the region with higher potential return in the autonomous exploration process by dynamically adjusting the attention weight of each region, thereby accelerating the exploration efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments in the technical solutions of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0052] Figure 1 is the flow chart of the automatic mapping method based on deep reinforcement learning in the embodiments of the present application;

[0053] Figure 2is an architecture diagram of an encoder-decoder of a policy network in an embodiment of the application;

[0054] Figure 3 is a step flow chart of a robot automatic mapping method in an unknown open environment in an embodiment of the application. DETAILED DESCRIPTION

[0055] Embodiments of the present application are described in detail below with reference to examples shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation on the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0056] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise explicitly limited, the words such as arrangement, installation, connection, etc. should be broadly understood, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0057] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0058] In the description of the present application, the meaning of several is one or more, and the meaning of multiple is more than two, greater than, less than, more than, etc. are not included in the number, and above, below, etc. are understood to include the number. If it is described as first, second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of indicated technical features.

[0059] In the description of the present application, the association relationship of the associated objects is described as "and / or", which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents a "or" relationship between the front and rear associated objects.

[0060] Explanation of terms:

[0061] Laser SLAM (Simultaneous Localization And Mapping) is a robotics perception technology used to solve the problem of robots simultaneously localizing and mapping in unknown environments.

[0062] Boundary-based autonomous exploration methods: This method was one of the earliest approaches to solving autonomous exploration problems. Its core idea is to evaluate and judge the robot's path to the next moment based on all boundary locations (i.e., the intersection of known and unknown areas) within a set of boundaries. In the pioneering work of this type of method, the robot only selects the nearest boundary as the target point. Although this idea is easy to understand, its limitations make boundary-based algorithms prone to greed. Therefore, when there are too many boundaries and the benefits of the boundaries are similar, such algorithms are prone to meaningless round trips. This is particularly evident in large-scale, open, and complex environments.

[0063] Sampling-based autonomous exploration methods generate a set of viewpoints by performing RRT or RRG sampling in free space. The optimal viewpoint is then selected for navigation based on a pre-designed benefit function, thereby maximizing observational information about the unknown environment. The benefit function here is often related to the number of boundaries surrounding the viewpoint and the change in the Shannon entropy of the entire environment after reaching the viewpoint. This method avoids calculating all boundaries and remains stable even when there are a large number of boundaries. However, it often performs poorly in environments where sampling is difficult. Like boundary-based methods, both rely on a greedy strategy, which can easily lead to the algorithm becoming trapped in local optima during exploration, resulting in inefficient exploration.

[0064] Autonomous exploration algorithms based on deep reinforcement learning: This approach designs the interaction process between the agent and the environment, guiding the agent to update its action selection strategy through a reasonable reward function until it learns the optimal strategy. This approach enables the agent to obtain longer-term rewards and, to a certain extent, avoids greed. Early related methods primarily used CNN architectures for network design. However, because CNNs require fixed network inputs, they generally generalize poorly to environments of varying scales. This makes these methods extremely unstable in practical applications, prone to incomplete or failed exploration.

[0065] Based on this, the application proposes a new autonomous exploration algorithm based on deep reinforcement learning, which introduces a graph neural network and an attention mechanism in network design. The structural advantages of the graph neural network are used to model the environment mapping problem as a graph structure, and the spatial dependence in the environment is modeled through the relationship between nodes, realizing the effective learning of the relationship between different regions by the robot in the complex environment. At the same time, the attention mechanism dynamically adjusts the attention weight of each region, helping the robot to preferentially select the region with higher potential return in the autonomous exploration process, thereby accelerating the exploration efficiency.

[0066] Specifically, the application provides a mobile robot autonomous exploration method in an unknown open environment. The method first models the environment as a graph structure using an efficient graph representation method, and uses the information transmission characteristics of the graph neural network based on the topological structure to enable the model to capture the topological features of the space while focusing on the features of the boundary region. The attention mechanism dynamically adjusts the exploration focus of different regions in the locally observed map according to the current state of the robot, which is beneficial to achieve a balance between the use of known information and the exploration of unknown information in the exploration task in an open environment. On the other hand, the strategy neural network of the application is trained and optimized by continuously interacting with the environment, which gives the robot the ability to infer the spatial structure of unknown regions based on known regions. At the same time, the model-free reinforcement learning method SAC adopted by the robot can analyze the impact of action strategies on future environmental states, which helps to solve the problem of short-sightedness in time. This overall process can enable the robot to efficiently search for unknown open environments and model unknown environments.

[0067] Embodiment 1

[0068] As shown in Figure 1 and Figure 3 , the embodiment provides a robot automatic mapping method in an unknown open environment, which solves the problems of incomplete local exploration and repeated exploration of known regions in the existing robot autonomous exploration method, resulting in low exploration efficiency. The method specifically includes the following steps:

[0069] S1, acquire sensor data and construct a local map according to the sensor data.

[0070] S2, topologize the local map to obtain a collision-free topological graph of the local map.

[0071] S3, perform sparse processing on the collision-free topological graph to obtain a sparse topological graph.

[0072] S4. Input the obtained sparse topology map into a policy network model based on deep reinforcement learning, and output a policy to control the robot's motion to move the robot. The policy network model includes an encoder and a decoder. The encoder extracts features from the input sparse topology map to obtain the topology map's features, which are used by the model to learn the spatial dependencies between nodes in the topology map. The decoder outputs a policy based on the topology map's features.

[0073] The above method is described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] (1) Local map topology

[0075] A robot autonomous exploration task refers to a process in which a robot, in an unknown environment ε, converts the observed sensor data o into a local map m through SLAM technology, and then performs path planning to gradually expand the local map m to the complete map M.

[0076] In this embodiment, the laser SLAM framework is used as the overall mapping method. A complete bounded two-dimensional map M can be represented as an x×y grid. The center of each grid is used as the coordinate, and the map can be represented as a set of points P as follows:

[0077]

[0078] According to different categories, M can also be divided into known spaces M k and unknown space M u , where M k It can be further divided into free space M f and occupies space M o .

[0079] Known Space M k and unknown space M u The grid at the junction of is defined as the boundary in the autonomous exploration task, represented by the set F as follows:

[0080]

[0081] According to the grid representation method of the global map mentioned above, it can be expressed in the local map M f Extract the collision-free topology graph in the current free space M f Perform uniform sampling to obtain a set of collision-free nodes and edges. Set V represents the node set, and set E represents the collision-free edge set, as follows:

[0082]

[0083] At this point, a collision-free topology graph G = (V, E) of the known local map is obtained.

[0084] For a node v i , its feature h i contains four dimensions, namely the coordinates (x i , y i ) of the node in M, the benefit value u i of the node, and the landmark value b i whether the node has been visited.

[0085] The calculation method of the benefit value is as follows:

[0086]

[0087] where f i is the boundary that can be observed by the robot at the current node, and the meaning of the benefit value is the ratio of the number of boundaries that can be observed by the robot at the current node position to the total number of boundaries, which reflects the importance of the node for exploring unknown areas to some extent.

[0088] (2) Sparse processing

[0089] Since the uniform sampling method may cause excessive algorithm calculation load in large-scale and open maps, the original topology graph is sparsified in this embodiment. The main method is to first group the nodes with a benefit value greater than a threshold according to the adjacency position relationship to obtain a boundary cluster set, then use the A* algorithm to search the path from the current robot position to each boundary cluster, and then sample the nodes on these paths to obtain path nodes. Finally, the boundary cluster nodes and the path nodes are retained, and the remaining nodes with a benefit value of 0 are removed, thereby obtaining a sparse topology graph G*. Although the sparse graph makes the nodes more sparse, the information it contains is still dense. Reducing the calculation load will hardly lose much feature information when the graph neural network is transmitted.

[0090] Finally, the local map observed by the agent after each action is converted into a sparse topology graph, and the feature of a single node and the input of the reinforcement learning network are respectively:

[0091] h i = (x i , y i , u i , b i ) (6)

[0092] h IN = (h1,..., h n ) (7)

[0093] (3) Strategy network model of deep reinforcement learning

[0094] As shown in Figure 2 , the policy network of deep reinforcement learning mainly consists of an encoder and a decoder. The encoder proposed in this embodiment mainly consists of a graph neural network component and a self-attention component, and the decoder mainly consists of a pointer network based on a multi-head attention layer. The actor and critic networks related to the reinforcement learning algorithm are derived from this structure.

[0095] The main function of the encoder is to model the topological graph input at the feature level to obtain the features of the topological graph, which will be used in the model's learning process of the spatial dependency between nodes in the topological graph.

[0096] The graph neural network component mentioned above mainly consists of two layers of multi-head graph attention network GAT and cross-layer residual connection mechanism. GAT is used to realize the aggregation of node and its adjacent node features. This feature aggregation method can enable the model to effectively learn the local topological relationship between nodes, which is conducive to the agent's understanding of the regional spatial characteristics of the current local map, thereby better assisting exploration. Due to the node transmission characteristics of the graph neural network, the nodes of the overall topological graph are prone to over-smoothing, so the number of layers should not be set too high. Through practice, this embodiment sets GAT to two layers, and the input of the second layer is the output of the first layer. The cross-layer residual connection mechanism is conducive to enhancing the gradient propagation and feature preservation capability. For each node in the topological graph, the feature update process of its GAT is as follows:

[0097]

[0098] where K is the number of attention heads in the multi-head graph attention mechanism, N(i) is the neighbor node set of node v i , h j is the feature vector of the neighbor node, W k is the learnable linear transformation matrix of the kth head, is the attention weight of node v j on node v i , and σ is the ReLU activation function.

[0099] The multi-head attention component mentioned above consists of six layers of standard multi-head attention components. When performing feature transmission, it receives the mask of the local topological graph relative to the global map to limit the attention mechanism to only focus on the observed part of the global map, which is represented as:

[0100]

[0101] The feature update process in the self-attention component is as follows:

[0102]

[0103] where, the output from the upper-layer attention component, is the learnable weight matrix of the kth attention head, u ij ,w i,j are the attention score of node i to node j and the normalized result, respectively, h′ i is the final output result of the self-attention component.

[0104] The decoder mainly consists of a pointer network similar to the single-layer multi-head self-attention layer, but it directly outputs the action probability with the obtained attention weight. The overall feature updating process is as follows:

[0105]

[0106] The strategy π θ is the learnable decoder output, representing the probability of the agent making an action a t at time step t given the observation o t , where j represents a node in the neighbor node set of the robot, which will be used as the navigation point of the robot at the next time.

[0107] (4) SAC optimization

[0108] After obtaining the neural network output of the encoder-decoder structure, a reinforcement learning algorithm needs to be designed to continuously optimize the strategy π θ . In this embodiment, the Soft Actor-Critic (SAC) algorithm is used as the main algorithm. The goal of this algorithm is to maximize the cumulative return of the agent and reduce the short-sightedness of the agent at the time level mentioned above. At the same time, a temperature factor is introduced to enhance the randomness in the reinforcement learning process, so as to improve the exploration ability of the algorithm.

[0109] After obtaining the neural network output of the encoder-decoder structure, a reinforcement learning algorithm needs to be designed to continuously optimize the strategy π θ . In this embodiment, the Soft Actor-Critic (SAC) algorithm is used as the main algorithm. The goal of this algorithm is to maximize the cumulative return of the agent and reduce the short-sightedness of the agent at the time level mentioned above. At the same time, a temperature factor is introduced to enhance the randomness in the reinforcement learning process, so as to improve the exploration ability of the algorithm.

[0110] The optimization goal of SAC is:

[0111]

[0112] where π* represents the optimal policy, i.e., the optimization objective; r t represents the immediate reward of the agent; γ∈(0,1) is the discount factor when calculating the total reward; α>0 is the temperature parameter, which is used to balance the exploration and exploitation in the reinforcement learning optimization process. represents the entropy of the policy, which is used to encourage the diversity of the policy.

[0113] In this embodiment, two Q networks are used as critic modules, which mainly function to evaluate the expected value of the state-action pair. The minimum value output by the double Q network is used as the optimization objective of the loss function, which can effectively alleviate the overestimation of the Q value. The state value V(o t+1 ) estimation process is as follows:

[0114]

[0115] wherein Q1(o and Q2(o represent the outputs of the two Q networks, respectively.

[0116] The loss function of each Q network adopts the MSE form:

[0117]

[0118] The loss function of the policy network, i.e., the actor network, is as follows:

[0119]

[0120] This loss encourages the policy to tend to select the action that makes the Q value larger in the current state, while maintaining the high-entropy characteristics of the policy distribution.

[0121] The temperature parameter α has an adaptive adjustment function and will gradually converge to the preset entropy value in the later stage of the algorithm to automatically balance the exploration and optimality:

[0122]

[0123] wherein H is the preset target entropy.

[0124] In summary, the deep reinforcement learning autonomous exploration method based on the combination of the graph neural network and the self-attention mechanism proposed in the present application constructs a unified encoder-decoder structure, captures the local spatial topological features through the graph attention mechanism, models the global observation information through the multi-layer self-attention network, and finally continuously optimizes the exploration behavior of the robot in the unknown environment through the reinforcement learning strategy. Compared with the prior art, the method of the present application improves the exploration coverage while reducing the path length and the number of turns of autonomous exploration, alleviates the problems of repeated exploration and the existence of exploration blind area, and has good generalization ability and scalability, and is suitable for various mobile robot platforms with autonomous exploration needs.

[0125] Embodiment 2

[0126] The embodiment of the present application also provides an electronic device, which comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for automatically mapping a robot in an unknown open environment as shown in any one of the above method embodiments. Figure 1 and / or Figure 3 The method for automatically mapping a robot in an unknown open environment.

[0127] It can be understood that the memory can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, codes, code sets or instruction sets. The memory can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above various method embodiments, etc.; and the data storage area can store data created according to the use of the server, etc.

[0128] The processor can include one or more processing cores. The processor connects various parts in the entire server through various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA) and a programmable logic array (PLA). The processor can be integrated with a combination of one or more of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes operating systems and application programs, etc.; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor, but be implemented by a separate chip.

[0129] Since the electronic device is an electronic device corresponding to the robot automatic mapping method in an unknown open environment according to the embodiments of the present application, and the principle of solving problems of the electronic device is similar to the method, the implementation of the electronic device can be seen from the implementation process of the above-mentioned method embodiments, and the repeated parts will not be repeated.

[0130] Embodiment 3

[0131] The embodiments of the present application also provide a computer readable storage medium, wherein at least one instruction, at least one program, a code set or an instruction set are stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement a robot automatic mapping method in an unknown open environment as shown in Figure 1 and / or Figure 3 .

[0132] Those skilled in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer readable storage medium including a Read-Only Memory (ROM), a Random Access Memory (RAM), a Programmable Read-only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), a One-time Programmable Read-Only Memory (OTPROM), an Electrically-Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical disk memories, magnetic disk memories, magnetic tape memories, or any other computer readable medium that can be used to carry or store data.

[0133] Since the storage medium is a storage medium corresponding to the robot automatic mapping method in an unknown open environment according to the embodiments of the present application, and the principle of solving problems of the storage medium is similar to the method, the implementation of the storage medium can be seen from the implementation process of the above-mentioned method embodiments, and the repeated parts will not be repeated.

[0134] Embodiment 4

[0135] In some possible implementation manners, each aspect of the method of the embodiment of the present application can also be implemented in the form of a program product, which includes program codes for causing a computer device to execute the steps of the method of automatically mapping a robot in an unknown open environment according to various exemplary embodiments of the present application described above in the specification when the program product is run on the computer device. The executable computer program codes or "codes" for executing each embodiment can be written in a high-level programming language such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (for example, Transact-SQL), Perl, or in various other programming languages.

[0136] It should be understood that parts of the present application can be realized in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be realized in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized in hardware, and as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application-specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0137] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0138] The above-described embodiments are only for the purpose of illustrating the technical concepts and characteristics of the present application, and the purpose is to enable those of ordinary skill in the art to understand the content of the present application and to implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made in accordance with the essence of the present application should be covered within the protection scope of the present application.

Claims

1. A method for automatic robot mapping in an unknown open environment, characterized in that: The following steps are involved: Obtain sensor data and build a local map based on the sensor data; Topologically transform the local map to obtain a collision-free topological map of the local map; Performing sparse processing on the collision-free topological graph to obtain a sparse topological graph; The obtained sparse topology map is input into the policy network model based on deep reinforcement learning, and the policy for controlling the robot's motion is output to move the robot. The policy network model includes an encoder and a decoder; the encoder is used to extract features from the input sparse topology graph to obtain the features of the topology graph; The decoder is used to output a strategy based on the characteristics of the topology graph.

2. The method for automatic robot mapping in an unknown open environment according to claim 1, characterized in that: The acquiring of sensor data and constructing a local map based on the sensor data includes: The sensor data o is obtained by a laser sensor set on the robot, and the sensor data o is converted into a local map through SLAM technology.

3. The method for automatic robot mapping in an unknown open environment according to claim 1, characterized in that: The topologically transforming the local map to obtain a collision-free topological map of the local map includes: Use x×y grid to represent the local map, and get the center coordinates of each grid (x i ,y i ), the local map is represented as a set of points P: Among them, according to different categories, the map is divided into known spaces M k and unknown space M u ; The known space M k Further divided into free space M f and occupied space M o ; The known space M k and unknown space M u The grid at the junction of is defined as the boundary in the autonomous exploration task, which is represented by the set F: Extract the collision-free topology graph from the local map and obtain a series of collision-free nodes and edges. Use set V to represent the node set and set E to represent the collision-free edge set: Finally, the collision-free topology graph G = (V, E) of the local map is obtained.

4. A method for automatic robot mapping in an unknown open environment according to claim 1 or 3, characterized in that: The node v in the collision-free topology graph i It includes four dimensional features, namely the coordinates of the node in the map (x i ,y i ), the benefit value of the node u i , and the landmark value b of whether the node has been visited i ; The benefit value u i The calculation method is as follows: Where, f i is the boundary that the robot can observe at the current node.

5. The method for automatic robot mapping in an unknown open environment according to claim 1, characterized in that: The performing of sparse processing on the collision-free topological graph to obtain a sparse topological graph includes: In the collision-free topology graph, nodes with benefit values ​​greater than a preset threshold are grouped according to their adjacent position relationships to obtain boundary clusters. Search for paths from the current robot position to each set of boundary clusters, sample nodes on these paths, and obtain path nodes; The boundary cluster nodes and path nodes are retained, and the remaining nodes with a benefit of 0 are removed to obtain a sparse topological graph.

6. The method for automatic robot mapping in an unknown open environment according to claim 1, characterized in that: The encoder consists of a two-layer multi-head graph attention network (GAT) with a cross-layer residual connection mechanism; GAT is used to aggregate the features of a node and its adjacent nodes, enabling the model to effectively learn the local topological relationship between nodes; The cross-layer residual connection mechanism is used to enhance gradient propagation and feature retention capabilities; For each node in the topology graph, the feature update process of its GAT is as follows: Where K is the number of attention heads in the multi-head graph attention mechanism, and Ν(i) is the number of attention heads in the node v i The neighbor node set, h j is the feature vector of the neighbor node, W k is the learnable linear transformation matrix of the k-th head, is the node v in the kth head j For node v i The attention weight is σ, and σ is the ReLU activation function.

7. The method for automatic robot mapping in an unknown open environment according to claim 6, characterized in that: The multi-head graph attention mechanism consists of six layers of standard multi-head attention components. When transferring features, it receives a mask of the local topology map relative to the global map, thereby limiting the attention mechanism to focus only on the observed parts of the global map.

8. The method for automatic robot mapping in an unknown open environment according to claim 1, characterized in that: The decoder consists of a pointer network based on a multi-head attention layer. The pointer network outputs action probabilities based on the obtained attention weights. The overall feature update process is as follows: Where, strategy π θ is the learnable decoder output, representing the agent's observation o at time step t t Next, make action a t =j is completely determined by the attention score, where j represents a node in the robot's neighbor node set, which will serve as the robot's navigation point at the next moment; softmax j represents the probability of selecting neighbor node j calculated by the softmax function, W Q is the learnable weight matrix of query, W K is the learnable weight matrix of the key vector; h c is the feature vector of the current node; is the feature vector of neighbor node j; d is the feature vector dimension; T is the transpose.

9. The method for automatic robot mapping in an unknown open environment according to claim 1, characterized in that: The robot automatic mapping method further includes an optimization step: Using the flexible actor-critic algorithm (SAC), the strategy π θ Optimize; the optimization goal of SAC is: Where, π * represents the optimal strategy; r t represents the agent's immediate reward; α is the temperature parameter; represents the entropy of the strategy; γ t represents the product of the discount factor γ at time t; T is the maximum value of the decision time step.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 9.

Images