Robot automatic mapping method and related equipment in unknown open environment

Abstract

The application discloses a robot automatic mapping method in an unknown open environment and related equipment, and the method comprises the following steps: acquiring sensor data, constructing a local map according to the sensor data; topologizing the local map to obtain a collision-free topological map of the local map; performing sparse processing on the collision-free topological map to obtain a sparse topological map; inputting the obtained sparse topological map into a strategy network model based on deep reinforcement learning, and outputting a strategy for controlling the motion of a robot, so as to move the robot. The application models the environment mapping problem as a graph structure, models the spatial dependence in the environment through the relationship between nodes, and realizes the effective learning of the relationship between different regions of the robot in a complex environment. In addition, the attention mechanism dynamically adjusts the attention weight of each region, helps the robot to preferentially select a region with a higher potential return in the autonomous exploration process, and thus accelerates the exploration efficiency. The application can be widely applied to the technical field of robots.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method and related equipment for automatic mapping of robots in unknown open environments. Background Technology

[0002] In recent years, robots have been required to perform increasingly diverse tasks, and the types of robots have also become more numerous, such as cleaning robots, inspection robots, food delivery robots, search and rescue robots, and so on. Point cloud maps, as a mainstream form of environmental representation, provide robots with basic localization capabilities and are the foundation for downstream planning, obstacle avoidance, and other functions. Currently, the construction process of point cloud maps mainly relies on manually controlled robots gradually traversing the environment, resulting in insufficient automation of the overall process. Furthermore, subsequent updates to point cloud maps often require secondary human intervention. Therefore, improving the autonomy and efficiency of robot map building has gradually become an important research topic. Autonomous exploration technology refers to robots building maps in completely unknown environments by using sensor data to model local environments, then planning, and finally traversing the entire environment.

[0003] Existing traditional active SLAM techniques, such as boundary-based and sampling-based methods, are heavily influenced by their greedy strategies. When applied to large-scale, open, and complex environments, they often suffer from low mapping efficiency and may even fail due to repeated round trips caused by poorly designed benefit functions. With the continuous development of artificial intelligence in recent years, deep reinforcement learning methods have been proposed and applied to autonomous exploration tasks. Because they can utilize the experience gained from the agent's historical interactions with the environment and do not rely on manually designed rules, they can significantly improve the agent's performance in autonomous exploration tasks in unknown open environments. However, traditional network designs are mainly based on CNN architectures. Since CNNs require a fixed network input, their generalization ability to different scales is poor. This makes such methods unstable in practical applications, prone to incomplete exploration or exploration failure. Summary of the Invention

[0004] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to provide a method and related equipment for automatic mapping of robots in unknown open environments.

[0005] The first technical solution adopted in this invention is:

[0006] A method for automatic mapping of robots in unknown open environments includes the following steps:

[0007] Acquire sensor data and construct a local map based on the sensor data;

[0008] Topologicalize the local map to obtain a collision-free topological map of the local map;

[0009] The collision-free topology graph is sparsified to obtain a sparse topology graph.

[0010] The obtained sparse topology graph is input into a policy network model based on deep reinforcement learning, which outputs a policy to control the robot's movement in order to move the robot.

[0011] The policy network model includes an encoder and a decoder. The encoder is used to extract features from the input sparse topological graph to obtain the features of the topological graph. These features are used by the model to learn the spatial dependencies between nodes in the topological graph. The decoder is used to output the policy based on the features of the topological graph.

[0012] Furthermore, the acquisition of sensor data and the construction of a local map based on the sensor data include:

[0013] Sensor data o is acquired by laser sensors mounted on the robot, and SLAM technology is used to convert the sensor data o into a local map.

[0014] Further, the step of topologicalizing the local map to obtain a collision-free topological map of the local map includes:

[0015] The local map is represented using an x×y raster format, and the center coordinates (x, y) of each raster are obtained. i ,y i The local map is represented as a set of points P:

[0016]

[0017] Based on different categories, the map is divided into known space M. k and unknown space M u ;where the known space M k Further divided into free space M f and occupy space M o ; Given the space M k and unknown space M u The grid at the boundary is defined as the boundary in the autonomous exploration task, denoted by the set F:

[0018]

[0019] Extract a collision-free topology graph from the local map to obtain a set of collision-free nodes and edges. Let set V represent the node set and set E represent the collision-free edge set.

[0020]

[0021] Finally, a collision-free topological graph G = (V, E) of the local map is obtained.

[0022] Furthermore, node v in the collision-free topology graph i It includes four dimensional features, namely the coordinates of the node in the map (x, y, s, ...). i ,y i The benefit value u of the node i And the landmark value b indicating whether the node has been visited. i ;

[0023] Among them, the benefit value u i The calculation method is as follows:

[0024]

[0025] In the formula, f i This represents the boundary that the robot can observe at the current node.

[0026] Further, the step of sparsifying the collision-free topology graph to obtain a sparse topology graph includes:

[0027] In a collision-free topology graph, nodes with benefit values ​​greater than a preset threshold are grouped according to their adjacency relationships to obtain boundary clusters.

[0028] The A* algorithm is used to search for the path from the current robot position to each group of boundary clusters, and node sampling is performed on these paths to obtain the path nodes.

[0029] By retaining the boundary cluster nodes and path nodes and removing the remaining nodes with zero benefit, a sparse topology graph is obtained.

[0030] Furthermore, the encoder consists of a two-layer multi-head graph attention network (GAT) and a cross-layer residual connection mechanism;

[0031] GAT is used to aggregate the features between a node and its neighboring nodes, enabling the model to effectively learn the local topological relationships between nodes.

[0032] The cross-layer residual connection mechanism is used to enhance gradient propagation and feature preservation capabilities;

[0033] For each node in the topology graph, the feature update process of its GAT is as follows:

[0034]

[0035] In the formula, K represents the number of attention heads in the multi-head graph attention mechanism, and N(i) represents the number of nodes v. i The set of neighboring nodes, h j W is the feature vector of the neighboring nodes. kLet be the learnable linear transformation matrix of the k-th head. For the k-th head node v j For node v i The attention weights are σ, where σ is the ReLU activation function.

[0036] Furthermore, the multi-head graph attention mechanism consists of six layers of standard multi-head attention components. When performing feature propagation, it receives a mask of the local topology graph relative to the global map, thereby restricting the attention mechanism to only focus on the parts of the global map that have already been observed.

[0037] Furthermore, the decoder consists of a pointer network based on a multi-head attention layer. The pointer network outputs action probabilities using the obtained attention weights, and its overall feature update process is as follows:

[0038]

[0039] In the formula, strategy π θ It is the output of the learnable decoder, representing the agent's observation o at time step t. t Next, make the action a. t The probability of =j is entirely determined by the attention score, where j represents a node in the robot's set of neighboring nodes that will serve as the robot's navigation point at the next moment; softmax j W represents the probability of selecting neighbor node j calculated by the softmax function. Q W is a learnable weight matrix for the query. K h is the learnable weight matrix of the key vector; c This is the feature vector of the current node; Let d be the feature vector of neighbor node j; d is the dimension of the feature vector; T is the transpose.

[0040] Furthermore, the robot automatic mapping method also includes an optimization step:

[0041] The Flexible Actor-Critic Algorithm (SAC) is used to apply the strategy π. θ Continuous optimization is being carried out; the optimization objective of SAC is:

[0042]

[0043] In the formula, π * Represents the optimal strategy; r t This represents the agent's immediate feedback; α is the temperature parameter. Entropy representing the strategy; γ t This represents the product of discount factors γ at time t; T is the maximum value of the decision time step.

[0044] The second technical solution adopted in this invention is:

[0045] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to realize an automatic mapping method for robots in an unknown open environment as described above.

[0046] The third technical solution adopted in this invention is:

[0047] A computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement an automatic mapping method for robots in an unknown open environment as described above.

[0048] The fourth technical solution adopted in this invention is:

[0049] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method for automatic mapping of a robot in an unknown open environment.

[0050] The beneficial effects of this invention are as follows: This invention models the environment mapping problem as a graph structure, and models the spatial dependencies in the environment through the relationships between nodes, enabling the robot to effectively learn the relationships between different regions in complex environments. Furthermore, the attention mechanism dynamically adjusts the attention weights of each region, helping the robot prioritize regions with higher potential rewards during autonomous exploration, thereby accelerating exploration efficiency. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of the automatic graph construction method based on deep reinforcement learning in an embodiment of the present invention;

[0053] Figure 2This is an architecture diagram of the encoder-decoder of the policy network in an embodiment of the present invention;

[0054] Figure 3 This is a flowchart illustrating the steps of an automatic mapping method for robots in an unknown open environment, as described in an embodiment of the present invention. Detailed Implementation

[0055] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0056] The terminology used in the embodiments of this application is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of this application. The singular forms "a," "described," and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise expressly limited, terms such as "set," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0057] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0058] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0059] In the description of this application, "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0060] Terminology Explanation:

[0061] Laser SLAM (Simultaneous Localization and Mapping) is a robot perception technology primarily used to address the problem of robots simultaneously performing localization and mapping in unknown environments.

[0062] Boundary-based autonomous exploration methods: This method was the earliest applied to solve autonomous exploration problems. Its core idea is to evaluate and judge the robot's next path point based on all boundary locations, that is, the intersection of known and unknown regions, within the set of boundaries. In pioneering work of this type of method, the robot only selects the nearest boundary as the target point. Although this idea is easy to understand, its limitations lead to greedy behavior in boundary-based algorithms. Therefore, this type of algorithm is prone to meaningless round trips when there are too many boundaries and the boundary rewards are similar, which is particularly evident in large-scale, open, and complex environments.

[0063] Sampling-based autonomous exploration methods generate a set of viewpoints by performing RRT or RRG sampling in free space. The optimal viewpoint is then selected for navigation based on a pre-designed benefit function, maximizing the acquisition of observational information about the unknown environment. This benefit function is often related to the number of boundaries around the viewpoint and the change in Shannon entropy of the entire environment after reaching the viewpoint. This type of method avoids calculating all boundaries and remains stable even with a large number of boundaries. However, it often performs poorly in environments where sampling is difficult. Like boundary-based methods, it relies on a greedy strategy, making it prone to getting trapped in local optima and resulting in low exploration efficiency.

[0064] Autonomous exploration algorithms based on deep reinforcement learning: These methods design the interaction process between the agent and the environment, guiding the agent to update its action selection strategy through a reasonable reward function until the optimal strategy is learned. This approach allows the agent to obtain longer-term rewards and avoids the greedy behavior to some extent. Early related methods mainly used CNN architectures, but because CNNs need to maintain a fixed network input, their generalization ability to environments of different scales is poor. This makes these methods unstable in practical applications, prone to incomplete exploration or exploration failure.

[0065] Based on this, this invention proposes a novel autonomous exploration algorithm based on deep reinforcement learning. It introduces graph neural networks and an attention mechanism into the network design. Leveraging the structural advantages of graph neural networks, the environment mapping problem is modeled as a graph structure. Spatial dependencies in the environment are modeled through the relationships between nodes, enabling the robot to effectively learn the relationships between different regions in complex environments. Simultaneously, the attention mechanism dynamically adjusts the attention weights of each region, helping the robot prioritize regions with higher potential rewards during autonomous exploration, thereby accelerating exploration efficiency.

[0066] Specifically, this invention proposes a method for autonomous exploration of mobile robots in unknown open environments. First, this method employs an efficient graph representation method to model the environment as a graph structure. Utilizing the information transmission characteristics of graph neural networks based on topological structures, the model learns by capturing spatial topological features while focusing on boundary region characteristics. An attention mechanism dynamically adjusts the exploration focus of different areas within the currently observed local map based on the robot's current state. This helps achieve a balance between utilizing known information and exploring unknown information in open environment exploration tasks. On the other hand, the policy neural network of this invention is trained and optimized through continuous interaction with the environment, endowing the robot with the ability to infer the spatial structure of unknown regions based on known regions. Simultaneously, the model-free reinforcement learning method SAC enables the robot to analyze the impact of action strategies on future environmental states, which helps address temporal shortsightedness. This overall process allows the robot to efficiently search for and model unknown open environments.

[0067] Example 1

[0068] like Figure 1 and Figure 3 As shown, this embodiment provides an automatic mapping method for robots in unknown open environments, solving the problems of incomplete local exploration and low exploration efficiency caused by repeated exploration of known areas when multiple exploration directions yield similar benefits in existing autonomous robot exploration methods. The method specifically includes the following steps:

[0069] S1. Acquire sensor data and construct a local map based on the sensor data.

[0070] S2. Perform topology transformation on the local map to obtain a collision-free topology map of the local map.

[0071] S3. Perform sparsification on the collision-free topology graph to obtain a sparse topology graph.

[0072] S4. Input the obtained sparse topology graph into a deep reinforcement learning-based policy network model, and output a policy to control the robot's movement to move the robot. The policy network model includes an encoder and a decoder; the encoder extracts features from the input sparse topology graph, obtaining its features, which are used by the model to learn the spatial dependencies between nodes in the topology graph; the decoder outputs the policy based on the features of the topology graph.

[0073] The above method will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] (1) Local map topology

[0075] Autonomous exploration tasks refer to the process by which a robot, in an unknown environment ε, converts sensor data o into a local map m using SLAM technology, and then performs path planning to gradually expand the local map m into a complete map M.

[0076] In this embodiment, a laser SLAM framework is used as the overall mapping method. A complete bounded two-dimensional map M can be represented as an x×y grid. Using the center of each grid as coordinates, the map can be represented as a set of points P, as follows:

[0077]

[0078] Depending on the category, M can also be divided into known space M k and unknown space M u , where M k It can also be further divided into free space M f and occupy space M o .

[0079] Given space M k and unknown space M u The grid at the boundary is defined as the boundary in the autonomous exploration task, denoted by set F, as follows:

[0080]

[0081] Based on the grid-based representation of the global map mentioned above, we can also represent it in the local map M. f Extract a collision-free topology graph in the current free space M. f Uniform sampling is performed to obtain a series of collision-free nodes and edges. Let set V represent the set of nodes and set E represent the set of collision-free edges, as follows:

[0082]

[0083] Thus, the collision-free topological graph G = (V, E) of the known local map is obtained.

[0084] For node v i In other words, its characteristic h i It contains four dimensions, namely the coordinates of the node in M ​​(x, y, z). i ,y i The benefit value u of the node i And the landmark value b indicating whether the node has been visited. i .

[0085] The calculation method for benefit value is as follows:

[0086]

[0087] Among them, f i The benefit value represents the number of boundaries that the robot can observe at the current node. It is the ratio of the number of boundaries that the robot can observe at the current node to the total number of boundaries, and to some extent reflects the importance of the node for exploring unknown areas.

[0088] (2) Sparsification treatment

[0089] Since uniform sampling methods can lead to excessive computational loads in large-scale, open maps, this embodiment sparsifies the original topology graph. The main method involves first grouping nodes with benefit values ​​greater than a threshold based on their adjacency relationships to obtain boundary clusters. Then, the A* algorithm is used to search for paths from the current robot position to each boundary cluster. Node sampling is performed along these paths to obtain path nodes. Finally, the boundary cluster nodes and path nodes are retained, while nodes with a benefit of 0 are removed, resulting in a sparse topology graph G*. Although the sparse graph makes the nodes sparser, it still contains dense information, reducing computational load while minimizing feature loss during transmission in the graph neural network.

[0090] Ultimately, the local map observed after each action of the agent is transformed into a sparse topological graph, where the features of individual nodes and the input to the reinforcement learning network are as follows:

[0091] h i =(x i ,y i ,u i ,b i (6)

[0092] h IN =(h1,...,h n (7)

[0093] (3) Policy network model of deep reinforcement learning

[0094] like Figure 2 As shown, the policy network of deep reinforcement learning mainly consists of an encoder and a decoder. The encoder proposed in this embodiment mainly consists of graph neural network components and self-attention components, and the decoder mainly consists of a pointer network based on a multi-head attention layer. The subsequent actor and critic networks related to the reinforcement learning algorithm are derived from this structure.

[0095] The main function of the encoder is to perform feature-level modeling on the input topology graph mentioned above, and obtain the features of the topology graph. These features will be used by the model to learn the spatial dependencies between nodes in the topology graph.

[0096] The aforementioned graph neural network component mainly consists of a two-layer multi-head graph attention network (GAT) and a cross-layer residual connection mechanism. GAT is used to aggregate features between a node and its neighboring nodes. This feature aggregation method allows the model to effectively learn the local topological relationships between nodes, which is beneficial for the agent to understand the regional spatial characteristics of the current local map, thus better assisting exploration. Due to the inter-node propagation characteristics of graph neural networks, the nodes in the overall topological graph are prone to oversmoothing; therefore, the number of layers should not be set too high. Based on practical experience, this embodiment sets the GAT to two layers, with the input of the second layer being the output of the first layer. The cross-layer residual connection mechanism enhances gradient propagation and feature preservation capabilities. For each node in the topological graph, the feature update process of its GAT is as follows:

[0097]

[0098] Where K is the number of attention heads in the multi-head graph attention mechanism, and N(i) is the node v i The set of neighboring nodes, h j W is the feature vector of the neighboring nodes. k Let be the learnable linear transformation matrix of the k-th head. For the k-th head node v j For node v i The attention weights are σ, where σ is the ReLU activation function.

[0099] The aforementioned multi-head attention component consists of six standard multi-head attention components. During feature propagation, it receives a mask of the local topology graph relative to the global map, thereby restricting the attention mechanism to focus only on the observed parts of the global map, as shown below:

[0100]

[0101] The feature update process in the self-attention component is as follows:

[0102]

[0103] in, Derived from the output of the attention component in the upper-layer graph. It is the learnable weight matrix of the k-th attention head, u ij ,w i,j Let h′ be the attention score of node i to node j and the normalized result, respectively. i This is the final output of the self-attention component.

[0104] The decoder mainly consists of a pointer network, which is similar to a single-layer multi-head self-attention layer, but it directly outputs the action probability using the obtained attention weights. Its overall feature update process is as follows:

[0105]

[0106] Strategy π θ It is the output of the learnable decoder, representing the agent's observation o at time step t. t Next, make the action a. t The probability of =j is determined entirely by the attention score, where j represents a node in the robot's set of neighboring nodes that will serve as the robot's navigation point at the next moment.

[0107] (4) SAC optimization

[0108] After obtaining the output of the encoder-decoder neural network, a reinforcement learning algorithm needs to be designed to apply the policy π. θ Through continuous optimization, this embodiment adopts the Soft Actor-Critic (SAC) algorithm as the main algorithm. The goal of this algorithm is to maximize the cumulative reward of the agent, reduce the short-sightedness of the agent at the time level mentioned earlier, and introduce a temperature factor to enhance the randomness in the reinforcement learning process, thereby improving the algorithm's exploration ability.

[0109] After obtaining the output of the encoder-decoder neural network, a reinforcement learning algorithm needs to be designed to apply the policy π. θ Through continuous optimization, this embodiment adopts the Soft Actor-Critic (SAC) algorithm as the main algorithm. The goal of this algorithm is to maximize the cumulative reward of the agent, reduce the short-sightedness of the agent at the time level mentioned earlier, and introduce a temperature factor to enhance the randomness in the reinforcement learning process, thereby improving the algorithm's exploration ability.

[0110] The optimization objective of SAC is:

[0111]

[0112] Where π* This represents the optimal strategy, i.e., the optimization objective; r t γ represents the agent's immediate reward; γ∈(0,1) is the discount factor when calculating the total reward; α>0 is the temperature parameter, used to balance exploration and exploitation in the reinforcement learning optimization strategy process; Entropy represents the strategy and is used to encourage strategy diversity.

[0113] This embodiment employs two Q-networks as the commentator module, whose main function is to evaluate the expected value of state-action pairs. The minimum value output by the dual Q-network is used as the optimization objective of the loss function, which can effectively alleviate the overestimation of Q-values. Its state value V(o t+1 The estimation process is as follows:

[0114]

[0115] in These represent the outputs of the two Q-networks, respectively.

[0116] The loss function for each Q network takes the form of MSE:

[0117]

[0118] The loss function of the policy network, i.e., the actor network, is:

[0119]

[0120] This loss encourages the policy to tend to choose actions that maximize Q in the current state, while maintaining the high entropy of the policy distribution.

[0121] The temperature parameter α has an adaptive adjustment function, gradually converging to a preset entropy value in the later stages of the algorithm to automatically balance exploration and optimality:

[0122]

[0123] in The target entropy is preset.

[0124] In summary, the deep reinforcement learning-based autonomous exploration method proposed in this invention, which combines graph neural networks and self-attention mechanisms, constructs a unified encoder-decoder structure. It captures local spatial topological features through graph attention mechanisms, models global observation information through multi-layer self-attention networks, and ultimately continuously optimizes the robot's exploration behavior in unknown environments using reinforcement learning strategies. Compared to existing technologies, this method improves exploration coverage while reducing path length and number of turns during autonomous exploration, alleviating problems such as repetitive exploration and blind spots. It also possesses good generalization and scalability, making it suitable for various mobile robot platforms with autonomous exploration needs.

[0125] Example 2

[0126] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to achieve the following: Figure 1 and / or Figure 3 This paper presents an automatic mapping method for robots in an unknown open environment.

[0127] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0128] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory, and accessing data stored in memory to perform various server functions and process data. Optionally, the processor may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor may integrate one or more of the following: Central Processing Unit (CPU) and Modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0129] Since this electronic device is an electronic device corresponding to the robot automatic mapping method in an unknown open environment according to an embodiment of the present invention, and the principle of solving the problem by this electronic device is similar to that of this method, the implementation of this electronic device can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0130] Example 3

[0131] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to achieve the following: Figure 1 and / or Figure 3 This paper presents an automatic mapping method for robots in an unknown open environment.

[0132] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0133] Since this storage medium is the storage medium corresponding to a robot automatic mapping method in an unknown open environment according to an embodiment of the present invention, and the principle of the storage medium in solving the problem is similar to that of the method, the implementation of this storage medium can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0134] Example 4

[0135] In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of an automatic mapping method for a robot in an unknown open environment according to various exemplary embodiments of this application as described above. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0136] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0137] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0138] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for automatic mapping of robots in an unknown open environment, characterized in that, Includes the following steps: Acquire sensor data and construct a local map based on the sensor data; Topologicalize the local map to obtain a collision-free topological map of the local map; The collision-free topology graph is sparsified to obtain a sparse topology graph. The obtained sparse topology graph is input into a policy network model based on deep reinforcement learning, which outputs a policy to control the robot's movement in order to move the robot. The policy network model includes an encoder and a decoder; the encoder is used to extract features from the input sparse topological graph to obtain the features of the topological graph; the decoder is used to output the policy based on the features of the topological graph. The process of topologically transforming the local map to obtain a collision-free topological map of the local map includes: use The local map is represented in raster form, and the center coordinates of each raster are obtained. Representing a local map as a set of points : Based on different categories, the map is divided into known spaces. and unknown space ; where the known space Further divided into free space and occupy space ; to the known space and unknown space The grid at the intersection is defined as the boundary in the autonomous exploration task, using a set. express: Extracting a collision-free topology graph from a local map yields a series of collision-free nodes and edges, using a set... Represents a set of nodes, a collection Representing a collision-free edge set: Finally, a collision-free topology map of the local map is obtained. ; The process of sparsifying the collision-free topology graph to obtain a sparse topology graph includes: In a collision-free topology graph, nodes with benefit values ​​greater than a preset threshold are grouped according to their adjacency relationships to obtain boundary clusters. Search for the path from the current robot position to each group of boundary clusters, sample nodes on these paths, and obtain path nodes; By retaining the boundary cluster nodes and path nodes and removing the remaining nodes with zero benefit, a sparse topology graph is obtained. The encoder consists of a two-layer multi-head graph attention network (GAT) and a cross-layer residual connection mechanism; GAT is used to aggregate the features between a node and its neighboring nodes, enabling the model to effectively learn the local topological relationships between nodes. The cross-layer residual connection mechanism is used to enhance gradient propagation and feature preservation capabilities; For each node in the topology graph, the feature update process of its GAT is as follows: In the formula, This refers to the number of attention heads in a multi-head graph attention mechanism. For nodes The set of neighboring nodes, The feature vectors of neighboring nodes, For the first A learnable linear transformation matrix of size, For the first Node in the head For nodes Attention weights It is the ReLU activation function; The multi-head graph attention mechanism consists of six standard multi-head attention components, which receive a mask of the local topological graph relative to the global map during feature propagation. This limits the attention mechanism to only focusing on the parts of the global map that have already been observed.

2. The automatic mapping method for robots in an unknown open environment according to claim 1, characterized in that, The acquisition of sensor data and the construction of a local map based on the sensor data include: Sensor data is acquired by laser sensors mounted on the robot. Sensor data is processed using SLAM technology. Convert to a local map.

3. The automatic mapping method for robots in an unknown open environment according to claim 1, characterized in that, Nodes in the collision-free topology graph It includes four dimensions of features, namely the coordinates of the node in the map. The benefit value of the node And the landmark value indicating whether the node has been visited. ; Among them, the benefit value The calculation method is as follows: In the formula, This represents the boundary that the robot can observe at the current node.

4. The automatic mapping method for robots in an unknown open environment according to claim 1, characterized in that, The decoder consists of a pointer network based on a multi-head attention layer. The pointer network outputs action probabilities using the obtained attention weights. Its overall feature update process is as follows: In the formula, strategy It is the output of the learnable decoder, representing the agent's progress at time step [number missing]. Observations Next, make an action. The probability is entirely determined by the attention score, where This represents a node in the robot's set of neighboring nodes that will serve as the robot's navigation point at the next moment. express Selecting Neighbor Nodes in Function Computation The probability, For the learnable weight matrix of the query, The learnable weight matrix is ​​the key vector; This is the feature vector of the current node; Neighboring nodes eigenvectors; The dimension of the feature vector; This is a transpose.

5. The automatic mapping method for robots in an unknown open environment according to claim 1, characterized in that, The robot automatic mapping method also includes an optimization step: The flexible actor-critic algorithm (SAC) is used to evaluate the strategy. Optimization is performed; the optimization objective of SAC is: In the formula, Indicates the optimal strategy; This represents the agent's immediate feedback; It is a temperature parameter; The entropy of the strategy; express Current Discount Factor The product; It is the maximum value of the decision time step.

6. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 5.

Images