Robot navigation method, device, computer readable storage medium and robot

Abstract

The application belongs to the technical field of robots, and particularly relates to a robot navigation method and device, a computer readable storage medium and a robot. The method comprises the following steps: acquiring vision navigation information of a robot; processing the vision navigation information by using a preset vision navigation network to obtain navigation parameters of the robot; wherein the navigation parameters comprise a position change amount and a yaw angle change amount; determining motion control parameters of the robot according to the navigation parameters; and controlling the robot to act according to the motion control parameters. In the application, the navigation parameters comprising the position change amount and the yaw angle change amount are adopted, the difference between different robots is small, and the application has good universality.

Description

Technical Field

[0001] This application belongs to the field of robotics technology, and in particular relates to a robot navigation method, device, computer-readable storage medium, and robot. Background Technology

[0002] Vision- and topology-based robot navigation algorithms combine visual perception and topology map information to enable robot navigation in its environment. This approach fully utilizes environmental perception information provided by visual sensors while using a topology map to represent the high-level structure of the environment and for navigation planning. This vision- and topology-based navigation algorithm is suitable for large and complex environments because it relies on a high-level topology structure. It is also flexible enough to be used in different environments, and the method is relatively simple to construct and maintain maps, making it widely applicable and of high research value.

[0003] However, current navigation algorithms based on vision and topology maps use outputs that vary in altitude at the same speed, while different robots have significantly different speeds, resulting in poor versatility of the navigation algorithm. Summary of the Invention

[0004] In view of this, embodiments of this application provide a robot navigation method, apparatus, computer-readable storage medium, and robot to solve the problem of poor versatility of existing navigation methods.

[0005] A first aspect of this application provides a robot navigation method, which may include: Acquire multiple past image frames, current image frames, and target image frames of the robot; The first encoder of the preset visual navigation network extracts features from the multiple past image frames and the current image frame to obtain first feature information; the second encoder of the visual navigation network extracts features from the multiple past image frames, the current image frame, and the target image frame to obtain second feature information; the first feature information and the second feature information are superimposed to obtain superimposed feature information; the superimposed feature information is processed by a fully connected layer of the visual navigation network to obtain the navigation parameters of the robot; wherein, the navigation parameters include position change and yaw angle change; The motion control parameters of the robot are determined based on the navigation parameters; The robot is controlled to move according to the motion control parameters.

[0006] In one specific implementation of the first aspect, the visual navigation network may further include a diffusion model; After obtaining the navigation parameters, the process may further include: The navigation parameters are optimized using the diffusion model to obtain the optimized navigation parameters; Accordingly, determining the robot's motion control parameters based on the navigation parameters may include: The motion control parameters of the robot are determined based on the optimized navigation parameters.

[0007] In one specific implementation of the first aspect, before acquiring the robot's multiple past image frames, current image frames, and target image frames, the following may be included: Construct a topology map of the robot's current environment; Determine the robot's navigation start point and navigation end point; The navigation path of the robot from the navigation starting point to the navigation ending point is determined based on the topology map.

[0008] In one specific implementation of the first aspect, constructing the topology map of the robot's current environment may include: Obtain multiple motion trajectories in the robot's current environment; The multiple motion trajectories are fused to obtain the topology map.

[0009] In one specific implementation of the first aspect, before fusing the multiple motion trajectories to obtain the topology map, the method may further include: The multiple motion trajectories are each subjected to sparsification processing to obtain the multiple sparsed motion trajectories; Accordingly, the process of fusing the multiple motion trajectories to obtain the topology map includes: The sparse multiple motion trajectories are fused to obtain the topology map.

[0010] In one specific implementation of the first aspect, determining the robot's motion control parameters based on the navigation parameters may include: The navigation parameters are processed using a preset PD controller to obtain the motion control parameters of the robot.

[0011] A second aspect of this application provides a robot navigation device, which may include: The visual navigation information acquisition module is used to acquire multiple past image frames, current image frames, and target image frames of the robot; The visual navigation network processing module is used to extract features from the multiple past image frames and the current image frame using a first encoder of the preset visual navigation network to obtain first feature information; to extract features from the multiple past image frames, the current image frame, and the target image frame using a second encoder of the visual navigation network to obtain second feature information; to superimpose the first feature information and the second feature information to obtain superimposed feature information; and to perform fully connected processing on the superimposed feature information using a fully connected layer of the visual navigation network to obtain the navigation parameters of the robot; wherein, the navigation parameters include position change and yaw angle change. A motion control parameter determination module is used to determine the motion control parameters of the robot based on the navigation parameters; A motion control module is used to control the robot to perform actions according to the motion control parameters.

[0012] In one specific implementation of the second aspect, the visual navigation network may further include a diffusion model; The visual navigation network processing module may further include: An optimization processing unit is used to optimize the navigation parameters using the diffusion model to obtain the optimized navigation parameters; Accordingly, the motion control parameter determination module can be specifically used to: determine the motion control parameters of the robot based on the optimized navigation parameters.

[0013] In one specific implementation of the second aspect, the robot navigation device may further include: The topology graph construction module is used to construct a topology graph of the robot's current environment. The start and end point determination module is used to determine the robot's navigation start point and navigation end point; The navigation path determination module is used to determine the navigation path of the robot from the navigation starting point to the navigation ending point based on the topology map.

[0014] In one specific implementation of the second aspect, the topology graph construction module may include: The trajectory acquisition unit is used to acquire multiple motion trajectories in the environment where the robot is currently located. The trajectory fusion unit is used to fuse the multiple motion trajectories to obtain the topology map.

[0015] In one specific implementation of the second aspect, the topology graph construction module may further include: A sparsification processing unit is used to perform sparsification processing on the multiple motion trajectories respectively to obtain the multiple motion trajectories with sparsification. Accordingly, the trajectory fusion unit can be specifically used to: fuse the sparse multiple motion trajectories to obtain the topology map.

[0016] In one specific implementation of the second aspect, the motion control parameter determination module may be specifically used to: process the navigation parameters using a preset PD controller to obtain the motion control parameters of the robot.

[0017] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the robot navigation methods described above.

[0018] A fourth aspect of this application provides a robot including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the robot navigation methods described above.

[0019] The fifth aspect of this application provides a computer program product that, when run on a robot, causes the robot to perform the steps of any of the robot navigation methods described above.

[0020] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment acquires the visual navigation information of the robot; processes the visual navigation information using a preset visual navigation network to obtain the robot's navigation parameters; wherein, the navigation parameters include position change and yaw angle change; determines the robot's motion control parameters based on the navigation parameters; and controls the robot to act according to the motion control parameters. In this application embodiment, navigation parameters including position change and yaw angle change are used, which have small differences between different robots and have good versatility. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of one embodiment of a robot navigation method according to the present application. Figure 2 A schematic diagram of a possible network structure for a visual navigation network; Figure 3A schematic flowchart illustrating the processing of visual navigation information using a visual navigation network; Figure 4 A schematic diagram of another possible network structure for visual navigation networks; Figure 5 A schematic flowchart illustrating a complete robot navigation process; Figure 6 This is a structural diagram of one embodiment of a robot navigation device according to the present application. Figure 7 This is a schematic block diagram of a robot according to an embodiment of this application. Detailed Implementation

[0023] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0025] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0028] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] Vision- and topology-based robot navigation algorithms combine visual perception and topology map information to enable robot navigation in its environment. This approach fully utilizes environmental perception information provided by visual sensors while using a topology map to represent the high-level structure of the environment and for navigation planning. This vision- and topology-based navigation algorithm is suitable for large and complex environments because it relies on a high-level topology structure. It is also flexible enough to be used in different environments, and the method is relatively simple to construct and maintain maps, making it widely applicable and of high research value.

[0030] However, current navigation algorithms based on vision and topology maps use outputs that vary in altitude at the same speed, while different robots have significantly different speeds, resulting in poor versatility of the navigation algorithm.

[0031] To address this technical problem, the embodiments of this application employ navigation parameters that include position change and yaw angle change, which have small differences between different robots and good versatility.

[0032] The execution subject of this application embodiment is a robot, which may include, but is not limited to, humanoid robots, wheeled robots, or other types of robots.

[0033] Please see Figure 1 One embodiment of a robot navigation method in this application may include: Step S101: Obtain the robot's visual navigation information.

[0034] In this embodiment, the visual navigation information may include multiple past image frames, the current image frame, and the target image frame. The robot can acquire each image frame using an image acquisition device such as a camera.

[0035] Here, past image frames refer to image frames acquired by the robot up to the current moment. The specific number of past image frames can be set according to the actual situation. For example, five consecutive image frames acquired up to the current moment can be used as past image frames. Of course, more or fewer frames can also be set for past image frames. This application embodiment does not specifically limit this. In particular, when the number of acquired past image frames is less than the set number of frames, the current image frame can be used to replace the missing past image frames.

[0036] The current image frame is the image frame captured by the robot at the current moment.

[0037] The target image frame is the image frame corresponding to the target node of the robot navigation, that is, the image frame that should be collected when the robot navigates to the target node.

[0038] Step S102: Process the visual navigation information using a preset visual navigation network to obtain the robot's navigation parameters.

[0039] In this embodiment of the application, navigation parameters may include position change amount. and yaw change ,in, and These represent the robot's position changes along various preset coordinate axes. Based on the output of the visual navigation network. The robot can decode these parameters into motion control parameters, and then control the robot to move and complete navigation tasks.

[0040] By adopting this new representation of navigation parameters, data collected from robots at different speeds can be used to train the visual navigation network, and the output of the visual navigation network can also be applied to robots at different speeds, demonstrating good versatility.

[0041] In one specific implementation, navigation parameters may also include time distance. d This refers to the time required for the robot to navigate from its current location to the target node.

[0042] Time distance based on visual navigation network output d The robot can determine whether it has navigated to the target node, for example, when the time distance... d If the value is less than a preset threshold, it can be determined that the robot has navigated to the target node. The specific value of this threshold can be set according to the actual situation, and this application embodiment does not impose a specific limitation on it.

[0043] Robots can also be based on time distance d Operations such as sparsifying topology nodes and merging similar nodes are performed. For details on the construction process of the topology graph, please refer to the following text. They will not be repeated here.

[0044] In this embodiment, the visual navigation network can adopt any existing neural network structure capable of image processing. This embodiment does not specifically limit its specific type or network structure.

[0045] Figure 2 The diagram illustrates a possible network structure for a visual navigation network. As shown, the visual navigation network may include a first encoder, a second encoder, and a fully connected layer. Based on this visual navigation network, step S102 may specifically include, as follows: Figure 3 The processing procedure shown is as follows: Step S1021: Use the first encoder to extract features from multiple past image frames and the current image frame to obtain the first feature information.

[0046] In this embodiment, the first encoder can adopt any existing neural network structure capable of image feature extraction, and this embodiment does not specifically limit its type or network structure. As an example, the first encoder can adopt a convolutional neural network (CNN).

[0047] Step S1022: Use the second encoder to extract features from multiple past image frames, the current image frame, and the target image frame to obtain second feature information.

[0048] In this embodiment, the second encoder can adopt any existing neural network structure capable of image feature extraction, and this embodiment does not specifically limit its type or network structure. As an example, the second encoder can adopt a convolutional neural network (CNN).

[0049] Step S1023: Superimpose the first feature information and the second feature information to obtain superimposed feature information.

[0050] Step S1024: Use a fully connected layer to process the superimposed feature information to obtain navigation parameters.

[0051] Figure 4 The diagram shows another possible network structure for a visual navigation network. As shown, the visual navigation network can include a first encoder, a second encoder, a fully connected layer, and a diffusion model. Figure 2 A diffusion model was added to the network structure shown.

[0052] The processing procedures of the first encoder, the second encoder, and the fully connected layer are as follows: Figure 3 The process shown is the same, and you can refer to the detailed description of steps S1021 to S1024 for details, which will not be repeated here. However, after using a fully connected layer to process the superimposed feature information to obtain navigation parameters, a diffusion model can be used to optimize the navigation parameters to obtain optimized navigation parameters.

[0053] Through the diffusion model, it is possible to base the initial results on... Perform multiple denoising iterations to generate a series of intermediate results with gradually decreasing noise levels. This continues until a noise-free final result is obtained. The specific noise reduction process is shown in the following formula:

[0054] Where K is the number of noise reduction attempts. and These are preset hyperparameters, and their specific values ​​can be set according to actual conditions. This application does not impose specific limitations on them. For neural networks that predict noise, To overlay feature information, The expected value is 0 and the variance is 0. The Gaussian distribution.

[0055] In one specific implementation of this application, the visual navigation network can be obtained through a supervised training process based on a pre-generated dataset. The dataset used for training the visual navigation network may include real datasets and / or simulation datasets, wherein the real dataset is a dataset generated in a real scene, and the simulation dataset is a dataset generated in a simulation environment. The simulation dataset can test the performance of the visual navigation network in a simulation environment, thereby providing effective safety assurance.

[0056] Taking the generation process of simulation datasets as an example, the robot's motion trajectory can be collected in robot simulation software, and the dataset can be obtained by processing the motion trajectory.

[0057] For example, a home environment can be added to robot simulation software, and then the robot can be controlled to walk in various rooms, saving visual images and robot odometry information to obtain the motion trajectory. For humanoid robots without a dedicated odometry, inertial measurement unit (IMU) information and the position of the robot's waist in the world coordinate system of the robot simulation software can be used as robot odometry information.

[0058] Specifically, in the initial stage of simulation, the coordinate axes of the world coordinate system in the robot simulation software can be aligned with the robot's coordinate axes based on the robot's initial orientation. The robot's position in the world coordinate system is then used as the odometry position reading. The negative value of the robot's initial IMU yaw reading can be added to the IMU yaw reading as an IMU bias. That is, the initial robot yaw reading is 0, and the yaw value after robot movement is the value after adding the bias. This processed IMU yaw is then used as the odometry yaw reading. Simultaneously, the robot's visual image can be saved.

[0059] After obtaining the motion trajectory including visual images and robot odometry information, the motion trajectory can be sampled at a preset sampling frequency to obtain an image sequence and an odometry sequence. The specific value of the sampling frequency can be set according to actual conditions; for example, it can be set to 4Hz or other values. This embodiment does not impose a specific limitation on it. The velocity between adjacent frames can be calculated based on the odometry sequence. If the velocity is less than 0, the trajectory is cut off at that frame to ensure that the velocity of adjacent frames of each trajectory is greater than 0. At this time, for any image frame, the position change, yaw angle change, and time distance between that image frame and the first subsequent image frame can be calculated and used as... and The label.

[0060] The process of generating the real dataset is similar to that of generating the simulation dataset described above, but there is no need to transform the position of the world coordinate system or add an offset to the IMU readings. Other processes can be referred to the specific description of generating the simulation dataset described above, and will not be repeated in this embodiment.

[0061] Because the humanoid robot's waist sways from side to side with its swinging legs during movement, this affects the accuracy of tag calculations. The data volume is relatively large, and since data has a significant impact on the learning effect of the network, a wheel-based method can be used to collect data. In a simulation environment, a supervisor controller from robot simulation software can be used to control the robot's movement in the environment and collect data. The supervisor controller can perform operations that can usually only be performed by human operators and not by robots. In real-world scenarios, cameras and GPS devices can be fixed to a person to collect data.

[0062] In one specific implementation of this application, any loss function in the prior art can be used during the training process of the visual navigation network, and no specific limitation is made here.

[0063] For example, the Mean Squared Error (MSE) loss function can be used as shown in the following formula:

[0064] in, For the number of data points, The predicted value output by the visual navigation network. It is the actual value.

[0065] The mean squared error loss function is commonly used in regression prediction tasks. It measures the performance of a network by calculating the square of the distance (i.e., the error) between the predicted and actual values. That is, the closer the predicted and actual values ​​are, the smaller their mean squared error; conversely, the further apart the predicted and actual values ​​are, the larger their mean squared error.

[0066] After training the visual navigation network based on the dataset and loss function, the trained visual navigation network can be used to process the visual navigation information to obtain the robot's navigation parameters.

[0067] Step S103: Determine the robot's motion control parameters based on the navigation parameters.

[0068] The motion control parameters may include linear velocity v and angular velocity w.

[0069] In this embodiment, a parameter mapping relationship from navigation parameters to motion control parameters can be pre-constructed for different robots. After obtaining the navigation parameters output by the visual navigation network, the robot can determine the motion control parameters corresponding to the navigation parameters based on the robot's parameter mapping relationship. In one specific implementation, the robot can use a preset proportional-derivative (PD) controller to process the navigation parameters to obtain the robot's motion control parameters.

[0070] It should be noted that if step S102 uses... Figure 2 The visual navigation network shown refers to the navigation parameters in step S103. Figure 2 The navigation parameters output by the fully connected layer in step S102, if the one used is... Figure 4 The visual navigation network shown refers to the navigation parameters in step S103. Figure 4 The optimized navigation parameters are output by the diffusion model in the model.

[0071] Step S104: Control the robot to move according to the motion control parameters.

[0072] It is easy to understand that the above process only performs single-step navigation of the robot. Based on this, a complete robot navigation process can include, for example, Figure 5 The steps shown are as follows: Step S501: Construct a topology map of the robot's current environment.

[0073] In this embodiment of the application, multiple motion trajectories in the robot's current environment can be obtained separately, and then the multiple motion trajectories can be fused to obtain a topology map.

[0074] In one specific implementation of this application, before trajectory fusion, multiple motion trajectories can be sparsified to obtain sparse multiple motion trajectories; correspondingly, the trajectory fusion process is to fuse the sparse multiple motion trajectories to obtain a topology map.

[0075] For each motion trajectory, when performing sparsification, the trajectory can first be sampled at a preset sampling frequency to obtain an image sequence of that trajectory. Then, a visual navigation network can be used to calculate the temporal distance between every two adjacent image frames in the image sequence. If time distance If the time interval between two adjacent image frames is less than or equal to a preset first threshold, then the later image frame in two adjacent image frames can be deleted from the image sequence. This way, in the remaining image sequence, the time interval between any two adjacent image frames is reduced. If all values ​​are greater than the first threshold, each image frame in the retained image sequence is used as a node in the sparsed motion trajectory, with the time distance... This serves as the connection weight between nodes. The specific value of the first threshold can be set according to actual circumstances, and this embodiment does not impose specific limitations on it.

[0076] When performing trajectory fusion on multiple motion trajectories, the time distance between each node of different motion trajectories can be calculated. If the time distance between two nodes with different motion trajectories If the time distance between node 1 in trajectory A and node 2 in trajectory B is less than a preset second threshold, then the two nodes can be merged into one node. The specific method of node merging can be set according to the actual situation, and this application embodiment does not impose specific limitations on it. In one possible implementation, one of the two nodes can be randomly selected as the merged node. For example, if the time distance between node 1 in trajectory A and node 2 in trajectory B... If the value is less than the second threshold, then node 1 and node 2 can be merged into one node. The merged node can be either the original node 1 or the original node 2. The merged node connects motion trajectory A and motion trajectory B into a whole. After completing the node fusion, the time distance between the merged node and other nodes can be recalculated. This is then used as the new connection weight. The specific value of the second threshold can be set according to actual circumstances; this embodiment does not impose specific limitations on it.

[0077] Step S502: Determine the robot's navigation start point and navigation end point.

[0078] In this embodiment of the application, an image frame captured by the robot at the start of navigation can be obtained, the image frame can be compared with each node in the topology map to calculate the image similarity, and then the node in the topology map with the highest similarity to the image frame can be used as the starting point of the robot's navigation.

[0079] Similarly, a desired image frame for the robot can be acquired, and its image similarity can be calculated between the desired image frame and each node in the topology map. The node in the topology map with the highest similarity to the desired image frame is then used as the robot's navigation endpoint. The desired image frame is the image frame that should be captured when the robot navigates to its final destination, and it can be preset by relevant personnel according to the actual situation.

[0080] Step S503: Determine the navigation path of the robot from the navigation starting point to the navigation ending point based on the topology map.

[0081] In the embodiments of this application, the navigation path of the robot from the navigation starting point to the navigation ending point can be determined in the topology map according to any shortest path search algorithm in the prior art. For example, it may include, but is not limited to, Dijkstra's algorithm, Bellman-Ford algorithm, Floyd algorithm and SPFA algorithm, etc. The embodiments of this application do not specifically limit it.

[0082] Step S504: Perform single-step navigation.

[0083] The specific process of single-step navigation can be found in the detailed description of steps S101 to S104, and will not be repeated here.

[0084] It should be noted that the target nodes for single-step navigation are the nodes along the navigation path in sequence. For example, if the nodes along the navigation path are, in sequence, the navigation start point, intermediate node 1, intermediate node 2, intermediate node 3, ..., intermediate node N, and the navigation end point, where N is the total number of intermediate nodes between the navigation start point and the navigation end point, then at the beginning of the navigation phase, the target node for single-step navigation is intermediate node 1. After the robot reaches intermediate node 1, the target node for single-step navigation becomes intermediate node 2, and so on. After the robot reaches intermediate node 2, the target node for single-step navigation becomes intermediate node 3, and so on. After the robot reaches intermediate node N, the target node for single-step navigation becomes the navigation end point.

[0085] After completing this single-step navigation, it can be determined whether the robot has reached the navigation destination. If it has not yet reached the navigation destination, the next single-step navigation will continue until the robot has reached the navigation destination, at which point the navigation process will end.

[0086] In summary, the embodiments of this application acquire the robot's visual navigation information; process the visual navigation information using a preset visual navigation network to obtain the robot's navigation parameters; wherein, the navigation parameters include position change and yaw angle change; determine the robot's motion control parameters based on the navigation parameters; and control the robot to perform actions according to the motion control parameters. In the embodiments of this application, navigation parameters including position change and yaw angle change are used, which show minimal differences between different robots and have good versatility.

[0087] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0088] Corresponding to the robot navigation method described in the above embodiments, Figure 6 This illustration shows a structural diagram of one embodiment of a robot navigation device provided in this application.

[0089] In this embodiment, a robot navigation device may include: The visual navigation information acquisition module 601 is used to acquire the robot's visual navigation information; The visual navigation network processing module 602 is used to process the visual navigation information using a preset visual navigation network to obtain the navigation parameters of the robot; wherein, the navigation parameters include position change and yaw angle change. The motion control parameter determination module 603 is used to determine the motion control parameters of the robot based on the navigation parameters; The motion control module 604 is used to control the robot to perform actions according to the motion control parameters.

[0090] In one specific implementation of this application, the visual navigation information may include multiple past image frames, the current image frame, and the target image frame; the visual navigation network may include a first encoder, a second encoder, and a fully connected layer. The visual navigation network processing module may include: The first feature information extraction unit is used to extract features from the multiple past image frames and the current image frame using the first encoder to obtain first feature information; The second feature information extraction unit is used to extract features from the multiple past image frames, the current image frame, and the target image frame using the second encoder to obtain second feature information; The feature information superposition unit is used to superimpose the first feature information and the second feature information to obtain superimposed feature information; A fully connected processing unit is used to perform fully connected processing on the superimposed feature information using the fully connected layer to obtain the navigation parameters.

[0091] In one specific implementation of this application embodiment, the visual navigation network may further include a diffusion model; The visual navigation network processing module may further include: An optimization processing unit is used to optimize the navigation parameters using the diffusion model to obtain the optimized navigation parameters; Accordingly, the motion control parameter determination module can be specifically used to: determine the motion control parameters of the robot based on the optimized navigation parameters.

[0092] In one specific implementation of this application embodiment, the robot navigation device may further include: The topology graph construction module is used to construct a topology graph of the robot's current environment. The start and end point determination module is used to determine the robot's navigation start point and navigation end point; The navigation path determination module is used to determine the navigation path of the robot from the navigation starting point to the navigation ending point based on the topology map.

[0093] In one specific implementation of this application embodiment, the topology graph construction module may include: The trajectory acquisition unit is used to acquire multiple motion trajectories in the environment where the robot is currently located. The trajectory fusion unit is used to fuse the multiple motion trajectories to obtain the topology map.

[0094] In one specific implementation of this application embodiment, the topology graph construction module may further include: A sparsification processing unit is used to perform sparsification processing on the multiple motion trajectories respectively to obtain the multiple motion trajectories with sparsification. Accordingly, the trajectory fusion unit can be specifically used to: fuse the sparse multiple motion trajectories to obtain the topology map.

[0095] In one specific implementation of this application, the motion control parameter determination module can be specifically used to: process the navigation parameters using a preset PD controller to obtain the motion control parameters of the robot.

[0096] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0097] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0098] Figure 7 A schematic block diagram of a robot provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0099] like Figure 7 As shown, the robot 7 in this embodiment includes a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, it implements the steps described in the various robot navigation method embodiments above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of modules 601 to 604 are shown.

[0100] For example, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 72 in the robot 7.

[0101] Those skilled in the art will understand that Figure 7 This is merely an example of robot 7 and does not constitute a limitation on robot 7. It may include more or fewer parts than shown, or combine certain parts, or different parts. For example, robot 7 may also include input / output devices, network access devices, buses, etc.

[0102] The processor 70 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0103] The memory 71 can be an internal storage unit of the robot 7, such as a hard drive or memory. The memory 71 can also be an external storage device of the robot 7, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 71 can include both internal and external storage units of the robot 7. The memory 71 is used to store the computer program and other programs and data required by the robot 7. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] In the embodiments provided in this application, it should be understood that the disclosed devices / robots and methods can be implemented in other ways. For example, the device / robot embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0111] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A robot navigation method, characterized in that, include: Acquire multiple past image frames, current image frames, and target image frames of the robot; The first encoder of the preset visual navigation network extracts features from the multiple past image frames and the current image frame to obtain first feature information; The second encoder of the visual navigation network is used to extract features from the multiple past image frames, the current image frame, and the target image frame to obtain second feature information; The first feature information and the second feature information are superimposed to obtain superimposed feature information; The superimposed feature information is processed by the fully connected layer of the visual navigation network to obtain the navigation parameters of the robot; wherein, the navigation parameters include position change and yaw angle change. The motion control parameters of the robot are determined based on the navigation parameters; The robot is controlled to move according to the motion control parameters.

2. The robot navigation method according to claim 1, characterized in that, The visual navigation network also includes a diffusion model; After obtaining the navigation parameters, the process also includes: The navigation parameters are optimized using the diffusion model to obtain the optimized navigation parameters; Accordingly, determining the motion control parameters of the robot based on the navigation parameters includes: The motion control parameters of the robot are determined based on the optimized navigation parameters.

3. The robot navigation method according to claim 1, characterized in that, Before acquiring multiple past image frames, current image frames, and target image frames of the robot, the following steps are also included: Construct a topology map of the robot's current environment; Determine the robot's navigation start point and navigation end point; The navigation path of the robot from the navigation starting point to the navigation ending point is determined based on the topology map.

4. The robot navigation method according to claim 3, characterized in that, The construction of the topology map of the robot's current environment includes: Obtain multiple motion trajectories in the robot's current environment; The multiple motion trajectories are fused to obtain the topology map.

5. The robot navigation method according to claim 4, characterized in that, Before fusing the multiple motion trajectories to obtain the topology map, the process also includes: The multiple motion trajectories are each subjected to sparsification processing to obtain the multiple sparsed motion trajectories; Accordingly, the process of fusing the multiple motion trajectories to obtain the topology map includes: The sparse multiple motion trajectories are fused to obtain the topology map.

6. The robot navigation method according to any one of claims 1 to 5, characterized in that, Determining the robot's motion control parameters based on the navigation parameters includes: The navigation parameters are processed using a preset PD controller to obtain the motion control parameters of the robot.

7. A robot navigation device, characterized in that, include: The visual navigation information acquisition module is used to acquire multiple past image frames, current image frames, and target image frames of the robot; The visual navigation network processing module is used to extract features from the multiple past image frames and the current image frame using a first encoder of a preset visual navigation network to obtain first feature information; The second encoder of the visual navigation network is used to extract features from the multiple past image frames, the current image frame, and the target image frame to obtain second feature information; The first feature information and the second feature information are superimposed to obtain superimposed feature information; The superimposed feature information is processed by the fully connected layer of the visual navigation network to obtain the navigation parameters of the robot; wherein, the navigation parameters include position change and yaw angle change. A motion control parameter determination module is used to determine the motion control parameters of the robot based on the navigation parameters; A motion control module is used to control the robot to perform actions according to the motion control parameters.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the robot navigation method as described in any one of claims 1 to 6.

9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the robot navigation method as described in any one of claims 1 to 6.

Images