Robot navigation method, apparatus, and computer-readable storage medium

Abstract

The present disclosure relates to a robot navigation method, device and computer readable storage medium, wherein the method comprises: in response to receiving a navigation task, determining coordinates of a robot in an activity scene and coordinates of a target place carried in the navigation task in the activity scene through a plurality of cameras installed on the robot; wherein the visual range of the plurality of cameras covers at least the ground of the entire activity scene, and each camera has a common visual area with at least one other camera, and the ground is covered by the visual range of two or more cameras; planning a navigation path for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, and a 2D grid map of the activity scene; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of robots, and in particular, to a robot navigation method, device and computer readable storage medium. BACKGROUND

[0002] At present, robots are widely used in medical treatment, shopping malls, hotels, catering and other industries. The robot may be a traditional wheeled robot or a tracked robot, and such robots have great limitations in use scenarios. The traditional wheeled robot has poor obstacle crossing ability, poor terrain adaptability, low turning efficiency, or large turning radius, is easy to slip, and is not stable enough. The robot may also be a biped robot, which can almost adapt to various complex terrains, can cross obstacles, and has good degrees of freedom, flexible and stable motion.

[0003] Robots are usually equipped with cameras or sensors and rely on the cameras or sensors to walk. However, when the robot is walking, it is difficult to complete path planning and avoid obstacles when completing tasks only by relying on the sensors installed on the robot. At the same time, it is also very difficult to accurately position the robot when it is just started. When the robot needs to complete higher-precision positioning and obstacle avoidance, it often needs more powerful computing power, but it is difficult for a biped robot to carry a large device to ensure computing power. SUMMARY

[0004] To overcome the problems in the related art, the embodiments of the present disclosure provide a robot navigation method, device and computer readable storage medium. The technical solution is as follows:

[0005] According to a first aspect of the embodiments of the present disclosure, a robot navigation method is provided, comprising:

[0006] In response to receiving a navigation task, the coordinates of the robot in the activity scene and the coordinates of a target place carried in the navigation task in the activity scene are determined by a plurality of cameras installed in the activity scene of the robot; wherein the visible range of the plurality of cameras installed in the activity scene of the robot covers at least the ground of the entire activity scene, and each camera has a common viewing area with at least one other camera, and the ground is covered by the visible range of two or more cameras;

[0007] According to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, a 2D grid map of the activity scene, a navigation path for the robot is planned; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras installed in the activity scene of the robot.

[0008] In an embodiment, the method further comprises: generating a 2D grid map of the activity scene:

[0009] determining a first camera of the plurality of cameras whose visual range covers the origin of the active scene;

[0010] determining a three-dimensional coordinate of each first camera in the active scene;

[0011] determining a three-dimensional coordinate of the rest of the plurality of cameras in the active scene according to the relationship between each first camera and the camera with the common view area;

[0012] establishing a 3D model of the active scene according to the three-dimensional coordinates of all the cameras and the common view area;

[0013] determining the ground of the active scene as a passable area according to the 3D model of the active scene;

[0014] converting the determined passable area to generate a 2D grid map.

[0015] In an embodiment, determining the coordinate of the robot in the active scene comprises:

[0016] determining a second camera of the plurality of cameras whose visual range covers the robot;

[0017] determining the coordinate of the robot in the active scene according to the three-dimensional coordinate of the second camera;

[0018] determining the coordinate of the target place carried in the navigation task in the active scene comprises:

[0019] determining a third camera of the plurality of cameras whose visual range covers the target place;

[0020] determining the coordinate of the target place in the active scene according to the three-dimensional coordinate of the third camera.

[0021] In an embodiment, the method further comprises:

[0022] determining whether the target place is in the passable area according to the coordinate of the target place in the active scene;

[0023] if the target place is in the passable area, planning a navigation path according to the coordinate of the robot and the coordinate of the target place and the 2D grid map of the active scene.

[0024] In an embodiment, the method further comprises:

[0025] determining whether there is an obstacle on the navigation path on which the robot is walking through the plurality of cameras;

[0026] If it is determined that an obstacle exists on the navigation path, a message is sent to the robot, wherein the message comprises at least one of the following: a reminder message reminding the robot that an obstacle exists in front of the robot; and a new navigation path.

[0027] According to a second aspect of the embodiments of the present disclosure, a robot navigation device is provided, comprising:

[0028] A positioning module is configured to, in response to receiving a navigation task, determine coordinates of the robot in the activity scene and coordinates of a target place carried in the navigation task in the activity scene by using a plurality of cameras installed on the robot in the activity scene; wherein a visual range of the plurality of cameras installed on the robot in the activity scene covers at least a ground of the entire activity scene, and each camera has a common visual area with at least one other camera, and the ground is covered by the visual ranges of two or more cameras.

[0029] A navigation module is configured to plan a navigation path for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, and a 2D grid map of the activity scene; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras installed on the robot in the activity scene.

[0030] In an embodiment, the device further comprises a generation module configured to generate the 2D grid map of the activity scene; the generation module comprises:

[0031] A first determination unit is configured to determine a first camera in the plurality of cameras whose visual range covers a coordinate origin of the activity scene.

[0032] A second determination unit is configured to determine a three-dimensional coordinate of each first camera in the activity scene.

[0033] A third determination unit is configured to determine a three-dimensional coordinate of a remaining camera in the plurality of cameras in the activity scene according to a relationship between each first camera and a camera having a common visual area.

[0034] An establishment unit is configured to establish a 3D model of the activity scene according to the three-dimensional coordinates of all the cameras and the common visual areas photographed.

[0035] A fourth determination unit is configured to determine a ground of the activity scene as a passable area according to the 3D model of the activity scene.

[0036] A conversion unit is configured to convert the determined passable area to generate a 2D grid map.

[0037] In an embodiment, the positioning module comprises:

[0038] a first positioning unit configured to determine a second camera among the plurality of cameras whose visual range covers the robot; and determine coordinates of the robot in the activity scene according to three-dimensional coordinates of the second camera;

[0039] a second positioning unit configured to determine a third camera among the plurality of cameras whose visual range covers the target location; and determine coordinates of the target location in the activity scene according to three-dimensional coordinates of the third camera.

[0040] In an embodiment, the navigation module comprises:

[0041] a fifth determining unit configured to determine whether the target location is in the passable area according to the coordinates of the target location in the activity scene;

[0042] a navigation unit configured to, if the target location is in the passable area, plan a navigation path according to the coordinates of the robot and the coordinates of the target location, and the 2D grid map of the activity scene.

[0043] In an embodiment, the device further comprises:

[0044] a determining module configured to determine whether there is an obstacle on a navigation path on which the robot is walking through the plurality of cameras;

[0045] a prompting module configured to, if it is determined that there is an obstacle on the navigation path, send a prompt message, wherein the prompt message comprises at least one of the following: a message prompting that there is an obstacle in front of the robot; and a re-planned navigation path.

[0046] According to a third aspect of embodiments of the present disclosure, a robot navigation device is provided, comprising:

[0047] a processor;

[0048] a memory for storing processor-executable instructions;

[0049] wherein the processor is configured to:

[0050] in response to receiving a navigation task, determine coordinates of the robot in the activity scene and coordinates of a target location carried in the navigation task in the activity scene through a plurality of cameras installed on the robot in the activity scene; wherein the visual range of the plurality of cameras installed on the robot in the activity scene covers at least the ground of the entire activity scene, and each camera has a common visual area with at least one other camera, and the ground is covered by the visual range of two or more cameras;

[0051] The robot is provided with a 2D grid map of the activity scene, which is converted from a 3D model of the activity scene established by a plurality of cameras installed in the activity scene of the robot.

[0052] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions. The instructions are executed by a processor to implement the steps of any of the above methods.

[0053] The technical scheme provided in the present application has the following beneficial effects: the 3D model of the activity scene is established by the plurality of cameras installed in the activity scene of the robot, the positioning of the robot and the target place is completed, and the path planning for the robot is completed. The reliable positioning information and navigation path are provided for the robot, especially for the biped robot, and the problem that the robot is difficult to position due to the large change of the scene is solved. The system does not need to be installed on the body of the robot, and the demand for too many sensors and high computing power of the biped robot is avoided.

[0054] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0056] Figure 1 is a schematic diagram of a robot according to an exemplary embodiment.

[0057] Figure 2 is a flowchart of a robot navigation method according to an exemplary embodiment.

[0058] Figure 3 is a flowchart of a robot navigation method according to an exemplary embodiment.

[0059] Figure 4 is a flowchart of a robot navigation method according to an exemplary embodiment.

[0060] Figure 5 is a flowchart of a robot navigation method according to an exemplary embodiment.

[0061] Figure 6 is a structural diagram of a robot navigation device according to an exemplary embodiment.

[0062] Figure 7is a structural schematic diagram of a robot navigation device according to an example embodiment.

[0063] Figure 8 is a structural schematic diagram of a robot navigation device according to an example embodiment.

[0064] Figure 9 is a structural schematic diagram of a robot navigation device according to an example embodiment. DETAILED DESCRIPTION

[0065] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to represent the same elements in different drawings. The embodiments described in the following example embodiments do not represent all the implementations consistent with the present disclosure. Instead, they only represent examples of devices and methods consistent with some aspects of the present disclosure, as detailed in the appended claims.

[0066] The technical solutions of the present application are mainly applied to a walkable robot, for example, a biped robot, Figure 1 is a schematic diagram of a robot according to an example embodiment. The biped robot 100 can realize the walking of the robot and be more humanized.

[0067] Figure 2 is a flowchart of a robot navigation method according to an example embodiment. The execution subject of the method can be a device capable of providing a navigation path for the robot, such as a server, terminal, etc. capable of communicating with the robot and the camera installed in the activity scene, or the robot itself. As shown in Figure 2 The method includes the following steps 201-202:

[0068] In step 201, in response to receiving a navigation task, the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene are determined by a plurality of cameras installed in the activity scene of the robot; wherein the visual range of the plurality of cameras installed in the activity scene of the robot covers at least the ground of the entire activity scene, and each camera has a common visual area with at least one other camera, and the ground is covered by the visual range of two or more cameras.

[0069] In an embodiment of the present application, the navigation task may, for example, be uploaded to the server by the robot after receiving the voice command of the user. In another embodiment of the present application, the navigation task may, for example, be sent directly to the server by the user through the terminal APP. The target place name, target article name, etc. may be carried in the navigation task. After receiving the navigation task, the server searches and locates in the activity scene according to the target place name, target article name, etc. through the plurality of cameras, and determines the coordinates of the target place or the target article in the activity scene. In an embodiment of the present application, the 3D model of the pre-established activity scene may also be displayed in the terminal used by the user, and the target place may be directly selected by the user in the visualized 3D model. At this time, the coordinates of the target place in the activity scene may be directly carried in the navigation task sent.

[0070] In an embodiment of the present application, the activity scene of the robot is an indoor scene, and a plurality of cameras may be installed on the top of the indoor scene. The positions of the cameras are, for example, the corners of the roof: the angles are obliquely downward, and the center of the roof: the angles are directly downward. When the cameras are installed, the following requirements are met: the sum of the visual ranges of the plurality of cameras covers at least the entire floor of the indoor scene, and each camera has a common visual area with at least one other camera. All the scene floor is covered by the visual ranges of two or more cameras, so that each part of the scene floor can be photographed by two or more cameras at the same time.

[0071] This step is to determine the coordinates of the robot in the activity scene and the coordinates of the target place in the activity scene carried in the navigation task by using the plurality of cameras installed in the activity scene of the robot, so that the robot and the target place can be accurately positioned.

[0072] In step 202, a navigation path is planned for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place in the activity scene carried in the navigation task, and a 2D grid map of the activity scene.

[0073] In this step, when two cameras simultaneously photograph an object (i.e. there is a common visual area), and the distance between the two cameras is known, the three-dimensional information (the distance from the object to the camera) of all objects in the common visual area can be measured, and then the world coordinates of all objects in the common visual area can be obtained through coordinate conversion according to the world coordinates of the cameras. Therefore, the 3D model of the activity scene, especially the floor of the activity scene, can be obtained by using the plurality of cameras installed in the activity scene of the robot.

[0074] Since the 3D model is not conducive to navigation, the 3D model can be converted into a 2D grid map, and the 2D grid map is used to plan a navigation path.

[0075] In an embodiment of the present application, a path from the coordinates of the robot to the coordinates of the target place can be planned by using the A* algorithm. The A* algorithm is a direct search algorithm most effective for solving the shortest path in a static network. In the field of robots, the A* algorithm is often used for path planning of mobile robots, and will not be described here.

[0076] The technical solution of the present application proposes to establish a 3D model of the activity scene by a plurality of cameras installed in the activity scene, complete positioning of the robot and the target place, and complete path planning for the robot. Reliable positioning information and navigation path are provided for the robot, especially for a biped robot, solving the problem of difficult positioning caused by large changes in the scene, and avoiding the need to install too many sensors on the biped robot. In addition, when the method is executed by a system, device, or equipment other than the robot, the robot does not need to have high computing power.

[0077] In an embodiment of the present application, the method can generate a 2D grid map of the activity scene in advance in response to receiving a navigation task. Specifically, as shown in Figure 3 The method for generating a 2D grid map of the activity scene can include the following steps 301-306:

[0078] In step 301, a first camera in the plurality of cameras is determined, which can visually cover the coordinate origin of the activity scene.

[0079] The coordinate origin of the activity scene is a point selected in advance in the scene as the coordinate origin of the activity scene. It can be marked with a color label, such as a red tape on the ground. This step can determine all first cameras that can shoot the coordinate origin of the activity scene by traversing each camera. The number of determined first cameras should be at least greater than or equal to 2.

[0080] In step 302, the three-dimensional coordinates of each first camera in the activity scene are determined.

[0081] Since there is a common viewing area between two cameras, the 3D positions of all pixel points in the common viewing area relative to the cameras can be calculated. Since the coordinate origin of the activity scene is in the common viewing area, its position relative to the cameras can be calculated. Then, the position of each first camera in the activity scene, i.e., the three-dimensional coordinates, is back calculated.

[0082] In step 303, the three-dimensional coordinates of the remaining cameras in the plurality of cameras in the activity scene are determined according to the existing common viewing area and the relationship between each camera and the cameras with the existing common viewing area.

[0083] Since each first camera has a co-view area with at least one other camera, the relative poses between the cameras can be calculated by adding the epipolar constraint and the distance between the cameras with the co-view area, and the positions of the cameras with the co-view area with the first camera in the activity scene are restored, so as to determine the three-dimensional coordinates of the remaining cameras in the activity scene. Wherein, the epipolar constraint is an algorithm for estimating the relative pose relationship of binocular cameras according to two-dimensional plane information of images, which will not be described here.

[0084] In step 304, a 3D model of the activity scene is established according to the three-dimensional coordinates of all cameras and the co-view area.

[0085] Through the above steps, the extrinsic parameters of all cameras are actually known, and the depth points in the visible range of the cameras can be calculated through the visible content, that is, the 3D model of the activity scene can be restored.

[0086] In step 305, the ground of the activity scene is determined as a passable area according to the 3D model of the activity scene.

[0087] For example, the ground of the activity scene can be determined as a passable area by using a V disparity algorithm. The road surface area is obtained by detecting the inclined straight line formed by the projection of the ground plane in the V disparity map. The specific steps are as follows: 1) disparity estimation: first, the SGBM (semi-global block matching) method built-in opencv (cross-platform computer vision and machine learning software library based on Apache2.0 license (open source)) is used to estimate the disparity of the camera data to obtain the original disparity map. 2) V disparity estimation: the number of different disparities in each row of the original disparity map is projected horizontally to obtain a V disparity map. 3) ground plane line detection: LSD (Line Segment Detection) line detection is used, and the ground is mapped as an inclined straight line. The parameters of the straight line, such as slope and offset, can be calculated by merging the obtained line segments. 4) The intersection of each column in the V disparity and the inclined straight line of the ground is obtained through the above steps, and the initial mask of the ground is obtained according to the intersection. 5) The initial mask of the ground obtained contains some noise points, and the RANSAC (Random Sample Consensus) method is used to estimate the plane parameters. 6) The fitted plane still contains some small noise blocks, which can be removed by finding the contour and then finding the largest area block to obtain the final mask of the ground. 7) Finally, the length of the retained ground mask is compared with the maximum width of the current object, and the ground mask with a length greater than 1.5 times the width is retained to obtain the road passable area in the actual activity scene.

[0088] In step 306, the determined passable area is converted to generate a 2D grid map.

[0089] For example, the octomap (a three-dimensional map creation tool based on octree) can be used to convert the passable area to a 2D grid map:

[0090] 1) Install octomap (a three-dimensional map creation tool based on octree) and map_server module (a tool for storing and providing a grid map in the form of a service) in an ubuntu (a Linux operating system) system;

[0091] 2) Publish and execute the three-dimensional map publishing program in the form of a topic through the publish of ROS;

[0092] 3) Write an octomap_server service to set the grid resolution, the range of the intercepted three-dimensional point cloud, and the topic information of the world coordinate system;

[0093] 4) Save the 2D grid map through the map_server service.

[0094] Through the above embodiment, the 2D grid map of the activity scene can be generated in advance to provide accurate data for the positioning and navigation of the robot.

[0095] In an embodiment of the present application, the determination of the coordinates of the robot in the activity scene in step 201 can include the following steps A1-A2:

[0096] In step A1, a second camera of the plurality of cameras is determined, which can cover the robot.

[0097] This step can be achieved by traversing each camera to find a camera that can shoot the robot among all the cameras as the second camera.

[0098] In step A2, the coordinates of the robot in the activity scene are determined according to the three-dimensional coordinates of the second camera.

[0099] Since the extrinsic parameters of all the cameras are determined when the 3D model of the activity scene is established, the coordinates of the robot in the activity scene can be calculated by triangulation method combined with the extrinsic parameters of the cameras, so as to complete the accurate positioning of the robot in the activity scene.

[0100] In an embodiment of the present application, when the target place name, target object name, etc. are carried in the navigation task, the determination of the coordinates of the target place in the activity scene carried in the navigation task can further include the following steps B1-B2:

[0101] In step B1, a third camera in the plurality of cameras is determined, which can cover the target place in the visual range.

[0102] This step can be achieved by traversing each camera to find a camera that can always capture the target place or target object as the third camera.

[0103] In step B2, the coordinates of the target place in the activity scene are determined according to the three-dimensional coordinates of the third camera.

[0104] Since the extrinsic parameters of all cameras are determined when the 3D model of the activity scene is established, the coordinates of the target place in the activity scene are calculated by triangulation in combination with the camera extrinsic parameters, so as to complete the accurate positioning of the target place in the activity scene.

[0105] In an embodiment of the present application, in order to ensure the feasibility of the planned path, step 202 can further include steps C1-C2:

[0106] In step C1, whether the target place is in a passable area is determined according to the coordinates of the target place in the activity scene.

[0107] In step C2, if the target place is in the passable area, a navigation path is planned according to the coordinates of the robot and the target place, and the 2D grid map of the activity scene.

[0108] In the embodiment, before planning the navigation path, it is determined whether the target place is in the passable area, and if not, the task is terminated, which can avoid wasting resources.

[0109] In an embodiment of the present application, when the method is executed by a system, device or apparatus other than the robot, after the navigation path is planned, the navigation path can be sent to the robot to make the robot walk according to the navigation path.

[0110] On the basis of the above-mentioned embodiments, the present application can also assist the robot to avoid obstacles in the process of walking according to the navigation path, for example, Figure 4 As shown in the figure, the method further includes steps 203-204:

[0111] In step 203, whether an obstacle appears on the navigation path on which the robot is walking is determined by the plurality of cameras.

[0112] In this step, the three-dimensional information of the ground in front of the walking robot body can be acquired in real time by the camera installed in the active scene, and whether the ground has a protrusion and whether the range of the protrusion is greater than a preset range are determined according to the three-dimensional information of the ground, and if the range of the protrusion is greater than the preset range, it can be determined that there is an obstacle on the navigation path. The range of the protrusion can be the height of the protrusion and / or the width of the protrusion.

[0113] In step 204, if it is determined that there is an obstacle on the navigation path, a prompt message is sent to the robot, wherein the prompt message includes at least one of the following: a message prompting that there is an obstacle in front of the robot, and a re-planned navigation path.

[0114] In an embodiment of the present application, if it is determined that there is an obstacle on the navigation path, a prompt message can be sent to the robot, and after receiving the prompt message, the robot can immediately stop advancing or can perform a preset detour action. In an embodiment of the present application, the navigation path can also be re-planned according to the above steps, and when a new navigation path is re-planned, the detected obstacle can be avoided. After the new navigation path is re-planned, the re-planned navigation path is sent to the robot, so that the robot walks according to the re-planned navigation path.

[0115] The technical solutions proposed in the present application are described below with specific embodiments. Figure 5 is a flowchart of a robot navigation method according to an exemplary embodiment. As shown in Figure 5 , the method comprises:

[0116] Step 501, a 3D model of the active scene is established by a plurality of cameras installed on the robot in the active scene.

[0117] In this step, the camera needs to be installed first. When installing, the sum of the visual ranges of the plurality of cameras is ensured to cover at least the entire indoor scene ground, and each camera has a common visual area with at least one other camera, and all scene grounds are covered by the visual ranges of two or more cameras. Then, the distance between each two cameras in the plurality of cameras is measured, and the origin of the active scene coordinates is set, such as using a red tape on the ground. Then, the external parameter calibration of the plurality of cameras is performed.

[0118] Step 502, a passable area is determined according to the 3D model of the active scene.

[0119] Step 503, the determined passable area is converted to generate a 2D grid map.

[0120] Then, the server can provide a navigation path for the robot in response to a navigation task.

[0121] Step 504, receiving a navigation task, the navigation task carrying coordinates of a target place.

[0122] Step 505, determining coordinates of the robot in the activity scene through a plurality of cameras installed on the robot.

[0123] Step 506, planning a navigation path for the robot in a 2D grid map according to the coordinates of the robot and the coordinates of the target place.

[0124] Step 507, determining whether there is an obstacle on the navigation path on which the robot is walking through the plurality of cameras when the robot walks according to the navigation path.

[0125] Step 508, if it is determined that there is an obstacle on the navigation path, sending a prompt message to the robot, wherein the prompt message includes a message prompting that there is an obstacle in front of the robot and a re-planned navigation path.

[0126] The following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiment of the present disclosure.

[0127] Figure 6 is a structural schematic diagram of a robot navigation device according to an exemplary embodiment; the device can be implemented in various ways, for example, implementing all components of the device in equipment capable of providing a navigation path for the robot, or implementing components in the device in a coupled manner on the side of equipment capable of providing a navigation path for the robot; the device can realize the method related to the present disclosure described above through software, hardware or a combination of both, such as Figure 6 As shown in the figure, the robot navigation device includes:

[0128] A positioning module 601 is configured to, in response to receiving a navigation task, determine coordinates of the robot in an activity scene and coordinates of a target place in the activity scene carried in the navigation task through a plurality of cameras installed on the robot in the activity scene; wherein a visual range of the plurality of cameras installed on the robot in the activity scene covers at least the ground of the entire activity scene, and each camera has a common visual area with at least one other camera, and the ground is covered by the visual range of two or more cameras.

[0129] A navigation module 602 is configured to plan a navigation path for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place in the activity scene carried in the navigation task, and a 2D grid map of the activity scene; wherein the 2D grid map is converted from a 3D model of the activity scene established through the plurality of cameras installed on the robot in the activity scene.

[0130] The device provided by the embodiment of the present disclosure can be used to execute the method of the present disclosure. Figure 2The technical solutions, implementation manners and beneficial effects of the embodiments are similar, and thus are not described herein.

[0131] In a possible implementation, as shown in Figure 7 The apparatus further includes a generating module 603 configured to generate a 2D grid map of the activity scene; the generating module 603 includes:

[0132] a first determining unit configured to determine a first camera of the plurality of cameras whose visual range covers a coordinate origin of the activity scene;

[0133] a second determining unit configured to determine a three-dimensional coordinate of each first camera in the activity scene;

[0134] a third determining unit configured to determine, according to a relationship between each first camera and a camera having a common view area, a three-dimensional coordinate of a remaining camera of the plurality of cameras in the activity scene;

[0135] a establishing unit configured to establish a 3D model of the activity scene according to the three-dimensional coordinates of all the cameras and the common view area captured.

[0136] a fourth determining unit configured to determine, according to the 3D model of the activity scene, that a ground of the activity scene is a passable area;

[0137] a converting unit configured to convert the determined passable area to generate a 2D grid map.

[0138] In a possible implementation, the positioning module includes:

[0139] a first positioning unit configured to determine a second camera of the plurality of cameras whose visual range covers the robot; and determine a coordinate of the robot in the activity scene according to a three-dimensional coordinate of the second camera;

[0140] a second positioning unit configured to determine a third camera of the plurality of cameras whose visual range covers the target place; and determine a coordinate of the target place in the activity scene according to a three-dimensional coordinate of the third camera.

[0141] In a possible implementation, the navigation module includes:

[0142] a fifth determining unit configured to determine, according to the coordinate of the target place in the activity scene, whether the target place is in a passable area;

[0143] a navigation unit configured to, if the target place is in the passable area, plan a navigation path according to the coordinate of the robot and the coordinate of the target place and the 2D grid map of the activity scene.

[0144] In a possible implementation, as shown in Figure 8 the apparatus further includes:

[0145] determining whether there is an obstacle on a navigation path of the robot;

[0146] sending a prompt message if it is determined that there is an obstacle on the navigation path, wherein the prompt message comprises at least one of the following: a message prompting that there is an obstacle in front of the robot; a re-planned navigation path.

[0147] Figure 9 is a block diagram of a robot navigation device 90 according to an exemplary embodiment. The robot navigation device 90 can be implemented in various ways, for example, implementing all components of the device in an apparatus capable of providing a navigation path for a robot, or implementing components of the device in a coupled manner on the side of an apparatus capable of providing a navigation path for a robot; see Figure 9 The robot navigation device 90 comprises:

[0148] a processor 901;

[0149] a memory 902 for storing processor-executable instructions;

[0150] The processor 901 is configured to:

[0151] in response to receiving a navigation task, determining coordinates of the robot in an activity scene and coordinates of a target place carried in the navigation task in the activity scene through a plurality of cameras installed on the robot in the activity scene; wherein the visual range of the plurality of cameras installed on the robot in the activity scene covers at least the ground of the entire activity scene, and each camera has a common visual area with at least one other camera, and the ground is covered by the visual range of two or more cameras;

[0152] planning a navigation path for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, and a 2D grid map of the activity scene; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras installed on the robot in the activity scene.

[0153] In an embodiment, it is also configured to generate a 2D grid map of the activity scene:

[0154] determining a first camera in the plurality of cameras whose visual range covers the coordinate origin of the activity scene;

[0155] determining the three-dimensional coordinates of each first camera in the activity scene;

[0156] determine three-dimensional coordinates of the rest of the plurality of cameras in the activity scene according to the relationship between each first camera and the camera with the co-view region;

[0157] establish a 3D model of the activity scene according to the three-dimensional coordinates of all the cameras and the co-view region captured;

[0158] determine a passable region of the activity scene according to the 3D model of the activity scene;

[0159] convert the determined passable region to generate a 2D grid map.

[0160] In an embodiment, determining the coordinates of the robot in the activity scene comprises:

[0161] determining a second camera of the plurality of cameras whose visual range covers the robot;

[0162] determining the coordinates of the robot in the activity scene according to the three-dimensional coordinates of the second camera;

[0163] determining the coordinates of a target place carried in the navigation task in the activity scene comprises:

[0164] determining a third camera of the plurality of cameras whose visual range covers the target place;

[0165] determining the coordinates of the target place in the activity scene according to the three-dimensional coordinates of the third camera.

[0166] In an embodiment, the method further comprises:

[0167] determining whether the target place is in the passable region according to the coordinates of the target place in the activity scene;

[0168] if the target place is in the passable region, planning a navigation path for the robot according to the coordinates of the robot and the coordinates of the target place and the 2D grid map of the activity scene.

[0169] In an embodiment, the method is further configured to:

[0170] determining whether there is an obstacle on the navigation path on which the robot is walking through the plurality of cameras;

[0171] if it is determined that there is an obstacle on the navigation path, sending a message to the robot, wherein the message comprises at least one of the following: a reminder message reminding the robot that there is an obstacle in front of the robot; a new navigation path.

[0172] With regard to the apparatus in the above-described embodiments, in which the specific manner in which each module performs an operation has been described in detail in the embodiments relating to the method, no detailed elaboration will be given here.

[0173] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by an apparatus capable of providing a navigation path for a robot, enable the apparatus to perform a grasping method, the method comprising:

[0174] In response to receiving a navigation task, determining coordinates of the robot in the activity scene and coordinates of a target place carried in the navigation task in the activity scene by a plurality of cameras installed in the activity scene of the robot; wherein the visual range of the plurality of cameras installed in the activity scene of the robot covers at least the ground of the entire activity scene, and each camera has a common view area with at least one other camera, and the ground is covered by the visual range of two or more cameras;

[0175] According to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, a 2D grid map of the activity scene is used to plan a navigation path for the robot; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras installed in the activity scene of the robot.

[0176] In an embodiment, the method further comprises: generating a 2D grid map of the activity scene:

[0177] Determining a first camera in the plurality of cameras whose visual range covers the coordinate origin of the activity scene;

[0178] Determining the three-dimensional coordinates of each first camera in the activity scene;

[0179] According to the relationship between each first camera and the camera with a common view area, determining the three-dimensional coordinates of the remaining cameras in the plurality of cameras in the activity scene;

[0180] According to the three-dimensional coordinates of all cameras and the common view area captured, a 3D model of the activity scene is established;

[0181] According to the 3D model of the activity scene, the ground of the activity scene is determined as a passable area;

[0182] The determined passable area is converted to generate a 2D grid map.

[0183] In an embodiment, determining the coordinates of the robot in the activity scene comprises:

[0184] Determining a second camera in the plurality of cameras whose visual range covers the robot;

[0185] determining the coordinate of the target place carried in the navigation task in the activity scene according to the three-dimensional coordinate of the second camera;

[0186] determining the coordinate of the target place carried in the navigation task in the activity scene according to the three-dimensional coordinate of the second camera;

[0187] determining the third camera in the plurality of cameras covering the target place in the visual range;

[0188] determining the coordinate of the target place in the activity scene according to the three-dimensional coordinate of the third camera.

[0189] In an embodiment, the method further comprises:

[0190] determining whether the target place is in the passable area according to the coordinate of the target place in the activity scene;

[0191] if the target place is in the passable area, planning the navigation path according to the coordinate of the robot and the coordinate of the target place, and the 2D grid map of the activity scene.

[0192] In an embodiment, the method further comprises:

[0193] determining whether there is an obstacle on the navigation path on which the robot is walking through the plurality of cameras;

[0194] if it is determined that there is an obstacle on the navigation path, sending a message to the robot, wherein the message comprises at least one of the following: a reminder message reminding the robot that there is an obstacle in front of the robot; a new navigation path.

[0195] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the aspects disclosed herein. This application is intended to cover any variations, uses or adaptations of the disclosure other than those expressly disclosed herein. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure is indicated by the following claims.

[0196] It should be understood that the present disclosure is not limited to the precise construction that has been described and illustrated herein and that various modifications and changes can be made therein without departing from the scope thereof. The scope of the present disclosure is indicated by the appended claims rather than by the description and examples.

Claims

1. A method of robot navigation, characterized by, The method comprises: in response to receiving the navigation task, determining the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene through the plurality of cameras installed on the robot in the activity scene; wherein the visual range of the plurality of cameras installed on the robot in the activity scene covers at least the ground of the entire activity scene, and each camera has a common view area with at least one other camera, and the ground is covered by the visual range of two or more cameras; planning a navigation path for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, and a 2D grid map of the activity scene; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras installed on the robot in the activity scene; The method further comprises: determining a first camera in the plurality of cameras whose visual range covers the coordinate origin of the activity scene; determining the three-dimensional coordinates of each first camera in the activity scene; determining the three-dimensional coordinates of the remaining cameras in the plurality of cameras in the activity scene according to the relationship between each first camera and the cameras with a common view area; establishing a 3D model of the activity scene according to the three-dimensional coordinates of all cameras and the common view area captured; determining the ground of the activity scene as a passable area according to the 3D model of the activity scene; converting the determined passable area to generate a 2D grid map; determining the coordinates of the robot in the activity scene comprises: determining a second camera in the plurality of cameras whose visual range covers the robot; determining the coordinates of the robot in the activity scene according to the three-dimensional coordinates of the second camera; determining the coordinates of the target place carried in the navigation task in the activity scene comprises: determining a third camera in the plurality of cameras whose visual range covers the target place; determining the coordinates of the target place in the activity scene according to the three-dimensional coordinates of the third camera.

2. The method of claim 1, wherein, The method further comprises: determining whether the target place is in the passable area according to the coordinates of the target place in the activity scene; if the target place is in the passable area, planning a navigation path according to the coordinates of the robot and the target place, and the 2D grid map of the activity scene.

3. The method of claim 1, wherein, The method further comprises: determining whether there is an obstacle on the navigation path on which the robot is walking through the plurality of cameras; if it is determined that there is an obstacle on the navigation path, sending a prompt message to the robot, wherein the prompt message comprises at least one of the following: a message prompting the robot that there is an obstacle in front of it; a re-planned navigation path.

4. A robot navigation device, characterized in that The device comprises: The positioning module is configured to, in response to receiving the navigation task, determine coordinates of the robot in the activity scene and coordinates of a target place carried in the navigation task in the activity scene by a plurality of cameras installed on the robot in the activity scene; wherein the visual range of the plurality of cameras installed on the robot in the activity scene covers at least the ground of the entire activity scene, and each camera has a common view area with at least one other camera, and the ground is visible to two or more cameras The navigation module is configured to plan a navigation path for the robot according to the coordinates of the robot in the activity scene and the coordinates of the target place in the activity scene carried in the navigation task, and a 2D grid map of the activity scene; wherein the 2D grid map is converted from a 3D model of the activity scene established by the plurality of cameras installed on the robot in the activity scene; The device further comprises a generation module configured to generate a 2D grid map of the activity scene; the generation module comprises: A first determination unit configured to determine a first camera in the plurality of cameras whose visual range covers the coordinate origin of the activity scene; A second determination unit configured to determine the three-dimensional coordinates of each first camera in the activity scene; A third determination unit configured to determine the three-dimensional coordinates of the remaining cameras in the plurality of cameras in the activity scene according to the relationship between each first camera and the cameras having a common view area; An establishment unit configured to establish a 3D model of the activity scene according to the three-dimensional coordinates of all the cameras and the common view areas photographed; A fourth determination unit configured to determine the ground of the activity scene as a passable area according to the 3D model of the activity scene; A conversion unit configured to convert the determined passable area to generate a 2D grid map; The positioning module comprises: A first positioning unit configured to determine a second camera in the plurality of cameras whose visual range covers the robot; and determine the coordinates of the robot in the activity scene according to the three-dimensional coordinates of the second camera; A second positioning unit configured to determine a third camera in the plurality of cameras whose visual range covers the target place; and determine the coordinates of the target place in the activity scene according to the three-dimensional coordinates of the third camera. The navigation module comprises:

5. The apparatus of claim 4, wherein, A fifth determination unit configured to determine whether the target place is in a passable area according to the coordinates of the target place in the activity scene; A navigation unit configured to, if the target place is in the passable area, plan a navigation path according to the coordinates of the robot and the coordinates of the target place, and the 2D grid map of the activity scene. The device further comprises:

6. The apparatus of claim 4, wherein, A determination module configured to determine whether there is an obstacle on the navigation path on which the robot is walking by the plurality of cameras; A prompt module configured to, if it is determined that there is an obstacle on the navigation path, send a prompt message, wherein the prompt message comprises at least one of the following: a message prompting that there is an obstacle in front of the robot; a re-planned navigation path. Comprise:

7. A robot navigation device, characterized in that A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to: ​ In response to receiving the navigation task, determining the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene by a plurality of cameras installed on the robot in the activity scene; wherein the visual range of the plurality of cameras installed on the robot in the activity scene covers at least the ground of the entire activity scene, and each camera has a common view area with at least one other camera, and the ground is covered by the visual range of two or more cameras; According to the coordinates of the robot in the activity scene and the coordinates of the target place carried in the navigation task in the activity scene, The 2D grid map of the activity scene plans a navigation path for the robot; wherein the 2D grid map is converted from the 3D model of the activity scene established by the plurality of cameras installed on the robot in the activity scene; Wherein the 2D grid map of the activity scene is generated by: Determining a first camera in the plurality of cameras whose visual range covers the coordinate origin of the activity scene; Determining the three-dimensional coordinates of each first camera in the activity scene; According to the relationship between each first camera and the camera with a common view area, determining the three-dimensional coordinates of the remaining cameras in the plurality of cameras in the activity scene; According to the three-dimensional coordinates of all cameras and the common view area photographed, establishing a 3D model of the activity scene; According to the 3D model of the activity scene, determining the ground of the activity scene as a passable area; Converting the determined passable area to generate a 2D grid map; Determining the coordinates of the robot in the activity scene, comprising: Determining a second camera in the plurality of cameras whose visual range covers the robot; Determining the coordinates of the robot in the activity scene according to the three-dimensional coordinates of the second camera; Determining the coordinates of the target place carried in the navigation task in the activity scene, comprising: Determining a third camera in the plurality of cameras whose visual range covers the target place; Determining the coordinates of the target place in the activity scene according to the three-dimensional coordinates of the third camera. The instructions are executed by the processor to implement the steps of the method of any one of claims 1-3.

8. A computer readable storage medium having stored thereon computer instructions, wherein, ​

Images