Multi-layer map creation method, device and robot

Abstract

The present invention discloses a multi-layer map creation method, device and robot, the method comprising: responding to control operations, controlling the robot to realize cross-floor movement between multiple floors based on the elevator; for each floor, establishing a world map of the floor based on the world coordinate system, updating the world map of the floor during the robot movement, marking the identified elevator stop area on the floor, binding multiple target points on the movement trajectory and the business operations corresponding to each target point to the world map of the floor, and executing this step repeatedly until the world map of all floors to be reached by the robot is created, wherein the target points include: an initial position point, at least one waiting point, at least one boarding point, at least one exit point, and at least one destination point. According to the above technical solution, there is no need for excessive participation of technical personnel and the use of special mapping tools, which improves the intelligence level and convenience of the robot.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a multi-layer map creation method, device and robot. Background Art

[0002] In hotels, supermarkets, shopping malls, factories, hospitals and other places, robots are gradually replacing manual labor to perform delivery, disinfection, cleaning and other tasks with their efficient, accurate and continuous working capabilities. However, hotels, supermarkets, shopping malls, factories, hospitals and other work scenarios often require cross-floor work, and robots based on simple two-dimensional work cannot meet the needs, and robots are required to achieve cross-floor mapping and working capabilities.

[0003] At present, the robot's multi-layer map construction method across floors is mainly post-processing. Based on a special mapping method, the robot builds a scene map. The map and trajectory are divided into multiple parts according to business needs, and the waiting and entering elevator target points are marked on each map. The robot's motion trajectory and map are processed in segmented and sub-graph forms. The mapping method is complex, and the constructed map has poor environmental adaptability and cannot adapt to local environmental changes. Summary of the invention

[0004] The main purpose of the present invention is to disclose a multi-layer map creation method, device and robot, so as to at least solve the problems in the related art of multi-layer map construction across floors, that is, the robot motion trajectory and map are processed in the form of segments and graphs, the map construction method is complex, the constructed map has poor environmental adaptability, and cannot adapt to local environmental changes.

[0005] According to one aspect of the present invention, a multi-layer map creation method is provided.

[0006] The multi-layer map creation method according to the present invention includes: responding to control operations, controlling the robot to realize cross-floor movement between multiple floors based on the elevator; for each floor, establishing a world map of the floor based on the world coordinate system, updating the world map of the floor during the movement of the robot, marking the identified stop area of ​​the elevator on the floor, binding multiple target points on the movement trajectory of the robot on the floor, and the business operations corresponding to each of the target points to the world map of the floor, and repeating this step until the world maps of all the floors to be reached by the robot are created, wherein the bound target points in the world maps of all the floors include: an initial position point, at least one elevator waiting point, at least one elevator boarding point, at least one elevator exit point, and at least one destination point.

[0007] According to another aspect of the present invention, a multi-layer map creation device is provided.

[0008] The multi-layer map creation device according to the present invention includes: a control module, which is used to respond to control operations and control the robot to realize cross-floor movement between multiple floors based on the elevator; a mapping module, which establishes a world map of each floor based on the world coordinate system, and updates the world map of the floor during the movement of the robot, and binds the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor, and the business operations corresponding to each of the target points to the world map of the floor, and executes this step repeatedly until the world maps of all the floors to be reached by the robot are created, wherein the bound target points in the world maps of all the floors include: an initial position point, at least one elevator waiting point, at least one elevator boarding point, at least one elevator exit point, and at least one destination point.

[0009] According to yet another aspect of the present invention, a robot is provided.

[0010] The robot according to the present invention comprises: a memory and a processor, wherein the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions stored in the memory, so that the robot executes any of the above-mentioned multi-layer map creation methods.

[0011] According to the present invention, in response to the control operation, the robot is controlled to realize cross-floor movement between multiple floors based on the elevator; for each floor, a world map of the floor is established based on the world coordinate system, and during the robot movement, the world map of the floor is updated, and the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor (including but not limited to: initial position point, at least one waiting point, at least one boarding point, at least one exit point, at least one destination point), and the business operations corresponding to each of the above target points are bound to the world map of the floor, and this step is executed repeatedly until the world map of all floors to be reached by the robot is created. Therefore, by controlling the robot to realize cross-floor movement between multiple floors based on the elevator, convenient mapping of multi-layer scenes can be realized, without excessive participation of technical personnel and the use of special mapping tools, which improves the intelligence level and convenience of the robot. The constructed map has strong environmental adaptability and can be updated in time when local environmental changes occur. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flowchart of a multi-layer map creation method according to an embodiment of the present invention;

[0013] Figure 2 is a flowchart of a multi-layer map creation method according to a preferred embodiment of the present invention;

[0014] Figure 3is a structural block diagram of a multi-layer map creation device according to an embodiment of the present invention;

[0015] Figure 4 is a structural block diagram of a robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0017] According to an embodiment of the present invention, a multi-layer map creation method is provided.

[0018] Figure 1 FIG. 1 is a flow chart of a method for creating a multi-layer map according to an embodiment of the present invention. Figure 1 As shown, the multi-layer map creation method includes:

[0019] Step S101: responding to the control operation, controlling the robot to realize cross-floor movement between multiple floors based on the elevator;

[0020] Step S102: For each floor, a world map of the floor is established based on the world coordinate system. During the movement of the robot, the world map of the floor is updated, and the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor, and the business operations corresponding to each of the target points are bound to the world map of the floor. This step is repeated until the world maps of all the floors to be reached by the robot are created, wherein the bound target points in the world maps of all the floors include: an initial position point, at least one elevator waiting point, at least one elevator boarding point, at least one elevator exit point, and at least one destination point.

[0021] exist Figure 1In the method shown, in response to the control operation, the robot is controlled to realize cross-floor movement between multiple floors based on the elevator (for example, manually pushed, controlled by remote control, etc.); for each floor, a world map of the floor is established based on the world coordinate system, and the world map of the floor is updated during the movement of the robot, and the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor (including but not limited to: initial position point, at least one waiting point, at least one boarding point, at least one exit point, at least one destination point), and the business operations corresponding to each of the above target points are bound to the world map of the floor, and this step is executed repeatedly until the world map of all floors to be reached by the robot is created. Therefore, by controlling the robot to realize cross-floor movement between multiple floors based on the elevator, convenient mapping of multi-layer scenes can be realized, without too much involvement of technical personnel and the use of special mapping tools, which improves the intelligence level and convenience of the robot. The constructed map has strong environmental adaptability and can be updated in time when local environmental changes occur. In addition, the above map is also bound to multiple target points and the business operations corresponding to each target point. During the robot's automated operation, when the robot runs to the corresponding target point, it can execute the business operations corresponding to the current target point, thereby more effectively realizing the entire cross-floor operation business process.

[0022] It should be noted that the above steps S101 and S102 are not executed in sequence. The robot is controlled to move on each floor, and the robot waits for the elevator, takes the elevator, and exits the elevator between the floors are synchronized. A world map of the floor is established based on the world coordinate system. During the movement of the robot, the world map of the floor is updated, and the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor, and the business operations corresponding to each of the above target points are bound to the world map of the corresponding floor until the world maps of all floors to be reached by the robot are created.

[0023] Preferably, when the target points bound in the world map of all the floors include at least one elevator waiting point, in the step S102, binding the multiple target points on the motion trajectory of the robot on the floor and the business operations corresponding to each of the target points to the world map of the floor may further include the following processing: before controlling the robot to arrive at at least one elevator on the current floor, setting the elevator waiting position of the robot to one or more elevator waiting points among the at least one elevator waiting point, wherein each elevator corresponds to one elevator waiting point; binding the location information of the one or more elevator waiting points to the simultaneous localization and mapping (SLPMAP) The invention relates to a method for realizing a floor-level automatic mapping (SLAM) key frame, and optimizing the world map of the floor and the one or more elevator waiting points when the SLAM closed-loop optimization occurs; controlling the robot to interact with the elevator control system at the one or more elevator waiting points through wireless communication to perform an elevator call operation; after controlling the robot to enter the elevator, identifying the position and orientation of the elevator on the world map during the movement of the robot, and optimizing the position information of the one or more elevator waiting points and the orientation information of the robot at the one or more elevator waiting points based on the identification results; associating the elevator call operation with the one or more elevator waiting points, and binding them to the world map of the floor.

[0024] In the preferred implementation process, during the robot movement, the position information of the waiting point is bound to the SLAM key frame corresponding to the waiting point position, and stored in a key frame database including SLAM key frames of multiple historical position points. The current frame is compared with the key frame in the key frame database. For example, the image features of the image frame at the current position and the key frame in the key frame database can be extracted using a convolutional neural network pre-training model, and the fully connected layer output of the convolutional neural network pre-training model is used as the extracted image feature vector. The high-dimensional image feature vector is reduced in dimension and whitened using the zero-phase component analysis (ZCA) whitening method, and the noise in the similarity matrix is ​​eliminated using a median filter. Then, the similarity between the image frame at the current position and each key frame in the key frame database is calculated using the defined similarity matrix.

[0025] When the similarity value between the image frame at the current position and a key frame in the key frame database is greater than or equal to a predetermined threshold value (for example, 70%), it is preliminarily determined that the predetermined closed-loop condition is met, and the key frame in the key frame database is locked. Then, based on this key frame, the N frames before and the M frames after the key frame (N and M can be dynamically set according to actual conditions, N can be equal to M or not, and M and N are both greater than or equal to 1) are compared with the image frame at the current position respectively. If in all comparison results, each similarity value is greater than or equal to the predetermined threshold value (for example, 70%), it is determined that the closed loop is established, and a closed-loop signal is provided to the SLAM module, so that the SLAM module can directly use the closed-loop detection result to perform image optimization operations, specifically, optimize the world map of the floor and the above-mentioned one or more elevator waiting points.

[0026] It can be seen that when the image frame at the current position is compared with a key frame in the key frame database and meets the predetermined closed-loop condition, the closed-loop is not directly determined to be established. Instead, the image frame at the current position is comprehensively compared with one or more key frames before and after the above-mentioned key frame in the key frame database for image comparison detection. Among all the comparison results, the closed-loop is determined to be established only when each similarity value is greater than or equal to the predetermined threshold, thereby greatly improving the accuracy of closed-loop detection.

[0027] Preferably, when the target points bound in the world maps of all the above-mentioned floors include: at least one boarding point, in the above-mentioned step S102, binding the multiple target points on the movement trajectory of the above-mentioned robot on the floor and the business operations corresponding to each of the above-mentioned target points to the world map of the floor can further include the following processing: after controlling the above-mentioned robot to enter the elevator corresponding to the current waiting point, the elevator control system notifies the above-mentioned robot that it has arrived at the boarding point, and records the current position of the above-mentioned robot as one of the above-mentioned at least one boarding point; after the above-mentioned elevator control system controls the elevator door to close, the above-mentioned robot scans the horizontal two-dimensional size of the elevator, and optimizes the position information of the above-mentioned one boarding point according to the scanning result; the above-mentioned robot's scanning elevator operation is associated with the above-mentioned one boarding point, and is bound to the world map of the starting floor corresponding to the elevator and the world map of the destination floor.

[0028] Preferably, when the target points bound in the world maps of all the above floors include: at least one exit point, in the above step S102, binding the multiple target points on the movement trajectory of the above robot on the floor and the business operations corresponding to each of the above target points to the world map of the floor can further include the following processing: when the elevator entered by the above robot arrives at the current destination floor, after receiving the arrival notification from the elevator control system, triggering the execution of the map building operation or map update operation of the current destination floor; after receiving the elevator door opening completion notification from the elevator control system, triggering the execution of the robot's exit operation, and recording the current exit position of the robot as one of the above at least one exit point; associating the robot's exit operation with the above one exit point, and binding the world map of the current destination floor.

[0029] The following combination Figure 2 The above preferred embodiments are further described.

[0030] Figure 2 FIG. 1 is a flowchart of a method for creating a multi-layer map according to a preferred embodiment of the present invention. Figure 2 As shown, the multi-layer map creation method includes:

[0031] Step S201: At the initial moment, the robot's initial position is selected on the current floor (eg, F1 floor) and started, the robot establishes a world coordinate system, and based on the world coordinate system, establishes a world map, which is recorded as Map1.

[0032] Step S202: Control the movement of the robot (for example, manually push, remote control, etc.). During the movement of the robot, the world map is continuously updated based on the SLAM algorithm; at the same time, the elevator is identified based on the deep learning algorithm.

[0033] Step S203: Control the robot to move on the F1 layer and perform business operations. The robot constructs the Map1 scene map in real time and binds the robot's movement trajectory and business operations to the Map1 scene map based on real-time coordinates.

[0034] Step S204: After the map of this floor is created and the business operations are completed, the robot is controlled to stop in front of the elevator, and this point is set as waiting point 1. At the same time, the position of waiting point 1 is bound to the SLAM key frame. When SLAM closed-loop optimization occurs, the scene map and the position of waiting point 1 are optimized. For scenes with multiple elevators, multiple waiting points can be set, and each elevator corresponds to a waiting point.

[0035] Step S205: Interact with the elevator control system through wireless communication (including but not limited to WIFI, 3G, 4G, 5G, Lora, etc.) to perform elevator call operations.

[0036] Step S206: After the elevator responds to the robot's elevator call operation, it controls the elevator to reach the robot's floor F1, and after opening the elevator door, it notifies the robot through wireless communication that the elevator door is open and it can enter, and then controls the robot to enter the elevator; uses a deep learning algorithm to identify the position and orientation of the elevator on the world map during the robot's movement, and optimizes the position of the manually set elevator waiting point 1 and the orientation of the robot at the elevator waiting point 1 based on the recognition result, so that the elevator waiting point 1 is more suitable for the robot; at the same time, the robot records the world map built on this floor as F1 layer Map1, and the F1 layer world map is bound to the F1 layer business scenario (for example, multiple target points on the robot's movement trajectory on this floor, and the business operations corresponding to each of the above target points) and the spatial map when the elevator stops at this floor.

[0037] It should be noted that if there are multiple elevators, each elevator corresponds to a waiting point (of course, multiple waiting points can be set for one elevator, depending on the actual scenario), then there are multiple waiting points. For each set waiting point, the above steps need to be performed, so that each waiting point is optimized, making each waiting point more suitable for the robot to use.

[0038] Step S207: After the robot enters the elevator, it notifies the elevator control system that it has reached the boarding point, and the robot records the boarding point as boarding point 1. The elevator control system then controls the elevator door to close and moves to the predetermined floor; after learning that the elevator is closed, the robot scans the horizontal two-dimensional size of the elevator and records it as length L and width W, and optimizes the boarding point to an elevator area with a length of L and a width of W.

[0039] It should be noted that if there are multiple elevators, each elevator corresponds to a boarding point (of course, multiple boarding points can be set for one elevator, depending on the actual scenario), then there are multiple boarding points. For each set boarding point, the above steps need to be performed so that each boarding point is optimized, making each waiting point more suitable for the robot to use.

[0040] Step S208: After the elevator reaches the predetermined floor F2, the elevator control system notifies the robot that it has reached the predetermined first destination floor, and the robot triggers the elevator exit operation. At this time, a world map Map2 corresponding to the floor F2 is established.

[0041] Step S209: After the elevator completes door opening on the F2 floor, the elevator control system notifies the robot that the door opening is complete and controls the robot to execute the expected business logic on the F2 floor. The robot builds the Map2 scene map in real time and binds the robot's motion trajectory and business operations to the scene map Map2 based on real-time coordinates. The Map2 world map includes the business scene of the F2 floor and the spatial map of the elevator when it stops at this floor.

[0042] Step S210: After the robot completes the F2 layer business operation, it is controlled to stop in front of the elevator and set this point as elevator waiting point 2. At the same time, the position of elevator waiting point 2 is bound to the SLAM key frame. When SLAM closed-loop optimization occurs, the scene map and the position of elevator waiting point 2 are optimized; for scenes with multiple elevators, multiple elevator waiting points can be set.

[0043] Step S211: Interact with the elevator control system via wireless communication to perform elevator call operations.

[0044] Step S212: After the elevator responds to the robot's elevator call operation, it controls the elevator to reach the robot's floor F2, and after opening the elevator door, it notifies the robot through wireless communication that the elevator door is open and it can enter. At this time, the robot is manually controlled to enter the elevator; the deep learning module continuously identifies the position and direction of the elevator on the world map during the robot's movement, and optimizes the position of the set elevator waiting point 2 and the robot's orientation at the elevator waiting point 2 based on the recognition results, so that the elevator waiting point 2 is more suitable for the robot to use.

[0045] Step S213: According to business needs, the operations from steps S206 to S211 are executed repeatedly until the mapping of the entire floor is completed.

[0046] It should be noted that when the selected floor is a new floor (i.e. a floor that the robot has not mapped), a new world map (scene map) will be created and bound to the floor. When the selected floor is a floor that has already been mapped, update and replace operations can be performed. Update means to update the existing map of the floor in real time based on the latest scene; while replace operation deletes the previously built map and directly creates a new map.

[0047] Preferably, when the target points bound in the world maps of all the above-mentioned floors may further include: at least one waiting point, then in step S102, binding the multiple target points on the movement trajectory of the above-mentioned robot on the floor and the business operations corresponding to each of the above-mentioned target points to the world map of the floor may further include the following processing: before controlling the above-mentioned robot to arrive at at least one gate on the current floor, setting the waiting point position of the above-mentioned robot to one or more waiting points among the above-mentioned at least one waiting point, wherein each gate corresponds to a waiting point; binding the position information of the above-mentioned one or more waiting points to the SLAM key frame, and optimizing the world map of the floor and the above-mentioned one or more waiting points when the SLAM closed-loop optimization occurs; controlling the above-mentioned robot to interact with the gates corresponding to the above-mentioned one or more waiting points through wireless communication at the above-mentioned one or more waiting points to perform the gate opening operation; after receiving the gate opening completion notification, controlling the above-mentioned robot to pass through the gate, and identifying the position and orientation of the above-mentioned corresponding gate on the world map of the floor during the movement of the above-mentioned robot, and optimizing the position information of the above-mentioned one or more waiting points and the orientation information of the above-mentioned robot at the above-mentioned one or more waiting points based on the identification result.

[0048] In the preferred implementation process, during the execution of the above steps S201 to S212, when one or more automatic gates appear during the robot movement, it is necessary to control the robot to be stationary in front of the automatic gate, and set the above position points as one or more waiting points. The following takes a waiting point as an example for explanation. When an automatic gate appears, the robot is controlled to be stationary in front of the automatic gate, and the position is determined as waiting point 1. The position of waiting point 1 is bound to the SLAM key frame. When the SLAM closed-loop optimization occurs, the scene map and the position of waiting point 1 are optimized. ; interact with the automatic gate machine through wireless communication to perform the gate opening operation; after the automatic gate machine responds to the robot's gate opening operation, it controls the automatic gate machine to open the gate door, and after opening the gate door, it notifies the robot through wireless communication that the automatic gate machine has been opened and can pass through, and then controls the robot to pass through the automatic gate machine; the robot's deep learning module identifies the position and direction of the automatic gate machine on the world map during the robot's movement, and optimizes the position of the set waiting point 1 and the direction of the robot at the waiting point 1 based on the recognition result, so that the elevator waiting point 1 is more suitable for the robot to use.

[0049] Preferably, after creating a world map of all floors to be reached by the robot, the following processing may also be included: using the target points and the world map to generate a fully supported topological map; extracting the main path of the fully supported topological map, and deleting the paths other than the main path in the fully supported topological map; judging whether the distance between each two adjacent target points in the target points of the main path is less than a predetermined distance threshold, if the distance between the two adjacent target points is less than the predetermined distance threshold, judging whether there is a connection path between the two adjacent target points corresponding to the distance, if there is no connection path, supplementing the connection path between the two adjacent target points, and repeating this step until a final topological map is formed; binding the paths in the final topological map to the world map.

[0050] As mentioned above, through the above processing, multiple target points are set and bound to the world map. Therefore, a fully supported topological map can be generated based on the above target points and the world map. For example, the A* algorithm or Dijkstar algorithm in the relevant technology can be used to automatically generate a fully supported topological map including multiple paths. After that, the minimum support map can be obtained based on the fully supported topological map, that is, the trunk path of the fully supported topological map is extracted, and the paths other than the trunk path in the fully supported topological map are deleted; finally, it is determined whether the connection path needs to be supplemented according to whether the distance between each two adjacent target points is less than a predetermined distance threshold (for example, 5 meters), thereby reasonably supplementing the links between the adjacent target points, and finally forming a basic available topological map, and the paths in the topological map are also bound to the world map for robot work. In the process of automated operation, the robot uses a pre-planned path bound to the world map to more conveniently realize path navigation.

[0051] Preferably, after the world map of all floors to be reached by the robot is created in step S102, the following processing may also be included: during the automated operation of the robot, when it is necessary to take an elevator, if there are multiple elevators on the current floor, the elevator with the shortest path to the current position is determined as the elevator to be selected; the number of objects and the object density in the elevator waiting area where the elevator waiting point corresponding to the elevator to be selected is located are detected to determine whether the number of objects in the elevator waiting area is greater than a predetermined first threshold and whether the object density is greater than a predetermined second threshold. When the number of objects is greater than the predetermined first threshold or the object density is greater than the predetermined second threshold, the elevator with the shortest path to the current position among the optional elevators of the multiple elevators is determined as the elevator to be selected. This step is executed repeatedly. When the number of objects in the elevator waiting area is less than or equal to the predetermined first threshold and the object density is less than or equal to the predetermined second threshold, it is determined that the elevator to be waited for is found, the robot moves to the elevator waiting point of the elevator, and performs the business operation corresponding to the elevator waiting point.

[0052] In the preferred implementation process, during the robot's automated working process, when the elevator riding operation is to be performed, if there is only one elevator on the current floor, the elevator waiting area (the elevator waiting area can be a designated area centered on the elevator waiting point and surrounding the elevator waiting point, for example, a circular area formed by a predetermined number as the radius with the elevator waiting point as the center) is identified and analyzed based on the deep learning algorithm to identify the number of objects and the density of objects in the elevator waiting area, wherein the above-mentioned objects include: biological objects and non-biological objects, biological objects include: pedestrians, animals, etc., and non-biological objects include: goods, other robots, etc. When the number of objects and the density of objects are small and are both lower than or equal to the predetermined threshold, the robot determines to move to the elevator waiting point corresponding to the elevator to wait for the elevator. If the elevator waiting point is occupied by other objects, it can move to the area in the elevator waiting area where the elevator can be waited for the elevator according to the predetermined strategy. When the number of objects or the density of objects is large, and one value is greater than the predetermined threshold, it continues to wait, and performs the elevator waiting operation when it is detected that the number and density of pedestrians are lower than or equal to the predetermined threshold.

[0053] In the preferred implementation process, during the robot's automated working process, when performing an elevator operation, if there are multiple elevators on the current floor, the elevator with the shortest path from the current position can be selected as the first priority elevator based on the principle of minimum path distance, and before entering the waiting area of ​​the first priority elevator, the waiting area is identified and analyzed based on the deep learning algorithm to identify the number of objects and the density of objects in the waiting area. When the number of objects and the density of objects are both lower than or equal to a predetermined threshold, the robot determines to move to the waiting point corresponding to the elevator to wait for the elevator. If the waiting point is occupied by other objects, the robot can move to an area in the waiting area where it can wait for the elevator according to a predetermined strategy. If one of the number of objects or the density of objects is higher than a predetermined threshold, then search whether there is an alternative elevator available. If there is an alternative elevator, then based on the principle of the minimum path distance, continue to select the elevator with the shortest path as the second preferred elevator from the currently available elevators, and before entering the waiting area of ​​the second priority elevator, identify and analyze the waiting area based on the deep learning algorithm to identify the number of objects and the density of objects in the waiting area. When the number of objects and the density of objects are both lower than or equal to the predetermined threshold, the robot determines to move to the waiting point corresponding to the elevator to wait for the elevator. If the waiting point is occupied by other objects, it can move to the waiting area in the waiting area according to the predetermined strategy to wait for the elevator. If one of the number of objects or the density of objects is higher than the predetermined threshold, then continue to search whether there is an alternative elevator available, and repeat this step until an elevator that meets the conditions is found.

[0054] Preferably, after creating a world map of all floors to be reached by the above-mentioned robot, the following processing may also be included: during the above-mentioned robot's automated operation, when the elevator that the above-mentioned robot is waiting for arrives, the number and density of objects in the elevator are detected, and the space occupancy status of the elevator is analyzed; when the number of objects in the elevator is less than or equal to a predetermined third threshold, and the density of objects in the elevator is less than or equal to a fourth threshold, and there is unoccupied space in the elevator that can accommodate the above-mentioned robot, the above-mentioned robot determines to perform an elevator entry operation and performs corresponding business operations at the elevator boarding point; when the number of objects in the elevator is greater than the predetermined third threshold, or the density of objects in the elevator is greater than the fourth threshold, or the unoccupied space in the elevator cannot accommodate the above-mentioned robot, the above-mentioned robot determines to abandon the elevator entry operation, and continues to perform the elevator call operation after the elevator leaves the parking position on the current floor.

[0055] In the preferred implementation process, when the elevator arrives successfully after waiting for the elevator, the robot first analyzes the number and density of objects in the elevator based on the deep learning algorithm, and analyzes the elevator space occupancy based on stereo vision, Lidar and other sensors. When the number and density of objects are both lower than or equal to the predetermined threshold, and the unoccupied space in the elevator can accommodate the robot, the elevator entry operation is performed, and the corresponding business operation is performed at the elevator boarding point, for example, triggering the above-mentioned robot to perform the elevator scanning operation; when the number or density of objects is higher than the predetermined threshold, or the unoccupied space in the elevator cannot accommodate the robot, the elevator entry operation is abandoned, and the elevator call operation is continued after the elevator leaves the parking position on the current floor.

[0056] Preferably, after creating a world map of all floors to be reached by the robot, the following processing may also be included: during the automated operation of the robot, when the robot reaches the destination floor by elevator, the number and density of objects in the elevator are detected, and the occupancy status of the elevator space is analyzed; when it is determined that there is no object blocking the robot from exiting the elevator, a corresponding exit operation is triggered at the exit point of the elevator; when it is determined that objects less than a predetermined number threshold are blocking the robot from exiting the elevator, a prompt operation is performed, and when it is determined that there is no object blocking the robot from exiting the elevator, a corresponding exit operation is triggered at the exit point of the elevator; when it is determined that objects greater than or equal to a predetermined number threshold are blocking the robot from exiting the elevator, a prompt operation is performed, and communication with the elevator control system is performed to control the elevator door opening time, and when it is determined that there is no object blocking the robot from exiting the elevator, a corresponding exit operation is triggered at the exit point of the elevator.

[0057] In the preferred implementation process, when the robot takes the elevator to the predetermined destination floor, the robot first analyzes the number and density of pedestrians in the elevator based on a deep learning algorithm, and analyzes the elevator space occupancy based on stereo vision, Lidar, etc. When there is no object blocking the robot from exiting the elevator, the robot performs the exit operation; when there are a small number of objects (for example, pedestrians, etc.), but the robot still has space to exit the elevator, it can first prompt the pedestrians (for example, voice broadcast reminders), and when the reminder is effective and the pedestrians give way, the robot performs the exit operation; when there are a large number of pedestrians and the robot has no space to perform the exit operation, it can first prompt the pedestrians (for example, voice broadcast reminders), and communicate with the elevator control system within a certain period of time to control the length of time the elevator is open, and when the reminder is effective and the pedestrians give way, the robot performs the exit, and when the pedestrians do not give way, the robot abandons the exit operation and re-executes the elevator call operation after the elevator door closes.

[0058] Preferably, after creating the world maps of all floors to be reached by the robot, the following processing may also be included: during the automated operation of the robot, when it is necessary to load the world map of the current floor, determine whether the data volume of the world map corresponding to the current floor is greater than a predetermined data threshold; when the data volume is less than or equal to the predetermined data threshold, perform a global map loading operation of the world map when the robot is taking the elevator, perform local area map relocation based on the global map after the robot exits the elevator, and perform corresponding business operations when the robot moves to each target point according to the relocation result; when the data volume is greater than the predetermined data threshold, perform a local map loading operation of the world map when the robot is taking the elevator, perform local area map relocation based on the local map after the robot exits the elevator, and perform corresponding business operations when the robot moves to each target point according to the relocation result, and gradually realize the loading of the world map during the movement of the robot.

[0059] In the preferred implementation process, when the robot moves across floors, it is necessary to load the world map of the destination floor. In order to reduce resource consumption when loading the map, the data volume of the world map is judged. When the data volume of the map is less than or equal to a predetermined threshold, the full map of the destination floor is loaded while the robot is riding the elevator, and after the robot exits the elevator, the local area map is relocated based on the global map, and the robot business operations are performed based on the relocation results; when the data volume of the map is greater than the predetermined threshold, the robot loads the local map of the destination floor while the robot is riding the elevator, and relocates the map based on the local map after the robot exits the elevator, and performs corresponding business operations when the robot moves to each target point on the floor based on the relocation results; and the entire map is gradually loaded during the robot's movement.

[0060] According to another aspect of the present invention, a multi-layer map creation device is provided.

[0061] Figure 3 is a structural block diagram of a multi-layer map creation device according to an embodiment of the present invention. Figure 3As shown, the multi-layer map creation device includes: a control module 30, which is used to respond to control operations and control the robot to realize cross-floor movement between multiple floors based on the elevator; a mapping module 32, for each floor, establishes a world map of the floor based on the world coordinate system, and during the movement of the robot, updates the world map of the floor, and binds the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor, and the business operations corresponding to each of the target points to the world map of the floor, and repeats this step until the world maps of all floors to be reached by the robot are created, wherein the bound target points in the world maps of all the floors include: an initial position point, at least one elevator waiting point, at least one elevator boarding point, at least one elevator exit point, and at least one destination point.

[0062] Figure 3 In the device shown, the control module 30 responds to the control operation and controls the robot to realize cross-floor movement between multiple floors based on the elevator (for example, manually pushed, controlled by remote control, etc.); the mapping module 32 establishes a world map of the floor based on the world coordinate system for each floor, and updates the world map of the floor during the robot movement process, and binds the identified stop position of the elevator on the floor, multiple target points on the movement trajectory of the robot on the floor (including but not limited to: initial position point, at least one waiting point, at least one elevator point, at least one exit point, at least one destination point), and the business operations corresponding to each of the above target points to the world map of the floor, and executes this step repeatedly until the world map of all floors to be reached by the robot is created. Therefore, by controlling the robot to realize cross-floor movement between multiple floors based on the elevator, convenient mapping of multi-layer scenes can be realized, without too much participation of technical personnel and the use of special mapping tools, which improves the intelligence level and convenience of the robot. The constructed map has strong environmental adaptability and can be updated in time when local environmental changes occur. In addition, the above map is also bound to multiple target points and the business operations corresponding to each target point. During the robot's automated operation, when the robot runs to the corresponding target point, it can execute the business operations corresponding to the current target point, thereby realizing the business logic of the entire cross-floor operation.

[0063] Preferably, the mapping module 32 is further used to bind at least one gate-waiting point on the movement trajectory of the robot and the business operations corresponding to each of the at least one gate-waiting point to the world map of all the floors.

[0064] It should be noted that the preferred implementation of the combination of the modules in the above multi-layer map creation device can be found in Figure 1 to Figure 2 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0065] According to an embodiment of the present invention, a robot is provided.

[0066] Figure 4 is a structural block diagram of a robot according to an embodiment of the present invention. Figure 4 As shown, the robot according to the present invention includes: a memory 40 and a processor 42, wherein the memory 40 is used to store computer-executable instructions; the processor 42 is used to execute the computer-executable instructions stored in the memory, so that the robot executes the multi-layer map creation method provided in the above embodiment.

[0067] The processor 42 may be a central processing unit (CPU). The processor 102 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0068] The memory 40 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the target object recognition method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory.

[0069] The memory 40 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 40 may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0070] The one or more modules are stored in the memory 40 and when executed by the processor 42, the following steps are performed: Figure 1 and Figure 2 A multi-layer map creation method in the illustrated embodiment.

[0071] For details of the above robots, please refer to Figure 1 and Figure 2 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0072] In summary, with the above embodiments provided by the present invention, the robot can be controlled to realize cross-floor movement between multiple floors based on the elevator, so as to realize convenient mapping of multi-layer scenes, without too much participation of technical personnel and the use of special mapping tools, thereby improving the intelligence level and convenience of the robot. The constructed map has strong environmental adaptability and can be updated in time when local environmental changes occur. The above map is also bound with multiple target points and business operations corresponding to each target point. During the robot's automated operation, when the robot runs to the corresponding target point, the business operation corresponding to the current target point can be executed, thereby more effectively realizing the business process of the entire cross-floor operation. The positions of the waiting points and the waiting gate points in the target points and the orientation of the robot at the waiting points and the waiting gate points are optimized based on the deep learning algorithm, so that the waiting points and the waiting gate points are more suitable for the robot to use. In addition, the robot waiting process is realized by detection and analysis based on the deep learning algorithm, and the robot entry process and robot exit process are realized by detection and analysis based on the deep learning algorithm, and the logical strategy of space occupancy analysis such as stereoscopic vision and Lidar.

[0073] The above disclosures are only several specific embodiments of the present invention; however, the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A multi-layer map creation method, characterized in that: include: In response to the control operation, the robot is controlled to realize cross-floor movement between multiple floors based on the elevator; For each floor, a world map of the floor is established based on the world coordinate system. During the robot movement, the world map of the floor is updated, the identified elevator stop area on the floor is marked, and multiple target points on the robot's movement trajectory on the floor and the business operations corresponding to each of the target points are bound to the world map of the floor. This step is repeated until the world maps of all floors to be reached by the robot are created, wherein the bound target points in the world maps of all floors include: an initial position point, at least one elevator waiting point, at least one elevator boarding point, at least one elevator exit point, and at least one destination point; When the target points bound in the world map of all floors include at least one boarding point, binding multiple target points on the robot's motion trajectory on the floor and the business operations corresponding to each target point to the world map of the floor includes: after controlling the robot to enter the elevator corresponding to the current waiting point, the elevator control system notifies the robot that it has arrived at the boarding point, and records the robot's current position as one of the at least one boarding points; after the elevator control system controls the elevator door to close, the robot scans the horizontal two-dimensional size of the elevator, and optimizes the position information of the one boarding point according to the scanning result; the robot's scanning elevator operation is associated with the one boarding point, and bound to the world map of the starting floor corresponding to the elevator and the world map of the destination floor.

2. The multi-layer map creation method according to claim 1, characterized in that: When the target points bound in the world map of all floors include at least one waiting point, binding multiple target points on the motion trajectory of the robot on the floor and the business operations corresponding to each of the target points to the world map of the floor includes: Before the robot is controlled to arrive at at least one elevator on the current floor, the elevator waiting position of the robot is set to one or more elevator waiting points among the at least one elevator waiting point, wherein each elevator corresponds to one elevator waiting point; Binding the location information of the one or more elevator waiting points to the real-time positioning and mapping SLAM key frame, and optimizing the world map of the floor and the one or more elevator waiting points when the SLAM closed-loop optimization occurs; Controlling the robot to interact with the elevator control system at the one or more elevator waiting points through wireless communication to perform an elevator call operation; After controlling the robot to enter the elevator, identifying the position and orientation of the elevator on the world map during the movement of the robot, and optimizing the position information of the one or more elevator waiting points and the orientation information of the robot at the one or more elevator waiting points based on the identification result; The elevator call operation is associated with the one or more elevator waiting points and bound to the world map of the floor.

3. The multi-layer map creation method according to claim 1, characterized in that: When the target points bound in the world map of all floors include at least one exit point, binding multiple target points on the motion trajectory of the robot on the floor and the business operations corresponding to each of the target points to the world map of the floor includes: When the elevator that the robot enters reaches the current destination floor, after receiving the arrival notification from the elevator control system, a map building operation or a map update operation of the current destination floor is triggered; When receiving the elevator door opening completion notification from the elevator control system, triggering the robot to execute the elevator exit operation, and recording the current elevator exit position of the robot as one of the at least one elevator exit point; The robot's exit operation is associated with the one exit point and bound to the world map of the current destination floor.

4. The multi-layer map creation method according to claim 1, characterized in that: The target points bound in the world map of all floors also include: at least one gate waiting point, and binding multiple target points on the motion trajectory of the robot on the floor and the business operations corresponding to each of the target points on the world map of the floor includes: Before the robot reaches at least one gate on the current floor, the gate waiting position of the robot is set to one or more gate waiting points among the at least one gate waiting point, wherein each gate corresponds to one gate waiting point; Binding the location information of the one or more gate waiting points to the SLAM key frame, and optimizing the world map of the floor and the one or more gate waiting points when the SLAM closed-loop optimization occurs; Control the robot to interact with gate machines corresponding to the one or more gate waiting points through wireless communication to perform gate opening operations; After receiving the notification that the gate is opened, the robot is controlled to pass through the gate, and the world map position and orientation of the corresponding gate on the floor are identified during the movement of the robot, and the position information of the one or more waiting points and the orientation information of the robot at the one or more waiting points are optimized based on the identification results.

5. The multi-layer map creation method according to claim 1, characterized in that: After creating a world map of all floors to be reached by the robot, it also includes: Using the target point and the world map, generating a fully supported topological map; Extracting the trunk path of the fully supported topology map, and deleting the paths other than the trunk path in the fully supported topology map; Determine whether the distance between each two adjacent target points in the target points of the trunk path is less than a predetermined distance threshold; if the current distance between two adjacent target points is less than the predetermined distance threshold, determine whether there is a connection path between the two adjacent target points corresponding to the distance; if there is no connection path, supplement the connection path between the two adjacent target points, and repeat this step until a final topological map is formed; Binding the paths in the final topological map to the world map.

6. The multi-layer map creation method according to claim 1, characterized in that: After creating a world map of all floors to be reached by the robot, it also includes: During the automated operation of the robot, when it is necessary to take an elevator, if there are multiple elevators on the current floor, the elevator with the shortest path to the current location is determined as the elevator to be selected; The number and density of objects in the elevator waiting area where the elevator waiting point corresponding to the elevator to be selected is detected, and it is determined whether the number of objects in the elevator waiting area is greater than a predetermined first threshold value and whether the object density is greater than a predetermined second threshold value. When the number of objects is greater than the predetermined first threshold value or the object density is greater than the predetermined second threshold value, the elevator with the shortest path to the current position among the optional elevators of the multiple elevators is determined as the elevator to be selected, and this step is executed repeatedly. When the number of objects in the elevator waiting area is less than or equal to the predetermined first threshold value and the object density is less than or equal to the predetermined second threshold value, it is determined that the elevator to be waited for is found, and the robot moves to the elevator waiting point of the elevator to perform the business operation corresponding to the elevator waiting point.

7. The multi-layer map creation method according to claim 1, characterized in that: After creating a world map of all floors to be reached by the robot, it also includes: During the automated operation of the robot, when the elevator that the robot is waiting for arrives, the number and density of objects in the elevator are detected, and the occupancy status of the elevator space is analyzed; When the number of objects in the elevator is less than or equal to a predetermined third threshold, and the density of objects in the elevator is less than or equal to a fourth threshold, and there is no occupied space in the elevator that can accommodate the robot, the robot determines to perform an elevator entry operation and performs a corresponding business operation at the elevator boarding point; When the number of objects in the elevator is greater than a predetermined third threshold, or the density of objects in the elevator is greater than a fourth threshold, or the unoccupied space in the elevator cannot accommodate the robot, the robot determines to give up the elevator entry operation, and continues to perform the elevator call operation after the elevator leaves the parking area on the current floor.

8. The multi-layer map creation method according to claim 1, characterized in that: After creating a world map of all floors to be reached by the robot, it also includes: During the automated operation of the robot, when the robot arrives at the destination floor by elevator, the number and density of objects in the elevator are detected, and the occupancy status of the elevator space is analyzed; When it is determined that there is no object blocking the robot from exiting the elevator, triggering the corresponding exiting operation at the exiting point of the elevator; When it is determined that objects less than a predetermined number threshold value hinder the robot from exiting the elevator, a prompt operation is performed; when it is determined that no objects hinder the robot from exiting the elevator, a corresponding exit operation is triggered at the exit point of the elevator; When it is determined that objects greater than or equal to a predetermined number threshold are obstructing the robot from exiting the elevator, a prompt operation is performed, and communication with the elevator control system is carried out to control the length of time the elevator door is opened. When it is determined that no objects are obstructing the robot from exiting the elevator, the corresponding exit operation is triggered at the elevator exit point.

9. The multi-layer map creation method according to claim 1, characterized in that: After creating a world map of all floors to be reached by the robot, it also includes: During the automated operation of the robot, when a world map of a destination floor needs to be loaded, determining whether the amount of data in the world map corresponding to the destination floor is greater than a predetermined data threshold; When the data volume is less than or equal to a predetermined data threshold, a global map loading operation of the world map is performed when the robot is riding an elevator, and a local area map is relocated based on the global map after the robot exits the elevator, and corresponding business operations are performed when the robot moves to each target point according to the relocation result; When the data volume is greater than the predetermined data threshold, a local map loading operation of the world map is performed when the robot is taking the elevator, and a local area map is relocated based on the local map after the robot exits the elevator. According to the relocation result, corresponding business operations are performed when the robot moves to each target point, and the loading of the world map is gradually realized during the movement of the robot.

10. A multi-layer map creation device, characterized in that: include: A control module, used for responding to control operations and controlling the robot to realize cross-floor movement between multiple floors based on the elevator; A mapping module, for each floor, establishes a world map of the floor based on the world coordinate system, updates the world map of the floor during the robot movement, binds the identified elevator stop area on the floor, multiple target points on the robot's movement trajectory on the floor, and the business operations corresponding to each target point to the world map of the floor, and executes this step repeatedly until the world maps of all floors to be reached by the robot are created, wherein the target points bound in the world maps of all floors include: an initial position point, at least one elevator waiting point, at least one elevator boarding point, at least one elevator exit point, and at least one destination point; When the target points bound in the world map of all floors include at least one boarding point, binding multiple target points on the robot's motion trajectory on the floor and the business operations corresponding to each target point to the world map of the floor includes: after controlling the robot to enter the elevator corresponding to the current waiting point, the elevator control system notifies the robot that it has arrived at the boarding point, and records the robot's current position as one of the at least one boarding points; after the elevator control system controls the elevator door to close, the robot scans the horizontal two-dimensional size of the elevator, and optimizes the position information of the one boarding point according to the scanning result; the robot's scanning elevator operation is associated with the one boarding point, and bound to the world map of the starting floor corresponding to the elevator and the world map of the destination floor.

11. The multi-layer map creation device according to claim 10, characterized in that: The mapping module is further used to bind at least one gate waiting point on the movement trajectory of the robot and the business operations corresponding to each of the at least one gate waiting point to the world map of all floors.

12. A robot comprising: A memory and a processor, characterized in that: Memory for storing computer-executable instructions; A processor is used to execute the computer-executable instructions stored in the memory, so that the robot performs the method according to any one of claims 1 to 9.

Images