System and method for autonomous mobile robot to ride and co-share elevator with human(s)

Abstract

To provide a system and method for an autonomous mobile robot to ride and co-share an elevator with human(s).SOLUTION: Upon an elevator riding task is started, a human detection and location finding module, and a human identification and state estimation module detect and count at least one human inside and / or outside the elevator, and a human-robot-interaction module interacts with the at least one human. An elevator confined space positioning module carries out a space positioning inside the elevator according to a result of detecting and counting the at least one human through the human detection and location finding module and the human identification and state estimation module, and chooses to enter the elevator or restart another elevator riding task.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a system for an autonomous mobile robot to enter an elevator, and more particularly to a system and method for an autonomous mobile robot to ride with a human in an elevator. boarding

Background Art

[0002] Autonomous mobile robots (AMRs) are increasingly being used in various industries and applications to automate the movement and handling of various goods and materials.However, in most applications, AMRs can only operate within a single floor space or level within a building.This is because AMRs have limited ability to access, enter and exit, and operate an elevator.

[0003] Recently, smart elevators that can be controlled by AMRs via various communication channels have been introduced in modern urban buildings.These include communication interfaces for obtaining elevator information, elevator calls, elevator status monitoring, and call monitoring.

[0004] There are still challenges for AMRs to access most elevators in urban buildings that do not have a communication interface like existing elevators.Recent research has proposed AMRs that can operate with existing elevators, but they require human intervention (e.g., a human pressing a button) or an AMR - dedicated elevator that is not shared with humans. boarding

[0005] Therefore, in order to overcome the drawbacks of the prior art, there is a need to provide a system and method for an autonomous mobile robot to ride with a human in an existing elevator without human intervention or a communication interface with an elevator control system.

Summary of the Invention

Problems to be Solved by the Invention

[0006] The object of the present invention is to provide a system and method for an autonomous mobile robot (AMR) to ride in an elevator with humans without human intervention or a communication interface with an elevator control system. An AMR equipped with the system and method of the present invention can ride in an elevator with a crowd of humans. boarding It can interact with humans inside and respond to various exceptional cases. These functions are for elevators spanning multiple floors. boarding It supports the entire operating lifecycle. This allows the AMR to use the elevator without requiring an API (Application Programming Interface) for communication. Since most elevators do not have a smart communication interface, the AMR with the system and method of the present invention can interact with most types of elevators, eliminating the need to modify the elevator.

[0007] Another object of the present invention is to provide a system and method for an autonomous mobile robot (AMR) to ride in an elevator with a human without human intervention or a communication interface with the elevator control system. Its core software module can be used in existing AMR systems or newly created AMRs. An AMR equipped with the system and method of the present invention provides various functions, particularly the ability to perceive its surroundings and locate landmarks (e.g., button panels inside and outside the elevator), activate buttons, determine door status, elevator movement status (by sensors such as cameras, LiDAR, pressure sensors, barometers, and IMUs), standby position, space occupancy and clearance, and entry / exit routes, as well as interact with humans for safety and communicate intended movements. These features and functions enable the AMR to perform the necessary steps to ride in an elevator under normal circumstances, and to enable human interaction and exception handling. [Means for solving the problem]

[0008] According to one aspect of the present invention, a system is provided for an autonomous mobile robot (AMR) to ride in an elevator with a human without human intervention or a communication interface with an elevator control system. The system includes a human detection and positioning module, a human identification and state estimation module, a human-robot interaction module, and an elevator enclosed space positioning module. The human detection and positioning module is configured to detect and position at least one human relative to the AMR. The human identification and state estimation module is connected to the human detection and positioning module and is configured to identify and estimate the state of the at least one human. The human-robot interaction module is connected to the human identification and state estimation module. The elevator enclosed space positioning module is connected to the human detection and positioning module, the human identification and state estimation module, and the human-robot interaction module. boarding When the task is initiated, the human detection and positioning module and the human identification and state estimation module are configured to detect and count the at least one human inside and / or outside the elevator, and the human-robot interaction module is configured to interact with the at least one human. The elevator enclosed space positioning module is configured to perform spatial positioning within the elevator according to the results of detecting and counting the at least one human via the human detection and positioning module and the human identification and state estimation module, whether entering the elevator or another elevator. boarding Choose whether to resume the task.

[0009] In one embodiment, the system further includes a sensing and perception module configured to preprocess sensor data from a perception source, integrate information from the sensor data, and transmit the sensor data.

[0010] In one embodiment, the system further includes an elevator landmark detection and positioning module connected to the sensing and perception module, the elevator landmark detection and positioning module being configured to determine the location of the elevator door and the elevator buttons inside and outside the elevator according to the sensor data.

[0011] In one embodiment, the system further includes an elevator actuator module, the elevator actuator module is configured to operate the elevator button.

[0012] In one embodiment, the sensing and perceiving module is further configured to receive the perceiving source for filtering and fusion.

[0013] In one embodiment, the perception source is captured via a 2D / 3D camera, a 2D / 3D LiDAR, a sensor array, or a combination thereof.

[0014] In one embodiment, the human detection and localization module and the human identification and state estimation module are connected to the sensing and perception module to receive human characteristics and are configured to cooperate in providing the human-robot interaction module with human posture and the number of humans based on the human characteristics.

[0015] In one embodiment, the human-robot interaction module includes a human-machine interface (HMI) (e.g., a touchscreen panel) that allows a user to interact with the autonomous mobile robot, and is configured to provide input and receive visual displays or aids (e.g., facial expressions, prompts / captions).

[0016] In one embodiment, the human-robot interaction module includes an audio input / output array (speaker, microphone) for voice interaction with the at least one human.

[0017] In one embodiment, the human-robot interaction module includes LED signal indicators for additional visual display or assistance (e.g., different LED colors indicate the operating status of the AMR).

[0018] According to one aspect of the present invention, a method is provided for an autonomous mobile robot (AMR) to ride in an elevator with a person without human intervention or a communication interface with an elevator control system. The method includes (a) navigating the AMR to the elevator lobby, detecting and locating at least one person relative to the AMR, and having the AMR identify and estimate the state of the at least one person; and (b) having the AMR press a button on a call panel to the elevator boarding (c) the autonomous mobile robot detects whether the elevator doors are open or closed; (d) in response to the elevator doors being open, the autonomous mobile robot detects and counts at least one person inside or outside the elevator; and (e) performs spatial positioning inside the elevator according to the results of detecting and counting the at least one person via the autonomous mobile robot, and enters the elevator or enters another elevator. boarding This includes a step to choose whether to resume the task.

[0019] In one embodiment, the method further includes the step of (f1) determining the at least one person inside or outside the elevator, and determining the location and number of the at least one person.

[0020] In one embodiment, the method further includes the step of (f2) estimating the occupancy status of the elevator by referring to a 2D map of the space inside the elevator and determining which elevator floor panel to use.

[0021] In one embodiment, the method further includes (f3) navigating the autonomous mobile robot into the elevator.

[0022] In one embodiment, the method further includes (f4) determining an available position having a passable path and determining an elevator floor panel to use.

[0023] In one embodiment, the method further includes (f5) determining an optimal position to wait within the elevator.

[0024] In one embodiment, the method further includes (f6) interacting with the at least one human for exception handling in one or more situations where the at least one human is blocking the call panel, the at least one human is entering or exiting the elevator, or the at least one human is blocking the floor panel.

[0025] In one embodiment, interacting with the at least one human includes providing a visual display or an audio command on the HMI / LED.

[0026] In one embodiment, the method further includes (g1) stopping operation in response to the at least one human entering a safety stop zone of the autonomous mobile robot and warning the at least one human using a visual display on the HMI / LED or the audio command.

[0027] In one embodiment, the method further includes (g2) notifying the at least one human within the elevator of the intended movement of the autonomous mobile robot using a visual display on the HMI / LED or the audio command.

Brief Description of the Drawings

[0028] The above-mentioned aspects of the present invention will become more readily apparent to those skilled in the art upon examination of the following detailed description and accompanying drawings.

[0029] [Figure 1] This is a schematic diagram illustrating a system for an autonomous mobile robot to ride in an elevator with a human, according to one embodiment of the present invention.

[0030] [Figure 2] This diagram schematically illustrates how an autonomous mobile robot equipped with the system of the present invention determines the open state of an elevator door. [Figure 3] This diagram schematically illustrates how an autonomous mobile robot equipped with the system of the present invention determines the closed state of an elevator door.

[0031] [Figure 4A] The following schematic diagram illustrates a random case after an autonomous mobile robot equipped with the system of the present invention scans its vicinity inside an elevator and counts the number of people. [Figure 4B] The following schematic diagram illustrates a random case after an autonomous mobile robot equipped with the system of the present invention scans its vicinity inside an elevator and counts the number of people.

[0032] [Figure 5A] Figure 4A schematically illustrates how an autonomous mobile robot determines the occupancy status of an elevator. [Figure 5B] Figure 4B schematically illustrates how an autonomous mobile robot determines the occupancy status of an elevator.

[0033] [Figure 6A] Figure 5A schematically illustrates how the autonomous mobile robot determines available locations with traversable paths and selects the elevator floor panel to be used. [Figure 6B] Figure 5B schematically illustrates how the autonomous mobile robot determines available locations with traversable paths and selects the elevator floor panel to be used.

[0034] [Figure 7A] The following schematic diagram illustrates a random case after an autonomous mobile robot equipped with the system of the present invention scans its vicinity inside an elevator and counts the number of people. [Figure 7B] The following schematic diagram illustrates a random case after an autonomous mobile robot equipped with the system of the present invention scans its vicinity inside an elevator and counts the number of people.

[0035] [Figure 8A] Figure 7A schematically illustrates how the autonomous mobile robot determines the optimal waiting position inside the elevator. [Figure 8B] Figure 7B schematically illustrates how the autonomous mobile robot determines the optimal waiting position inside the elevator.

[0036] [Figure 9] This is a flowchart showing a method for an autonomous mobile robot to ride in an elevator with a human, according to one embodiment of the present invention. [Modes for carrying out the invention]

[0037] The present invention will be described more specifically with reference to the following embodiments. Note that the following description of preferred embodiments of the present invention is provided herein for illustrative and explanatory purposes only. It is not intended to be exhaustive or to be limited to the exact form disclosed.

[0038] Figure 1 is a schematic diagram showing a system for an autonomous mobile robot to ride in an elevator with a human, according to one embodiment of the present invention. The present invention provides a system 1 for an autonomous mobile robot (AMR) to ride in an elevator with a human without human intervention or a communication interface with the elevator control system. Therefore, system 1 can be applied to conventional / existing elevators, smart elevators, etc. System 1 includes a human detection and positioning module 40, a human identification and state estimation module 50, a human-robot interaction module 60, and an elevator enclosed space positioning module 70. The human detection and positioning module 40 is configured to detect and position at least one human relative to the autonomous mobile robot (AMR). The human identification and state estimation module 50 is connected to the human detection and positioning module 40 and is configured to identify and estimate the state of at least one human. The human-robot interaction module 60 is connected to the human identification and state estimation module 50. The elevator enclosed space positioning module 70 is connected to the human detection and positioning module 40, the human identification and state estimation module 50, and the human-robot interaction module 60.

[0039] System 1 further includes a sensing and perception module 10 for preprocessing sensor data (e.g., filtering), information integration (e.g., joining of two LiDAR point clouds, color and depth image alignment), information preprocessing (e.g., image feature extraction and processing of pressure sensor data), and transmission of the original sensors via communication. The sensing and perception module 10 is preferably, but not limited to, connected to a human detection and localization module 40 and a human identification and state estimation module 50 and configured to receive perception sources for filtering and fusion. Perception sources can be captured via 2D / 3D cameras, 2D / 3D LiDAR, sensor arrays, or a combination thereof. The present invention is not limited thereto. In this embodiment, the human detection and localization module 40 and the human identification and state estimation module 50 are connected to the sensing and perception module 10 to receive human features and collaborate to provide the human-robot interaction module 60 with human posture and human count based on the human features.

[0040] System 1 also includes an elevator landmark detection and positioning module 20 connected to the sensing and perception module 10, and an elevator actuator module 30 connected to the elevator landmark detection and positioning module 20. The elevator landmark detection and positioning module 20 can locate the elevator doors and the elevator buttons inside and outside. The elevator actuator module 30 can activate / press the elevator buttons. With the above modules of System 1, the AMR can detect landmarks in the elevator lobby, move in front of the elevator call button panel and press the elevator call button, detect the elevator door and determine the door status, wait for the elevator door to open and enter the elevator, change its orientation inside the elevator to detect the elevator floor buttons, move to the elevator floor button panel and press the elevator floor buttons, change its orientation inside the elevator to detect the elevator door, wait for the elevator door to open and exit the elevator. In other words, the AMR can ride the elevator under normal conditions.

[0041] In particular, in this embodiment, the human-robot interaction module 60 enables the AMR to interact with humans in a safe, efficient, and effective manner. The human-robot interaction module 60 preferably includes, but is not limited to, 1) a human-machine interface (HMI) 61 (e.g., a touchscreen panel) on which a user or passenger can interact with the AMR, provide input, and receive visual displays or aids (e.g., facial expressions, prompts / captions); 2) an audio input / output array 63 (e.g., a microphone, a speaker) for voice interaction; and 3) an LED signal indicator 62 (e.g., different LED colors indicating the operating status of the AMR) for additional visual displays or aids. These modules of System 1 enable the AMR to interact with humans for exception handling.

[0042] A method for an autonomous mobile robot to ride in an elevator with a human being is also disclosed based on System 1 of the present invention. Refer to the embodiment in Figure 2. In order to ride the elevator, it is necessary to navigate AMR 1a to the elevator lobby and locate the elevator doors and elevator buttons (inside and outside). In this embodiment, AMR 1a is enabled to detect and locate at least one human being to AMR 1a via the human detection and location module 40 and to identify and estimate the state of at least one human being via the human identification and state estimation module 50. Preferably, AMR 1a with System 1 approaches the call panel 81 and presses a button on the call panel 81 to enter the elevator 8 boarding Tasks can be started, but are not limited to them. (elevator) boarding Once the task is initiated, the AMR 1a can detect the elevator 8 and determine whether the elevator doors are open or closed. Furthermore, as shown in Figure 2, when the elevator doors are open, at least one person 9 inside and outside the elevator 8 is detected and counted via the person detection and positioning module 40 and the person identification and state estimation module 50. Subsequently, the elevator enclosed space positioning module 70 performs spatial positioning within the elevator 8 according to the results of detecting and counting at least one person via the person detection and positioning module 40 and the person identification and state estimation module 50, and determines whether to enter the elevator 8 or another elevator in the elevator 8. boarding The user chooses whether to resume the task. On the other hand, as shown in Figure 3, if the elevator 8 is in a closed position or if the elevator door is blocked by at least one person 9, the AMR 1a can interact with at least one person 9 via the human-robot interaction module 60 for safety and communicate the intended movement. Further details will be provided later.

[0043] In this embodiment, the AMR 1a can determine the location and number of at least one person 9 inside and outside the elevator 8. To effectively share the elevator 8 with at least one person 9, the AMR 1a needs to position itself within the elevator 8 depending on where the person(s) 9 are standing inside the elevator 8. In particular, after the elevator enclosed space positioning module 70 performs spatial positioning within the elevator 8, the AMR 1a can scan its vicinity within the elevator 8 and count the person(s) 9. After the scan, the AMR 1a can perform elevator occupancy estimation to identify available and passable space within the elevator 8. Figures 4A-4B show two random cases after the AMR 1a has scanned its vicinity within the elevator 8 and counted the person(s). In this embodiment, the AMR 1a estimates the elevator occupancy by referring to a 2D map and determines which elevator floor panel to use. Furthermore, the AMR 1a determines available locations with traversable paths and determines which elevator floor panel to use. In this embodiment, the elevator 8 has a plurality of vertical elevator floor panels P2, P4 and horizontal elevator floor panels P1, P3. The AMR 1a may use only one selected panel, or predetermine which panels to use. In the random case of Figure 4A, the AMR 1a performs elevator occupancy estimation to identify passable space T (as shown in Figure 5A). In the random case of Figure 4B, the AMR 1a performs elevator occupancy estimation to identify empty space V and passable space T. Preferably, based on the elevator occupancy, the AMR 1a can determine the optimal waiting position (i.e., the best position to wait in the elevator 8) by considering a predefined priority panel (e.g., the vertical left elevator floor panel P4), a safe distance or maximum distance from human passengers, and the shortest distance from the current AMR posture, but is not limited to this. The optimal waiting position can be determined based on the lowest decision cost, and the present invention is not limited thereto.In the random cases shown in Figures 4A and 5A, AMR 1a determines that elevator floor panel P2 is the best option and can access elevator floor panel P2 (as shown in Figure 6A). In the random cases shown in Figures 4B and 5B, AMR 1a is predetermined to use only elevator floor panel P4. As shown in Figure 6B, after calculation, elevator floor panel P4 becomes inaccessible, and AMR 1a interacts with the human 9 via the human-robot interaction module 60 for exception handling, for example by providing a visual indication on the HMI 61 / LED signal indicator 62 or by providing an audio output via the voice input / output array 63 to notify the passenger to move aside.

[0044] After determining the target destination floor, AMR 1a is navigated into the elevator. To navigate AMR 1a into the elevator, it can wait at the elevator entrance, midway between the elevator door and the elevator center. Of course, the navigation route can be adjusted according to actual requirements, and the present invention is not limited thereto.

[0045] In this embodiment, the AMR 1a needs to position itself within the elevator depending on where one or more people are standing inside. Once the AMR 1a estimates the elevator's occupancy status by referring to a 2D map and determines available positions, it can wait along passable paths and determine the optimal position to take. In this embodiment, the AMR 1a can determine the optimal position to wait within the elevator by considering a predefined preferred position (e.g., center position), a safe distance or maximum distance from human passengers, and the AMR's next planned position (based on the next task). Figures 7A and 7B show two random cases after the AMR 1a scans its vicinity within the elevator 8 and counts people 9. As shown in the random case of Figure 7A, the AMR 1a performs elevator occupancy status estimation to identify passable space T. As shown in the random case of Figure 7B, the AMR 1a performs elevator occupancy status estimation to identify empty space V and passable space T. In the random case shown in Figure 7A, AMR 1a can move from its current orientation F1 to the optimal standby orientation F2 and wait for the next task F3 (as shown in Figure 8A). Similarly, in the random case shown in Figure 7B, AMR 1a can move from its current orientation F1 to the optimal standby orientation F2 and wait for the next task F3 (as shown in Figure 8B). Of course, AMR 1a can position itself within the elevator according to the practical requirements for effectively sharing the elevator with humans, and the present invention is not limited thereto.

[0046] When AMR 1a arrives at the destination floor, it navigates from the elevator to the elevator lobby of that floor. In the elevator lobby, AMR 1a switches to a map of the destination floor and adjusts its orientation and position based on the map. Of course, the present invention is not limited to this.

[0047] In particular, elevators boardingThroughout the entire process, the AMR needs to detect, count, and locate any people (one or more) in its vicinity (e.g., elevator lobby, inside the elevator). This is necessary when sharing the elevator with people or for handling related exception cases. Possible scenarios include: a passenger(s) blocking the elevator call button panel; a passenger(s) entering the elevator (AMR enters the elevator); a passenger(s) exiting the elevator (AMR enters the elevator); a passenger(s) blocking the elevator entrance; the elevator being full (AMR enters the elevator); a passenger(s) blocking the elevator floor button panel; the AMR determining the optimal position to wait in the elevator (depending on where the person(s) are standing); the central / priority position in the elevator being occupied; a passenger(s) entering the elevator (AMR exits the elevator); a passenger(s) exiting the elevator (AMR exits the elevator); and a passenger(s) blocking the elevator entrance. If any of the above scenarios occur in which the AMR needs to interact with a person for safety reasons and communicate intended movements, the AMR equipped with System 1 of the present invention will... boarding Throughout its entire operational lifecycle, it can interact with humans (one or more) for exception handling.

[0048] If a passenger(s) is blocking the elevator call button panel, the AMR equipped with System 1 of the present invention will identify the current state of the button by using a visual display on the HMI / LED of the Human-Robot Interaction Module 60 or a voice command (voice output) to interact with the human, asking the passenger(s) to move aside so that they can see the panel, or by notifying the passenger(s).

[0049] Furthermore, when the AMR is pressing an elevator call button or an elevator floor button, the AMR can also assist people in pressing the necessary elevator buttons via the human-robot interaction module 60. The AMR can ask passengers if there is a button they want to press and, based on their response, can press the necessary button. When passengers enter the elevator (when the AMR enters or exits the elevator), or when passengers block the elevator floor button panel, the AMR can notify passengers by using visual displays on the HMI / LEDs of the human-robot interaction module 60 or by using voice commands (voice output) to interact with humans. The AMR can wait for passengers to enter or exit first (for safety and collision avoidance purposes), assess the remaining time until the doors close and the elevator's current occupancy capacity, and then initiate action (e.g., enter the next elevator or board the next elevator). If the middle / priority position in the elevator is occupied, the AMR can notify passengers using visual displays on the HMI / LEDs or by using voice commands to inform passengers of the AMR's intended movement (e.g., move left or move right). In this embodiment, the AMR can use visual displays on the HMI / LED or voice commands to notify passengers that the AMR is moving to a specific location in the elevator (e.g., the center) and request them to move to the side. In this embodiment, exiting the elevator takes precedence over entering it, and the AMR will voice-inform passengers that it is exiting the elevator and request them to move to the side. The AMR will begin to descend from the elevator once passengers have moved, but will stop moving if passengers do not move aside. This is for safety and collision avoidance. In this embodiment, if the AMR needs to interact with humans for safety reasons and communicate its intended movements, the AMR can notify the person(s) in the elevator of its intended movement (e.g., move left or move right).When a person enters a safety stop zone (for example, within 30 cm around the AMR, depending on the AMR's safety scheme), the AMR stops operating and warns the person(s) via visual display on the HMI / LED or audio command. Of course, an AMR equipped with System 1 of the present invention can perform many functions for riding in an elevator with a person without human intervention or a communication interface with the elevator control system. The present invention is not limited to the embodiments described above, and redundant descriptions are omitted below.

[0050] Figure 9 is a flowchart illustrating a method for an autonomous mobile robot to ride in an elevator with a human, according to one embodiment of the present invention, applicable to the system shown in Figure 1. As shown in Figure 9, the method includes steps S01 to S05. In step S01, the autonomous mobile robot is navigated to the elevator lobby. In this embodiment, the autonomous mobile robot can detect and locate at least one human relative to the autonomous mobile robot, and identify and estimate the state of at least one human. In step S02, the autonomous mobile robot presses a button on the call panel to enter the elevator boarding The task is initiated. In step S03, the autonomous mobile robot detects whether the elevator doors are open or closed. In step S04, if the elevator doors are open, the autonomous mobile robot further detects and counts at least one person inside or outside the elevator. In step S05, the autonomous mobile robot performs spatial positioning within the elevator according to the results of detecting and counting at least one person, and determines whether to enter the elevator or another elevator. boarding Choose whether to resume the task. By performing the necessary steps S01-S05, the autonomous mobile robot will return to the elevator in its normal state. boarding This enables interaction with humans and exception handling. As a result, autonomous mobile robots can use elevators without modifying them.

[0051] In summary, the present invention provides a system and method for an autonomous mobile robot (AMR) to ride in an elevator with humans without human intervention or a communication interface with an elevator control system. An AMR equipped with the system and method of the present invention can ride in an elevator with a crowd of humans. boarding It can interact with people inside and respond to various exceptional cases. These functions are for elevators spanning multiple floors. boarding The system supports the entire operational lifecycle. This allows the AMR to use the elevator without requiring an API (Application Programming Interface) for communication. Given that most elevators do not have such a smart communication interface, the AMR with the system and method of the present invention does not require modification of the elevator, as it enables interaction with most types of elevators. Its core software module can be used in existing AMR systems or newly created AMRs. The AMR with the system and method of the present invention provides various functions, particularly the ability to perceive the surroundings and locate landmarks (e.g., button panels inside and outside the elevator), activate buttons, determine door status, elevator movement status (by sensors such as cameras, LiDAR, pressure sensors, barometers, and IMUs), determine standby positions, space occupancy and clearance, and determine entry and exit routes, as well as interact with humans for safety and communicate intended movements. These features and functions enable the AMR to perform the necessary steps to board the elevator under normal circumstances, and to enable human interaction and exception handling.

[0052] While the present invention has described what is currently considered to be the most practical and preferred embodiment, it should be understood that the present invention is not limited to the disclosed embodiment. On the contrary, the present invention is intended to cover a variety of modifications and similar configurations that fall within the spirit and scope of the appended claims, and should be interpreted in the broadest way to encompass all such modifications and similar configurations.

Claims

1. This is a system for autonomous mobile robots to ride elevators together with humans. A human detection and localization module configured to detect and locate at least one human for an autonomous mobile robot (AMR), A human identification and state estimation module connected to the human detection and location module and configured to identify and estimate the state of at least one human, A human-robot interaction module connected to the aforementioned human identification and state estimation module, The elevator enclosed space positioning module connected to the human detection and location module, the human identification and state estimation module, and the human-robot interaction module, Includes, When the elevator boarding task is initiated, the human detection and location module and the human identification and state estimation module are configured to detect and count the at least one human inside and / or outside the elevator, and the human-robot interaction module is configured to interact with the at least one human. The elevator enclosed space positioning module performs spatial positioning within the elevator according to the results of detecting and counting the at least one person via the person detection and location module and the person identification and state estimation module, identifies available space and passable space to determine the optimal waiting position, the AMR further determines available positions with passable paths and determines the elevator floor panel to use, and according to the available positions with passable paths and the elevator floor panel to use, the AMR is configured to determine the optimal position to wait in the elevator and choose whether to enter the elevator or resume another elevator boarding task. system.

2. The system according to claim 1, further comprising a sensing and perceptual module configured to receive sensor data for filtering and fusion.

3. The system according to claim 2, further comprising an elevator landmark detection and localization module connected to the sensing and perception module and configured to determine the location of the elevator door and elevator buttons inside and outside the elevator according to the sensor data.

4. The system according to claim 3, further comprising an elevator actuator module configured to operate the elevator buttons.

5. The system according to claim 2, wherein the sensor data is captured via a 2D / 3D camera, a 2D / 3D LiDAR, a sensor array, or a combination thereof.

6. The system according to claim 2, wherein the human detection and localization module and the human identification and state estimation module are connected to the sensing and perception module to receive human characteristics and are configured to cooperate in providing the human-robot interaction module with human posture and the number of humans based on the human characteristics.

7. The system according to claim 1, wherein the human-robot interaction module includes a human-machine interface (HMI) and / or voice input / output array that enables a user to interact with the autonomous mobile robot, and is configured to provide input and receive visual displays or voice assistance.

8. The system according to claim 7, wherein the human-robot interaction module includes an LED signal indicator for additional visual display or assistance.

9. A method for an autonomous mobile robot to ride in an elevator with a human, (a) Navigating an autonomous mobile robot (AMR) to the elevator lobby, detecting and locating at least one human being relative to the AMR, and having the AMR identify and estimate the state of the at least one human being; (b) The step of pressing a button on the call panel via the autonomous mobile robot to start the elevator boarding task, (c) The autonomous mobile robot detects whether the elevator doors are open or closed, (d) In response to the elevator being in the open door state, the autonomous mobile robot detects at least one person inside or outside the elevator and counts the at least one person; (e) Performing spatial positioning within the elevator according to the results of detecting and counting the at least one person via the autonomous mobile robot, identifying available space and passable space to determine an optimal waiting position, the AMR further determines an available position with a passable path, determines the elevator floor panel to use, and, according to the available position with a passable path and the elevator floor panel to use, the AMR determines the optimal position to wait in the elevator and selects whether to enter the elevator or resume another elevator boarding task, method.

10. (f1) The method according to claim 9, further comprising the step of determining the at least one person inside or outside the elevator, and determining the location and number of the at least one person.

11. (f2) The method of claim 9, further comprising the step of estimating the occupancy status of the elevator by referring to a 2D map of the space inside the elevator and determining the elevator floor panel to be used.

12. (f3) The method according to claim 9, further comprising the step of navigating the autonomous mobile robot into the elevator.

13. The method of claim 9, wherein in step (e), the AMR determines the optimal waiting position according to a predefined priority panel, a safe distance or maximum distance from human passengers, and the shortest distance from the current position of the AMR, the predefined priority panel being for determining the elevator floor panel to use.

14. The method of claim 9, wherein in step (e), the AMR determines the optimal position to wait in the elevator according to a predefined priority position, a safe distance or maximum distance from human passengers, and the AMR's next scheduled position based on the AMR's next task.

15. The method of claim 9, wherein in step (e), the AMR determines the optimal position to wait in the elevator based on a plurality of circumstances, the plurality of circumstances including where a person is standing, whether a predefined priority position is occupied, and whether a passenger is entering or exiting the elevator, blocking the entrance to the elevator, or whether the AMR is interacting with the AMR while it is getting out of the elevator.

16. (f4) The method of claim 9, further comprising the step of interacting with the at least one person for exception handling in one or more situations in which the at least one person is blocking the call panel, the at least one person is entering or exiting the elevator, or the at least one person is blocking the floor panel.

17. The method according to claim 16, wherein the step of interacting with at least one human being includes providing a visual display on an HMI / LED or an audio command.

18. (g1) The method according to claim 17, further comprising the step of stopping operation in response to the at least one human entering the safe stop zone of the autonomous mobile robot and warning the at least one human by a visual display on the HMI / LED or by using the voice command.

19. (g2) The method of claim 17, further comprising the step of notifying the at least one person in the elevator of the intended movement of the autonomous mobile robot by means of a visual display on the HMI / LED or the voice command.

Images