Robot control system and robot control method

Abstract

This invention provides a robot control system and a robot control method that enable robots to perform tasks efficiently. [Solution] The robot control system according to this embodiment is a robot control system for controlling an autonomous mobile robot, which sets a plurality of waypoints on a map that are candidates for waypoints for the autonomous mobile robot, assigns attributes to the waypoints for setting robot movements, searches for a path to the destination based on the plurality of waypoints, controls the autonomous mobile robot to pass through the waypoints to the destination in order, and when it arrives at a waypoint to which an attribute has been assigned, the autonomous mobile robot performs a robot movement.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a robot control system and a robot control method. BACKGROUND ART

[0002] Patent Document 1 discloses a path generation device that generates a path for a mobile robot. This path generation device divides corridors, rooms, and the like that form the path of the robot into appropriate units, and stores map data including node elements that are information of each of the divided units, and connectors that are information for connecting adjacent node elements between which the robot can move. When a movement start point and a movement target point are given, the path generation device generates a list of connectors that the robot uses to pass through. The list of connectors includes information on door opening and closing operations when there is a door, and information for pressing a required button when the robot boards an elevator. PRIOR ART DOCUMENTS PATENT DOCUMENTS

[0003] PATENT DOCUMENT 1 Japanese Unexamined Patent Publication No. 2006-259963 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] In such a device, there is a demand to control the robot so that the robot can efficiently execute tasks. MEANS FOR SOLVING THE PROBLEMS

[0005] The robot control system according to this embodiment is a robot control system for controlling an autonomous mobile robot, which sets a plurality of waypoints on a map that are candidates for waypoints for the autonomous mobile robot, assigns attributes to the waypoints for setting robot movements, searches for a path to a destination based on the plurality of waypoints, controls the autonomous mobile robot to pass through the waypoints to the destination in order, and when it arrives at a waypoint to which the attributes have been assigned, the autonomous mobile robot performs a robot movement.

[0006] The robot control method according to this embodiment is a robot control method for controlling an autonomous mobile robot, wherein a plurality of waypoints that are candidates for waypoints for the autonomous mobile robot are set on a map, attributes for setting robot movements are assigned to the waypoints, a path to a destination is searched based on the plurality of waypoints, the autonomous mobile robot is controlled to pass through the waypoints to the destination in order, and when the autonomous mobile robot arrives at a waypoint to which the attributes have been assigned, the autonomous mobile robot performs a robot movement. [Effects of the Invention]

[0007] This disclosure provides a robot control system and a robot control method that can control a robot to perform tasks efficiently. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing the overall configuration of the robot control system according to this embodiment. [Figure 2] This is a block of the system's control system. [Figure 3] This table shows the correspondence between waypoint attributes and actions. [Figure 4] This diagram explains the restricted areas and waypoints set in the facility's map information. [Figure 5] This is a flowchart showing the control method. [Modes for carrying out the invention]

[0009] The present invention will be described below through embodiments of the invention, but the invention as claimed is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means of solving the problem.

[0010] (Overall structure) The control system according to this embodiment is a system for managing autonomous mobile units. Figure 1 is a schematic diagram showing the configuration of the control system 1. The control system 1 comprises a control device 100, a robot 200, a camera 500, a network 600, a user terminal 400, and an accessory unit 700. The control system 1 is a system for managing multiple robots 200. The control device 100 manages the movement and tasks of the multiple robots 200.

[0011] Robot 200 is an autonomous mobile device that performs tasks such as transport. Robot 200 autonomously moves around medical and welfare facilities such as hospitals, rehabilitation centers, nursing homes, and elderly care facilities. Robot 200 is used to transport pharmaceuticals, medical equipment, meals, tableware, medical records, supplies, specimens, linens, and people. The transported object may also be a person, such as a patient. Furthermore, the system according to this embodiment can also be used in commercial facilities such as shopping malls. Robot 200 has wheels, a chassis, motors, sensors, a battery, a controller, etc.

[0012] Robot 200 is equipped with distance measuring sensors such as LiDAR. These sensors measure the distance to surrounding objects, such as people, around Robot 200. Robot 200 then autonomously navigates to avoid contact with surrounding objects. For example, Robot 200 has a virtual bumper function. With the virtual bumper function, if a surrounding object approaches Robot 200 to a predetermined distance or less, Robot 200 will slow down or stop.

[0013] At least one of the robots 200 is of a different type. All robots 200 may be of the same type. Each robot 200 is assigned a unique identification number (ID). Figure 1 shows three robots 200, but the number of robots is not particularly limited as long as there are multiple robots.

[0014] Furthermore, at least one of the robots 200 may perform tasks other than transport tasks. These other tasks may include cleaning, security, and guidance tasks. The robot 200 may perform multiple tasks such as cleaning, security, and guidance using the accessory units 700, or it may perform tasks independently. For example, the robot 200 can perform various tasks by using the accessory units 700 in combination with the robot 200. The robot 200 may be equipped with different accessory units depending on the task. By changing the accessory units 700, the robot 200 becomes a multitasking robot capable of performing multiple tasks.

[0015] For transport tasks, the auxiliary unit 700 is a wheeled cart or wagon for carrying the transported items. For cleaning tasks, the auxiliary unit 700 has a vacuum cleaner for sucking up dirt and debris. For security tasks, the auxiliary unit 700 has sensors such as LiDAR (Light Detection And Ranging) and cameras. For guidance tasks, the auxiliary unit 700 has signs or displays. In the following description, the robot 200 will primarily perform the transport tasks.

[0016] User U1 or User U2 can use the user terminal 400 to submit task requests, such as requests for the transport of goods. For example, the user terminal 400 may be a tablet computer or a smartphone. The user terminal 400 can be any information processing device capable of wireless or wired communication.

[0017] In the present embodiment, the robot 200 and the user terminal 400 are connected to the control device 100 via the network 600. The network 600 is a wired or wireless LAN (Local Area Network) or WAN (Wide Area Network). Furthermore, the control device 100 is connected to the network 600 by wire or wirelessly. For communication between the respective devices, for example, one conforming to a general-purpose communication standard such as Wi-Fi (registered trademark) can be used.

[0018] Various signals transmitted from the user terminals 400 of the users U1 and U2 are once sent to the control device 100 via the network 600, and are transferred from the control device 100 to the target robot 200. Similarly, various signals transmitted from the robot 200 are once sent to the control device 100 via the network 600, and are transferred from the control device 100 to the target user terminal 400. The control device 100 is a server connected to each device, and collects data from each device. Further, the control device 100 is not limited to a physically single device, and may include a plurality of devices that perform distributed processing. In addition, the control device 100 may be arranged dispersedly in edge devices such as the robot 200. For example, a part or all of the control system 1 may be mounted on the robot 200.

[0019] The robot 200 includes a drive motor, wheels, a battery, and the like. Furthermore, the robot 200 includes a sensor such as a camera and LiDAR, and an arithmetic processing unit such as a processor. The robot 200 estimates its own position based on the detection result of the sensor. The robot 200 autonomously moves along a route from a departure point to a destination on a map based on its own position. The departure point is the current position of the robot, and the destination is the transport destination of the transported object. In addition, route search may be performed using a transport source of the transported object or the like as a transit point. Note that the control device 100 may perform route search from the departure point to the destination, or the robot 200 may perform the route search.

[0020] The user terminal 400 and the robot 200 may transmit and receive signals without going through the control device 100. For example, the user terminal 400 and the robot 200 may directly transmit and receive signals via wireless communication. In addition, the control device 100 may collect data from the camera 500. The camera 500 is a monitoring camera, a security camera, or the like. Further, the control device 100 may collect data from communication devices and sensors not shown in the drawings.

[0021] It is assumed that a plurality of types of robots 200 are used in a facility. The control device 100 assigns a task to each of the robots 200. Each of the robots 200 may be equipped with an attachment unit 700 corresponding to the assigned task, and execute the task. The task executed by the robot 200 may be input by the user U1 or the user U2, or may be scheduled in advance. For example, the user U1 or the like operates the user terminal 400 to make a task request. The user U1 or the like can input the type of task to be executed. The user U1 or the like may input an area, a time zone, and the like for executing the task. The control device 100 creates a schedule for the robot 200 to efficiently execute the task.

[0022] The user U1 or the user U2 may operate the user terminal 400 to request a transport task. In this case, the user U1 or the user U2 inputs information related to the transported object. Further, the user U1 or the user U2 may input arrival schedule information indicating the expected arrival of the transported object. The control device 100 assigns a robot to execute the transport task based on the arrival schedule information. Then, the control device 100 transmits a control signal for the robot to execute the task. The control signal may include a route to the destination and transported object information indicating the transported object and the like.

[0023] In such an overall configuration, each element of the control system 1 can be distributed to the robot 200, the user terminal 400, and the control device 100 to construct the control system 1 as a whole. It is also possible to collect and construct substantial elements for realizing the transportation of the transported object in a single device.

[0024] (Control system) The control device 100 has a server computer or the like and performs calculations to control and manage the robot 200. The control device 100 can be implemented as a device capable of executing programs, such as a computer's central processing unit (CPU). The functions described later can also be realized by program. The control device 100 manages the transported object ID and the robot 200 robot ID. The robot 200 is equipped with a sensor 201 that detects marker M and a recognition unit 202 that recognizes marker M.

[0025] For example, the control device 100 manages the schedules of multiple robots 200 so that the robots 200 can perform tasks efficiently. For instance, when the control device 100 receives a task request from a user terminal 400 or the like, it selects one robot 200 from the multiple robots 200 and instructs the robot 200 to perform the task. Alternatively, the control device 100 instructs the robot 200 to use an auxiliary unit 700.

[0026] Figure 2 is a block diagram showing the control system of the control device 100 according to this embodiment. As shown in Figure 2, the control device 100 includes a map information storage unit 111, a robot information storage unit 112, a transported object information storage unit 113, a task management unit 114, and a route planning unit 115. The control device 100 also includes an area setting unit 121, a waypoint setting unit 122, a robot control unit 125, and a communication unit 140. Furthermore, the control device 100 includes a right-of-way granting unit 123. The functional blocks shown in Figure 2 are examples, and the control device 100 and the robot 200 may have other functional blocks. Alternatively, the control device 100 may not have some of the functional blocks shown in Figure 2. Moreover, some functions may be performed on the robot 200 side. The control device 100 acquires image information from one or more cameras 500. The image information may include image data, or it may include information extracted from the image data.

[0027] The map information storage unit 111 stores map information that shows the floor map of the facility (also simply called the map). This map information may include information about restricted areas and waypoints, which will be described later. The map information may be pre-created. Furthermore, the map information may not be a floor map of the entire facility, but rather map information that partially includes the area where the service is scheduled to be performed. Each robot autonomously navigates to its destination by referring to the map information. The map information may be generated based on architectural drawings, CAD (Computer-Aided Design) data, BIM (Building Information Modeling) data, etc. Alternatively, the map information may be generated based on measurement results from a distance measuring sensor such as LiDAR.

[0028] The robot information storage unit 112 stores robot information. The robot information includes information about the robots 200 operating in the facility. The robot information includes information about the robot 200's model number, executable services, types of transportable objects, and attachable accessory units 700. The robot information storage unit 112 stores the robot information as a database in which various information is stored for each robot ID. The robot information may include the robot's current position, movement path, information indicating whether it is performing a task or is paused, and information about the task being performed. The robot information may also include information about accessory units in use and objects being transported.

[0029] The transport item information storage unit 113 stores transport item information. For example, the transport item information includes information such as the transport item's identification number (ID), contents (type) of the transport item, source, destination, pickup time, or arrival time. Transport item information 126 indicates whether the transport item is a pharmaceutical product, medical device, meal, tableware, medical record, equipment, specimen, linen, or person. Transport item information may also include information such as the transport item's size or weight. Transport item information may also include information indicating the status, such as being transported, before transport (before loading), or transported. The transport item information storage unit 113 stores the transport item information as a database where this information is associated with each transport item ID. When a new transport request for transport is received from the user terminal 400, transport item information is added. After transport is completed, the information regarding that transport item may be deleted from the list.

[0030] The task management unit 114 manages the tasks that the robots 200 perform. For example, user U1 inputs transport information indicating the transported items, source, destination, etc., and requests a transport task. The task management unit 114 assigns that transport task to the robots 200. For example, the task management unit 114 extracts robots that are capable of performing the task. For example, if some of the robots 200 are unable to transport the requested items, the task management unit 114 excludes those robots 200 and extracts the remaining robots 200.

[0031] The task management unit 114 extracts robots 200 capable of transporting the transported object from among multiple robots 200. Then, the task management unit 114 assigns the transport task to the extracted robot 200. If there are two or more robots capable of transporting the transported object, the task management unit 114 selects a robot to perform the transport service more efficiently. For example, the task management unit 114 assigns the task to a robot that is near the transport source. Alternatively, the task management unit 114 assigns the task to an available robot that is not performing another task. In this way, tasks can be performed efficiently.

[0032] The task management unit 114 manages tasks performed, tasks currently being performed, and tasks scheduled to be performed by each robot 200. Furthermore, the task management unit 114 may store various task information for each task as a database. The task management unit 114 may also store information indicating whether each robot 200 is performing or has completed a task as task information. In addition, the task management unit 114 may store the transport start time when transport begins and the scheduled end time when a task currently being performed will be completed as task information. Transport information regarding the transported object may also be included in the task information. For example, the task information may include information such as the type of transported object, the destination, and the source of transport. Alternatively, the task management unit 114 may store information indicating whether the auxiliary unit 700 is in use or not.

[0033] The route planning unit 115 plans the route for executing the task. For example, the route planning unit 115 searches for the route from the source to the destination of the robot 200 to which the task has been assigned. Specifically, it searches for the route from the robot 200's current position to the source. The source is the loading location where the transported object is loaded. Furthermore, the route planning unit 115 searches for the route from the source to the destination.

[0034] Route planning uses waypoints set on a map. Waypoints are set on the map as candidate intermediate points that the robot will pass through. Waypoints will be described later. The route found by the route planning unit 115 is transmitted to the robot 200. At least some of the processing of the route planning unit 115 may be executed by the robot 200. Furthermore, if a congested area is identified based on images from a surveillance camera or the like, the route planning unit 115 may search for a route that avoids the congested area. The route planning unit 115 may also search for a route that can be traveled in the shortest time or the shortest distance.

[0035] When waypoints corresponding to the departure point, destination, and intermediate points are input, the route planning unit 115 performs a route search. The route search identifies the waypoints to be passed and the order in which they will be passed. Then, the communication unit 140 transmits the ID information and location of the waypoints to be passed to the robot 200.

[0036] The area setting unit 121 divides the map into multiple areas. Based on these multiple areas, the area setting unit 121 sets restricted areas and unrestricted areas. The area setting unit 121 sets restricted areas on the map. Restricted areas are regions where the movement of the robot 200 is restricted. For example, restricted areas include areas with intersections or junctions. Alternatively, narrow passages where the robot 200 cannot travel in both directions may be restricted areas. In order for the robot 200 to pass through a restricted area, it needs the right of passage. In other words, a robot 200 that has not been granted the right of passage will not be able to enter the restricted area and will have to wait before it. On the map, areas other than restricted areas are designated as unrestricted areas. In unrestricted areas, the robot 200 can pass through without the right of passage. The area setting unit 121 sets one or more restricted areas on the map. The area setting unit 121 also sets one or more unrestricted areas on the map.

[0037] The area setting unit 121 assigns an area ID to each of the restricted and unrestricted areas. The area setting unit 121 associates boundary lines and boundary coordinates with the area ID and stores them as area information. The area information may also include information indicating the attributes of the restricted and unrestricted areas. For example, attributes may include passageways, intersections, junctions, elevator halls, halls, rooms, entrances and exits, package receiving areas, loading areas, loading and unloading areas, waiting areas, charging areas, etc. The area information may also include information indicating the number of robots that can pass through at the same time. Restrictions on passage are not limited to prohibiting simultaneous passage; one-way passage, one-way traffic, etc., may also be included. Furthermore, the area setting unit 121 may change the area settings depending on the time of day or situation.

[0038] For example, the area setting unit 121 divides each room into different areas. Then, the area setting unit 121 determines whether or not to designate each room as a restricted area, depending on its purpose, size, arrangement, and layout. The area setting unit 121 designates intersections and junctions in a passageway as restricted areas. The area setting unit 121 sets the parts other than intersections and junctions as unrestricted areas. For example, the area setting unit 121 sets the straight sections of a passageway as unrestricted areas.

[0039] The control device 100 or another computer may set restricted areas and unrestricted areas through calculation processing, or the user may set them. Furthermore, after setting restricted or unrestricted areas through calculation processing, the user may manually adjust the area settings. For example, a computer such as the control device 100 may set one or more restricted areas or one or more unrestricted areas by executing a program. The user may manually set one or more restricted areas or one or more unrestricted areas.

[0040] By using a program for area definition, simple area definition becomes possible. Specifically, the computer divides corridors and rooms based on architectural drawing data, map layouts, and BIM data. When the computer identifies intersections, junctions, entrances and exits, waiting areas, corners, narrow passages, etc., it designates those locations as restricted areas. For automatic area definition, AI (Artificial Intelligence) models generated through supervised learning can be used.

[0041] Image data of architectural drawings or BIM data are used as input to a machine learning model. The machine learning model uses a segmentation algorithm to identify areas that cannot be passed through, such as walls and fixtures, and divides rooms, corridors, etc. The machine learning model determines whether or not passage should be restricted and sets restricted areas. For example, the machine learning model sets narrow corridors, entrances and exits, waiting areas, junctions and their surroundings as restricted areas. The machine learning model sets areas on the map other than the restricted areas as unrestricted areas.

[0042] Of course, the area setting unit 121 may set areas using algorithms other than the model obtained by machine learning. Alternatively, it may monitor the robot's movement after the robot has been put into operation. If areas where the robot is likely to be unable to pass are identified, the area setting unit 121 may set those areas as restricted areas. Furthermore, manual and automatic settings may be combined. The area setting unit 121 may divide the map into multiple areas using Voronoi partitioning.

[0043] Furthermore, the area setting unit 121 sets exclusion areas on the map. Exclusion areas are areas affected by sunlight in marker recognition. For example, the robot 200 has a sensor 201 for recognizing markers M. Sensor 201 is an optical sensor such as a camera. Markers M are installed on transported objects and facilities. The robot 200 performs various actions by recognizing markers M. For example, it may perform self-position estimation and movement control based on the marker recognition result. Here, the exclusion areas are areas where the accuracy of marker recognition may decrease due to sunlight. The robot 200 determines where to recognize markers M based on the exclusion areas. The robot 200 detects markers M from outside the exclusion areas using sensor 201. This prevents the accuracy of marker recognition from decreasing due to the influence of sunlight.

[0044] The waypoint setting unit 122 sets waypoints on the map. Waypoints are points that the robot 200 will pass through. Waypoints are used for route planning. For example, the route planning unit 115 sets waypoints and their order from the starting point to the destination. The route planning unit 115 determines the order of the waypoints that will be passed through. The robot 200 autonomously drives to pass through the waypoints in the set order. For example, if the path is divided at a junction or intersection, the waypoint setting unit 122 sets waypoints at the junction, intersection, corner and their surroundings as appropriate. The waypoint setting unit 122 also sets waypoints at the boundaries of areas.

[0045] The waypoint setting unit 122 sets multiple waypoints on the map. The waypoint setting unit 122 assigns a waypoint ID to each waypoint. The waypoint setting unit 122 associates the coordinates and IDs of the waypoints and stores them as waypoint information. The waypoint information may also include the attributes of the waypoints. The waypoint setting unit 122 may also set chargers, elevator interiors, elevator boarding / alighting positions, luggage pick-up locations, luggage loading locations, and in front of automatic doors as waypoints.

[0046] In this way, the waypoint setting unit 122 sets multiple waypoints on the map. Furthermore, the waypoint setting unit 122 may assign attributes to the waypoints. The waypoint setting unit 122 classifies the waypoints into multiple categories based on map information and assigns attributes to them. Robot actions (also called robot movements) are assigned according to the attributes of the waypoints. Attributes and actions may be pre-associated. In this case, the waypoint setting unit 122 stores a table or similar that associates attributes with actions. When the waypoint setting unit 122 performs the process of assigning attributes to waypoints, it refers to the table and assigns an action according to that attribute.

[0047] Figure 3 is a table showing the attributes of a waypoint and the actions associated with those attributes. In Figure 3, the registered waypoint attributes are: general waypoint, charger, return point, door, inside EV, boarding EV, disembarking EV, wagon load, wagon unload, waiting in wagon lane, waiting for a right, and releasing a right. Note that a single waypoint may be assigned two or more attributes. Specifically, in addition to the general waypoint attribute, one or more other attributes may be assigned.

[0048] General waypoints serve as intermediate points in route planning. Robot 200 autonomously moves through general waypoints towards the next waypoint. Chargers indicate the location of Robot 200's charger. When the battery level falls below a certain value, Robot 200 moves to the charger waypoint as its destination. When Robot 200 arrives at the charger waypoint, Robot 200 performs relative position correction. For example, the charger is equipped with a marker, and relative position correction is performed by the camera on Robot 200 capturing an image of the marker. Then, Robot 200 connects to the charger and begins charging. Once charging is complete, Robot 200 detaches from the charger.

[0049] The return point corresponds to a location where a marker for self-position recognition is provided. For example, a marker is installed on a wall, and when the robot 200 arrives at the waypoint of the return point, it captures an image of the marker with a sensor 201 (see Figure 2), such as a camera. The recognition unit 202 recognizes the marker M based on the image of the marker M. The robot 200 determines its relative position to the marker based on the captured image of the marker. Since the marker position on the map is known, the robot estimates its own position from its relative position to the marker. In this way, the self-position estimation error accumulated by odometry can be corrected. The door corresponds to the location where the robot 200 sends a signal requesting the automatic door to open. Alternatively, the door corresponds to the location where the robot waits until the automatic door opens. Inside the elevator corresponds to a position inside the elevator car, where the robot 200 stops while the elevator is moving up or down. At this position, the robot 200 switches the floor map to the target floor.

[0050] EV boarding corresponds to the boarding and alighting positions of the elevator (EV). In other words, it corresponds to the position where the elevator is waiting. When robot 200 arrives at the EV boarding waypoint, it calls for the elevator car and detects people and obstacles inside the car. If there is an empty elevator car, robot 200 performs the boarding action. If there is no empty elevator car, robot 200 speaks to indicate that it will see the passenger off. EV alighting corresponds to the boarding and alighting positions of the elevator. In other words, when robot 200 arrives at the EV alighting waypoint, it speaks to alert people in the surrounding area before alighting. The wagon load corresponds to the loading location of the wagon, which is the auxiliary unit 700 shown in Figure 1. When robot 200 arrives near a waypoint on the wagon load, robot 200 uses a camera to image a marker placed on the wagon. Based on the image of the marker, robot 200 determines its relative position to the wagon. After correcting the relative position, robot 200 moves underneath the wagon and lifts it up. The wagon unloading process corresponds to the location where the wagon, as an auxiliary unit 700 shown in Figure 1, will be unloaded by the robot. When the robot 200 arrives near the wagon unloading waypoint, it uses sensors to detect whether there are any obstacles at the unloading location. If there are no obstacles, it moves to the unloading position and lifts down the wagon. The wagon lane corresponds to an area where multiple wagons are lined up. The waypoints of the wagon lane are, for example, located within a restricted area where the number of robots that can enter is predetermined. The robot waits until it receives permission to enter from the control device 100, which is a server, or from a preceding robot 200.

[0051] As described above, various actions are assigned to waypoints according to their attributes. When robot 200 arrives at or near a waypoint, it performs the assigned action. General waypoints are assigned the action of autonomous movement. Charger waypoints are assigned the actions of marker recognition, charger connection, and charger disconnection. When robot 200 arrives at a charger waypoint, it performs the actions in the order of marker recognition, charger connection, and charger disconnection. Return point waypoints are assigned the action of self-position recognition based on marker recognition. Waypoints inside the EV are assigned the action of switching the destination floor map. EV boarding waypoints are assigned actions such as calling the cart, detecting people or obstacles inside the cart, boarding, and speaking. EV disembarking waypoints are assigned actions such as disembarking and speaking.

[0052] At the wagon loading waypoints, the robot 200 is assigned the actions of correcting its relative position to the wagon, crawling under the wagon, and lifting the wagon. When the robot 200 arrives at a wagon loading waypoint, it performs the actions in the following order: correcting its relative position to the wagon based on marker recognition, crawling under the wagon, and lifting the wagon. At the wagon unloading waypoints, the robot is assigned the actions of detecting obstacles in the wagon storage area, moving to the wagon unloading position, and lifting the wagon down. When the robot 200 arrives at a wagon unloading waypoint, it performs the actions in the following order: detecting obstacles in the wagon storage area, moving to the wagon unloading position, and lifting the wagon down. At the wagon lane waiting waypoints, the robot is assigned the action of waiting until it receives permission to enter from the server (control device 100) or a preceding robot.

[0053] Waypoints waiting for access are assigned the action of requesting access to the entry area at the waypoint before the restricted area. Waypoints releasing access are assigned the action of notifying the server that the restricted area has been passed. Requesting and releasing access are actions for managing access rights to restricted areas, which will be described later. Note that Figure 3 is an example showing waypoints, attributes, and actions, and other attributes and actions may be set besides those shown in Figure 3. Also, at least some of the attributes and actions shown in Figure 3 may not be used. Multiple attributes may be assigned to a single waypoint; that is, one or more attributes may be assigned to a single waypoint. Also, multiple actions may be assigned to a single waypoint or a single attribute.

[0054] The control device 100 or another computer may set waypoints through calculation processing, or the user may set them. Furthermore, after waypoints have been set through calculation processing, the user may manually adjust the waypoint settings. For example, one or more waypoints may be set by a computer such as the control device 100 executing a program. The user may also manually set waypoints.

[0055] By using a program for setting waypoints, simple waypoint setting becomes possible. Specifically, when the computer identifies intersections, etc., based on architectural drawing data, map layouts, CAD data, and BIM data, the waypoint setting unit 122 sets waypoints at those locations. Alternatively, the waypoint setting unit 122 sets waypoints at the boundary between restricted and unrestricted areas. For automatic waypoint setting, an AI model generated by supervised learning or similar methods can be used.

[0056] Image data and BIM data representing architectural drawings are used as input to the machine learning model. Furthermore, area-defined map data is also used as input to the machine learning model. The machine learning model sets waypoints using a segmentation algorithm. A combination of manual and automatic setting may also be used.

[0057] Furthermore, the control device 100 has a robot control unit 125. The robot control unit 125 controls multiple types of robots 200. The robot control unit 125 controls the robots 200 so that they pass through waypoints in the order set by the path search. As described above, the route planning unit 115 searches for a path to the destination based on multiple waypoints. This selects waypoints to be passed through from among multiple waypoints registered on the map. Furthermore, the route planning unit 115 determines the order in which the selected waypoints are passed. The robots are controlled so that they pass through the waypoints to the destination in order. The robot control unit 125 may also control the robots 200's entry into restricted areas based on right of way.

[0058] Figure 4 will be used to explain the restricted areas, unrestricted areas, and waypoints set on the map. Figure 4 is a schematic plan view of the map. In Figure 4, the map includes restricted areas RA1 and RA2 and unrestricted areas FA1, FA2, and FA3. Also, for clarity of explanation, Figure 4 shows a two-dimensional XY Cartesian coordinate system. Furthermore, Figure 4 shows three robots 200, which will be identified as robots 200A, 200B, and 200C. If robots 200A, 200B, and 200C are not identified, they will be described collectively as robot 200.

[0059] In Figure 4, a passage B extending in the X direction is provided on the -Y side of the map. A three-way intersection T1 is located along passage B. Passage B is divided into an unrestricted area FA1, a restricted area RA1, and an unrestricted area FA2. Restricted area RA1 is located between unrestricted area FA1 and unrestricted area FA2. Unrestricted area FA1 is located on the -X side of restricted area RA1, and unrestricted area FA2 is located on the +X side. In restricted area RA1, the number of robots that can pass through at the same time is limited to one. Restricted area RA1 corresponds to intersection T1.

[0060] Hall H is located on the +Y side of restricted area RA1. Branch point T1 is the path from passage B to Hall H. Hall H includes, for example, an elevator hall where elevator EV is installed. Restricted area RA1 corresponds to the branch point where Hall H and passage B connect. Hall H is the unrestricted area FA3.

[0061] A dispensing room D is located on the +Y side of the unrestricted area FA2 and on the +X side of the unrestricted area FA3. Dispensing room D serves as a loading and unloading area. In other words, dispensing room D is a room where goods are loaded onto robot 200, or a room where goods are received from robot 200. Dispensing room D can also be described as a waiting area for receiving or loading goods.

[0062] An entrance EN to pharmacy room D is provided at the boundary between the unrestricted area FA2 and the restricted area RA2. Robot 200 enters pharmacy room D from passage B through entrance EN. Then, transport items such as pharmaceuticals are loaded onto robot 200. An exit EX from pharmacy room D is provided at the boundary between the unrestricted area FA3 and the restricted area RA2. Robot 200 exits pharmacy room D through exit EX to hall H. Then, robot 200 moves to the designated transport destination and transports the items. The number of robots that can pass through pharmacy room D simultaneously is limited to 4.

[0063] In Figure 4, 19 waypoints WP1 to WP19 are set on the map. The waypoint setting unit 122 registers a unique waypoint ID for each waypoint WP1 to WP19. Furthermore, the waypoint setting unit 122 associates the XY coordinates on the map with the waypoint ID. The waypoint setting unit 122 also associates attributes and actions with each waypoint WP1 to WP19.

[0064] Waypoints WP1-WP8 are set in corridor B. Waypoints WP9-WP12 and WP19 are set in hall H. Waypoints WP13-WP17 are set in pharmacy room D. Waypoint WP18 is set inside elevator EV. Waypoints WP1, WP2, WP5, and WP6 are set in unrestricted area FA1. Waypoints WP3, WP4, WP7, and WP8 are set in unrestricted area FA2. Waypoints WP9-WP12 and WP19 are set in unrestricted area FA3. Waypoints WP13-WP17 are set in restricted area RA2. Note that no waypoints are set in restricted area RA1, but they may be set there.

[0065] Since passage B is wide enough for robots 200 to pass each other, waypoints are set in two rows. Here, we will explain assuming that robot 200 travels through passage B in right-hand direction. Waypoints WP1 to WP4 are passing points when traveling through passage B in the +X direction. Waypoints WP5 to WP8 are passing points when traveling through passage B in the -X direction. For example, if the planned route includes a path that proceeds through passage B in the +X direction, the route planning unit 115 plans the route so that robot 200A passes through waypoints WP1, WP2, WP3, and WP4 in that order.

[0066] Since passage B has a width that allows passage in both the +X and -X directions, waypoints WP1 and WP5 are located at the same position in the X direction, but offset in the Y direction. This makes it possible for the two robots 200 to pass each other.

[0067] Waypoints WP2 and WP6 correspond to the branching points of path B. Therefore, waypoints WP2 and WP6 are set at the boundary between restricted area RA1 and unrestricted area FA1. In other words, waypoints WP2 and WP6 are set near the boundary line between restricted area RA1 and unrestricted area FA1. Here, waypoints WP2 and WP6 are set on the unrestricted area FA1 side of the area boundary.

[0068] Waypoints WP3 and WP7 correspond to the branching points of path B. Waypoints WP3 and WP7 are set at the boundary between restricted area RA1 and unrestricted area FA2. In other words, waypoints WP3 and WP7 are set near the boundary line between restricted area RA1 and unrestricted area FA2. Here, waypoints WP3 and WP7 are set on the unrestricted area FA2 side of the area boundary.

[0069] Waypoints WP9 and WP10 correspond to the branching points of passage B. Waypoints WP9 and WP10 are set at the boundary between restricted area RA1 and unrestricted area FA3. In other words, waypoints WP9 and WP10 are set near the boundary line between restricted area RA1 and unrestricted area FA3. Here, waypoints WP9 and WP10 are set on the unrestricted area FA3 side of the area boundary.

[0070] Waypoints WP11 and WP12 correspond to the waiting positions of the elevator EV. Waypoint WP1 is set in front of the elevator EV. For example, when robot 200 boards the elevator EV, waypoint WP11 or waypoint WP12 is included in the route. Then, robot 200 waits at waypoint WP11 or waypoint WP12 until the elevator EV arrives.

[0071] Waypoints WP13 to WP15 correspond to waiting areas for loading and unloading of transported goods. Robot 200 waits at waypoint WP13, etc., until loading of transported goods is complete. Alternatively, robot 200 waits at waypoint WP13, etc., until unloading of transported goods is complete. Waypoint WP16 corresponds to a charging area for charging.

[0072] All waypoints WP1 to WP19 have the attributes of a general waypoint. Waypoints WP1, WP4, and WP5 have only the attributes of a general waypoint. At waypoints WP1, WP4, and WP5, robot 200 autonomously moves to the next waypoint. At waypoints WP2, WP7, and WP10, the attribute is pending right of passage. At waypoints WP2, WP7, and WP10, robot 200 requests the right of passage to enter restricted area RA1. The right of passage granting unit, described later, grants the right of passage to robot 200 upon receiving the request.

[0073] Waypoint WP3 has the attributes of "right released" and "right pending." At waypoint WP3, robot 200 notifies control device 100 that it has passed through restricted area RA1. In other words, robot 200 releases its right of passage to restricted area RA1, which it has passed through. Also, robot 200 entering pharmacy room D requests the right of passage to enter restricted area RA2 at waypoint WP3. Thus, the target area is set for waypoints that are "right pending" and "right released." Note that if robot 200 moves straight down passage B, robot 200 does not request a right of passage at waypoint WP3. In this way, robot 200 may determine whether or not to perform a request action to request a right of passage depending on the path it is moving.

[0074] The attributes of waypoints WP6 and WP9 are set to "Right Released". At waypoints WP6 and WP9, robot 200 notifies control device 100 that it has passed through restricted area RA1. In other words, robot 200 releases its right of passage to restricted area RA1, which it has passed through. The attribute of waypoint WP8 is set to "Right Awaited". Robot 200 entering pharmacy room D requests the right of passage to enter restricted area RA2 at waypoint WP8. However, if robot 200 proceeds straight down passage B, robot 200 does not request a right of passage at waypoint WP8. Thus, robot 200 may determine whether or not to perform a request action to request a right of passage depending on the path it is traveling.

[0075] The attribute of waypoints WP11 and WP12 is "Elevator EV boarding". Therefore, robot 200 at waypoints WP11 and WP12 will perform an elevator call as the elevator boarding action.

[0076] The attributes of waypoints WP13 to WP15 are wagon load and wagon unload. Robot 200 at waypoints WP13 to WP15 performs actions related to loading or unloading wagons. The wagon is one of the auxiliary units 700 shown in Figure 1 and contains the transported goods.

[0077] For example, the wagon is equipped with a marker M for identifying the transported object. The marker M is a one-dimensional barcode or a two-dimensional barcode, and has a unique pattern. The marker M may also be a QR code (registered trademark). The marker M may include information indicating the position on a map or information indicating the transported object. As shown in Figure 2, the robot 200 has a sensor 201. The sensor 201 is an optical sensor such as a camera, and it captures an image of the marker. The recognition unit 202 recognizes the marker based on the captured image of the marker M. Based on the recognition result, the robot 200 determines its relative position to the wagon. For example, the distance and direction of the robot 200 to the wagon are determined by the size, position, and inclination of the marker M image in the captured image. Based on the relative position determined by the imaging result, the robot 200 approaches the wagon. This allows the robot to appropriately perform operations such as lifting the wagon.

[0078] Furthermore, at least one of the waypoints WP13 to WP16 may have the "waiting in the wagon lane" attribute. Alternatively, a waypoint with the "waiting in the wagon lane" attribute may be added to pharmacy room D.

[0079] Waypoint WP16 is located near charger BC. Robot 200, upon arriving at waypoint WP16, is charged by charger BC. Markers are provided at and around charger BC. Robot 200 approaches charger BC based on its relative position identified by imaging results. This allows robot 200 to connect properly to charger BC. Additionally, at least one of waypoints WP13 to WP16 may have the attribute of waiting in the wagon lane.

[0080] Waypoint WP17 is located inside pharmacy room D. Waypoint WP17 is located near door AD at exit EX of pharmacy room D. Door AD is installed at exit EX. The attribute of waypoint WP17 is the action to open door AD. Therefore, when robot 200 arrives at waypoint WP17, it performs the action of opening door AD. For example, door AD is a remotely operated automatic door. Then, robot 200 at waypoint WP17 sends a signal to open and close door AD. As a result, door AD opens, allowing robot 200 to exit pharmacy room D. Robot 200, equipped with a cart or other equipment, autonomously moves towards its destination.

[0081] Waypoint WP19 is located inside Hall H. Waypoint WP19 is located near Door AD. The attribute of Waypoint WP19 is "Open". Therefore, when Robot 200 arrives at Waypoint WP19, it notifies Control Unit 100 that it has passed through Restricted Area RA2.

[0082] Waypoint WP18 is located inside the elevator car. The attributes of Waypoint WP18 include "inside the elevator" and "exiting the elevator." For example, robot 200 at Waypoint WP18 performs an action to switch the destination floor map. That is, robot 200 switches from the floor map of the boarding floor to the floor map of the exiting floor while riding in the elevator. Waypoint WP18 may also have the attribute of "boarding the elevator." In this case, robot 200 at Waypoint WP18 performs an action to detect obstacles inside the elevator.

[0083] Of course, the waypoint attributes and actions listed above are just examples, and some waypoints, attributes, or actions may not be set. Alternatively, other waypoints, attributes, or actions may be set.

[0084] Returning to the explanation of Figure 2, the right-of-way granting unit 123 grants the robot 200 the right of way. The right of way is a resource that allows the robot 200 to pass through the restricted area. The right-of-way granting unit 123 grants the right of way to each robot. Furthermore, the right-of-way granting unit 123 manages the right of way for each restricted area.

[0085] The right-of-way granting unit 123 grants the robot 200 a right of way in response to a request from the robot 200. For example, when robot 200A arrives at waypoint WP2 or its vicinity, robot 200A sends a request signal to the control device 100 to request the right of way to restricted area RA1. When the communication unit 140 of the control device 100 receives the request signal, the right-of-way granting unit 123 grants robot 200A the right of way to restrict area RA1. Specifically, the communication unit 140 sends a right-of-way granting signal to robot 200A. The right-of-way signal includes the area ID of the restricted area that can be traversed, etc.

[0086] Alternatively, the right-of-way granting unit 123 may determine whether or not to grant a right of way. When the robot 200A transmits its own position on the map to the control device 100, the right-of-way granting unit 123 determines whether or not it has arrived in front of the restricted area RA1. For example, if the robot 200A arrives at waypoint WP2, the right of way is granted to the robot 200A. Note that the location where the right of way is granted may be a location other than a waypoint. The coordinates of the location where the right of way is granted may be set in advance. For example, when the robot 200 reaches a predetermined right-of-way granting location near the restricted area, the right-of-way granting unit 123 grants the robot 200 a right of way. In the following description, the right-of-way granting location is described as being the same location as the waypoint closest to the restricted area, but the right-of-way granting location may have different coordinates than the waypoint.

[0087] The right-of-way granting unit 123 releases the right of way when the robot 200 passes through the restricted area. For example, if robot 200A passes through the restricted area RA1 and enters the unrestricted area FA2, the right-of-way granting unit 123 releases robot 200A's right of way with respect to the restricted area RA1. Alternatively, robot 200A may send a signal to the control device 100 to release the right of way based on its own position. Waypoints for releasing the right of way may also be set.

[0088] For example, suppose the route planning unit 115 plans routes so that robot 200A moves along route PH1 and robot 200B moves along route PH2. In this case, if robots 200A and 200B enter restricted area RA1 at the same time, there is a risk that robots 200A and 200B will face each other within restricted area RA1. This could prevent robots 200A and 200B from passing through restricted area RA1, or they would need to take a detour. Therefore, restricted area RA1 is provided to restrict the passage of robots 200. In this way, multiple robots 200 can move efficiently.

[0089] The communication unit 140 transmits various data and signals to each of the robots 200. The communication unit 140 receives request signals from the robots 200 to request a right of passage and release signals to release a right of passage. The communication unit 140 transmits a right of passage grant signal to the robots 200 to grant them a right of passage. As described above, since the attributes of waypoints WP2 and WP10 are pending a right of passage, the robots 200 perform a right of passage request action at waypoints WP2 and WP10 to request a right of passage.

[0090] First, with none of the robots 200A to 200C granted the right of passage to the restricted area RA1, robot 200A arrives at waypoint WP2. Based on the attributes of waypoint WP2, robot 200A requests the control device 100 for the right of passage. Upon receiving the request from robot 200A, the right of passage granting unit 123 grants robot 200A the right of passage.

[0091] Assume that robot 200B arrives at waypoint WP10 before robot 200A passes through restricted area RA1. Based on the attributes of waypoint WP10, robot 200B requests a right of passage from control device 100. Then, robot 200B waits at waypoint WP10 until the right of passage is granted.

[0092] When robot 200A passes through restricted area RA1 and arrives at waypoint WP3, robot 200A performs a right-of-way release action to release its right of way. When control unit 100 receives a signal from robot 200A indicating the release of the right of way, the right-of-way granting unit 123 grants the right of way to the next robot 200B. Control unit 100 manages the right of way in the order of the right of way requests.

[0093] Thus, waypoints near the restricted area have attributes such as waiting for a right of passage or having a right of passage released. When robot 200 moves to a waypoint just before the restricted area, robot 200 sends a request signal requesting a right of passage. When communication unit 140 receives the request signal, the right of passage granting unit 123 determines whether passage is possible. That is, it determines whether the number of robots in the restricted area is less than the limit. If the number of robots is less than the limit, it sends a right of passage to robot 200. If the number of robots is equal to or greater than the limit, it does not send a right of passage granting signal. That is, it waits until it receives a release signal from robot 200 in the restricted area to release (cancel) its right of passage. When communication unit 140 receives a release signal from robot 200 in the restricted area, the right of passage granting unit 123 grants a right of passage to the next robot 200. Therefore, communication unit 140 sends a right of passage granting signal to robot 200.

[0094] The control device 100 and the robot 200 may use machine learning models such as deep learning for route planning and drive control. Furthermore, machine learning models such as deep learning, including RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network), may also be used for detecting surrounding objects.

[0095] The control method using the control system according to this embodiment will be explained with reference to Figure 5. Figure 4 is a flowchart showing the robot control method according to this embodiment. First, the area setting unit 121 sets restricted areas and unrestricted areas on the map (S11). This divides the map into restricted areas and unrestricted areas. The area setting unit 121 may set restricted areas and unrestricted areas using a machine learning model such as VLM.

[0096] The waypoint setting unit 122 sets multiple waypoints (S13). Here, the waypoint setting unit 122 sets waypoints in the restricted area and the unrestricted area. Furthermore, the waypoint setting unit 122 assigns attributes and actions to each waypoint (S13). The waypoint setting unit 122 may use a machine learning model such as VLM. The area setting unit 121 and the waypoint setting unit 122 may execute steps S11 to S13 simultaneously. For example, if the waypoint setting unit 122 uses a machine learning model such as VLM, when architectural drawing data is input, the machine learning model outputs the coordinates of each waypoint.

[0097] The task management unit 114 assigns a task to a robot (S14). For example, when a user registers a task to be performed, the task management unit 114 determines which robot will perform the task based on the robot's availability, current location, etc.

[0098] Next, the route planning unit 115 performs route planning (S15). For example, the route planning unit 115 obtains the current position of the robot to perform the task from the robot information storage unit 112. The route planning unit 115 searches for a path from the current position to waypoints and the destination. For example, the route planning unit 115 obtains the type of robot from the robot information. Based on this, it determines the waypoints to be passed through from among multiple waypoints. Furthermore, the route planning unit 115 determines the order in which the waypoints will be passed.

[0099] The robot 200 autonomously moves to a waypoint along the path (S16). For example, the robot control unit 125 controls the robot 200 so that it sequentially passes through the waypoints to the destination. Thus, the robot 200 autonomously moves toward the next waypoint along the path.

[0100] When robot 200 arrives at a waypoint, robot 200 executes the action set for that waypoint (S17). Robot 200 may execute the action while passing through the waypoint, or it may execute the action while stopped at the waypoint. The action here includes processes such as granting a right of way. This allows the robot to be controlled appropriately, enabling efficient task execution.

[0101] Robot 200 determines whether or not it has arrived at its destination (S18). That is, robot 200 determines whether or not it has arrived at the last waypoint on the route. Note that this determination may also be made by the control device 100. If robot 200 has not arrived at its destination (NO in S18), the process returns to step S16, and robot 200 autonomously moves to the next waypoint. If robot 200 has arrived at its destination (YES in S18), the process ends. Of course, the control of another robot 200 may be performed in the same way. In this case, the process from step S15 onwards will be executed by the other robot 200.

[0102] Furthermore, some or all of the processing in the robot 200 and control device 100 described above can be implemented as a computer program. Such programs can be stored using various types of non-temporary computer-readable media and supplied to a computer. Non-temporary computer-readable media include various types of tangible recording media. Examples of non-temporary computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Programs may also be supplied to a computer using various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0103] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. [Explanation of Symbols]

[0104] 1. Robot control system 100 Control device

Claims

1. A robot control system for controlling an autonomous mobile robot, Multiple waypoints that can serve as transit points for the autonomous mobile robot are set on the map. Assign attributes to the aforementioned waypoints to set robot movements, Based on the multiple waypoints, a route to the destination is searched. The autonomous mobile robot is controlled to sequentially pass through the aforementioned waypoints to the destination. A robot control system in which the autonomous mobile robot performs robotic actions when it arrives at a waypoint with the aforementioned attributes.

2. The robot control system according to claim 1, wherein the robot operation includes at least one of the following: boarding an elevator, disembarking from an elevator, opening a door, recognizing a marker, boarding an EV, and autonomous movement.

3. In the aforementioned map, restricted areas are set to limit the passage of autonomous mobile robots. Granting the right of passage that allows passage through the aforementioned restricted area The robot control system according to claim 2, wherein the robot operation includes a right-request operation to request the right of passage and a right-release operation to release the right of passage.

4. The aforementioned autonomous mobile robot performs a transport task of transporting an object. A robot control system according to claim 2 or 3, which includes recognizing a marker provided on the transported object and performing a loading operation to place the transported object based on the recognition result of the marker.

5. A robot control method for controlling an autonomous mobile robot, Multiple waypoints that can serve as transit points for the autonomous mobile robot are set on the map. Assign attributes to the aforementioned waypoints to set robot movements, Based on the multiple waypoints, a route to the destination is searched. The autonomous mobile robot is controlled to sequentially pass through the aforementioned waypoints to the destination. A robot control method in which the autonomous mobile robot performs robotic actions when it arrives at a waypoint given the aforementioned attributes.

Images