Robot, robot control device, robot control system, and robot control method

Abstract

To provide a technology that enables a robot to autonomously use methods other than detouring or stopping around an obstacle when there is an obstacle on a movement path thereof.SOLUTION: An autonomously movable robot includes an environment detection unit that detects objects on a movement path of the robot and an action determination unit that selects one of the following actions of: an action to detour the object detected by the robot; an action to remove the object detected by the robot; an action to cause an object removal resource other than the robot to remove the detected object; and an action to remove the object detected by the robot with the object removal resource other than the robot.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a robot, a robot control device, a robot control system, and a robot control method that determine the robot's behavior in an environment where obstacles may appear on the movement path. [Background technology]

[0002] Transport robots are used in logistics warehouses and other facilities. Transport robots move along a path, but their movement may be restricted by obstacles on the path. Obstacles include, for example, luggage, workers, or other transport robots. Even in such cases, if the transport robot is made to perform an appropriate avoidance operation, it can return to its assigned task in a short time, thereby maintaining a high operational rate for the transport robot.

[0003] For example, Patent Document 1 discloses an autonomous mobile robot for a chicken coop that autonomously travels inside the chicken coop, stating that "when the obstacle in contact with the belt is not detected by the sensor, the belt starts rotational travel across the plurality of rotating bodies due to the frictional force generated between the belt and the obstacle, and while eliminating the obstacle from the traveling direction of the autonomous mobile robot, when the obstacle in contact with the belt is detected by the sensor, the control unit stops the movement of the autonomous mobile robot, providing an autonomous mobile robot." Specifically, in paragraphs 0013 to 0015 of the same document, "Next, the operation of the bumper unit 2 will be described. The operation is different when the obstacle 10 is a light object such as a chicken (about 3 kg or less) and when the obstacle 11 is a heavy object such as a block (about 10 kg or more). First, the case where the obstacle 10 is a light object will be described. FIG. 7A shows a case where an obstacle 10 exists on the traveling path of the autonomous mobile robot 1. FIG. 7A is a view seen from the ceiling direction, and FIG. 7B is a view seen from the side. It is shown as a schematic diagram for easier explanation. When the bumper unit 2 comes into contact with the obstacle 10 while the autonomous mobile robot 1 is traveling in the V2 direction, a frictional force Ff1 proportional to the weight is generated on the obstacle 10, and due to this frictional force Ff1, the compression spring 912 contracts by D1 = Ff1 (frictional force) / K (spring constant). When the weight of the obstacle 10 is a light object, the amount of deflection D1 is also small, so the sensor 910 does not react. Incidentally, if the amount of deflection required until the sensor 910 reacts is D, the relational expression D > D1 holds. If the autonomous mobile robot 1 continues to move as it is, a frictional force F1 is generated at the contact portion between the belt 201 and the obstacle 10. Generally, the rolling friction coefficient of the bearing is much smaller than the static friction coefficient, so the belt 201 starts to rotate due to this frictional force F1. When the autonomous mobile robot 1 is traveling with a propulsion force stronger than the frictional force Ff1, the obstacle 10 moves in the F1 direction as seen from the autonomous mobile robot 1, reaching the state shown in FIG. 8. Next, the case where the obstacle 11 is a heavy object will be described using FIG. 9. When the weight of the obstacle 11 is a heavy object, the amount of deflection D2 also becomes large, and as a result, the sensor 910 switches to ON. Incidentally, it can be expressed by the relational expression D < D2. When the sensor 910 switches to ON, the control unit 5 sends a command to the carriage unit 3 to stop the travel.""The disclosure states: [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-128772 Summary of the Invention [Problem to be solved by the invention]

[0005] However, Patent Document 1 describes that if the robot detects an obstacle, it will remove the obstacle, and that if the robot detects an obstacle, it will stop, but does not take into consideration whether to remove or detour around the detected obstacle.

[0006] Therefore, an object of the present invention is to provide a technique that enables a robot to autonomously use methods other than detouring around or stopping an obstacle when an obstacle exists on its movement path. [Means for solving the problem]

[0007] The above problem can be solved by a robot that includes, for example, an environment detection unit that detects objects on the robot's movement path, and an action determination unit that determines one of the following actions: an action to go around the object detected by the environment detection unit, an action to remove the object detected by the robot, an action to have an object removal resource other than the robot remove the detected object, or an action to remove the object detected by the robot and an object removal resource other than the robot. [Effects of the Invention]

[0008] According to the present invention, even when an obstacle exists on the movement path, the robot can autonomously use methods other than going around or stopping the obstacle. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram illustrating a configuration of a robot control system according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing an example of a working environment of a robot. [Figure 3] A diagram showing the working environment in Figure 2 as input information for reinforcement learning. [Figure 4] 3 is a flowchart showing a data processing procedure of the robot control device according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example of a list of removal method candidates according to the first embodiment. [Figure 6] FIG. 3 is a diagram showing an example of a value function table according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of an optimal control law according to the first embodiment. [Figure 8] 10 is a flowchart showing a data processing procedure of a robot control device according to a second embodiment. [Figure 9] 10 is an example of a display screen of a mobile device when the obstacle is a worker. [Figure 10] 10 is an example of the display screen on the display device when requesting support from an administrator. [Figure 11] FIG. 10 is a diagram illustrating the configuration of a robot control system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of a robot control system according to the present invention will be described with reference to the drawings. [Example]

[0011] First, a first embodiment of a robot control system 100 according to the present invention will be described with reference to FIGS.

[0012] 1 is a diagram showing the configuration of a robot control system 100 according to this embodiment. This robot control system 100 is a system in which a robot control device 1, a robot 2, a sensor 3, an input device 4, a display device 5, a mobile terminal 6, and a cloud 7 are interconnected to enable wired or wireless communication, and is a system that autonomously controls the robot 2 using commands from the robot control device 1. Below, an overview of the robot 2 to the cloud 7 will be provided, followed by a detailed description of the robot control device 1.

[0013] The robot 2 is, for example, a transport robot that includes a robot arm 21 that can grasp a load, a drive mechanism 22 such as a caterpillar used for movement, and a loading platform 23 on which the load can be placed. Note that the robot 2 that is the control target of this embodiment is not limited to the transport robot exemplified in Fig. 1, and may be, for example, a robot that does not have a robot arm or a loading platform, a walking robot, or an autonomous vehicle.

[0014] The sensor 3 is a device that detects environmental information in the working environment of the robot 2, and is, for example, a camera, a distance sensor, a radar, a force sensor, a temperature sensor, an angle sensor, etc. This sensor 3 may be a fixed sensor such as a surveillance camera installed in the working environment of the robot 2, or may be a sensor mounted on the robot 2.

[0015] The input device 4 is a device, such as a keyboard, mouse, or joystick, that is used by an administrator who manages the robot control system 100 to input management information and commands to the robot control device 1. Details of the management information and commands will be described later.

[0016] The display device 5 is a device used by the robot control device 1 to present information to the manager, and is, for example, a liquid crystal display. Details of the information displayed on the display device 5 will be described later.

[0017] The mobile terminal 6 is a terminal such as a smartphone or tablet carried by a worker working in the working environment of the robot 2. Details of the information displayed on the mobile terminal 6 will be described later. If the worker also serves as the manager, the functions of the robot control device 1, input device 4, and display device 5 may be provided in the mobile terminal 6.

[0018] The cloud 7 is a server or the like that can substitute for the functions of a computing unit 14 and an optical disc 11a, which will be described later.

[0019] <Robot control device 1> The robot control device 1 is mainly composed of a data acquisition unit 11, a storage unit 12, a communication unit 13, a calculation unit 14, and an internal bus (not shown) connecting the various units. Specifically, the robot control device 1 is a computer equipped with hardware such as a calculation unit such as a CPU, a storage unit such as a semiconductor memory, and a communication unit. The calculation unit executes a predetermined program to realize each functional unit, but the following description will omit such well-known techniques as appropriate.

[0020] <<Data Acquisition Unit 11>> The data acquisition unit 11 is a functional unit that acquires data from an external recording medium, and is, for example, an optical disc drive that plays back an optical disc 11a, or a USB interface that reads a USB (registered trademark) memory 11b.

[0021] <<Storage section 12>> The memory unit 12 is a functional unit that stores various data, and is composed of a base map memory unit 12a, an estimated map memory unit 12b, a removal method memory unit 12c, an operation memory unit 12d, and a control law memory unit 12e. Details of each memory unit will be described later, but in summary, it has the following functions.

[0022] The base map storage unit 12a is a functional unit that stores a base map acquired from an external recording medium such as the optical disk 11a or the cloud 7. The base map is, for example, basic map information used when identifying the shortest travel path for the robot 2, and is a map that stores information such as the travel surface, walls, and fixed obstacles in the working environment of the robot 2. The travel path is the path along which the robot 2 travels. There are several methods for creating a travel path based on the base map before the robot 2 travels to the destination, and there are also methods in which the robot 2 autonomously creates a travel path to the destination based on its own position and the surrounding environment.

[0023] The estimated map storage unit 12b is a functional unit that stores an estimated map estimated by the environment detection unit 14c (described later). The estimated map is map information of the work environment estimated based on the output information of the sensor 3, and differs from a base map in that it successively reflects changes over time in the work environment, such as the occurrence of unexpected obstacles.

[0024] The removal method storage unit 12c is a functional unit that stores candidate methods for removing an unexpected obstacle when the obstacle is found on the travel route.

[0025] The action storage unit 12d is a functional unit that stores actions required when the robot removes an obstacle or when the robot goes around the obstacle.

[0026] The control law storage unit 12e is a functional unit that stores the robot control law, which is the calculation result of the control law determination unit 14g described below. The control law is used to make the control variable, which is the output of the controlled object, follow a target value, which is a desired value, in order to achieve the control objective. For example, if the objective is for the robot to remove an obstacle, the control law is a law that controls the robot's motion to remove the obstacle. A control law stored in the control law storage unit 12e may be selected and input directly to the robot. Alternatively, if an optimal control law is selected from the control laws stored in the control law storage unit 12e, an action based on the optimal control law may be extracted from the action storage unit 12d and the robot may be instructed to perform that action. Alternatively, if the robot can be instructed using only the action storage unit 12d, the control law storage unit 12e may be unnecessary.

[0027] <<Communication Unit 13>> The communication unit 13 is a functional unit that communicates with the robot 2, the sensor 3, the input device 4, the display device 5, the mobile terminal 6, and the cloud 7. This communication unit 13 is connected to the memory unit 12 and the calculation unit 14 via an internal bus, so that the memory unit 12 and the calculation unit 14 can send and receive information to and from the robot 2, etc. via the communication unit 13.

[0028] <<Arithmetic section 14>> The calculation unit 14 is a functional unit that generates information necessary for autonomous control of the robot 2, and includes an input control unit 14a, an output control unit 14b, an environment detection unit 14c, an object information acquisition unit 14d, a cost calculation unit 14e, an operation determination unit 14f, a control law determination unit 14g, and a robot control unit 14h. Of these, the environment detection unit 14c to the control law determination unit 14g may be located on the cloud 7. Details of each unit will be described later, but in summary, each unit has the following functions.

[0029] The input control unit 14 a is a functional unit for transferring data input via the data acquisition unit 11 or the communication unit 13 to an appropriate functional unit of the storage unit 12 or the calculation unit 14 .

[0030] The output control unit 14b is a functional unit for transferring data output from each part of the memory unit 12 or the calculation unit 14 to an appropriate one of the robot 2, the display device 5, the mobile terminal 6, or the cloud 7 via the communication unit 13.

[0031] The environment detection unit 14c is a functional unit that detects the working environment of the robot 2 and unexpected obstacles on the movement path of the robot 2 based on environmental information acquired from the sensors, and generates or updates the aforementioned estimated map. The environmental information is, for example, information on the distance between the robot and the obstacle. An obstacle is, for example, an object that is not registered in the base map. In this embodiment, the term obstacle is used, but an obstacle can also be rephrased as an object on the path along which the robot moves.

[0032] The object information acquisition unit 14d extracts obstacle information required for determining whether to remove an obstacle detected by the environment detection unit 14c, based on the environmental information acquired from the sensor 3. For example, the shape and volume of the obstacle can be estimated from the image information output by the camera and the distance information output by the distance sensor. In addition, the type of obstacle (such as an object or a person) can be estimated from the temperature information output by the temperature sensor.

[0033] The cost calculation unit 14e has a function of estimating one or more candidate obstacle removal methods from the obstacle information extracted by the environment detection unit 14c and storing these candidates in the removal method storage unit 12c. The cost calculation unit 14e also has a function of calculating the removal cost required to remove the obstacle from the travel path and the detour cost required to bypass the obstacle. The cost calculation unit 14e also has a function of comparing the detour cost and the removal cost and selecting the lower cost. The removal cost includes at least one of the time, workload, and economic cost required for the work. A method for calculating the removal time, which is part of the removal cost, will be described later.

[0034] The operation determination unit 14f determines the operation of the robot based on the removal cost and detour cost calculated by the cost calculation unit 14e. When it is determined that the removal cost is low, the operation of the robot may be determined based on the removal method stored in the removal method storage unit 12c. The operation determination unit 14f may transmit an optimal control law to the robot via the communication unit.

[0035] The control law determination unit 14g selects an optimal control law from the robot movements determined by the movement determination unit 14f and stores the optimal control law in the control law storage unit 12d. The control law determination unit 14g may transmit the optimal control law to the robot via the communication unit.

[0036] The robot control unit 14h calculates the operation amount of the robot 2 using the optimal control law stored in the control law storage unit 12d. The robot control unit 14h may also calculate the operation amount of the robot 2 based on the movement determined by the movement determination unit 14f. By referring to the operation amount based on the optimal control law calculated by the robot control unit 14h, the robot 2 can continue efficient autonomous control to achieve its goal even in a situation where an unexpected obstacle has occurred. In this embodiment, the robot 2 is provided with an output control unit 14b, a communication unit 13, etc., and outputs or communicates with the robot 2, but the output control unit 14b may also control the robot 2 based on the operation amount.

[0037] <Example of robot working environment> Next, an example of the operation of the robot 2 controlled by the robot control device 1 will be described with reference to Fig. 2. In the following, movement of the robot 2 in the positive direction (rightward) of the X axis in Fig. 2 will be referred to as "rightward movement" for convenience, movement in the negative direction (leftward) of the X axis will be referred to as "leftward movement" for convenience, movement in the positive direction (upward) of the Y axis will be referred to as "upward movement" for convenience, and movement in the negative direction (downward) of the Y axis will be referred to as "downward movement" for convenience.

[0038] In this example, Robot 2, which is located near coordinates "X-axis 0m, Y-axis 0m," aims to transport a package to a destination near coordinates "X-axis 6m, Y-axis 6m." However, there is an obstacle, a pile of cardboard boxes, on Robot 2's planned path, making it difficult to pass through the area near coordinates "X-axis 3m, Y-axis 1m." However, since this obstacle can be removed, Robot 2 has two options for moving to its destination: remove the obstacle and continue on the planned path, or use a detour.

[0039] 2, it is assumed that reinforcement learning technology is applied as a means for calculating the optimal control law in the control law determination unit 14g. Note that, in addition to reinforcement learning, means for calculating the optimal control law in the control law determination unit 14g include deep reinforcement learning and genetic algorithms, but the present invention does not limit the optimization method to these examples.

[0040] When optimizing control rules using reinforcement learning, state S is defined as input information for reinforcement learning. In the problem of optimizing the behavior of Robot 2, for example, the coordinate position can be defined as a state, as shown in Figure 3. As an example of a state, when the coordinates on both the X and Y axes are in the range of 0m to 2m, it is state S1, and when the coordinates on both the X and Y axes are in the range of 4m to 6m, it is state S9. Note that state S1 is the starting position of Robot 2, state S2 is the position of the stacked cardboard boxes (obstacle), state S3 is not directly related to the explanation here, but may be the location of a worker who can assist in removing the obstacle in some cases, and state S9 is Robot 2's destination.

[0041] <Processing flow> Next, with reference to the flowchart of FIG. 4, the details of the processing executed by the robot control device 1 mainly through cooperation between the storage unit 12 and the calculation unit 14 when controlling the robot 2 will be sequentially explained.

[0042] First, in step St1, the environment detection unit 14c generates or updates map information (estimated map) relating to the travel path, such as the road surface, walls, and obstacles in the surrounding environment, based on the output information of the sensor 3, and stores the information in the estimated map storage unit 12b. At the same time, the environment detection unit 14c estimates the self-position of the robot 2 on the estimated map and the base map using the output information of the sensor 3.

[0043] In step St2, the environment detection unit 14c detects obstacles on the movement path of the robot 2 using the output information of the sensor 3. The obstacles detected here are registered in the estimation map.

[0044] In step St3, the environment detection unit 14c determines whether the obstacle detected on the movement path is movable. This allows the robot to autonomously determine whether it is a target for removal. An example of a method for determining whether it is movable is to compare the estimated map with the base map and detect the difference. If an obstacle recorded in the estimated map exists in the base map, it can be determined that the obstacle is likely to be difficult to move because it is fixed to a wall, floor, etc. Conversely, if the obstacle does not exist in the base map, it can be determined that the obstacle is likely to be relatively easy to move. Therefore, if the obstacle does not exist in the base map, proceed to step St4, and if it does exist, proceed to step St11.

[0045] In step St4, the object information acquisition unit 14d acquires information necessary to remove the obstacle detected by the environment detection unit 14c. Examples of information necessary to remove an obstacle include the shape, volume, weight, and strength of the obstacle, as well as the tools required for removal. For example, by analyzing image information from a camera, which is a type of sensor 3, the unit distinguishes between an autonomously moving object such as a worker or another robot 2 and an inactive object such as a cardboard box or a cart. For inactive objects, in addition to image analysis, information such as the volume and weight of the obstacle can be acquired by contacting the obstacle, such as by pushing or lifting it. Another method involves reading information such as the volume and weight from a mark containing information, such as an icon or barcode, attached to the obstacle. The content and method of the obstacle removal information collected by the object information acquisition unit 14d are not limited to these examples.

[0046] In step St5, the cost calculation unit 14e determines whether or not the obstacle on the movement path of the robot 2 is of a type that can be removed. As an example of a method for determining whether or not the obstacle can be removed, there is a method in which, among the information extracted by the object information acquisition unit 14d, if the obstacle is an object that does not move autonomously, it is determined that the obstacle can be removed, and if the obstacle is an object that moves autonomously, it is determined that the obstacle cannot be removed. Examples of objects that do not move autonomously include cardboard boxes and carts. Examples of objects that move autonomously include workers and other robots.

[0047] The reason why workers and other robots are determined to be unremovable is that it is not appropriate for robot 2 to remove humans from a safety standpoint, and it is also appropriate to classify other robots as unremovable in consideration of the possibility of damage. However, if workers and other robots can be requested to evacuate from the movement route, they may be determined to be removable. Note that the method of determining whether an obstacle can be removed in the present invention is not limited to these examples. If the obstacle can be removed, proceed to step St6, and conversely, if it cannot be removed, proceed to step St11.

[0048] In step St6, the cost calculation unit 14e estimates an appropriate obstacle removal method using the obstacle information acquired by the object information acquisition unit 14d, and generates a removal method candidate list. The generated removal method candidate list is stored in the removal method storage unit 12c. An example of the removal method candidate list is shown in FIG. 5.

[0049] As an example of a removal technique, when stacked cardboard boxes are obstacles, as shown in the example of FIG. 2, robot 2 may use robot arm 21 or the like to remove the cardboard boxes from the movement path, or may request obstacle removal resources, such as nearby workers or other robots, to remove the cardboard boxes. For example, when there are a large number of cardboard boxes and the removal cost can be reduced by having robot 2 cooperate with nearby workers and obstacle removal resources, such as other robots, rather than removing the cardboard boxes alone, nearby workers and obstacle removal resources, such as other robots, may be requested to remove the cardboard boxes. Even for robots without robot arms, if the obstacle is a cart, a removal technique can be used in which the cart (obstacle) is removed from the movement path by pushing it with the robot's bumper. Note that automatic estimation methods include estimation methods based on image recognition using neural network technology, but the optimization technique in the present invention is not limited to these examples.

[0050] In step St7, if the candidate obstacle removal methods stored in the removal method storage unit 12c include a method of requesting removal from an obstacle removal resource such as a worker, the cost calculation unit 14e calculates the time from the time of the request until the obstacle is removed by the worker, etc. This removal time can be calculated using information such as the distance to a waiting location for the worker, etc., or the average value of past removal times, but the present invention is not limited to these examples.

[0051] In step St8, the cost calculation unit 14e refers to the list of candidate obstacle removal methods stored in the removal method storage unit 12c and calculates the removal time required to execute each candidate method. The cost calculation unit 14e also calculates the detour time required to take a route that bypasses the obstacle. FIG. 5 shows the predicted time required to move the stacked cardboard boxes from the movement route using the robot arm 21 in FIG. 2. In the example of FIG. 5, the time required to remove the obstacle using the robot arm 21 is calculated to be 120 seconds. The removal time can be calculated using information such as the average value of past removal times, but the present invention is not limited to these examples. The detour time can be calculated by calculating the time required to take the detour from the robot's current position or the extra time required from the difference between when the detour is taken and when the detour is not taken.

[0052] In step St9, the control law determination unit 14g generates an evaluation function such as an objective function or a reward function. Since the control law determination unit 14g of this embodiment employs reinforcement learning, it generates a reward function. In this way, it generates an evaluation function appropriate for the optimization technique employed by the control law determination unit 14g. An example of the reward function generated by the control law determination unit 14g in the environment of FIGS. 2 and 3 is shown in the following (Equation 1).

[0053]

number

[0054] In (Equation 1), s' is the state after the operation of the robot 2 (the object of operation), ω is the weighting coefficient, T rem is the obstacle removal time, R is the reward function, γ is the decay rate, and a is the operation amount. In (Equation 1), when the state transitions to state S9, which is the destination in Figures 2 and 3, the reward function = 1 is output, and when the state transitions to state S2, which has an obstacle, the weight coefficient ω, the decay rate γ, and the removal time T rem The value calculated from the time Tstp required for one state transition is output as a reward function. In other words, (Equation 1) functions as a function that imparts a penalty according to the removal time when transitioning to state S2 where an obstacle exists. Note that the removal time Trem refers to the removal time in the list of candidate removal methods for obstacles stored in the removal method storage unit 12c. If there are multiple candidate removal methods in the list, for example, the shortest removal time is set as the removal time T rem If the frequency of use is recorded in a list, the removal time of the most frequently used removal method is referred to as the removal time T rem However, in the present invention, the method of selecting the removal time to be referenced is not limited to these examples.

[0055] In step St10, the control law determination unit 14g uses the evaluation function generated in step St9 to optimize the control law of the robot 2 based on reinforcement learning technology, and stores the optimal control law in the control law storage unit 12d. In reinforcement learning technology, the optimal control law is calculated by updating the value function each time a state transition is experienced. An example of a value function table is shown in Figure 6. The value function in Figure 6 is a table that stores evaluation values ​​for each state and operation amount, and is updated based on an update formula. An example of the update formula for the value function is shown in (Formula 2) below.

[0056]

number

[0057] In (Equation 2), Q is the value function, a * is the maximum value operation amount at state s' after the transition, α is the learning rate, and γ is the decay rate. (Equation 2) is a general update formula for a method called Q-learning, a type of reinforcement learning technology. By using the reward function R generated in step St9 in (Equation 2), the value of state S, where an obstacle is present, decreases depending on the removal time. Therefore, by comparing the value with states that take detours (states S4, S7, and S8), it is possible to determine the fastest route to the destination (state S9) by either taking the detour or removing the obstacle. For example, assume that the value of the operation amount to transition to state S7 at state S4 is approximately 0.66, the value of the operation amount to remove the obstacle at state S2 and transition to state S5 is 0.73, and the reward at state S2 is -0.1. Note that selecting an operation to transition to state S4 in state S1 will take a route that takes a detour, and conversely, selecting an operation to transition to state S2 in state S1 will take a route that removes an obstacle. In this case, from (Equation 2), the value of the operation amount to take the detour is approximately 0.59, and the value of the operation amount to take the route that removes the obstacle is 0.66, and it can be determined that the operation amount to take the route that removes the obstacle will allow the destination (state S9) to be reached the fastest.

[0058] Next, we will explain the case where the operation amount is determined by comparing the detour cost and removal cost using the same example as above. First, the detour cost is calculated by calculating the difference between the value of the operation amount to take the shortest route in an obstacle-free situation and the value of the operation amount to take the detour. In this example, the value of the operation amount to take the shortest route in an obstacle-free situation is the value (0.83) of the operation amount to remove the obstacle in state S2 and transition to state S5 (0.73) minus the reward (-0.1) for state S2. Therefore, the difference in value is 0.83 - 0.66 = 0.17. On the other hand, the removal cost is the absolute value of the reward in state S2, which is 0.1. Therefore, since the removal cost is lower than the detour cost, it can be determined that the operation amount to take the route that removes the obstacle will result in the fastest arrival at the destination (state S9).

[0059] When a route is calculated using a method other than reinforcement learning technology, an example of an evaluation formula for comparing and evaluating a detour route and a route that removes an obstacle is shown below (Formula 3).

[0060]

number

[0061] In Equation 3, E is the evaluation value of the route, γ is the decay rate, Trt is the time required to reach the goal, Trem is the removal time, and Tstp is the time required for one state transition (control period). For example, if there is no obstacle on the detour and the removal time Trem is 0 seconds, the time required to reach the goal Trt is 300 seconds, the control period Tstp is 60 seconds, and the decay rate is 0.9, the evaluation value E is approximately 0.59. Similarly, if the time required to reach the goal on a route that removes obstacles Trt is 300 seconds and the removal time Trem is 60 seconds, the evaluation value E is approximately 0.66. Therefore, the route that removes obstacles has a higher evaluation value E and can be identified as the route that reaches the destination (state S9) the fastest. The evaluation value here is an evaluation index when the robot removes obstacles and when the robot detours around obstacles.

[0062] In the example of the value function in FIG. 6, the evaluation value of the operation amount (moving right) from the initial state S1 to state S2 where an obstacle exists is 0.6, while the evaluation value of the operation amount (moving up) to state S4, which is a detour, is 0.5. In this case, it can be seen that the optimal operation is to select the operation amount (moving right) with the higher evaluation value, remove the obstacle, and move toward the destination. In this case, the method for removing the obstacle is the removal method selected in step St9 from the list of candidate obstacle removal methods stored in the removal method storage unit 12c. Note that in FIG. 6, an evaluation value of 0.0 indicates an operation amount that cannot be selected due to the influence of an obstacle such as a wall, and the circles in the figure indicate the best operation amount to be selected for each state S.

[0063] In this way, in reinforcement learning, a table storing the operation amount with the highest evaluation value in each state is set as the optimal control law, and a table such as that shown in FIG. 7 is stored in the control law storage unit 12d as an example of the optimal control law.

[0064] In step St11, the control law determination unit 14g plans a new detour route based on the base map and the estimated map if the obstacle is a fixed object or the like that cannot be moved. If it is determined that there is no detour route, the robot control unit 14h requests support from a manager or worker. The method for requesting support will be described in detail later.

[0065] In step St12, the robot control unit 14h controls the robot 2 based on the optimal control law stored in the control law storage unit 12d.

[0066] According to the robot control system of this embodiment described above, if an unexpected obstacle is present on the movement path, the robot can use methods other than bypassing the obstacle, such as removing the obstacle by itself or utilizing obstacle removal resources around the robot, as necessary, to optimize the movement path of the robot and determine a control rule that enables the robot to arrive at its destination early. [Example]

[0067] Next, a second embodiment of the robot control system 100 of the present invention will be described with reference to Fig. 8. Note that a duplicated description of points common to the first embodiment will be omitted.

[0068] In Example 1, the operation of removing an obstacle and the operation of taking a detour were compared and evaluated by adding information about the removal time to the reward function, as in the middle equation of (Equation 1). In this case, there was a problem in that the magnitude of the weighting coefficient ω had to be set appropriately.

[0069] Therefore, in this embodiment, in order to achieve substantially the same effect as in the first embodiment without using the weighting coefficient ω, information on the removal time is added to the update formula to compare and evaluate the merits of taking a detour route and removing the obstacle.

[0070] 8 is a diagram showing the flow of processing performed by the robot control device 1 of this embodiment. Since the processing is the same as that of FIG. 4 except that step St9 is replaced with step St9a, only step St9a will be described below.

[0071] In step St9a, the control law determination unit 14g defines an update equation for the value function. An example of the definition of the update equation is shown in the following (Equation 4).

[0072]

number

[0073] Comparing (Equation 2), which is the update equation for the value function in Example 1, with (Equation 4), which is the update equation for the value function in this example, the difference lies in whether or not the decay rate γ in the equation is raised to the power of τ. Here, τ in (Equation 4) is a removal time coefficient, and is defined by the following (Equation 5).

[0074]

number

[0075] In (Equation 5), T stp is the time required for one state transition (control period), and the brackets are ceiling functions. Therefore, the removal time coefficient τ is a value obtained by replacing the time required to remove an obstacle with the number of state transitions. Therefore, similar to the reward function in Example 1, it has the function of lowering the value of the state S where an obstacle exists according to the removal time. Note that the removal time T rem The method of referring to is the same as in step St9. In addition, since the calculation of the reward function in this embodiment does not require the removal time, (Equation 1) in the first embodiment can be simplified as shown in the following (Equation 6).

[0076]

number

[0077] In this way, by using (Equation 6), which is a calculation formula for the reward function that does not take removal time into consideration, it is possible to obtain the same effect as in Example 1 without having to perform the complicated process of adjusting the weight coefficient ω as in Example 1. [Example]

[0078] Next, a third embodiment of the robot control system 100 of the present invention will be described with reference to Figures 9 and 10. Note that a duplicated description of points common to the above-mentioned embodiments will be omitted.

[0079] As described in the first embodiment, the robot control system 100 of the present invention can request support from a manager or a worker as needed. In this embodiment, an example of a display screen on the mobile terminal 6 when requesting support from a worker and an example of a display screen on the display device 5 when requesting support from a manager will be described.

[0080] <Example of display screen on mobile device 6> FIG. 9 shows an example in which, when the object information acquisition unit 14d of the robot control device 1 determines that an obstacle on the movement path to the destination is a worker, the output control unit 14b displays a movement request to the worker on the screen of the mobile terminal 6 held by the worker.

[0081] As shown here, the screen of the mobile terminal 6 displays messages and diagrams encouraging the worker to move from the movement route of the robot 2. The screen also displays buttons for responding to the request to move to the robot control device 1, and the worker can select a button using an input device such as a touch panel. In the example of Figure 9, three buttons are displayed as possible responses to the robot control device 1: "Difficult to move," "Plan to move," and "Contact the person in charge."

[0082] When the "Contact Administrator" button 6a is pressed, the robot control device 1 sends a notification to the administrator of the robot 2, enables communication between the worker and the administrator, and has the robot 2 wait in place until a new button is pressed.

[0083] When the "Difficult to move" button 6b is pressed, the robot control device 1 deletes the candidate related to the request to move the worker from the list of candidate obstacle removal methods stored in the removal method storage unit 12c.

[0084] When the "Planned Movement" button 6c is pressed, the robot control device 1 makes the robot 2 wait until the worker has completed moving from the movement route. As a means for confirming that the worker has completed moving, for example, a "Movement Completed" button may be provided on the screen. The estimated time required to complete the movement can also be entered on the screen, and in the example shown, the required time can be set using a scroll bar.

[0085] <Example of display screen of display device 5> FIG. 10 shows an example in which, when the robot control unit 14h of the robot control device 1 determines that support should be requested from a manager, information requesting support from the manager is displayed on the screen of the display device 5.

[0086] As shown here, the screen of the display device 5 displays a message and a diagram informing the manager that the robot 2 requires human support. The diagram may also display sensor information such as the robot 2's location information and camera images. Buttons for response options to the request for movement are displayed on the screen, allowing the manager to send a response option to the robot 2. In the example of Figure 10, four buttons are displayed as response options to the robot control device 1: "Detailed information," "Waiting request," "Remote support," and "Move to site."

[0087] When the "Detailed Information" button 5a is selected, the robot control device 1 transmits sensor information other than the information currently being displayed and various information stored in the memory unit 12 to the display device 5, allowing the administrator to check the details.

[0088] When the "Wait request" button 5b is selected, the robot control device 1 causes the robot 2 to wait until a new instruction is input.

[0089] When the "remote support" button 5c is selected, the robot control device 1 switches the operation mode of the robot 2 from autonomous control in cooperation with the robot control device 1 to remote control by manual operation by the administrator.

[0090] When the "Move to site" button 5d is selected, the robot control device 1 causes the robot 2 to wait until the manager's human support is completed. The estimated time required for the human support to be completed can also be entered on the screen, and in the example shown, the required time can be set using a scroll bar.

[0091] As described above, according to this embodiment, it is possible to appropriately achieve cooperation between the robot control system 100 and the worker or manager.

[0092] <Modification> In the first and second embodiments, the process by the robot control device 1 to remove an object on the path of the robot 2 has been described. In this modification, the robot 2 notifies a worker of where the object has been removed or allows the worker to search for the object later. The difference between this modification and the first and second embodiments is that an object search unit 14i has been newly added to the robot control system 100. FIG. 11 shows the robot control system 100 to which the object search unit 14i has been added. The object search unit 14i is processed by, for example, a CPU or processor within the robot control system 100. The process of the object search unit 14i will be described below. For example, the environment detection unit 14c adds the positions and object information of the object before and after removal to map information (an estimated map). When a worker searches for the object, the original position and object information of the object are input from a mobile terminal carried by the worker. The object search unit 14i compares the input object position and object information with the object position and object information added to the map information before removal, identifies the object's position after removal, and sends the position information to the worker's mobile device. This eliminates the worker's need to search for the removed object, further improving work efficiency. Furthermore, once the map information is added, the added map information (estimated map) may be output via the output control unit 14b to the worker's mobile device 6 or the manager's display device, thereby providing advance notification. [Explanation of symbols]

[0093] 100 Robot Control System 1. Robot control device 11 Data Acquisition Section 12 Storage section 12a Base map storage section 12b Estimation map memory section 12c Removal method storage section 12d Operation memory section 12e Control law memory section 13 Communications Department 14 Arithmetic section 14a Input control section 14b Output control section 14c Environmental detection unit 14d Object information acquisition unit 14e Cost Calculation Section 14f Action decision unit 14g Control law determination section 14h Robot control unit 14i Object search section 2. Robot 21 Robotic Arm 22 Drive mechanism 23 Cargo bed 3 sensors 4 Input Devices 5 Display device 6. Mobile devices 7. Cloud

Claims

1. An autonomously mobile robot, an environment detection unit that detects objects on the path of movement of the robot; a motion determination unit that determines one of an action to bypass the object detected by the environment detection unit, an action to remove the object detected by the environment detection unit, an action to have an object removal resource other than a robot remove the object detected by the environment detection unit, and an action to have an object removal resource other than the robot and the robot remove the object detected by the environment detection unit.

2. The robot according to claim 1, The robot further comprises a robot control unit that controls a removal operation based on a decision made by the operation decision unit when the environment detection unit determines that the object can be moved.

3. The robot according to claim 2, The environment detection unit determines whether the detected object is registered in a base map in which the robot's working environment is registered in advance, based on the base map, and determines that an object not registered in the base map is movable.

4. The robot according to claim 2, The environment detection unit detects the robot's working environment, generates an estimated map based on environmental information related to the robot's working environment, and determines whether the detected object can be moved based on the difference between the estimated map and a base map in which the robot's working environment is registered in advance.

5. The robot according to claim 3 or 4, The robot further comprises a cost calculation unit that calculates a removal cost for the object that the environment detection unit has determined to be movable.

6. The robot of claim 1, a cost calculation unit that calculates a removal cost that is a cost of removing the object and a detour cost that is a cost of the robot detouring around the object, The robot is characterized in that the operation determination unit compares the removal cost calculated by the cost calculation unit with the detour cost and determines the robot operation that will result in the lowest cost.

7. 7. The robot according to claim 6, The robot further comprises an object information acquisition unit that acquires object information regarding the object detected by the environment detection unit.

8. The robot according to claim 7, The robot is characterized in that the cost calculation unit calculates the removal cost based on the object information acquired by the object information acquisition unit.

9. The robot according to claim 7, The robot is characterized by having an output control unit that, when the object information acquisition unit determines that an object on the movement path is a worker, requests the worker to move via the display screen of a mobile device carried by the worker.

10. The robot according to claim 7, The object information includes at least one of the object's shape, volume, weight, strength, and tools required to remove the obstacle.

11. 7. The robot according to claim 6, The robot is characterized in that the operation determination unit calculates an objective function, an evaluation function, or a reward function for each operation using the removal cost and the detour cost as input values, and determines an operation.

12. The robot according to claim 11, The robot is characterized in that the operation determination unit calculates, for each operation, an evaluation value of an operation amount, a parameter of an update formula for the evaluation value of the operation amount, a value of the operation amount, or a parameter of an update formula for the value of the operation amount, and determines the operation.

13. 7. The robot according to claim 6, the operation determination unit calculates an evaluation value for a state transition of the robot when the robot removes the object and an evaluation value for a state transition of the robot when the robot detours around the object using the removal cost, the detour cost, and an arbitrary attenuation rate, compares the evaluation values, and determines an operation based on the comparison result.

14. The robot according to claim 1, The robot is characterized in that the environment detection unit determines whether or not a detected object can be moved.

15. The robot according to claim 1, The robot further comprises a robot control unit that, when the operation determination unit determines an operation to have an object removal resource other than the robot remove the object, and an operation by which the object removal resource other than the robot and the robot remove the object, requests support for the removal work from the object removal resource other than the robot.

16. The robot according to claim 1, when the operation determination unit determines an operation of causing the object removal resource to remove the object or an operation of an object removal resource other than the robot and the robot to remove the object, the environment detection unit adds the position of the object before removal, the position of the object after removal, and object information to an estimated map generated based on environmental information related to a working environment of the robot; an object search unit that acquires the object position and object information input by a worker or a manager, compares the input object position and object information with the assigned object position before removal and object information, and identifies the object position after removal; The robot further comprises an output control unit that outputs the position of the object identified by the object search unit after removal to a device owned by a worker or a manager.

17. A robot control device for controlling an autonomously mobile robot, an environment detection unit that detects objects on the path of movement of the robot; an action determination unit that determines one of an action of bypassing the object detected by the robot, an action of removing the object detected by the robot, an action of making an object removal resource other than the robot remove the detected object, and an action of removing the object detected by the object removal resource other than the robot and the robot; A robot control device comprising: a control unit that controls the robot based on the action determined by the action determination unit.

18. An autonomously mobile robot, a sensor that detects environmental information related to a working environment of the robot; a robot control device that controls the robot; A robot control system comprising: The robot control device an environment detection unit that detects objects on the path of movement of the robot; an action determination unit that determines one of an action of bypassing the object detected by the robot, an action of removing the object detected by the robot, an action of making an object removal resource other than the robot remove the detected object, and an action of removing the object detected by the object removal resource other than the robot and the robot; a control unit that controls the robot based on the action determined by the action determination unit.

19. 20. The robot control system of claim 18, A robot control system characterized in that the task of removing the object from the movement path is performed by either the robot arm of the robot, the bumper of the robot, or an object removal resource other than a robot.

20. A robot control method for controlling an autonomously mobile robot, comprising: an environment detection step of detecting an object on a moving path of the robot; an action determination step of determining one of an action of bypassing the detected object, an action of removing the object detected by the robot, an action of making an object removal resource other than the robot remove the detected object, and an action of removing the object detected by the object removal resource other than the robot and the robot; a robot control step of controlling the robot based on the determined motion.

Images