Multi-robot control method, multi-robot control device, and multi-robot control system

Abstract

[Problem] To provide a multi-robot control method that enables a single operator to control a plurality of robots. [Solution] A multi-robot control method according to the present disclosure comprises the steps of: receiving operation instructions for a plurality of robots from an operator; operating the plurality of robot models faster than the real time within the environment model based on the operation instruction by using the environment model corresponding to the plurality of robots and the plurality of robot models, and acquiring a predicted observation value obtained as a result; displaying the predicted observation value to an operator; accumulating task information containing the operation instruction and the prediction observation value; and transmitting the accumulated task information to the plurality of robots.

Description

Technical Field

[0001] The present disclosure relates to a method for controlling a plurality of robots, an apparatus for controlling a plurality of robots, and a system for controlling a plurality of robots. Background Art

[0002] Recently, the use of robots in service fields such as housework, nursing and physical assistance is being promoted. In such fields, multiple robots are usually operated simultaneously in a single working environment. In addition, in the case where it is difficult for robots to perform all tasks autonomously, robots remotely controlled by operators are usually used (for example, see Patent Document 1). However, it is inefficient in terms of cost for one robot to occupy the control system of one operator.

[0003] Reference List

[0004] Patent Literature

[0005] Patent document 1: Japanese Patent Application Publication No. H9-254065. Summary of the invention

[0006] Problems to be solved by the present invention

[0007] The present disclosure is to solve the above-mentioned problems and aims to provide a control method for a plurality of robots, a control device for a plurality of robots, and a control system for a plurality of robots, wherein the plurality of robots can be controlled by one operator.

[0008] Solution to the problem

[0009] In order to solve the above problems, a method for controlling multiple robots according to the present disclosure includes the following steps: receiving operation instructions for multiple robots from an operator; using multiple robot models and an environment model corresponding to the multiple robots, based on the operation instructions, causing the multiple robot models to operate in the environment model at a speed faster than the actual time, and obtaining predicted observation values ​​obtained as a result; displaying the predicted observation values ​​to the operator; accumulating task information including the operation instructions and the predicted observation values; and sending the accumulated task information to the multiple robots.

[0010] In addition, according to the contents of the present disclosure, an apparatus for controlling a plurality of robots includes: an input unit, which receives operation instructions for the plurality of robots from an operator; a simulation unit, which includes a plurality of robot models and an environment model corresponding to the plurality of robots, and causes the plurality of robot models to operate in the environment model at a speed faster than the actual time based on the operation instructions, and outputs predicted observation values ​​obtained as a result; a display unit, which displays the predicted observation values ​​to the operator; an accumulation unit, which accumulates task information including the operation instructions and the predicted observation values; and a sending unit, which sends the task information accumulated in the accumulation unit to the plurality of robots.

[0011] In addition, a system for controlling multiple robots according to the present disclosure includes: multiple robots; and a control device for the multiple robots, wherein the control device includes: an input unit, which receives operation instructions for the multiple robots from an operator; a simulation unit, which includes multiple robot models and an environment model corresponding to the multiple robots, simulates the operation of the multiple robot models in the environment model at a speed faster than the actual time based on the operation instructions, and outputs predicted observation values ​​obtained as a result; a display unit, which displays the predicted observation values ​​to the operator; a first accumulation unit, which accumulates task information including operation instructions and predicted observation values; and a sending unit, which sends the task information accumulated in the first accumulation unit to the multiple robots, and each of the multiple robots includes: a receiving unit, which receives task information from the control device; a second accumulation unit, which accumulates task information received by the receiving unit; and a control unit, which causes the robot to operate in an actual environment according to the operation instructions included in the task information accumulated in the second accumulation unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a diagram showing a configuration of a control system for a plurality of robots according to a first embodiment.

[0013] Figure 2 : is a diagram showing a specific configuration of a control device and a robot according to the first embodiment.

[0014] Figure 3 is a diagram showing a logical configuration of a simulator and an accumulation unit according to the first embodiment.

[0015] Figure 4 is a diagram showing a logical configuration of a robot and an accumulating unit according to the first embodiment.

[0016] Figure 5 is a flowchart for explaining the operation of the control system for a plurality of robots according to the first embodiment.

[0017] Figure 6 It is a timing diagram showing the transmission and execution of task information from the control device to the robot.

[0018] Figure 7 is a table showing an example of the definition of environment complexity.

[0019] Figure 8 is a diagram showing a configuration in which the simulator and the robot have a one-to-one correspondence.

[0020] Fig. 9 is a diagram showing a configuration in which the simulator and the robot have a 1-to-N correspondence relationship.

[0021] Fig.10 is a table showing an example of the definition of parameters of the autonomous driving robot.

[0022] Fig.11 : is a table showing an example of calculation of the maximum value of the shortening rate in the automatic driving robot. DETAILED DESCRIPTION

[0023] In the following description, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings, the same or corresponding elements are marked with the same reference numerals, and detailed descriptions thereof will not be repeated as appropriate.

[0024] (First Embodiment)

[0025] Figure 1 1 is a diagram showing a configuration of a control system 100 for multiple robots according to a first embodiment of the present disclosure. The control system 100 includes a control device 10 operated by an operator 1 and two robots 20a, 20b controlled by the control device 10. The control device 10 is set in an operation station B. The robot 20a is set in a work environment W1, and the robot 20b is set in a work environment W2. It is generally assumed that the operation station B and the work environments W1, W2 are located in different places, but they may be located in the same place.

[0026] The control device 10 and the robot 20a can communicate with each other via wireless communication. Similarly, the control device 10 and the robot 20b can communicate with each other via wireless communication. The operator 1 can control the robots 20a, 20b by operating the control device 10. That is, the operator 1 can control the robots 20a, 20b via the control device 10.

[0027] The robots 20a, 20b are not limited to any particular type, but are manipulator robots used for housework support at home as an example. Note that the number of robots controlled by the control device 10 is not necessarily two, but may be three or more. In addition, a plurality of robots may exist in one working environment.

[0028] (Configuration of control device)

[0029] Figure 2 2 is a diagram showing a specific configuration of the control device 10 and the robots 20a, 20b. The control device 10 includes an input unit 11, a simulation unit 12, a display unit 13, accumulation units 14a and 14b, a transmission unit 15, and a reception unit 16.

[0030] The input unit 11 is formed of, for example, a keyboard or a mouse, and receives operation instructions to the robots 20a, 20b from the operator 1. Specific examples of the operation instructions relate to forward movement, backward movement, direction change, acceleration, deceleration, etc. of the robot.

[0031] The simulation unit 12 is formed of, for example, a personal computer, etc. The simulation unit 12 simulates the operation of the robots 20a, 20b based on the operation instructions to the robots 20a, 20b received from the operator 1 via the input unit 11. Specifically, the simulation unit 12 includes a simulator 12a that simulates the operation of the robot 20a and a simulator 12b that simulates the operation of the robot 20b.

[0032] Figure 3 1 is a diagram showing the logical configuration of the simulators 12a, 12b and the accumulation units 14a, 14b which will be described later. The simulator 12a includes a robot model 121a corresponding to the robot 20a, and an environment model 122a corresponding to the work environment W1. Similarly, the simulator 12b includes a robot model 121b corresponding to the robot 20b and an environment model 122b corresponding to the work environment W2.

[0033] The simulator 12a operates the robot model 121b in the environment model 122a at a faster speed than the actual time based on the operation instruction from the operator 1, and outputs the predicted observation value obtained as a result. Specific examples of the predicted observation value include internal information such as the position, velocity, and acceleration of the robot model 121a, and external information such as the position, velocity, and acceleration of obstacles existing around the robot model 121a.

[0034] Likewise, the simulator 12b operates the robot model 121b in the environment model 122b at a faster speed than the actual time based on the operation instruction from the operator 1, and outputs the predicted observation value obtained as a result. Specific examples of the predicted observation value include internal information such as the position, velocity, and acceleration of the robot model 121b, and external information such as the position, velocity, and acceleration of obstacles existing around the robot model 121b.

[0035] In the environment models 122a, 122b, the robot models 121a, 121b operate faster than the real time. Specifically, when the real time, which is the time actually passed in the real environment, is represented by Treal, and the simulation time passed in the environment model is represented by Tsim, the following relationship is established between the two.

[0036] Shortening rate Kscale = Tsim / Treal.

[0037] In the above equation, the shortening rate Kscale is a real number of 1 or more. For example, in the case of shortening rate Kscale=10, when 100 seconds of simulation time has passed in the environment model, only 10 seconds of actual time has passed.

[0038] Return to reference Figure 2 , the display unit 13 is formed of, for example, a liquid crystal display or the like. The display unit 13 displays to the operator 1 the predicted observation value corresponding to the robot 20a output from the simulator 12a, and the predicted observation value corresponding to the robot 20b output from the simulator 12b. The time felt by the operator 1 is the actual time. Therefore, in the case of the shortening rate Kscale=10, for example, the operator 1 can feel the simulated time passing at a rate of 1 / 10 of the actual time in the environment model. Therefore, in principle, the operator 1 can simultaneously operate up to 10 robot models in the environment model.

[0039] The accumulation units 14a, 14b are formed of, for example, a storage device, etc. The accumulation unit 14a accumulates task information corresponding to the robot 20a, including operation instructions from the operator 1, predicted observation values ​​obtained by operating the robot model 121a in the environment model 122a based on the operation instructions, and time information in the environment model 122a (see Figure 3 ). Similarly, the accumulating unit 14b accumulates task information related to the robot 20b, including the operation instructions from the operator 1, the predicted observation values ​​obtained by operating the robot model 121b in the environment model 122b based on the operation instructions, and the time information in the environment model 122b (see Figure 3 ).

[0040] exist Figure 3 , the task information accumulated for the first time in the accumulation unit 14a is the task information at time Tst in the environment model 122a. The task information accumulated for the second time is the task information at time Tst+ΔT in the environment model 122a. The task information accumulated for the Nth time is the task information at time Tst+NΔT in the environment model 122a. Here, ΔT represents the time elapsed in the environment model 122a as the simulation time.

[0041] Likewise, the task information accumulated for the first time in the accumulation unit 14b is the task information at time Tst in the environment model 122b. The task information accumulated for the second time is the task information at time Tst+ΔT in the environment model 122b. The task information accumulated for the Nth time is the task information at time Tst+NΔT in the environment model 122b. Here, ΔT represents the time elapsed in the environment model 122b as the simulation time.

[0042] Return to reference Figure 2 The sending unit 15 is formed of, for example, a wireless communication interface, and collectively sends the task information accumulated in the accumulating unit 14a to the corresponding robot 20a. The sending unit 15 also collectively sends the task information accumulated in the accumulating unit 14b to the corresponding robot 20b.

[0043] The receiving unit 16 is formed of, for example, a wireless communication interface or the like, and receives abnormality detection information to be described later from the robot 20 a or the robot 20 b . The display unit 13 displays the abnormality detection information received by the receiving unit 16 to the operator 1 .

[0044] (Robot configuration)

[0045] Next, the configuration of the robots 20a and 20b will be described. However, the robots 20a and 20b have the same configuration, and the robot 20a is described here. The robot 20a includes a receiving unit 21a, an accumulating unit 22a, a control unit 23a, a driving motor 24a, an internal sensor 25a, an external sensor 26a, and an abnormality detecting unit 27a.

[0046] The receiving unit 21a is formed of, for example, a wireless communication interface or the like, and collectively receives the task information transmitted from the control device 10. The task information has been accumulated in the accumulation unit 14a of the control device 10.

[0047] The accumulation unit 22 a is formed of, for example, a storage device or the like, and accumulates the task information collectively received by the reception unit 21 a . Figure 4 2 is a diagram showing the robots 20a, 20b in a working environment as an actual environment, and a logical configuration of the accumulating units 22a, 22b thereof.

[0048] exist Figure 4, the task information accumulated for the first time in the accumulation unit 22a is the task information at time Tst in the actual environment. The task information accumulated for the second time is the task information at time Tst+Δτ in the actual environment. The task information accumulated for the Nth time is the task information at time Tst+NΔτ in the actual environment. Here, the time Tst in the actual environment is equal to the time Tst in the environment model. In addition, Δτ represents the time elapsed in the actual environment as the actual time, and the relationship is represented by Δτ=Kscale·ΔT.

[0049] Return to reference Figure 2 The control unit 23a is formed of, for example, a microcomputer. The control unit 23a reads the task information accumulated in the accumulation unit 22a, and controls the drive motor 24a and the like according to the operation instructions included in the task information to control the operation of the robot 20a in the actual environment, which is the working environment W1 (see Figure 4 Specific examples of the operation of the robot 20a include forward movement, backward movement, direction change, acceleration, deceleration, etc. of the robot 20a.

[0050] In addition, the control unit 23a acquires actual observation values ​​obtained due to the operation of the robot 20a via the internal sensor 25a and the external sensor 26a. Specific examples of the actual observation values ​​include internal information such as the position, speed and acceleration of the robot 20a, and external information such as the position, speed and acceleration of obstacles present around the robot 20a.

[0051] The abnormality detection unit 27a is formed of, for example, a microcomputer. The abnormality detection unit 27a detects the difference between the predicted observation value and the actual observation value as abnormal based on the comparison between the predicted observation value included in the task information and the actual observation value acquired via the internal sensor 25a and the external sensor 26a (see Figure 4 ). For example, when the difference between the position and speed of the surrounding obstacles in the predicted observation value and the position and speed of the surrounding obstacles in the actual observation value is equal to or greater than a predetermined threshold, the difference is determined to be abnormal. When the abnormality detection unit 27a detects an abnormality, the abnormality detection unit 27a sends abnormality detection information including the details of the abnormality to the control device 10 via wireless communication.

[0052] (Control system operation)

[0053] Next, refer to Figure 5 The flowchart shown in FIG. 1 illustrates the operation of the control system 100 for multiple robots according to the first embodiment. The operation of the control system 100 is divided into an operation phase of steps S11 to S16 implemented by the control device 10 and an execution phase of steps S21 to S27 implemented by the robots 20a, 20b.

[0054] In step S11 of the execution phase, the input unit 11 of the control device 10 receives an operation instruction from the operator 1 to the robot 20a. In addition, the input unit 11 of the control device 10 receives an operation instruction from the operator 1 to the robot 20b.

[0055] In step S12, the simulation unit 12 of the control device 10 simulates the operation of the robots 20a, 20b at a higher speed than the actual time based on the operation instruction received in the above step S11. Specifically, the simulator 12a operates the robot model 121a in the environment model 122a faster than the actual time based on an operation instruction, and outputs a predicted observation value obtained as a result. Similarly, the simulator 12b operates the robot model 121b in the environment model 122b faster than the actual time based on an operation instruction, and outputs a predicted observation value obtained as a result.

[0056] In step S13 , the display unit 13 of the control device 10 displays to the operator 1 one predicted observation value corresponding to the robot 20 a and one predicted observation value corresponding to the robot 20 b as the output in step S12 described above.

[0057] In step S14, the accumulation unit 14a of the control device 10 accumulates a set of task information, including the one operation instruction for the robot 20a received in step S11, the one predicted observation value corresponding to the robot 20a output due to the one operation instruction in step S12, and the time information in the environment model 122a. Similarly, the accumulation unit 14b of the control device 10 accumulates a set of task information, including the one operation instruction for the robot 20b received in step S11, the one predicted observation value corresponding to the robot 20b output due to the one operation instruction in step S12, and the time information in the environment model 122b.

[0058] In step S15, the input unit 11 of the control device 10 determines whether an operation instruction is continuously received from the operator 1. If an operation instruction is continuously received from the operator 1 (S15=No), the operation returns to the above-mentioned step S11, and if not (S15=Yes), the operation transfers to step S16.

[0059] In step S16, the transmission unit 15 of the control device 10 collectively transmits the task information accumulated in the accumulation unit 14a to the corresponding robot 20a. In addition, the transmission unit 15 of the control device 10 collectively transmits the task information accumulated in the accumulation unit 14b to the corresponding robot 20b. At this time, the operation phase implemented by the control device 10 ends and shifts to the execution phase to be implemented by the robots 20a, 20b.

[0060] In step S21 of the execution phase, the receiving unit 21a of the robot 20a collectively receives the task information from the control device 10. Similarly, the receiving unit 21b of the robot 20b collectively receives the task information from the control device 10.

[0061] In step S22, the accumulating unit 22a of the robot 20a accumulates the task information received collectively in step S21. At this time, the time information Tst+nΔT in the environment model 122a included in the task information is converted into the time information Tst+nΔτ in the actual environment. Here, n represents a natural number from 0 to N, and the relationship represented by Δτ=Kscale·ΔT holds.

[0062] Similarly, the accumulating unit 22b of the robot 20b accumulates the task information received collectively in step S21. At this time, the time information Tst+nΔT in the environment model 122b included in the task information is converted into the time information Tst+nΔτ in the actual environment. Here, n represents a natural number from 0 to N, and the relationship represented by Δτ=Kscale·ΔT holds.

[0063] In step S23, the control unit 23a of the robot 20a extracts a set of task information from the accumulation unit 22a, and controls the drive motor 24a, etc. according to an operation instruction included in the set of task information to control the operation of the robot 20a in the actual environment as the working environment W1. Specific examples of the operation of the robot 20a include forward movement, backward movement, direction change, acceleration, deceleration, etc. of the robot 20a.

[0064] Likewise, the control unit 23b of the robot 20b extracts a set of task information from the accumulation unit 22b, and controls the drive motor 24b, etc., according to an operation instruction included in the set of task information, to control the operation of the robot 20b in the actual environment as the work environment W2. Specific examples of the operation of the robot 20b include forward movement, backward movement, direction change, acceleration, deceleration, etc. of the robot 20b.

[0065] In step S24, the control unit 23a of the robot 20a acquires the one actual observation value obtained by the operation in step S23 via the internal sensor 25a and the external sensor 26a. Specific examples of the one actual observation value include internal information such as the position, velocity and acceleration of the robot 20a, and external information such as the position, velocity and acceleration of obstacles existing around the robot 20a.

[0066] Likewise, the control unit 23b of the robot 20b acquires the one actual observation value obtained as a result of the operation in step S23 described above via the internal sensor 25b and the external sensor 26b. Specific examples of the one actual observation value include internal information such as the position, velocity, and acceleration of the robot 20b, and external information such as the position, velocity, and acceleration of obstacles present around the robot 20b.

[0067] In step S25, the abnormality detection unit 27a of the robot 20a determines the difference between the predicted observation value and the actual observation value, or determines whether an abnormality exists, based on the comparison between one predicted observation value included in the set of task information extracted in step S23 and one actual observation value obtained in step S24. For example, in the case where the difference between the position and speed of the obstacle in the surrounding environment in the predicted observation value and the position and speed of the obstacle in the surrounding environment in the actual observation value is equal to or greater than a predetermined threshold, the difference is determined to be an abnormality.

[0068] If an abnormality is detected (S25=Yes), the abnormality detection unit 27a transmits abnormality detection information including details of the abnormality to the control device 10 via wireless communication (step S27), and the control on the robot 20a ends (END). Thereafter, the control on the robot 20a is transferred to real-time control to be performed by the operator 1. Thus, for example, the operator 1 can control an obstacle avoidance operation in real time. On the other hand, if no abnormality is detected (S25=No), the operation is transferred to step S26.

[0069] Likewise, the abnormality detection unit 27b of the robot 20b determines the difference between the predicted observation value and the actual observation value, or determines whether an abnormality exists, based on the comparison between one predicted observation value included in the set of task information extracted in step S23 and one actual observation value acquired in step S24. For example, in the case where the difference between the position and speed of the surrounding obstacles in the predicted observation value and the position and speed of the surrounding obstacles in the actual observation value is equal to or greater than a predetermined threshold, the difference is determined to be an abnormality.

[0070] If an abnormality is detected (S25=Yes), the abnormality detection unit 27b transmits abnormality detection information including details of the abnormality to the control device 10 via wireless communication (step S27), and the control on the robot 20b ends (END). Thereafter, the control on the robot 20b is transferred to real-time control to be performed by the operator 1. Thus, for example, the operator 1 can control the obstacle avoidance operation in real time. On the other hand, if no abnormality is detected (S25=No), the operation is transferred to step S26.

[0071] In step S26, the control unit 23a of the robot 20a determines whether there is task information remaining in the accumulation unit 22a. If there is task information remaining in the accumulation unit 22a (S26=No), the operation returns to the above-mentioned step S23. If the accumulation unit 22a is empty (S26=Yes), the control on the robot 20a ends (END).

[0072] Likewise, the control unit 23b of the robot 20b determines whether there is task information remaining in the accumulation unit 22b. If there is task information remaining in the accumulation unit 22b (S26=No), the operation returns to the above-mentioned step S23. If the accumulation unit 22b is empty (S26=Yes), the control on the robot 20b ends (END).

[0073] As described above, the control system for multiple robots according to the first embodiment includes multiple robot models and an environment model corresponding to the multiple robots. The control device operates the multiple robot models in the environment model faster than the actual time based on the operation instructions received from the operators of the multiple robots, and obtains the predicted observation values ​​obtained as a result. The control device accumulates task information including the operation instructions and the predicted observation values, and sends the accumulated task information to the multiple robots. Using such characteristics, the control system according to the first embodiment can enable one operator to control multiple robots.

[0074] Each robot accumulates the task information received from the control device and operates in the real environment according to the operation instructions included in the accumulated task information. Therefore, compared with the case where the task information is sequentially sent from the control device to each robot, the influence of the communication delay between the control device and each robot can be minimized.

[0075] Figure 6 The timing chart on the upper middle side is a timing chart in which the control device centrally transmits and executes the accumulated task information of the robot in the control system according to the first embodiment. Figure 6The timing diagram on the lower middle side is a timing diagram in which the control device sequentially sends and executes the task information of the robot, as seen in the conventional technology. As can be seen from the comparison between the two, the control system according to the first embodiment can minimize the influence of the communication delay between the control device and the robot.

[0076] Each robot acquires actual observation values ​​obtained by operation in the actual environment, and detects abnormalities based on the comparison between the predicted observation values ​​and the actual observation values. Therefore, each robot can detect the difference between the predicted observation values ​​and the actual observation values ​​as abnormalities in real time.

[0077] In the case where an abnormality is detected, the control of the robot in which the abnormality is detected is transferred to real-time control to be performed by the operator. Therefore, the operator can control the robot in real time when the abnormality is detected.

[0078] The relationship expressed as the shortening rate Kscale = Tsim / Treal holds between the actual time Treal felt by the operator and the simulated time Tsim passed in the environment model, and the shortening rate Kscale is a real number of 1 or greater. Therefore, the operator can feel the simulated time passed in the environment model as 1 / Kscale of the time. For example, in the case of the shortening rate Kscale = 10, the time required for the operator to operate a robot model for 100 seconds in the environment model is 100 seconds / 10 = 10 seconds. That is, one operator can operate up to 10 robot models at the same time. Therefore, one operator can effectively control multiple robots.

[0079] The shortening rate Kscale can be determined based on the frequency of occurrence of interference elements present in the robot working environment, which is the actual environment. For example, an obstacle with a high probability of a difference between the predicted observation value and the actual observation value is an interference element with a high frequency of occurrence. In the presence of such an interference element with a high frequency of occurrence, the shortening rate Kscale is set to a smaller value, so that a long time margin can be ensured from the time when an abnormality is detected to the time when control is transferred to real-time control and the operator takes measures for the abnormality.

[0080] The shortening rate Kscale can be determined based on the spatial density of interference elements present in the robot's working environment, which is an actual environment. For example, a working environment in which a relatively large number of obstacles exist is an environment with a high spatial density of interference elements. In the case where the spatial density of interference elements is high, the shortening rate Kscale is set to a smaller value, so that a long time margin can be ensured from when an abnormality is detected to when control is transferred to real-time control and the operator takes measures for the abnormality.

[0081] The shortening rate Kscale can be determined based on the difficulty level of dealing with interference elements existing in the working environment of the robot as the actual environment. For example, an obstacle with a relatively drastic change in speed is an interference element with a high level of difficulty in dealing with. In the presence of such an interference element with a high level of difficulty in dealing with, the shortening rate Kscale is set to a smaller value, so that a long time margin can be ensured from the time when an abnormality is detected to the time when control is transferred to real-time control and the operator takes measures for the abnormality.

[0082] The shortening rate Kscale can be determined based on a combination of the frequency of occurrence, spatial density, and difficulty level of interference elements present in the robot's operating environment as the actual environment. Figure 7 As shown, the environmental complexity based on the combination of the spatial density of interference elements and the difficulty of coping can be defined, and the shortening rate Kscale can be determined based on the environmental complexity.

[0083] In the case where the robot is a household robot (household mobile manipulator) operating in a private residence, the shortening rate Kscale can be determined based on the time attribute of the private residence. Specific examples of the time attribute of the private residence include parents and children going out (during the daytime on weekdays), children going out (during the daytime on holidays), and parents and children going out (at night), and the shortening rate Kscale increases in this order.

[0084] In the case where the robot is a trolley robot (wheeled mobile manipulator) operating in a factory or warehouse, the shortening rate Kscale can be determined based on the attributes of the type of factory or warehouse. Specific examples of the attributes of the type of factory or warehouse include unmanned factories, unmanned warehouses, manned factories, and manned warehouses, and the shortening rate Kscale increases in this order.

[0085] In the case where the robot is a store robot (store mobile manipulator) operating in a retail store, the shortening rate Kscale can be determined based on the time attribute in the retail store. Specific examples of the time attribute in the retail store include non-business hours, business hours (off-season), and business hours (peak season), and the shortening rate Kscale increases in this order.

[0086] In the case where the robot is an autonomous driving robot traveling on a road, the shortening rate Kscale may be determined based on the attributes of the type of road the robot is traveling on. Specific examples of the attributes of the road type include expressways, main roads, city roads, private roads, and parking lots, and the shortening rate Kscale increases in this order.

[0087] (Revise)

[0088] In the first embodiment, the operations of the two robots 20a, 20b are simulated by the two simulators 12a, 12b, respectively. That is, the simulators and the robots have a one-to-one correspondence, such as Figure 8 However, one simulator can simulate the operation of multiple robots. That is, the simulator and the robot can have a one-to-N correspondence. Fig. 9 In the example shown, the operations of two robots 20a, 20b arranged in the same working environment W1 are simulated by a single simulator 12a. Therefore, the computing resources required for the simulation can be saved.

[0089] (Formal representation)

[0090] In order to discuss the maximum value of the shortening rate Kscale, the concept of considering the operator's reaction rate limit is formally expressed. Since there is a limit to the human reaction rate, the shortening rate Kscale of the simulation speed has a limit that the operator cannot react.

[0091] As described above, the shortening rate Kscale can be expressed as follows using the actual time Treal that actually elapses in the actual environment and the simulation time Tsim that elapses in the environment model.

[0092] [Formula 1]

[0093] K scale =T sim / T real (1).

[0094] In addition, in order to prevent the operation of the robot from being hindered, the operator needs to implement appropriate operations on the corresponding robot after detecting or predicting the abnormality, and thus be able to recover from or avoid the abnormality. This can be expressed as follows, where Dthr represents the range within which a certain object (person / object) can significantly interfere with the robot.

[0095] [Formula 2]

[0096] |P follower (T)-P object (T)|≥D thr (2)

[0097] in

[0098]

[0099] Here, in the above expression, the positions of the robot and the object at time T1 when the abnormality is predicted are represented by PfollowerT1 and PobjectT1, respectively, and the positions of the robot and the object at the subsequent time t are represented by Pfollower(t) and Pobject(t), and the speeds are represented by Vfollower(Oopt(t), t) and Vobject(t). In addition, Oopt(t) represents the appropriate operation details when the operator takes measures for the abnormality.

[0100] In reality, proper operation cannot be started immediately after the object is detected. Instead, proper operation is started after a delay time Td, and there is a difference between the simulation time and the actual time according to expression (1). Therefore, making the shortening rate Kscale as high as possible within a range that does not interfere with the robot operation will lead to the following problems.

[0101] [Formula 3]

[0102] Maximize:K scale

[0103] S.t

[0104] |P"′ follower (T,K scale )-P object (T)|≥D thr

[0105] T≥T1

[0106] in

[0107]

[0108] (Specific example: self-driving robot)

[0109] In an autonomous driving robot whose speed is controlled by an operator, parameters such as the size of the safety zone (safety zone boundary), the operator's field of view, the avoidance maneuvers performed by the operator, and the operator's reaction rate are defined as follows: Fig.10 shown.

[0110] use Fig.10 With the parameters shown, the above expressions (3) and (4) can be specifically written as follows.

[0111] [Formula 4]

[0112]

[0113] Pv object (T) = P object_T1 +(T-T1)V object_max .

[0114] Fig.10 Each value of the parameters in can be set based on the attributes of the road type, such as highway, main road, or urban road. In addition, the object is a movable vehicle, and the collision with an oncoming vehicle (object) is assumed to be an anomaly. At this time, if an operation to change the traveling direction is not performed before the operator notices the anomaly, Omax represents a positive constant. In addition, if the oncoming vehicle is in front of the target operating robot (car), P'follower(T, Kscale) < Pobject(T), and the constraint expression can be modified as follows.

[0115] [Formula 5]

[0116] -(Pv″ follower (T, K scale ) + KT d O max -Pv object (T)) ≥ D thr

[0117] -Pv′ follower (T, K scale ) - KT d O max +Pv object (T) ≥ D thr

[0118] -Pv′ foilower (T, K scale ) - KT d O max +(P object_T1 +(T - T1)V object_max ) ≥ D thr

[0119] Pv′ follower (T, K scale ) ≤ -D thr -KT d O max +(P object_T1 +(T - T1)V object_max )

[0120] where

[0121]

[0122] Here, the indefinite integral of Oopt(t) is defined as Sopt(t) and decomposed into the form of a definite integral to obtain the following items.

[0123] [Formula 6]

[0124]

[0125] Here, it is assumed that an operation of going straight toward the object or not changing the traveling direction is performed until the operator performs the optimal operation. Therefore, Sopt(T1+KTd)=Omax(T1+KTd), and the following relationship holds.

[0126] [Formula 7]

[0127] P follower_T1 +S opt (T)-O max (T1+KT d )≤Pv′ follower (T, K)≤P follower_T1 +S opt (T)+O max (T1+KT d ).

[0128] When this relationship is applied to the above constraint expression, the following is obtained.

[0129] [Formula 8]

[0130] P follower_T1 +S opt (T)+O max (T1+KT d )≤-D thr -KT d O max +(P object_T1 +(T-T1)V object_max )

[0131] O max (T1+KT d )+KT d O max ≤-D thr +(P object_T1 +(T-T1)V object_max )-P follower_T1 -S opt (T)

[0132]

[0133] K≤G2(T)≤G(T) (5).

[0134] Specifically, when T1=0, the following results are obtained.

[0135] [Formula 9]

[0136]

[0137] Here, if Dthr=2.0m, Vobject_max=Omax=maximum operating speed, and Td=0.1 seconds, G2(T) is as follows.

[0138] [Formula 10]

[0139]

[0140] Since time T is later than T1, and T1 = 0, the numerical range is expressed as T ≥ T1 = 0. Therefore, according to expression (5), the maximum value that K can take is G2 (0).

[0141] [Formula 11]

[0142] K≤G2(0)≤G2(T)≤G(T)

[0143]

[0144] When G2(0) is calculated for expressways, main roads, and urban roads, the results are as follows: Fig.11 shown.

[0145] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the embodiments. These embodiments can be implemented in various other modes, and various omissions, substitutions, changes and combinations can be made without departing from the gist of the embodiments. These embodiments and modifications thereof are included in the scope and gist of the embodiments, and are included in the claims and their equivalents.

[0146] Note that the technology according to the present disclosure is not limited to a specific standard, and the exemplary arrangement can be appropriately changed. Note that each of the above-mentioned embodiments shows an example for implementing the present disclosure, and the present disclosure can be implemented in various other modes. For example, various modifications, substitutions, omissions, combinations, etc. can be made without departing from the gist of the present disclosure. Such modifications, substitutions, omissions, combinations, etc. are also included in the scope of the present disclosure and are included in the inventions disclosed in the claims and their equivalents.

[0147] In addition, the processing procedures described in the present disclosure may be considered as methods including a series of these processes. Alternatively, the process may be considered as a program for causing a computer to implement a series of these processes, or may be considered as a recording medium storing the program. In addition, the above-mentioned processing is performed by a processor such as a CPU of a computer. In addition, the type of recording medium does not affect the embodiments of the present disclosure and is therefore not limited to any specific type.

[0148] Note that each component shown in the present disclosure can be implemented by software, or can be implemented by hardware. For example, each component can be a software module implemented by software such as a microprogram, and each component can be implemented by a processor that executes the software module. Alternatively, each component can be implemented by a circuit block on a semiconductor chip (die), for example, or an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). In addition, the number of components and the number of hardware for implementing the components can be inconsistent with each other. For example, a processor or circuit can implement multiple components. On the contrary, a component can be implemented by multiple processors or circuits.

[0149] Note that the processors mentioned in the present disclosure are not limited to any specific type. For example, the processor may be a CPU, a microprocessing unit (MPU), a graphics processing unit (GPU), etc.

[0150] Note that the present disclosure can also provide configurations as described below.

[0151] [1](Control method for multiple robots)

[0152] A method for controlling a plurality of robots, the method comprising the following steps:

[0153] receiving operation instructions for the plurality of robots from an operator (S11);

[0154] Using a plurality of robot models corresponding to the plurality of robots and an environment model, based on the operation instruction, the plurality of robot models are caused to operate in the environment model at a speed faster than actual time, and a predicted observation value obtained as a result is acquired (S12);

[0155] Displaying the predicted observation value to the operator (S13);

[0156] accumulating task information including the operation instructions and the predicted observation values ​​(S14); and

[0157] The accumulated task information is transmitted to the plurality of robots (S16). [2]

[0159] The method according to [1] further comprises the following steps:

[0160] Receiving the task information (S21);

[0161] Accumulating the received task information (S22); and

[0162] According to the operation instructions included in the accumulated task information, the plurality of robots are caused to operate in the actual environment (S23). [3]

[0164] The method according to [2] further includes the following steps:

[0165] acquiring actual observation values ​​obtained as a result of causing the plurality of robots to operate in the actual environment (S24); and

[0166] Anomalies are detected based on a comparison between the predicted observation value and the actual observation value (S25). [4]

[0168] The method according to [3] further includes the step of: when the abnormality is detected, transferring control of the robot that detected the abnormality to real-time control to be implemented by the operator. [5]

[0170] A method according to any one of [1] to [4], wherein a relationship represented by shortening rate Kscale = Tsim / Treal holds between the actual time Treal felt by the operator and the simulated time Tsim passed in the environmental model, and the shortening rate Kscale is a real number of 1 or greater. [6]

[0172] The method according to [5], wherein the shortening rate is determined based on at least one of the frequency of occurrence, spatial density or difficulty of coping with interference elements existing in the actual environment. [7]

[0174] The method according to [5], wherein the robot comprises a home robot to be operated at a private residence, and the shortening rate is determined based on a time attribute at the private residence. [8]

[0176] The method according to [5], wherein the robot includes a cart-type robot to be operated in one of a factory or a warehouse, and the shortening rate is determined based on attributes of the type of the factory or the warehouse. [9]

[0178] The method according to [5], wherein the robot comprises a store robot to be operated in a store, and the shortening rate is determined based on a time attribute in the store.

[10]

[0180] The method according to

[10] , wherein the robot includes an autonomous driving robot to be driven on a road, and the shortening rate is determined based on attributes of the type of the road.

[0181]

[11] (Control device for multiple robots) A device for controlling multiple robots, the device comprising:

[0182] An input unit (11) receiving operation instructions for a plurality of robots (20a, 20b) from an operator (1);

[0183] a simulation unit (12), the simulation unit comprising a plurality of robot models (121a, 121b) corresponding to the plurality of robots and an environment model (122a, 122b), causing the plurality of robot models to operate in the environment model at a speed faster than actual time based on the operation instruction, and outputting predicted observation values ​​obtained as a result;

[0184] a display unit (13), wherein the display unit displays the predicted observation value to the operator;

[0185] an accumulation unit (14a, 14b) that accumulates task information including the operation instruction and the predicted observation value; and

[0186] A sending unit (15) that sends the task information accumulated in the accumulating unit to the plurality of robots.

[12]

[0188] The apparatus according to

[11] , wherein the simulation unit comprises a plurality of simulators (12a, 12b), and

[0189] Each simulator of the plurality of simulators simulates operation of at least one robot model of the plurality of robot models.

[0190]

[13] (Control system for multiple robots)

[0191] A system for controlling a plurality of robots, the system comprising:

[0192] The plurality of robots (20a, 20b) and a control device (10) for the plurality of robots, wherein:

[0193] The control device comprises:

[0194] An input unit (11) receiving operation instructions for the plurality of robots from an operator (1);

[0195] a simulation unit (12), the simulation unit comprising a plurality of robot models (121a, 121b) corresponding to the plurality of robots and an environment model (122a, 122b), simulating operations of the plurality of robot models in the environment model at a speed faster than actual time based on the operation instructions, and outputting predicted observation values ​​obtained as a result;

[0196] a display unit (13), wherein the display unit displays the predicted observation value to an operator;

[0197] a first accumulating unit (14a, 14b) that accumulates task information including the operation instruction and the predicted observation value, and

[0198] a sending unit (15) for sending the task information accumulated in the first accumulating unit to the plurality of robots;

[0199] Each robot of the plurality of robots comprises:

[0200] a receiving unit (21a, 21b) which receives the task information from the control device;

[0201] a second accumulating unit (22a, 22b) that accumulates the task information received by the receiving unit; and

[0202] A control unit (23a, 23b) causes the robot to operate in a real environment according to the operation instruction included in the task information accumulated in the second accumulation unit.

[0203] Reference Mark List

[0204] 1 Operator

[0205] 10 Control Device

[0206] 11 Input unit

[0207] 12 Analog Units

[0208] 12a Simulator

[0209] 12b Simulator

[0210] 12c Simulator

[0211] 12d simulator

[0212] 13 Display unit

[0213] 14a Accumulation Unit

[0214] 14b Accumulation Unit

[0215] 15 Sending unit

[0216] 16 Receiving Unit

[0217] 20a Robot

[0218] 20b Robot

[0219] 20c Robot

[0220] 20d Robot

[0221] 21a Receiving unit

[0222] 21b Receiving Unit

[0223] 22a Accumulation Unit

[0224] 22b Accumulation Unit

[0225] 23a Control unit

[0226] 23b Control unit

[0227] 24a Drive motor

[0228] 24b Drive motor

[0229] 25a Internal sensor

[0230] 25b Internal sensor

[0231] 26a External sensor

[0232] 26b External sensor

[0233] 27a Anomaly detection unit

[0234] 27b Anomaly Detection Unit

[0235] 100 Control Systems

[0236] 121a Robot Model

[0237] 121b Robot Model

[0238] 122a Environmental Model

[0239] 122b Environmental Model

[0240] B Operation table

[0241] W1 Working Environment

[0242] W2 Working Environment

[0243] W3 Operating Environment

Claims

1. A method for controlling a plurality of robots, the method comprising the following steps: receiving operation instructions for the plurality of robots from an operator; Using a plurality of robot models corresponding to the plurality of robots and an environment model, causing the plurality of robot models to operate in the environment model at a speed faster than actual time based on the operation instruction, and acquiring predicted observation values ​​obtained as a result; displaying the predicted observations to the operator; Accumulating task information including the operation instructions and the predicted observation values; as well as The accumulated task information is transmitted to the plurality of robots.

2. The method according to claim 1, further comprising the steps of: receiving the task information; Accumulating the received task information; and The plurality of robots are caused to operate in an actual environment according to the operation instructions included in the accumulated task information.

3. The method according to claim 2, further comprising the steps of: acquiring actual observation values ​​obtained as a result of causing the plurality of robots to operate in the actual environment; and Anomalies are detected based on a comparison between the predicted observations and the actual observations.

4. The method according to claim 3, further comprising the step of: when the abnormality is detected, transferring the control of the robot that detected the abnormality to real-time control to be implemented by the operator.

5. The method according to claim 1, wherein: Between the actual time Treal felt by the operator and the simulated time Tsim elapsed in the environment model, a relationship represented by a shortening rate Kscale=Tsim / Treal holds, and the shortening rate Kscale is a real number of 1 or more.

6. The method according to claim 5, wherein: The shortening rate is determined based on at least one of the occurrence frequency, spatial density, or difficulty of dealing with interference elements existing in the actual environment.

7. The method according to claim 5, wherein: The robot comprises a home robot to be operated at a private residence, and the shortening rate is determined based on an attribute of time at the private residence.

8. The method according to claim 5, wherein: The robot includes a cart-type robot to be operated in one of a factory or a warehouse, and the shortening rate is determined based on a property of a type of the factory or the warehouse.

9. The method according to claim 5, wherein: The robot comprises a store robot to be operated in a store, and the shortening rate is determined based on a time attribute in the store.

10. The method according to claim 5, wherein: The robot includes an autonomous driving robot to travel on a road, and the shortening rate is determined based on a property of a type of the road.

11. A device for controlling a plurality of robots, the device comprising: an input unit that receives operation instructions for the plurality of robots from an operator; a simulation unit including a plurality of robot models corresponding to the plurality of robots and an environment model, causing the plurality of robot models to operate in the environment model at a speed faster than actual time based on the operation instruction, and outputting predicted observation values ​​obtained as a result; a display unit, wherein the display unit displays the predicted observation value to the operator; an accumulation unit, the accumulation unit accumulating task information including the operation instruction and the predicted observation value; as well as A transmitting unit transmits the task information accumulated in the accumulating unit to the plurality of robots.

12. The device according to claim 11, wherein The simulation unit includes a plurality of simulators, and Each simulator of the plurality of simulators simulates operation of at least one robot model of the plurality of robot models.

13. A system for controlling a plurality of robots, the system comprising: The plurality of robots and the control device for the plurality of robots, wherein: The control device comprises: an input unit that receives operation instructions for the plurality of robots from an operator; a simulation unit including a plurality of robot models corresponding to the plurality of robots and an environment model, simulating operations of the plurality of robot models in the environment model at a speed faster than actual time based on the operation instructions, and outputting predicted observation values ​​obtained as a result; a display unit, wherein the display unit displays the predicted observation value to the operator; a first accumulating unit, which accumulates task information including the operation instruction and the predicted observation value; and a sending unit configured to send the task information accumulated in the first accumulating unit to the plurality of robots; Each robot of the plurality of robots comprises: a receiving unit, the receiving unit receiving the task information from the control device; a second accumulating unit that accumulates the task information received by the receiving unit; and A control unit that causes the robot to operate in an actual environment according to the operation instruction included in the task information accumulated in the second accumulation unit.

Images