Robot control device, robot control system, robot, and robot control method

Abstract

To suppress work efficiency from deteriorating as a whole, even when a robot s into a situation where it is hard to continue work, in a scene in which the robot and a mate work cooperatively.SOLUTION: A robot control device 11 comprises: a determining part 33 that determines a progress situation of a first work that is performed following a work instruction; a searching part 35 that when the progress situation of the first work shows that the work cannot be continued, searches a mate 17 that can provide specific information for enabling the progress situation of the first work to show that the work can be continued; an estimating part 37 that estimates a total cost of work including a second work for changing the progress situation of the first work to a progress situation where the work can be continued; a plan creating part 38 that creates a work plan including the second work for the mate 17, on the basis of the estimated total costs of work; and a control part 39 that makes the mate perform the work including the second work following the created work plan. The plan creating part 38 creates the work plan so that the total costs of work becomes minimum.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a robot control device for controlling the operation of a robot, a robot control system, a robot, and a robot control method. [Background technology]

[0002] In recent years, there has been an increasing demand for the automation of tasks using robots and for collaborative work between robots and humans in the fields of manufacturing, logistics, services, construction, and facility maintenance. For this type of robot, for example, if it encounters a situation where it is difficult to continue working, it is required that it quickly return to work. Patent Document 1 describes a technology for enabling a robot to quickly return to work. This publication describes a "robot control system comprising an instructor storage unit that stores information on multiple instructors, a selection unit that selects one of the multiple instructors stored in the instructor storage unit when support information is needed to allow the robot to continue work, an environment information transmission unit that transmits environmental information about the robot to the instructor selected by the selection unit via a network line, and a support information reception unit that receives the support information from the instructor selected by the selection unit via the network line." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 149414 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in a scenario where a robot works collaboratively with a partner (a person, another robot, etc.), if the robot encounters a situation where it is difficult to continue working, it is possible that the partner working collaboratively may be concentrating on a different task. In such a case, if the robot requests support from the partner who is concentrating on the task, such as through dialogue, the robot will be interrupted. As a result, there is a concern that work efficiency may be impaired. In other words, the robot control system of Patent Document 1 does not mention or suggest how to deal with situations in which a robot and a partner are working together and a robot finds itself in a situation where it is difficult to continue working, and does not consider the circumstances of the partner. As a result, there is a risk that the overall work efficiency will decrease.

[0005] The present invention has been made in consideration of the above-mentioned situation, and aims to suppress a decline in overall work efficiency in a scene where a robot and a partner are working together, even if the robot finds itself in a situation where it is difficult to continue working. [Means for solving the problem]

[0006] In order to solve the above problems, the robot control device according to the present invention comprises: A robot control device that causes a robot to perform a task in accordance with a predetermined task instruction, an acquisition unit that acquires the work instruction; a determination unit that determines a progress status of a first work in accordance with the work instruction; a search unit that searches for a partner that can provide specific information for enabling the progress of the first work to be continued when the determination result of the determination unit indicates that the progress of the first work cannot be continued; an estimation unit that, when the partner is extracted as a result of the search by the search unit, estimates a total task cost including a second task for changing the progress status of the first task from "cannot continue" to "can continue"; a plan creation unit that creates a work plan including the second work for the partner based on the total work cost estimated by the estimation unit; a control unit that causes work including the second work to be performed in accordance with the work plan created by the plan creation unit, The most important feature of the plan creation unit is that it creates the work plan so as to minimize the total work cost. [Effects of the Invention]

[0007] According to the present invention, in a scene where a robot and a partner are working together, even if the robot finds itself in a situation where it is difficult to continue working, a decrease in overall work efficiency can be suppressed. Problems, configurations, and effects other than those described above will be described in detail in the following embodiments. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic configuration diagram of a robot control system including a robot control device according to an embodiment of the present invention. [Figure 2] 1 is a functional block diagram of a robot control device according to an embodiment of the present invention. [Figure 3] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a robot control device according to an embodiment of the present invention. [Figure 4] 3 is a flowchart showing an operation procedure of a robot control device according to an embodiment of the present invention. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a display screen of specific information useful for making the progress of a first task in accordance with a task instruction continuable; DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, a robot control device, a robot control system, a robot, and a robot control method according to embodiments of the present invention will be described with reference to the appropriate drawings. In the description of the robot control device according to the embodiment of the present invention, components having common functions are given common reference numerals, and redundant description thereof will be omitted.

[0010] [General Configuration of Robot Control System 10] First, a schematic configuration of a robot control system 10 including a robot control device 11 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a schematic diagram of a robot control system 10 including a robot control device 11 according to an embodiment of the present invention.

[0011] As shown in FIG. 1, the robot control system 10 includes a robot 1, a robot control device 11, an input device 13, and an output device 15.

[0012] The robot 1 is configured to include a moving cart unit 3, an articulated arm unit 5, and a hand unit .

[0013] The movable carriage unit 3, the articulated arm unit 5, and the hand unit 7 are each connected to a robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other.

[0014] The articulated arm unit 5 is provided with a sensor 9 that monitors and measures the surroundings of the working environment of the robot 1. The sensor 9 is connected to a robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other.

[0015] The sensor 9 includes, for example, a camera, a distance sensor, and a microphone. Examples of distance sensors include LiDAR, a laser displacement meter, a stereo camera, an infrared distance meter, a laser distance meter, and a TOF distance sensor. The sensor 9 may be a wearable device that measures the physical data of a person (supervisor or collaborator), or may be a joint angle sensor, an acceleration sensor, or a tactile sensor built into the robot 1. The sensor 9 may be installed in a location where it can capture an image of the entire working environment around the robot 1.

[0016] The robot control device 11 according to the embodiment generates control commands by referring to the work instruction content and the monitoring results of the sensor 9. Each of the mobile cart unit 3, the articulated arm unit 5, and the hand unit 7 provided in the robot 1 operates according to the control commands of the robot control device 11 according to the embodiment.

[0017] The input device 13 is used when a user (for example, a supervisor of the robot 1) gives predetermined work instructions to the robot 1. As the input device 13, for example, a human interface device such as a keyboard, mouse, microphone, or liquid crystal display, a smartphone, a tablet, or a robot controller may be used as appropriate. The input device 13 is connected to the robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other.

[0018] The output device 15 is used when the robot 1 outputs specific information to the partner 17 to enable continuation of the progress of the work in accordance with the work instructions. As the output device 15, for example, a human interface device such as a speaker or a liquid crystal display, a smartphone, a tablet, etc. may be used as appropriate. The output device 15 is connected to the robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other. The output device 15 may be the same device as the input device 13, or may be a different device.

[0019] The partner 17 refers to a general entity that provides the robot 1 with specific information, including guidance and advice, to enable the robot 1 to continue the progress of the work (first work) in accordance with the work instructions when the robot 1 falls into a situation where it is difficult to continue the work.

[0020] Specifically, the partner 17 is not limited to a supervisor who gives predetermined work instructions to the robot 1, but may also be a coworker different from the supervisor. Furthermore, the partner 17 is not limited to a person, but may also be, for example, another robot or a search device for searching for the specific information. The "specific information" will be described in more detail later.

[0021] The work performed in accordance with the work instructions is not particularly limited, but may be, for example, the work of removing instructed products from product inventory storage shelves 20 and replenishing them on product display shelves (not shown) in a convenience store. In the example shown in Figure 1, bottles 22a, boxes 22b, and cans 22c are listed as products to be placed on product inventory storage shelves 20.

[0022] A task performed in accordance with a work instruction may be, for example, a single task such as "removing bottle 22a from product inventory storage shelf 20," or it may be a sequence of tasks such as "moving to back yard," "detecting bottle 22a," "grasping bottle 22a," "moving to product display shelf," and "placing bottle 22a" based on a work instruction such as "replenishing bottle 22a on product display shelf." The work performed in accordance with the work instructions corresponds to the "first work" of the present invention. In the following description, the work performed in accordance with the work instructions may be referred to as the "first work."

[0023] In the embodiment of the present invention, an example of a work instruction to "refill product display shelves with bottles 22a" will be described. In this case, the first work performed by the robot 1 in accordance with the work instruction is the series of works performed to refill product display shelves with bottles 22a.

[0024] As shown in Figure 1, upon receiving the work instruction, the robot 1 moves to the back yard by operating the mobile cart unit 3, detects the bottle 22a on the product inventory storage shelf 20 indicated by the sensor (camera) 9, grasps the detected bottle 22a by operating the multi-joint arm unit 5 and hand unit 7, moves to the specified product display shelf by operating the mobile cart unit 3 while grasping the bottle 22a, and places the bottle 22a in the specified location by operating the multi-joint arm unit 5 and hand unit 7, performing a series of tasks.

[0025] [Software configuration of the robot control device 11 according to the embodiment] Next, the software configuration of the robot control device 11 according to the embodiment of the present invention will be described with reference to FIG. FIG. 2 is a functional block diagram of the robot control device 11 according to the embodiment.

[0026] As shown in FIG. 2, the robot control device 11 according to the embodiment is configured to include a communication unit 18, an acquisition unit 31, a judgment unit 33, a search unit 35, a memory unit 23b, an estimation unit 37, a plan creation unit 38, and a control unit 39.

[0027] The communication unit 18 acquires the specific information for enabling the continuation of the progress of the first work between the mobile terminal 19 carried by the partner 17 and the mobile terminal 19 via a wireless communication medium.

[0028] The acquisition unit 31 acquires detection information from the sensor 9 and information related to specified work instructions from the input device 13, and also acquires the specific information sent from the mobile terminal 19 held by the partner 17 via the communication unit 16.

[0029] In detail, the acquisition unit 31 may acquire, for example, one task selected from a predetermined task group as a work instruction, or may acquire a sequence of tasks consisting of multiple tasks as a work instruction.

[0030] The acquisition unit 31 may also create a sequence of tasks consisting of multiple tasks as a work instruction based on information related to a predetermined work instruction from the input device 13. For example, the acquisition unit 31 may extract a sequence of tasks to be performed by performing natural language processing on a linguistic instruction such as "refill bottle 22a." Examples of natural language processing include topic models and large-scale language models such as ChatGPT.

[0031] Specifically, the acquisition unit 31 may create a series of tasks, such as "move to back yard," "detect bottle 22a," "grasp bottle 22a," "move to bottle display shelf," and "place bottle 22a," based on a task instruction such as "refill bottle 22a."

[0032] As an example of creating a task sequence, a task target and a task sequence may be linked and stored in advance in the storage device 23, and when a task target is input, a task sequence corresponding to this task target may be read from the storage device 23 and created, or a series of task sequences for achieving the task target may be planned. Examples of methods for planning task sequences toward a task target include dynamic programming, A*, RRT*, and active inference.

[0033] The acquisition unit 31 may also directly generate a sequence of tasks to be performed using a large-scale language model based on a vague human task instruction such as "I'm thirsty." Examples of such methods include ChatGPT for Robotics and RT-2.

[0034] The determination unit 33 determines the progress of the first task in accordance with the task instruction acquired by the acquisition unit 31, that is, whether or not the first task can be continued.

[0035] The determination unit 33 may determine whether or not to continue the work by setting a work error criterion in advance, such as "the object cannot be detected," and determining whether or not the detection state of the object satisfies the work error criterion. Alternatively, the determination unit 33 may set work start and end criteria in advance, and determine whether or not to continue the first work by determining whether or not the current state satisfies the work start and end criteria.

[0036] In addition, the judgment unit 33 may acquire in advance the detection data of the sensor 9 when it is possible to continue work, and by comparing the past detection data with the detection data of the sensor 9 acquired in the current work state, determine whether the work has fallen into an abnormal state and thereby judge whether the work can be continued.

[0037] Examples of such anomaly detection methods include anomaly detection methods using neural networks such as AutoEncoder and VAE. For example, if the sensor 9 is a camera, a neural network is trained on images such as RGB images, gray images, and infrared images captured by the camera during a previous task. Then, when the task is performed, the currently captured image is input to the trained neural network to determine whether the current task state is abnormal.

[0038] The data input to the neural network does not have to be images. For example, if the sensor 9 is a distance sensor, the neural network learns depth images, point cloud data, etc. measured by the distance sensor; if the sensor 9 is a microphone, the neural network learns audio data acquired by the microphone; and if the sensor 9 is a joint angle sensor, acceleration sensor, or tactile sensor built into the robot 1, the neural network learns joint angle data, acceleration data, and tactile data measured by each sensor 9. This allows the neural network to determine whether the current working state is abnormal.

[0039] The data to be learned does not necessarily have to be the information acquired by the sensor 9 itself, but may be information obtained by processing the information. For example, it may be differential information between measurement information such as the velocity and acceleration of joint angles, or optical flow calculated from time-series images.

[0040] Furthermore, the data to be learned may be, for example, an image that has been homography transformed to match the viewpoint as seen from a predetermined coordinate system, or information that has been normalized based on the average and standard deviation of previously obtained measurement information.

[0041] If the determination unit 33 determines that the progress of the first work in accordance with the work instructions cannot be continued, the search unit 35 searches for a partner 17 that can provide specific information to allow the progress of the first work to be continued.

[0042] More specifically, the search unit 35 detects a person who will become the partner 17 using a predetermined person detection method based on, for example, images and point cloud data acquired by the sensor 9. Examples of the person detection method include object detection methods using neural networks such as R-CNN, YOLO, SSD, and DETR, and person pose estimation methods such as OpenPose. Also, for example, a facial image of the person who gave the work instruction may be acquired using the input device 13, and a search may be performed using face recognition technology to identify the same person as the person who gave the work instruction as the partner 17.

[0043] The search unit 35 may also search for the partner 17 while moving using the mobile cart unit 3 provided in the robot 1. The search unit 35 may use the mobile cart unit 3 to move to a location where the partner 17 is likely to be, based on map information created by SLAM or the like and estimated self-position information, and perform the search.

[0044] In addition, when the sensor 9 includes multiple microphones, the search unit 35 may use a sound source localization method to estimate the person's position based on the person's speaking voice, and search for the partner 17 by moving to the estimated person's position using the mobile cart unit 3.

[0045] Furthermore, the search unit 35 may identify the position of a person from a beacon sensor or the like carried by a person in the vicinity, and search for the other person 17 by moving to the identified position of the person using the mobile cart unit 3.

[0046] Furthermore, when the work cost information includes schedule information of the partner 17, the search unit 35 may read the schedule information of the partner 17 from the memory contents (work cost information) of the memory unit 23b, estimate the location of the partner 17 by comparing the current time with the schedule information, and search for the partner 17 by moving to the estimated location using the mobile cart unit 3.

[0047] The partner 17 to be searched for by the search unit 35 is not limited to the person (supervisor) who gave the work instruction using the input device 13, but may also be another person (coworker). For example, if multiple people are detected by the person detection process, the person closest to the robot 1 may be regarded as the partner 17.

[0048] The memory unit 23b stores a task cost table in which task cost information related to each of a plurality of tasks performed by the partner 17 is associated with the task cost information. The task cost information related to the task means information related to added value associated with each of a plurality of different tasks. The task here is an expanded concept that includes all actions of the partner 17 (supervisor / co-worker) and the robot 1.

[0049] For example, if we consider a case where robot 1 is performing a first task and the progress of the first task becomes unsustainable, the value obtained by multiplying the added value associated with the first task (for example, based on the difficulty of the task, etc.) by the waiting time corresponds to the work cost associated with robot 1 waiting to perform the task.

[0050] Also, for example, if we consider a case where partner 17 is performing some work and has to interrupt his work to respond to a question and answer session by robot 1 (a request for specific information, or the acquisition of specific information in response to the request), the added value corresponding to partner 17's work (for example, according to the difficulty of the work, etc.) multiplied by the interruption time is equivalent to the work cost related to partner 17's interruption of work.

[0051] The variables used to calculate the work cost are not limited to the added value associated with each of the multiple work contents, but may be attribute values ​​associated with each of the work subjects (partner 17, robot 1, etc.). For example, a configuration may be adopted in which the attribute value associated with the partner 17 is the numerical value

[10] , and the attribute value associated with the robot 1 is the numerical value [5].

[0052] In addition, in cases where the main subject of the work is a partner 17, for example, a configuration may be adopted in which the numerical value

[20] is associated as the attribute value for the partner 17 (supervisor) and the numerical value

[10] is associated as the attribute value for the partner 17 (co-worker) depending on the partner 17's position, position, level of proficiency, etc.

[0053] When the progress status of the first work in accordance with the work instructions is "cannot be continued" and the search unit 35 extracts the partner 17 as a result of the search, the estimation unit 37 estimates the total work cost including the second work for changing the progress status of the first work from "cannot be continued" to "can be continued" based on the work cost table stored in the memory unit 23b.

[0054] The estimation unit 37 estimates the total work cost including the second work, for example, based on the work cost related to the standby time of the robot 1 and the work cost related to the workload of the partner 17 (for example, work interruption).

[0055] More specifically, the estimation unit 37 estimates the work content of the partner 17 searched for by the search unit 35, and estimates the work cost based on the work content of the partner 17. The work content of the partner 17 may be estimated using, for example, machine learning or a human activity recognition technique in computer vision.

[0056] The estimation result of the work content of the partner 17 by the estimation unit 37 may be a specific work content, and may further include the probability that the work content is a specific work content.

[0057] Furthermore, the estimation result of the work content of the partner 17 by the estimation unit 37 is not limited to a specific work content, and the progress status of the specific work may also be estimated. In this case, the estimation unit 37 may estimate the progress status of the specific work by using a method that incorporates a probabilistic transition model or person tracking into human behavior recognition.

[0058] In addition, if the work cost information includes schedule information of the partner 17, the estimation unit 37 may receive the schedule information of the partner 17 from the memory unit 23b and estimate the work content of the partner 17 by comparing the current time with the schedule information.

[0059] The estimation unit 37 compares the estimated work content of the partner 17 with the work cost information, thereby estimating the work cost corresponding to the work content.

[0060] Furthermore, if there are multiple candidates for the partner 17 and the task cost information differs for each of the multiple partners 17, the estimation unit 37 may estimate the task cost of the searched partner 17 by recognizing the appropriate partner 17 using the face recognition technology described above. Furthermore, if the task cost information depends on the location and time, the task cost according to the current location and time may be estimated by referring to the location and current time when the current self-location is calculated using SLAM.

[0061] Specifically, for example, when the robot 1 is performing a first task and the progress of the first task becomes unsustainable, a first case is considered in which specific information is acquired to change the progress of the first task from unsustainable to continuable by having the robot 1 immediately conduct a question and answer session (second task) with the partner 17 who is performing a certain task.

[0062] The total work cost in this first case is calculated based on the work cost associated with the waiting time of the robot 1 (the time until the first work can be continued) (for example, the value obtained by multiplying the added value associated with the first work by the waiting time) and the work cost associated with the interruption of the work of the partner 17 (for example, the value obtained by multiplying the added value associated with the partner 17 by the interruption time).

[0063] Next, for example, consider a second case in which, while the robot 1 is performing a first task, the progress status of the first task changes to "cannot continue" and, by referring to the schedule information of the partner 17 extracted by the search by the search unit 11c, it is known that the partner 17, who is performing a certain task, will complete the task if it waits for five minutes. After waiting for five minutes, the robot 1 then has the partner 17 conduct a question and answer session (second task) (waiting for the partner 17 to complete the task) to obtain specific information to change the progress status of the first task from "cannot continue" to "can continue."

[0064] The total work cost in this second case is calculated based on the work cost associated with the standby of the robot 1 (the time until the first work can be continued). In this case, the work of the partner 17 is not interrupted, so the work cost associated with the interruption of the work of the partner 17 is not taken into account (added).

[0065] The total work cost in the first case and the total work cost in the second case are referenced in the next-stage plan creation unit 38 when creating a work plan so that the total work cost for the second work required to change the progress status of the first work from ``cannot continue'' to ``can continue'' is minimized.

[0066] The plan creation unit 38 creates a work plan for the first work in accordance with the work instructions acquired by the acquisition unit 31. When the progress status of the first work is not continuable, the plan creation unit 38 creates a work plan so as to minimize the total work cost for the second work required to change the progress status of the first work from not continuable to continuable.

[0067] The work plan created by the plan creation unit 38 includes an operation of requesting the partner 17 to provide specific information (including helpful information) necessary to change the progress status of the first work from "cannot continue" to "can continue," and an operation of acquiring the specific information provided by the partner 17 based on the request. These operations correspond to the "second work" of the present invention.

[0068] The plan creation unit 38 may create a work plan that includes a series of tasks, for example, waiting for a partner 17 who is currently performing a certain task to complete the task, then requesting the partner 17 to provide specific information, and then acquiring the specific information provided by the partner 17.

[0069] In addition, the plan creation unit 38 may create a work plan that includes a series of tasks, for example, to request the partner 17 to provide specific information without waiting for the partner 17 to complete the work (by interrupting the work of the partner 17), and then to acquire the specific information provided by the partner 17.

[0070] Furthermore, after acquiring the specific information provided by the partner 17, the plan creation unit 38 may create a work plan to resume the first work in accordance with the original work instruction.

[0071] Furthermore, after the execution of work based on the specific information provided by a certain partner 17, if the result of the re-determination of the progress of the first work by the determination unit 33 is that the progress of the first work cannot be continued, the plan creation unit 38 may be configured to recreate a work plan that includes an operation of having the search unit 35 re-search for another partner that is different from the partner and can provide the specific information.

[0072] If another partner is extracted by the re-search, the estimation unit 37 may estimate the work cost of the other partner in the form of an expected value of the work cost, for example, based on the prior distribution of the work content and the work costs for each of multiple tasks.

[0073] Examples of methods for planning a series of work sequences include dynamic programming and tree search. Reinforcement learning methods such as Q-learning, DQN, and A3C may also be used to plan a series of work sequences.

[0074] Furthermore, the plan creation unit 38 may create a work plan that includes an operation of selecting a partner 17 based on the probability that the progress status of the first work will be able to continue, which is obtained through dialogue with each of the multiple partners 17. In this case, for example, by linking the error content, which is a factor in whether or not the work can be continued, with the specialized knowledge required to solve it, the probability that the work will be able to continue through dialogue with each partner 17 can be set depending on whether or not each of the multiple partners 17 has the specialized knowledge in question.

[0075] The plan creation unit 38 may also create a work plan based on values ​​calculated with reference to the probability distribution of work costs. Examples of values ​​calculated with reference to the probability distribution of work costs include expected values ​​for multiple work costs and risk-aware work costs calculated by adding a constant multiple of the standard deviation to the average work cost. Furthermore, the probability distribution of work costs may be directly used when planning work by utilizing techniques such as Bayesian inference. Examples of work planning techniques that utilize Bayesian inference include active inference and control as inference.

[0076] The control unit 39 causes the robot 1 to perform the first task by transmitting a control command to the robot 1 to perform the first task based on the task plan created by the plan creation unit 38. The control unit 39 also causes the robot 1 to perform, via the input device 13, a task of requesting the partner 17 to provide specific information for changing the progress status of the first task from "cannot continue" to "can continue," as a second task in accordance with the task plan created by the plan creation unit 38. In addition, the control unit 39 may be configured to cause the robot 1 to perform operations related to the second task by sending a control command to the robot 1 to execute the second task based on the task plan created by the plan creation unit 38.

[0077] Here, the specific information is a concept that includes all information that is useful for making the progress of the first task continuable. The specific information includes, for example, the contents of the task instructions, a description of the task that was performed, the contents of the error that prevented the task from continuing, the cause of the error, questions requesting solutions to those errors, answers to those questions, and instructions for additional tasks to continue the task.

[0078] For example, if the judgment unit 33 cannot detect the work object, the equipment used, or the partner 17 and judges that the work cannot be continued, examples of information that can be useful in allowing the work to be continued include detailed information such as the color and shape of the work object and the equipment used, the location where they are placed, the inventory status, and the appearance and location of people (supervisors and co-workers).

[0079] Furthermore, for example, when it is determined that continuation of the movement operation is not possible due to an obstacle, examples of the specific information include a route to avoid the obstacle and whether or not the obstacle can be removed.

[0080] Furthermore, for example, when it is determined that work cannot be continued due to a malfunction, examples of specific information include questions and answers regarding the location of the malfunction and the cause of the malfunction.

[0081] Also, for example, if it is determined that the work cannot be continued because there is no available equipment required for the work, examples of specific information include questions and answers regarding the available time of the equipment and the availability of other equipment.

[0082] Furthermore, the specific information may include additional information to explain the current progress of the first task. For example, it may include a description of the task currently being performed, or it may include data acquired via the sensor 9 (including the concept of measurement).

[0083] The specific information may also be a control input for the robot 1 required for returning to work. In this case, examples of the specific information include position information, joint angle information, and joint velocity information for directly controlling the robot 1.

[0084] The standby time of the robot 1 related to the work cost may be, for example, the standby time until the partner work is completed and the robot is ready to interact, or the standby time until the robot is ready to continue working. Furthermore, the standby time of the robot 1 related to the work cost may include the time required for interaction, travel time, etc. in addition to the standby time until the robot is ready to interact.

[0085] In addition, if the work plan created by the plan creation unit 38 includes the task of re-searching for the partner 17, the total time required for searching for the partner 17, estimating the work cost, planning the work, and interacting, which are repeatedly performed until the work can be continued, may also be used as the total time required.

[0086] The work cost may also include the work load of the robot 1. The work load of the robot 1 may be, for example, the amount of energy consumed for the work.

[0087] The workload of partner 17 related to work costs is the physical and psychological burden on partner 17, such as the time it takes to have a conversation, the time it takes for partner 17 to return to their own work after the conversation, the physical and psychological burden caused by multitasking when partner 17 has to have a conversation and do another task at the same time, and the psychological burden caused by work interruptions.

[0088] The workload of the partner 17 related to the work cost may be quantified using the subjective assessment of the partner 17 or using physical data acquired by vital signs sensing. Examples of physical data include heart rate, blood pressure, skin temperature, respiratory rate, electromyography, pupillary response, oxygen intake, electrodermal activity, and electroencephalography (EEG). Physical data may be estimated from images rather than directly measured. For example, work cost may be quantified using physical data estimated using facial expression analysis, pupil size analysis, and posture analysis. Furthermore, work cost may be estimated using a workload model instead of the physical data values ​​themselves. Examples of workload models include NASA-TLX, the Borg scale, heart rate variability analysis (HRV), EEG pattern detection, and pupillary light response.

[0089] Furthermore, the operation cost may be a combination of the standby time of the robot 1 and the workload of the partner 17, or may be a combination of the workloads of multiple types of partner 17.

[0090] In the task cost information, the task cost linked to the task content of the partner 17 may be a predetermined value quantified as described above, or may be a task cost range and probability distribution obtained from task cost data acquired multiple times. Examples of methods for calculating the probability distribution from the acquired task costs include maximum likelihood estimation and posterior distribution estimation using a Gaussian distribution or a Gaussian mixture distribution, or probability distribution estimation in a feature space using a deep learning method such as VAE.

[0091] The task cost information may also include task execution probability, which is the probability that each task will be performed by the partner 17. By using the task execution probability, it is possible to calculate the expected value of the task cost, etc.

[0092] Furthermore, the task cost information may be different for each of the multiple partners 17. For example, by defining the task cost for each task based on the job title and required specialized knowledge, the task cost can be estimated according to the job title and whether or not the partner 17 has specialized knowledge.

[0093] The task cost information may also include location and time. For example, by combining it with schedule information of the partners 17, the task execution probability at each time for each of the multiple partners 17 may be calculated.

[0094] [Hardware configuration of the robot control device 11 according to the embodiment] Next, the hardware configuration of the robot control device 11 according to the embodiment of the present invention will be described with reference to FIG. FIG. 3 is a block diagram illustrating an example of a hardware configuration of the robot control device 11 according to the embodiment.

[0095] As shown in FIG. 3, the robot control device 11 according to the embodiment is a computer including a processor 21 such as a CPU, a storage device 23 such as a semiconductor memory, an input unit 25 that receives information from the input device 13, an output unit 27 that transmits information to the output device 15, a communication unit 18 that performs two-way communication with the robot 1 and the sensor 9, and a bus 29 that connects these units. The processor 21 executes a predetermined robot control processing program 23a to realize the functions of the above-mentioned functional units.

[0096] [Operation of the robot control device 11 according to the embodiment] Next, the operation of the robot control device 11 according to the embodiment of the present invention will be described with reference to FIG. FIG. 4 is a flowchart illustrating the operation of the robot control device 11 according to the embodiment.

[0097] In step S11 shown in FIG. 4, the acquisition unit 31 acquires the information detected by the sensor 9 and the information related to a predetermined work instruction from the input device 13.

[0098] In step S12, the plan creation unit 38 creates a work plan for the first work in accordance with the work instruction based on the detection information from the sensor 9 acquired by the acquisition unit 31 and information related to the predetermined work instruction from the input device 13.

[0099] In step S13, the control unit 39 causes the robot 1 to perform the first task based on the task plan created by the plan creation unit 38 in step S12.

[0100] In step S14, the determination unit 33 determines whether the first work based on the work plan has been completed.

[0101] If the determination result in step S14 indicates that the first task has been completed, the determination unit 33 ends the series of processing flows. On the other hand, if the determination result in step S14 indicates that the first task has not been completed, the robot control device 11 according to the embodiment advances the processing flow to the next step S15.

[0102] In step S15, the determination unit 33 determines the progress of the first task based on the task plan (whether the first task can be continued).

[0103] If the determination in step S15 indicates that the first task has been completed, the determination unit 33 ends the series of processing flows.

[0104] On the other hand, if the result of the determination in step S14 is that the first task is not completed, in step S15, the determination unit 33 determines the progress of the first task (whether the first task can be continued).

[0105] If the result of the determination in step S15 is that the progress of the first task can be continued, the robot control device 11 according to the embodiment returns the processing flow to step S13 and causes the subsequent processing to be performed sequentially.

[0106] On the other hand, if the result of the determination in step S15 is that the progress of the first task is not to be continued, the robot control device 11 according to the embodiment advances the flow of the process to the next step S16.

[0107] In step S16, the search unit 35 searches for a partner 17 that can provide specific information for enabling the progress of the first task to be continued.

[0108] If the partner 17 is extracted as a result of the search in step S16, in step S17, the estimation unit 37 estimates the total work cost including the second work to change the progress status of the first work from ``cannot continue'' to ``can continue'' based on the work cost table stored in the memory unit 23b.

[0109] In step S18, the plan creation unit 38 creates a work plan so as to minimize the total work cost estimated in step S17 (the total work cost related to the second work to change the progress status of the first work from ``cannot continue'' to ``can continue''). Next, the robot control device 11 according to the embodiment returns the process flow to step S13 and causes the subsequent processes to be performed in sequence.

[0110] That is, in step S13, the control unit 39 causes the robot 1 to perform the second task based on the task plan created by the plan creation unit 38 in step S18. Here, the second task includes the task of requesting the partner 17 extracted by the search in step S16 to provide specific information in order to change the progress status of the first task from "cannot continue" to "can continue," and the task of receiving specific information from the partner 17. Thereafter, the robot control device 11 according to the embodiment causes the subsequent processes to be performed in sequence.

[0111] FIG. 5 is an explanatory diagram exemplarily showing a display screen of specific information useful for making the progress of the first work in accordance with the work instruction continuable.

[0112] In the example shown in Fig. 5, a question asking about the work object, "Which bottle is the one to be replenished?", is displayed as specific information using a window corresponding to the display unit 41. In addition, in the example shown in Fig. 5, the content of the work instruction and the data acquired via the sensor 9 are also displayed as specific information.

[0113] In addition, in the task of acquiring specific information from the partner 17, the acquisition unit 31 acquires the specific information via the input device 13 or via the mobile terminal 19 and communication unit 18 carried by the partner 17.

[0114] 5, the input device 13 and the output device 15 are configured as a common touch panel device. Therefore, the robot control device 11 acquires answers to questions displayed on the display unit 61 via operation inputs of the text input unit 51 and the control instruction icon 53. When an instruction to execute control is acquired via operation input of the control instruction icon 53, the robot control device 11 causes the robot 1 to perform work in accordance with a predetermined work plan.

[0115] 5, an example in which an answer to a question is acquired via text input unit 51 has been described, but it may also be acquired via voice, contact information on a touch panel, or an image. For example, when acquiring via an image, an identified gesture can be used as the answer to the question based on the posture estimation result of partner 17 in the image.

[0116] In addition, the judgment unit 33 may re-judge whether the progress of the first work has become continuable through the second work using the acquired specific information, and if it is not continuable, may repeat the question and answer work (request for specific information and acquisition of specific information in response to the request) until it becomes continuable.

[0117] For example, after the second task is performed using the acquired specific information, the judgment unit 33 may determine whether the progress of the first task, which has been determined to be unable to be continued, satisfies the start criteria, thereby determining whether the progress of the first task has become able to be continued.

[0118] Furthermore, the plan creation unit 38 may recreate the work plan based on the acquired specific information. For example, the plan creation unit 38 may be configured to create a new work plan by modifying the content of the first work that complies with the work instruction based on the acquired specific information.

[0119] In the robot control device 11 according to an embodiment of the present invention, the total work cost, which is the sum of the work costs of the robot 1 and the partner 17, is estimated based on the work content of the partner 17, and a work plan for returning to work is created so that the estimated total work cost is minimized. Therefore, a work plan can be created to return the first work of the robot 1, which has fallen into a state where it cannot be continued, to a state where it can be continued, taking into consideration the respective circumstances (work costs) of the robot 1 and the partner 17.

[0120] Therefore, according to the robot control device 11 of the embodiment of the present invention, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to suppress a decrease in overall work efficiency.

[0121] [Configuration of the robot control device 11 according to the first modified example of the present invention] Next, the configuration of a robot control device 11 according to a first modified example of the present invention will be described with reference to FIG. The robot control device 11 according to the embodiment of the present invention and the robot control device 11 according to the first modified example have a common basic configuration. Therefore, the following description will focus on the differences between the two and will serve as an explanation of the configuration of the robot control device 11 according to the first modified example.

[0122] In the robot control device 11 according to an embodiment of the present invention, if there is a bias in the work costs stored in advance in the memory unit 23b as work cost information, or if changes in the work cost information occur in response to changes in the work environment or the partner 17, there is a concern that a discrepancy will occur between the work cost estimated by the estimation unit 37 and the actual work cost.

[0123] Therefore, in the robot control device 11 according to the first variant, the estimation unit 37 estimates (in the first variant, this includes concepts of measurement or estimation) the work cost before and after the work (including the first work and the second concept) according to the work plan, and updates the work cost information stored in the memory unit 23b based on the comparison results of the estimated work costs.

[0124] An example of updating work cost information is to obtain (including the concept of measurement) the time spent waiting for partner 17 to complete the work and the time taken until the work can be continued, and then update the work cost information based on the obtained time.

[0125] Furthermore, the work cost information may be updated based on information about the workload of the partner 17 obtained by an interview during or after the dialogue with the partner 17. For example, if the work cost information indicates a probability distribution of the work cost associated with the work content of the partner 17, the obtained work cost may be used to calculate a posterior probability distribution of the work cost, and the work cost information may be updated using the obtained posterior probability distribution.

[0126] Furthermore, the task cost information (task execution probability) of the partner 17 may be updated as appropriate. For example, the task content of the partner 17 may be obtained (including the concept of recognition) through an interview during or after the dialogue with the partner 17, a posterior probability distribution may be calculated based on the obtained probability of each task, and the task cost information (task execution probability) of the partner 17 may be updated using the obtained posterior probability distribution.

[0127] In the robot control device 11 according to the first variant of the present invention, the estimation unit 37 is configured to estimate (in the first variant, this includes concepts of measurement or estimation) the work cost before and after the execution of work according to the work plan (including the first work and the second concept), and update the work cost information stored in the memory unit 23b based on the comparison results of the estimated work costs (before and after execution).

[0128] According to the robot control device 11 of the first variant of the present invention, the work cost information stored in the memory unit 23b is updated based on the comparison results of the work costs (before and after execution) estimated by the estimation unit 37. Therefore, even if the robot 1 falls into a situation where it is difficult to continue working in a scene where the robot 1 and its partner 17 are working together, in addition to the effect of suppressing a decrease in overall work efficiency, the effect of improving the accuracy of work cost estimation can be expected.

[0129] [Configuration, Functions, and Effects of the Robot Control Device 11 According to the Second Modification of the Present Invention] Next, a robot control device 11 according to a second modified example of the present invention will be described with reference to FIG. The robot control device 11 according to the embodiment of the present invention and the robot control device 11 according to the second modified example have a common basic configuration. Therefore, the following description will focus on the differences between the two and will serve as an explanation of the configuration of the robot control device 11 according to the second modified example.

[0130] In the robot control device 11 according to the embodiment of the present invention, if the variance of the operation costs estimated by the estimation unit 37 is large, there is a concern that a discrepancy will occur between the expected value of the operation costs and the actual operation costs.

[0131] Therefore, in the robot control device 11 according to the second variant, the plan creation unit 38 creates a work plan that includes the task of requesting the partner to provide actual information that will help reduce the uncertainty of the work cost, in other words, improve the accuracy of estimating the work cost, and the task of acquiring the actual information provided by the partner based on the request.

[0132] The control unit 39 causes the output device 15 to perform an operation of requesting the partner 17 to provide actual status information that is useful for improving the accuracy of estimating the work cost, as an operation in accordance with the work plan created by the plan creation unit 38. The control unit 39 also causes the input device 13 to perform an operation of acquiring the actual status information provided by the partner based on the request.

[0133] In addition, the control unit 39 may be configured to cause the robot 1 to perform operations related to the work by sending a control command to the robot 1 to cause the robot 1 to perform the work in accordance with the work plan created by the plan creation unit 38 (the work of requesting the partner 17 to provide actual information, and the work of obtaining specific information provided based on the request).

[0134] Here, actual status information is a concept that includes all information that is useful for improving the accuracy of estimating the work cost. The actual status information includes, for example, a question asking whether the partner 17 is currently performing the work, a question asking the content of the work that the partner 17 is performing, a question asking the estimated time of completion of the work, confirmation of whether or not communication is possible, and answers to these questions.

[0135] For example, if the output device 15 is a speaker and the input device 13 is a microphone, a question asking about the estimated time of completion of the work, such as "When will the work be finished?", is output via the output device 15, and an answer to the question, such as "10 minutes left," is obtained via the input device 13.

[0136] Furthermore, the plan creation unit 38 may recreate a work plan based on the re-estimated work cost. For example, if there is a work in a series of work sequences that is not dependent on the work determined not to be continued, the plan creation unit 38 may recreate a work plan to execute this work during the re-estimated standby time of the robot 1.

[0137] In the robot control device 11 according to the second modification of the present invention, a novel concept of actual status information useful for improving the accuracy of task cost estimation is used. In order to improve the accuracy of task cost estimation by the estimation unit 37, the plan creation unit 38 creates a task plan including an operation of requesting the partner 17 to provide actual status information useful for improving the accuracy of task cost estimation, and an operation of acquiring the actual status information provided by the partner 17 based on the request. The control unit 39 performs the operations of requesting the partner 17 to provide the actual status information based on the task plan, and acquiring the actual status information provided by the partner 17 based on the request. The acquisition unit 31 acquires the actual status information. The estimation unit 37 is configured to re-estimate the task cost based on the acquired actual status information.

[0138] According to the robot control device 11 of the second variant of the present invention, the estimation unit 37 re-estimates the work cost based on actual information that is useful for improving the accuracy of estimating the work cost. Therefore, even if the robot 1 finds itself in a situation where it is difficult to continue working in a scene where the robot 1 and its partner 17 are working together, it is possible to expect the effect of suppressing a decline in overall work efficiency as well as the effect of improving the accuracy of estimating the work cost.

[0139] [Configuration, Functions, and Effects of the Robot Control Device 11 According to the Present Invention] A robot control device 11 according to a first aspect includes: A robot control device 11 that causes a robot 1 to perform a task in accordance with a predetermined task instruction, an acquisition unit 31 that acquires the work instruction; a determination unit (33) for determining the progress of the first work in accordance with the work instruction; a search unit (35) that searches for a partner (17) that can provide specific information for enabling the progress of the first work to be continued when the progress of the first work is determined not to be continued as a result of the determination by the determination unit (33); an estimation unit (37) that, when the partner (17) is extracted as a result of the search by the search unit (35), estimates a total work cost including a second work for changing the progress status of the first work from "cannot continue" to "can continue"; a plan creation unit (38) that creates a work plan including the second work for the partner (17) based on the total work cost estimated by the estimation unit (37); a control unit (39) that causes work including the second work to be performed in accordance with the work plan created by the plan creation unit (38), The plan creation unit 38 creates the work plan so as to minimize the total work cost.

[0140] According to the robot control device 11 based on the first aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to provide a robot control device 11 that can suppress a decrease in overall work efficiency.

[0141] A robot control device 11 according to a second aspect is the robot control device 11 according to the first aspect, The estimation unit 37 estimates the total work cost based on the work cost related to the standby work of the robot 1 and the work cost related to the workload of the partner 17. The configuration may be adopted.

[0142] According to the robot control device 11 based on the second perspective, the estimation unit 37 estimates the total work cost based on the work cost associated with the standby work of the robot 1 and the work cost associated with the workload of the partner 17. Therefore, in addition to the effects of the robot control device 11 based on the first perspective, collaborative work between the robot 1 and the partner 17 can be appropriately and smoothly promoted while taking into consideration the respective circumstances (work costs) of the robot 1 and the partner 17.

[0143] A robot control device 11 according to a third aspect is the robot control device 11 according to the second aspect, The acquisition unit 31 further acquires the specific information, The plan creation unit 38 recreates a work plan for performing work based on the acquired specific information and work including the first work. The configuration may be adopted.

[0144] According to the robot control device 11 based on the third perspective, the plan creation unit 38 recreates a work plan for performing work based on the acquired specific information and work including the first work, and therefore, in addition to the effects of the robot control device 11 based on the second perspective, it is possible to promote the creation of an excellent work plan that takes into account work based on the specific information and work including the first work.

[0145] A robot control device 11 according to a fourth aspect is the robot control device 11 according to the third aspect, The determination unit 33 re-determines the progress of the first work in accordance with the re-created work plan. The configuration may be adopted.

[0146] According to the robot control device 11 based on the fourth aspect, the judgment unit 33 re-evaluates the progress of the first task in accordance with the recreated work plan, and therefore, in addition to the effects of the robot control device 11 based on the third aspect, it is possible to promote appropriate and smooth work taking into account the progress of the first task.

[0147] A robot control device 11 according to a fifth aspect is the robot control device 11 according to the fourth aspect, If the result of the re-determination of the progress status of the first work by the determination unit 33 is that the progress status of the first work is not to be continued, the plan creation unit 38 recreates the work plan including an operation of causing the search unit 35 to re-search for another partner 17 that can provide the specific information and is different from the partner 17. The configuration may be adopted.

[0148] According to the robot control device 11 based on the fifth aspect, if the result of the re-evaluation by the judgment unit 33 regarding the progress of the first task is that the progress of the first task should not be continued, the plan creation unit 38 recreates a work plan that includes an operation of having the search unit 35 re-search for another partner 17 that is different from the partner 17 and can provide the specific information. Therefore, compared to the effect of the robot control device 11 based on the fourth aspect, the effect of appropriate and smooth work promotion taking into account the progress of the first task can be further enhanced.

[0149] A robot control device 11 according to a sixth aspect is the robot control device 11 according to the second aspect, Further, a storage unit 23b is provided for storing work cost information in which the work contents of the partner 17 are associated with corresponding work costs, The estimation unit 37 acquires the work content of the partner 17 and estimates the work cost related to the work load of the partner 17 based on the acquired work content of the partner 17 and the work cost information. The configuration may be adopted.

[0150] According to the robot control device 11 based on the sixth aspect, the estimation unit 37 acquires the work content of the partner 17 and estimates the work cost related to the workload of the partner 17 based on the acquired work content of the partner 17 and the work cost information. Therefore, compared to the effect of the robot control device 11 based on the second aspect, the effect of promoting appropriate and smooth collaborative work that takes into consideration the circumstances of the partner 17 (work cost related to the workload) can be further enhanced.

[0151] A robot control device 11 according to a seventh aspect is the robot control device 11 according to the sixth aspect, The task cost information is information in which a probability distribution of the corresponding task cost is associated with each of the tasks of the partner. The configuration may be adopted.

[0152] The robot control device 11 based on the seventh aspect is expected to have the effect of improving the accuracy of the task cost estimation by the estimating unit 37, compared to the operational effect of the robot control device 11 based on the sixth aspect.

[0153] A robot control device 11 according to an eighth aspect is the robot control device 11 according to the sixth aspect, The task cost information is information in which the corresponding task cost is associated with each attribute of the partner 17. The configuration may be adopted.

[0154] According to the robot control device 11 based on the eighth aspect, similar to the operational effect of the robot control device 11 based on the sixth aspect, it is possible to expect the effect of improving the accuracy of the task cost estimation by the estimating unit 37.

[0155] A robot control device 11 according to a ninth aspect is the robot control device 11 according to the sixth aspect, The work cost information includes schedule information of the partner 17. The configuration may be adopted.

[0156] According to the robot control device 11 based on the ninth aspect, similar to the operational effect of the robot control device 11 based on the sixth aspect, it is possible to expect the effect of improving the accuracy of the task cost estimation by the estimating unit 37.

[0157] A robot control device 11 according to a tenth aspect is the robot control device 11 according to the ninth aspect, The estimation unit 37 estimates whether the partner 17 is currently performing work based on the work cost information including the schedule information of the partner 17, and estimates the work cost related to the workload of the partner 17 based on whether the partner 17 is currently performing work. The configuration may be adopted.

[0158] According to the robot control device 11 based on the tenth aspect, it is possible to expect the effect of further improving the accuracy of the estimation of the task cost by the estimating unit 37, compared to the operational effect of the robot control device 11 based on the ninth aspect.

[0159] A robot control device 11 according to an eleventh aspect is the robot control device 11 according to the seventh aspect, The estimation unit 37 estimates the work costs before and after the work according to the work plan is performed, and updates the work cost information stored in the storage unit 23b based on the comparison result of the estimated work costs. The configuration may be adopted.

[0160] According to the robot control device 11 based on the eleventh aspect, it is possible to expect the effect of further improving the accuracy of the task cost estimation by the estimating unit 37, compared to the operational effect of the robot control device 11 based on the seventh aspect.

[0161] A robot control device 11 according to a twelfth aspect is the robot control device 11 according to the seventh aspect, The plan creation unit 38 creates a work plan including an operation of requesting the partner 17 to provide actual information useful for improving the estimation accuracy of the work cost, and an operation of acquiring the actual information provided by the partner 17 based on the request, The control unit 39 performs an operation of requesting the partner 17 to provide the actual situation information based on the work plan, and an operation of acquiring the actual situation information provided by the partner 17 based on the request, The acquisition unit 31 acquires the actual status information, The estimation unit 37 re-estimates the work cost based on the acquired actual situation information. The configuration may be adopted.

[0162] According to the robot control device 11 based on the twelfth aspect, the control unit 39 performs the tasks of requesting the partner 17 to provide the actual information based on the work plan, and acquiring the actual information provided by the partner 17 based on the request, and the estimation unit 37 re-estimates the work cost based on the acquired actual information.Therefore, compared to the effect of the robot control device 11 based on the seventh aspect, it is expected that the accuracy of the work cost estimation by the estimation unit 37 will be further improved.

[0163] A robot control system 10 according to a thirteenth aspect comprises: A robot control system 10 including a robot 1 and a robot control device 11 that causes the robot 1 to perform a task in accordance with a predetermined task instruction, The robot control device 11 an acquisition unit 31 that acquires the work instruction; a determination unit (33) for determining the progress of the first work in accordance with the work instruction; a search unit (35) that searches for a partner (17) that can provide specific information for enabling the progress of the first work to be continued when the progress of the first work is determined not to be continued as a result of the determination by the determination unit (33); an estimation unit (37) that, when the partner (17) is extracted as a result of the search by the search unit (35), estimates a total work cost including a second work for changing the progress status of the first work from "cannot continue" to "can continue"; a plan creation unit (38) that creates a work plan including the second work for the partner (17) based on the total work cost estimated by the estimation unit (37); a control unit (39) that causes work including the second work to be performed in accordance with the work plan created by the plan creation unit (38), The plan creation unit 38 creates the work plan so as to minimize the total work cost.

[0164] According to the robot control system 10 based on the thirteenth aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to provide a robot control system 10 that can suppress a decrease in overall work efficiency.

[0165] A robot 1 according to a fourteenth aspect of the present invention comprises: A robot 1 that performs work according to predetermined work instructions, an acquisition unit 31 that acquires the work instruction; a determination unit (33) for determining the progress of the first work in accordance with the work instruction; a search unit (35) that searches for a partner (17) that can provide specific information for enabling the progress of the first work to be continued when the progress of the first work is determined not to be continued as a result of the determination by the determination unit (33); an estimation unit (37) that, when the partner (17) is extracted as a result of the search by the search unit (35), estimates a total work cost including a second work for changing the progress status of the first work from "cannot continue" to "can continue"; a plan creation unit (38) that creates a work plan including the second work for the partner (17) based on the total work cost estimated by the estimation unit (37); a control unit (39) that causes work including the second work to be performed in accordance with the work plan created by the plan creation unit (38), The plan creation unit 38 creates the work plan so as to minimize the total work cost.

[0166] According to the robot 1 based on the fourteenth aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to suppress a decline in overall work efficiency.

[0167] A robot control method according to a fifteenth aspect comprises: A robot control method used when causing a robot 1 to perform a task in accordance with a predetermined task instruction, obtaining the work instruction; determining a progress status of a first task in accordance with the task instruction; If the result of the determination is that the progress status of the first work is not to be continued, searching for a partner 17 that can provide specific information to make the progress status of the first work possible to be continued; When the partner 17 is extracted as a result of the search, a step of estimating a total work cost including a second work for changing the progress status of the first work from "cannot continue" to "can continue"; A step of creating a work plan including the second work for the partner 17 based on the estimated total work cost; performing work including the second work in accordance with the created work plan; In the step of creating a work plan, the work plan is created so as to minimize the total work cost.

[0168] According to the robot control method based on the 15th aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to suppress a decline in overall work efficiency.

[0169] Other Embodiments The above-described embodiments are merely examples of the present invention, and therefore the technical scope of the present invention should not be construed as being limited by them, as the present invention can be embodied in various forms without departing from the spirit or main characteristics thereof.

[0170] It is also possible to replace part of the configuration of one embodiment described here with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment.It is also possible to add, delete, or replace part of the configuration of each embodiment with another configuration. Furthermore, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected.

[0171] For example, the robot 1 according to the embodiment of the present invention may be, but is not limited to, an orthogonal robot. The robot 1 may also have a built-in robot control device 11. Furthermore, the robot 1 may also be a robot 1 in a fixed form.

[0172] Furthermore, in the description of the robot control device 11 according to the embodiment of the present invention, an example has been given in which the articulated arm unit 5 of the robot 1 is provided with a sensor 9 that monitors and measures the surroundings of the working environment of the robot 1, but the present invention is not limited to this example. The sensor 9 may be built into the robot 1.

[0173] Finally, in the robot control device 11 according to the embodiment of the present invention, some or all of the functional units, processing units, and processing means may be implemented in hardware, such as integrated circuits. The functional units, processing units, and processing means may also be implemented in software, with the processor 21 interpreting and executing programs that implement the respective functions. Information such as programs, tables, and files that implement the respective functions may be stored in a storage device, such as a memory, a hard disk, or an SSD (Solid State Drive), or in a storage medium, such as a flash memory card or a DVD (Digital Versatile Disk). [Explanation of symbols]

[0174] 1. Robot 3 Mobile cart section 5 Articulated arm 7 Hand section 9 Sensors 10 Robot Control System 11 Robot control device 13 Input Devices 15 Output Devices 17 Partner 18 Communications Department 19 Mobile devices 20 Product inventory storage shelves 21 processors 22a Bottle 23b Storage section 31 Acquisition Department 33 Judgment section 35 Search Department 37 Estimation part 38 Planning Department 39 Control Unit

Claims

1. A robot control device that causes a robot to perform a task in accordance with a predetermined task instruction, an acquisition unit that acquires the work instruction; a determination unit that determines a progress status of a first work in accordance with the work instruction; a search unit that searches for a partner that can provide specific information for enabling the progress of the first work to be continued when the determination result of the determination unit indicates that the progress of the first work cannot be continued; an estimation unit that, when the partner is extracted as a result of the search by the search unit, estimates a total task cost including a second task for changing the progress status of the first task from "cannot continue" to "can continue"; a plan creation unit that creates a work plan including the second work for the partner based on the total work cost estimated by the estimation unit; a control unit that causes work including the second work to be performed in accordance with the work plan created by the plan creation unit, The plan creation unit creates the work plan so as to minimize the total work cost. A robot control device characterized by:

2. The robot control device according to claim 1, The estimation unit estimates the total work cost based on a work cost related to the standby work of the robot and a work cost related to the workload of the partner robot. A robot control device characterized by:

3. The robot control device according to claim 2, The acquisition unit further acquires the specific information, The plan creation unit recreates a work plan for performing work based on the acquired specific information and work including the first work. A robot control device characterized by:

4. The robot control device according to claim 3, The determination unit re-determines the progress of the first work in accordance with the recreated work plan. A robot control device characterized by:

5. The robot control device according to claim 4, If the result of the re-determination of the progress status of the first work by the determination unit is that the progress status of the first work is not to be continued, the plan creation unit recreates the work plan including an operation in which the search unit re-searches for another partner that can provide the specific information and is different from the partner. A robot control device characterized by:

6. The robot control device according to claim 2, a storage unit that stores task cost information in which a corresponding task cost is associated with each task content of the partner; The estimation unit acquires the work content of the partner, and estimates a work cost related to the work load of the partner based on the acquired work content of the partner and the work cost information. A robot control device characterized by:

7. The robot control device according to claim 6, The task cost information is information in which a probability distribution of the corresponding task cost is associated with each of the tasks of the partner. A robot control device characterized by:

8. The robot control device according to claim 6, The task cost information is information in which the corresponding task cost is associated with each attribute of the partner. A robot control device characterized by:

9. The robot control device according to claim 6, The task cost information includes schedule information of the partner. A robot control device characterized by:

10. The robot control device according to claim 9, The estimation unit estimates whether the partner is currently performing work based on the work cost information including the partner's schedule information, and estimates the work cost related to the partner's workload based on whether the partner is currently performing work. A robot control device characterized by:

11. The robot control device according to claim 7, The estimation unit estimates the work costs before and after the work according to the work plan is performed, and updates the work cost information stored in the storage unit based on a comparison result of the estimated work costs. A robot control device characterized by:

12. The robot control device according to claim 7, the plan creation unit creates a work plan including an operation of requesting the partner to provide actual situation information useful for improving the estimation accuracy of the work cost, and an operation of acquiring the actual situation information provided by the partner based on the request; The control unit causes the operation of requesting the partner to provide the actual situation information based on the work plan and the operation of acquiring the actual situation information provided by the partner based on the request, The acquisition unit acquires the actual status information, The estimation unit re-estimates the task cost based on the acquired actual situation information. A robot control device characterized by:

13. A robot and a robot control device according to claim 1. A robot control system comprising:

14. A robot that performs work according to predetermined work instructions, A robot control device according to claim 1 is provided. A robot characterized by:

15. A robot control method used when causing a robot to perform a task in accordance with a predetermined task instruction, comprising: obtaining the work instruction; determining a progress status of a first task in accordance with the task instruction; If the result of the determination indicates that the progress of the first work cannot be continued, searching for a partner who can provide specific information for enabling the progress of the first work to be continued; When the partner is extracted as a result of the search, estimating a total task cost including a second task for changing the progress status of the first task from "cannot continue" to "can continue"; creating a work plan including the second work for the partner based on the estimated total work cost; performing work including the second work in accordance with the created work plan; A robot control method, wherein in the step of creating a work plan, the work plan is created so as to minimize the total work cost.

Images