Robot control device, robot control system, robot, and robot control method

Abstract

A robot control device (11) is provided with: a determining unit (33) for determining the progress status of first work in accordance with a work instruction; a search unit (35) for searching for a partner (17) capable of providing specific information for changing the progress status of the first work to a continuation-possible status if the progress status of the first work is a continuation-not-possible status; an estimating unit (37) for estimating the total work cost including second work for changing the progress status of the first work from the continuation-not-possible status to the continuation-possible status; a plan creating unit (38) for creating a work plan including the second work for the partner (17) on the basis of the estimated total work cost; and a control unit (39) for causing the work including the second work complying with the created work plan to be performed. The plan creating unit (38) creates the work plan so as to minimize the total work cost.

Description

Robot control device, robot control system, robot, and robot control method

[0001] The present invention relates to a robot control device for controlling the operation of a robot, a robot control system, a robot, and a robot control method.

[0002] In recent years, there has been an increasing demand for robot-based task automation and collaboration between robots and humans in fields such as manufacturing, logistics, services, construction, and facility maintenance. For example, when a robot encounters a situation where it is difficult to continue working, it is required to quickly return to work. Patent Document 1 (JP-A-2005-102626) describes a technology for enabling a robot to quickly return to work. This publication describes a "robot control system including: an instructor memory unit that stores information on multiple instructors; a selection unit that selects one of the multiple instructors stored in the instructor memory unit when support information is needed to allow the robot to continue working; an environment information transmission unit that transmits environmental information about the robot to the instructor selected by the selection unit via a network line; and a support information reception unit that receives the support information from the instructor selected by the selection unit via a network line."

[0003] International Publication No. 2020 / 149414

[0004] For example, in a scenario where a robot and a partner (such as a human or another robot) are working together, if the robot encounters a situation where it is difficult to continue working, the partner may be concentrating on a different task. In such a scenario, requesting support from the concentrating partner, such as through dialogue, would interrupt the partner's work. As a result, there is a concern that work efficiency may be impaired. In other words, the robot control system of Patent Document 1 does not describe or suggest how to deal with a situation where a robot encounters a situation where it is difficult to continue working in a scenario where a robot and a partner are working together, by taking the partner's circumstances into consideration. As a result, there is a risk of overall work efficiency decreasing.

[0005] The present invention has been made in consideration of the above-mentioned situation, and aims to suppress a decline in overall work efficiency in a scene where a robot and a partner are working together, even if the robot finds itself in a situation where it is difficult to continue working.

[0006] In order to solve the above problem, the robot control device of the present invention is a robot control device that causes a robot to perform work in accordance with predetermined work instructions, and includes: an acquisition unit that acquires the work instructions; a determination unit that determines the progress status of a first work in accordance with the work instructions; a search unit that searches for a partner that can provide specific information to change the progress status of the first work from ``cannot be continued'' to ``cannot be continued'' as a result of the determination by the determination unit; an estimation unit that estimates a total work cost including a second work to change the progress status of the first work from ``cannot be continued'' to ``cannot be continued'' as a result of the search by the search unit; a plan creation unit that creates a work plan including the second work for the partner based on the total work cost estimated by the estimation unit; and a control unit that causes a robot to perform work including the second work in accordance with the work plan created by the plan creation unit, and the most main feature of the plan creation unit is that it creates the work plan so as to minimize the total work cost.

[0007] According to the present invention, in a scene where a robot and a partner work together, even if the robot falls into a situation where it is difficult to continue working, it is possible to suppress a decrease in overall work efficiency. Other issues, configurations, and effects will be described in detail in the following embodiments.

[0008] 1 is a schematic configuration diagram of a robot control system including a robot control device according to an embodiment of the present invention; FIG. 2 is a functional block diagram of a robot control device according to an embodiment of the present invention; FIG. 3 is a block diagram illustrating an example of a hardware configuration of a robot control device according to an embodiment of the present invention; FIG. 4 is a flowchart illustrating an operation procedure of a robot control device according to an embodiment of the present invention; and FIG. 5 is an explanatory diagram illustrating an example of a display screen of specific information useful for making the progress status of a first task according to a task instruction continuable.

[0009] Hereinafter, a robot control device, a robot control system, a robot, and a robot control method according to embodiments of the present invention will be described with reference to the appropriate drawings. In the description of the robot control device according to the embodiments of the present invention, components having common functions are assigned common reference numerals, and duplicate descriptions thereof will be omitted.

[0010] [General Configuration of Robot Control System 10] First, a general configuration of a robot control system 10 including a robot control device 11 according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a general configuration diagram of a robot control system 10 including a robot control device 11 according to an embodiment of the present invention.

[0011] As shown in FIG. 1, the robot control system 10 includes a robot 1, a robot control device 11, an input device 13, and an output device 15.

[0012] The robot 1 is configured to include a moving carriage unit 3, an articulated arm unit 5, and a hand unit 7.

[0013] The mobile carriage unit 3, the articulated arm unit 5, and the hand unit 7 are each connected to the robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other.

[0014] The articulated arm unit 5 is provided with a sensor 9 that monitors and measures the surroundings of the working environment of the robot 1. The sensor 9 is connected to a robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other.

[0015] The sensor 9 includes, for example, a camera, a distance sensor, and a microphone. Examples of distance sensors include LiDAR, a laser displacement meter, a stereo camera, an infrared distance meter, a laser distance meter, and a time-of-flight distance sensor. The sensor 9 may be a wearable terminal that measures physical data of a person (supervisor or collaborator), or may be a joint angle sensor, an acceleration sensor, or a tactile sensor built into the robot 1. The sensor 9 may be installed in a location where it can capture an image of the entire working environment of the robot 1.

[0016] The robot control device 11 according to the embodiment generates control commands by referring to the work instruction content, the monitoring results by the sensor 9, etc. Each of the mobile cart unit 3, the articulated arm unit 5, and the hand unit 7 provided in the robot 1 operates according to the control commands of the robot control device 11 according to the embodiment.

[0017] The input device 13 is used when a user (e.g., a supervisor of the robot 1) gives predetermined work instructions to the robot 1. As the input device 13, for example, a human interface device such as a keyboard, mouse, microphone, or liquid crystal display, a smartphone, a tablet, or a robot controller may be appropriately used. The input device 13 is connected to the robot control device 11 according to the embodiment via a wired medium or a wireless medium so as to be able to communicate with each other.

[0018] The output device 15 is used when the robot 1 outputs specific information to the partner 17 to enable continuation of the progress of the work in accordance with the work instructions. As the output device 15, for example, a human interface device such as a speaker or a liquid crystal display, a smartphone, a tablet, etc. may be used as appropriate. The output device 15 is connected to the robot control device 11 according to the embodiment so as to be able to communicate with each other via a wired medium or a wireless medium. The output device 15 may be the same device as the input device 13, or may be a different device.

[0019] Partner 17 refers to any entity that provides robot 1 with specific information, including guidance and advice, to enable it to continue the progress of the work (first work) in accordance with the work instructions when robot 1 finds itself in a situation where it is difficult to continue the work.

[0020] Specifically, the partner 17 is not limited to a supervisor who gives predetermined work instructions to the robot 1, but may also be a coworker different from the supervisor. Furthermore, the partner 17 is not limited to a person, but may be, for example, another robot or a search device for searching for the specific information. The "specific information" will be described in more detail later.

[0021] The work performed in accordance with the work instructions is not particularly limited, but may be, for example, the work of removing instructed products from product inventory storage shelves 20 and replenishing them on product display shelves (not shown) in a convenience store. In the example shown in Figure 1, bottles 22a, boxes 22b, and cans 22c are listed as products to be placed on product inventory storage shelves 20.

[0022] The work performed in accordance with the work instruction may be, for example, a single work such as "removing bottle 22a from product inventory shelf 20," or may be a sequence of multiple work operations such as "moving to back yard," "detecting bottle 22a," "grasping bottle 22a," "moving to product display shelf," and "placing bottle 22a" based on the work instruction "replenishing bottle 22a on product display shelf." The work performed in accordance with the work instruction corresponds to the "first work" of the present invention. In the following description, the work performed in accordance with the work instruction may be referred to as the "first work."

[0023] In the embodiment of the present invention, an example of a work instruction to "refill the product display shelves with bottles 22a" will be described. In this case, the first work performed by the robot 1 in accordance with the work instruction is the series of work performed to refill the product display shelves with bottles 22a.

[0024] As shown in Figure 1, upon receiving the work instruction, the robot 1 operates the mobile cart unit 3 to move to the back yard, detects the bottle 22a on the product inventory storage shelf 20 indicated by the sensor (camera) 9, grasps the detected bottle 22a by operating the multi-joint arm unit 5 and hand unit 7, moves to the specified product display shelf by operating the mobile cart unit 3 while grasping the bottle 22a, and places the bottle 22a in the specified location by operating the multi-joint arm unit 5 and hand unit 7, performing a series of tasks.

[0025] [Software Configuration of Robot Control Device 11 According to Embodiment] Next, the software configuration of the robot control device 11 according to the embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a functional block diagram of the robot control device 11 according to the embodiment.

[0026] As shown in FIG. 2, the robot control device 11 according to the embodiment is configured to include a communication unit 18, an acquisition unit 31, a determination unit 33, a search unit 35, a memory unit 23b, an estimation unit 37, a plan creation unit 38, and a control unit 39.

[0027] The communication unit 18 acquires the specific information for enabling the continuation of the progress of the first work between the mobile terminal 19 carried by the partner 17 and the mobile terminal 19 via a wireless communication medium.

[0028] The acquisition unit 31 acquires detection information from the sensor 9 and information related to specified work instructions from the input device 13, and also acquires the specific information sent from the mobile terminal 19 held by the partner 17 via the communication unit 16.

[0029] In more detail, the acquisition unit 31 may acquire, for example, one task selected from a predetermined task group as a work instruction, or may acquire a sequence of tasks consisting of multiple tasks as a work instruction.

[0030] The acquisition unit 31 may also create a sequence of tasks consisting of multiple tasks as a work instruction based on information related to a specific work instruction from the input device 13. For example, the acquisition unit 31 may extract a sequence of tasks to be performed by performing natural language processing on a linguistic instruction such as "refill bottle 22a." Examples of natural language processing include topic models and large-scale language models such as ChatGPT.

[0031] Specifically, the acquisition unit 31 may create a series of tasks, such as "move to back yard," "detect bottle 22a," "grasp bottle 22a," "move to bottle display shelf," and "place bottle 22a," based on a task instruction such as "refill bottle 22a."

[0032] As an example of creating a task sequence, a task goal and a task sequence may be linked and stored in advance in the storage device 23, and when a task goal is input, a task sequence corresponding to this task goal may be read from the storage device 23 and created, or a series of task sequences for achieving the task goal may be planned. Examples of methods for planning task sequences toward a task goal include dynamic programming, A*, RRT*, and active inference.

[0033] The acquisition unit 31 may also directly generate a sequence of tasks to be performed using a large-scale language model based on a vague task instruction from a person, such as "I'm thirsty." Examples of such methods include ChatGPT for Robotics and RT-2.

[0034] The determination unit 33 determines the progress of the first task in accordance with the task instruction acquired by the acquisition unit 31, that is, whether or not the first task can be continued.

[0035] The determination unit 33 may determine whether or not to continue the work by setting a work error criterion in advance, such as "the object cannot be detected," and determining whether or not the detection state of the object satisfies the work error criterion. The determination unit 33 may also set work start and end criteria in advance, and determine whether or not to continue the first work by determining whether or not the current state satisfies the work start and end criteria.

[0036] In addition, the judgment unit 33 may acquire in advance the detection data of the sensor 9 when work can be continued, and by comparing the past detection data with the detection data of the sensor 9 acquired in the current work state, determine whether the work has fallen into an abnormal state and thereby judge whether work can be continued.

[0037] Examples of such anomaly detection techniques include anomaly detection techniques using neural networks such as AutoEncoder and VAE. For example, if the sensor 9 is a camera, images such as RGB images, gray images, and infrared images captured by the camera during a previous task are trained in a neural network. Then, when the task is performed, the currently captured image is input to the trained neural network to determine whether the current task state is abnormal.

[0038] The data input to the neural network does not have to be images. For example, if the sensor 9 is a distance sensor, the neural network learns depth images, point cloud data, etc. measured by the distance sensor; if the sensor 9 is a microphone, the neural network learns audio data acquired by the microphone; and if the sensor 9 is a joint angle sensor, acceleration sensor, or tactile sensor built into the robot 1, the neural network learns joint angle data, acceleration data, and tactile data measured by each sensor 9. This allows the neural network to determine whether the current working state is abnormal.

[0039] The data to be learned does not necessarily have to be the information itself acquired by the sensor 9, but may be information obtained by processing the information. For example, it may be differential information between measurement information such as the velocity and acceleration of joint angles, or optical flow calculated from time-series images.

[0040] Furthermore, the data to be learned may be, for example, an image that has been homography transformed to match the viewpoint as seen from a predetermined coordinate system, or information that has been normalized based on the average and standard deviation of previously obtained measurement information.

[0041] If the determination unit 33 determines that the progress of the first work in accordance with the work instructions cannot be continued, the search unit 35 searches for a partner 17 that can provide specific information to allow the progress of the first work to be continued.

[0042] More specifically, the search unit 35 detects the person who will become the partner 17 using a predetermined person detection method based on, for example, images and point cloud data acquired by the sensor 9. Examples of person detection methods include object detection methods using neural networks such as R-CNN, YOLO, SSD, and DETR, and person pose estimation methods such as OpenPose. In addition, for example, a facial image of the person who gave the work instructions may be acquired using the input device 13, and the same person as the person who gave the work instructions may be searched for as the partner 17 using facial recognition technology.

[0043] The search unit 35 may also search for the partner 17 while moving using the mobile cart unit 3 provided on the robot 1. The search unit 35 may use the mobile cart unit 3 to move to a location where the partner 17 is likely to be, based on map information created by SLAM or the like and estimated self-position information, for example, and perform the search.

[0044] In addition, when the sensor 9 includes multiple microphones, the search unit 35 may use a sound source localization method to estimate the person's position based on the person's speaking voice, and search for the partner 17 by moving to the estimated person's position using the mobile cart unit 3.

[0045] Furthermore, the search unit 35 may identify the position of a person from a beacon sensor or the like carried by a person in the vicinity, and search for the partner 17 by moving to the identified position of the person using the mobile cart unit 3.

[0046] Furthermore, if the work cost information includes schedule information of the partner 17, the search unit 35 may read the schedule information of the partner 17 from the contents (work cost information) stored in the memory unit 23b, estimate the location of the partner 17 by comparing the current time with the schedule information, and search for the partner 17 by traveling to the estimated location using the mobile cart unit 3.

[0047] The partner 17 to be searched for by the search unit 35 is not limited to the person (supervisor) who gave the work instruction using the input device 13, but may also be another person (co-worker). For example, if multiple people are detected by the person detection process, the person closest to the robot 1 may be considered to be the partner 17.

[0048] The memory unit 23b stores a task cost table that associates task cost information for each of a plurality of tasks performed by the partner 17. The task cost information for each task means information on added value associated with each of a plurality of different tasks. The term "task" here is an expanded concept that includes all actions of the partner 17 (supervisor / cooperative worker) and the robot 1.

[0049] For example, if we consider a case where robot 1 is performing a first task and the progress of the first task becomes unsustainable, the value obtained by multiplying the added value associated with the first task (for example, based on the difficulty of the task, etc.) by the waiting time corresponds to the work cost associated with robot 1 waiting to perform the task.

[0050] Also, for example, if we consider a case where partner 17 is performing some work and has to interrupt his work to respond to a question and answer session by robot 1 (a request for specific information and acquisition of specific information in response to the request), the added value corresponding to partner 17's work (for example, based on the difficulty of the work, etc.) multiplied by the interruption time is equivalent to the work cost related to partner 17's interruption of work.

[0051] The variables used to calculate the task cost are not limited to added value associated with each of a plurality of tasks, but may be attribute values ​​associated with each task subject (partner 17, robot 1, etc.). For example, a configuration may be adopted in which the attribute value associated with the partner 17 is a numerical value

[10] , and the attribute value associated with the robot 1 is a numerical value [5].

[0052] In addition, in cases where the main performer of the work is a partner 17, for example, a configuration may be adopted in which the numerical value

[20] is associated as the attribute value for partner 17 (supervisor) and the numerical value

[10] is associated as the attribute value for partner 17 (co-worker) depending on the partner 17's position, position, level of expertise, etc.

[0053] When the progress status of the first work in accordance with the work instructions is "cannot be continued" and the search unit 35 extracts the partner 17 as a result of the search, the estimation unit 37 estimates the total work cost including the second work for changing the progress status of the first work from "cannot be continued" to "can be continued" based on the work cost table stored in the memory unit 23b.

[0054] The estimation unit 37 estimates the total work cost including the second work, for example, based on the work cost associated with the waiting time of the robot 1 and the work cost associated with the workload of the partner 17 (for example, work interruptions).

[0055] More specifically, the estimation unit 37 estimates the work content of the partner 17 searched for by the search unit 35, and estimates the work cost based on the work content of the partner 17. The work content of the partner 17 may be estimated using, for example, machine learning or a human activity recognition technique in computer vision.

[0056] The estimation result of the work content of the partner 17 by the estimation unit 37 may be a specific work content, and may further include the probability that the work content is a specific work content.

[0057] Furthermore, the estimation result of the work content of the partner 17 by the estimation unit 37 is not limited to a specific work content, and the progress status of the specific work may also be estimated. In this case, the estimation unit 37 may estimate the progress status of the specific work by using a method that incorporates a probabilistic transition model or person tracking into human behavior recognition.

[0058] In addition, if the work cost information includes schedule information of the partner 17, the estimation unit 37 may receive the schedule information of the partner 17 from the memory unit 23b and estimate the work content of the partner 17 by comparing the current time with the schedule information.

[0059] The estimation unit 37 compares the estimated work content of the partner 17 with the work cost information, thereby estimating the work cost corresponding to the work content.

[0060] Furthermore, if there are multiple candidate partners 17 and the task cost information differs for each of the multiple partners 17, the estimation unit 37 can estimate the task cost of the searched partner 17 by recognizing the appropriate partner 17 using the face recognition technology described above. Furthermore, if the task cost information depends on the location and time, the task cost according to the current location and time can be estimated by referring to the location and current time when the current self-location is calculated using SLAM.

[0061] Specifically, for example, when the robot 1 is performing a first task and the progress of the first task becomes unsustainable, a first case is considered in which specific information is obtained to change the progress of the first task from unsustainable to continuable by having the robot 1 immediately conduct a question and answer session (second task) with its partner 17 who is performing a certain task.

[0062] The total work cost in this first case is calculated based on the work cost associated with the waiting time of robot 1 (the time until the first work can be continued) (for example, the value obtained by multiplying the added value associated with the first work by the waiting time) and the work cost associated with the interruption of the work of partner 17 (for example, the value obtained by multiplying the added value associated with partner 17 by the interruption time).

[0063] Next, for example, consider a second case in which, while the robot 1 is performing a first task, the progress status of the first task changes to "cannot be continued," and by referring to the schedule information of the partner 17 extracted by the search by the search unit 11c, it is known that the partner 17, who is currently performing a certain task, will complete the task if it waits for five minutes. After waiting for five minutes, the robot 1 then has the partner 17 conduct a question-and-answer session (second task) (waiting for the partner 17 to complete the task), thereby obtaining specific information to change the progress status of the first task from "cannot be continued" to "can be continued."

[0064] The total operation cost in this second case is calculated based on the operation cost associated with the standby of the robot 1 (the time until the first operation can be continued). In this case, the operation of the partner 17 is not interrupted, so the operation cost associated with the interruption of the operation of the partner 17 is not taken into account (added).

[0065] The total work cost in the first case and the total work cost in the second case are referenced in the next-technique plan creation unit 38 when creating a work plan so that the total work cost for the second work required to change the progress status of the first work from ``cannot continue'' to ``can continue'' is minimized.

[0066] The plan creation unit 38 creates a work plan for the first work in accordance with the work instructions acquired by the acquisition unit 31. When the progress status of the first work is not to be continued, the plan creation unit 38 creates a work plan so as to minimize the total work cost for the second work required to change the progress status of the first work from not to be continued to being continued.

[0067] The work plan created by the plan creation unit 38 includes an operation to request the partner 17 to provide specific information (including helpful information) necessary to change the progress status of the first work from "cannot continue" to "can continue," and an operation to obtain the specific information provided by the partner 17 based on the request. These operations correspond to the "second work" of the present invention.

[0068] The plan creation unit 38 may create a work plan that includes a series of tasks, for example, waiting for a partner 17 who is currently performing a certain task to complete the task, then requesting the partner 17 to provide specific information, and then acquiring the specific information provided by the partner 17.

[0069] In addition, the plan creation unit 38 may create a work plan that includes a series of tasks, for example, to request the partner 17 to provide specific information without waiting for the partner 17 who is currently performing a certain task to complete the task (by interrupting the work of the partner 17), and then to acquire the specific information provided by the partner 17.

[0070] Furthermore, after acquiring the specific information provided by the partner 17, the plan creation unit 38 may create a work plan to resume the first work in accordance with the original work instruction.

[0071] Furthermore, after the execution of work based on the specific information provided by a certain partner 17, if the result of the re-evaluation of the progress of the first work by the judgment unit 33 is that the progress of the first work cannot be continued, the plan creation unit 38 may be configured to recreate a work plan that includes an operation of having the search unit 35 re-search for another partner that is different from the partner and can provide the specific information.

[0072] If another partner is extracted by the re-search, the estimation unit 37 may estimate the work cost of the other partner in the form of an expected value of the work cost, for example, based on the prior distribution of the work content and the work costs for each of multiple tasks.

[0073] Examples of methods for planning a series of work sequences include dynamic programming and tree search. Furthermore, a reinforcement learning method such as Q-learning, DQN, or A3C may also be used as a method for planning a series of work sequences.

[0074] Furthermore, the plan creation unit 38 may create a work plan that includes an operation of selecting a partner 17 based on the probability that the progress status of the first work will be able to continue, which is obtained through dialogue with each of the multiple partners 17. In this case, for example, by linking the error content that is the factor that prevents the work from being continued with the specialized knowledge required to solve it, the probability that the work will be able to continue through dialogue with each partner 17 can be set depending on whether each of the multiple partners 17 has the specialized knowledge in question.

[0075] The plan creation unit 38 may also create a work plan based on values ​​calculated with reference to the probability distribution of work costs. Examples of values ​​calculated with reference to the probability distribution of work costs include an expected value for each of multiple work costs and a risk-aware work cost calculated by adding a constant multiple of the standard deviation to the average work cost. Furthermore, the probability distribution of work costs may be directly used when planning work by utilizing techniques such as Bayesian inference. Examples of work planning techniques that utilize Bayesian inference include active inference and control as inference.

[0076] The control unit 39 causes the robot 1 to perform the first task by transmitting a control command to the robot 1 to cause the robot 1 to perform the first task based on the task plan created by the plan creation unit 38. The control unit 39 also causes the robot 1 to perform, via the input device 13, a task of requesting the partner 17 to provide specific information for changing the progress status of the first task from "cannot continue" to "can continue," as a second task in accordance with the task plan created by the plan creation unit 38. Note that the control unit 39 may be configured to cause the robot 1 to perform an operation related to the second task by transmitting a control command to the robot 1 to perform the second task based on the task plan created by the plan creation unit 38.

[0077] Here, the specific information is a concept that includes all information that is useful for enabling the progress of the first task to be continued. The specific information includes, for example, the content of the task instructions, a description of the task that was performed, the content of the error that prevented the task from being continued, the cause of the error, questions requesting solutions to those errors, answers to those questions, and instructions for additional tasks to continue the task.

[0078] For example, if the judgment unit 33 is unable to detect the work object, the equipment being used, or the partner 17 and determines that work cannot be continued, examples of information that would be useful in allowing work to continue include detailed information such as the color and shape of the work object and the equipment being used, the location where they are placed, the inventory status, and the appearance and location of people (supervisors and co-workers).

[0079] Furthermore, for example, when it is determined that continuation of the movement operation is not possible due to an obstacle, examples of the specific information include a route to avoid the obstacle and whether or not the obstacle can be removed.

[0080] Furthermore, for example, if it is determined that work cannot be continued due to a malfunction, examples of the specific information include questions and answers regarding the location of the malfunction and the cause of the malfunction.

[0081] Also, for example, if it is determined that work cannot be continued because there is no available equipment required for the work, examples of specific information include questions and answers regarding the available time of the equipment and the availability of other equipment.

[0082] Furthermore, the specific information may include additional information for explaining the current progress of the first task, such as a description of the task currently being performed or data acquired via the sensor 9 (including the concept of measurement).

[0083] The specific information may also be a control input for the robot 1 required for returning to work. In this case, examples of the specific information include position information, joint angle information, and joint velocity information for directly controlling the robot 1.

[0084] The standby time of the robot 1 related to the work cost may be, for example, the standby time until the partner work is completed and the robot is ready to interact, or the standby time until the robot is ready to continue working. Furthermore, the standby time of the robot 1 related to the work cost may include the time required for interaction, travel time, etc. in addition to the standby time until the robot is ready to interact.

[0085] In addition, if the work plan created by the plan creation unit 38 includes the task of re-searching for the partner 17, the total time required for searching for the partner 17, estimating the work cost, planning the work, and interacting, which are repeatedly performed until the work can be continued, may also be used as the total time required.

[0086] The work cost may also include the work load of the robot 1. The work load of the robot 1 may be, for example, the amount of energy consumed for the work.

[0087] The workload of partner 17 related to work costs is the physical and psychological burden on partner 17, such as the time it takes to have a conversation, the time it takes for partner 17 to return to their own work after the conversation, the physical and psychological burden caused by multitasking when partner 17 has to have a conversation and do another task at the same time, and the psychological burden caused by work interruptions.

[0088] The workload of the partner 17 related to the work cost may be quantified using the subjective evaluation of the partner 17, or may be quantified using physical data acquired by vital signs sensing. Examples of the physical data include heart rate, blood pressure, skin temperature, respiratory rate, electromyography, pupillary response, oxygen intake, electrodermal activity, and electroencephalography (EEG). Physical data may be estimated from images or the like rather than being measured directly. For example, the work cost may be quantified using physical data estimated using facial expression analysis, pupil size analysis, and posture analysis. Furthermore, the work cost may be estimated using a workload model instead of the physical data values ​​themselves. Examples of workload models include the NASA-TLX, the Borg scale, heart rate variability analysis (HRV), EEG pattern detection, and pupillary light response.

[0089] Furthermore, the operation cost may be a combination of the standby time of the robot 1 and the workload of the partner 17, or may be a combination of the workload of multiple types of partner 17.

[0090] In the task cost information, the task cost linked to the task content of the partner 17 may be a predetermined value quantified as described above, or may be a range and probability distribution of task costs obtained from task cost data acquired multiple times. Examples of methods for calculating the probability distribution from the acquired task costs include maximum likelihood estimation and posterior distribution estimation using a Gaussian distribution or a Gaussian mixture distribution, or probability distribution estimation in a feature space using a deep learning method such as VAE.

[0091] The task cost information may also include a task execution probability, which is the probability that each task will be performed by the partner 17. By using the task execution probability, it is possible to calculate an expected value of the task cost, etc.

[0092] Furthermore, the task cost information may be different for each of the multiple partners 17. For example, by defining the task cost for each task based on the job title and required specialized knowledge, the task cost can be estimated according to the job title and whether or not the partner 17 has specialized knowledge.

[0093] The task cost information may also include location and time. For example, by combining it with schedule information of the partners 17, the task execution probability at each time for each of the multiple partners 17 may be calculated.

[0094] [Hardware Configuration of Robot Control Device 11 According to the Embodiment] Next, the hardware configuration of the robot control device 11 according to the embodiment of the present invention will be described with reference to Fig. 3. Fig. 3 is a block diagram illustrating an example of the hardware configuration of the robot control device 11 according to the embodiment.

[0095] 3, the robot control device 11 according to the embodiment is a computer including a processor 21 such as a CPU, a storage device 23 such as a semiconductor memory, an input unit 25 that receives information from the input device 13, an output unit 27 that transmits information to the output device 15, a communication unit 18 that performs two-way communication with the robot 1 and the sensor 9, and a bus 29 that connects these units. The processor 21 executes a predetermined robot control processing program 23a to realize the functions of the above-mentioned units.

[0096] [Operation of the robot control device 11 according to the embodiment] Next, the operation of the robot control device 11 according to the embodiment of the present invention will be described with reference to Fig. 4. Fig. 4 is a flowchart illustrating the operation of the robot control device 11 according to the embodiment.

[0097] In step S11 shown in FIG. 4, the acquisition unit 31 acquires information detected by the sensor 9 and information related to a predetermined work instruction from the input device 13.

[0098] In step S12, the plan creation unit 38 creates a work plan for the first work in accordance with the work instruction based on the detection information from the sensor 9 acquired by the acquisition unit 31 and information related to the specified work instruction from the input device 13.

[0099] In step S13, the control unit 39 causes the robot 1 to perform the first task based on the task plan created by the plan creation unit 38 in step S12.

[0100] In step S14, the determination unit 33 determines whether the first work based on the work plan has been completed.

[0101] If the determination result in step S14 indicates that the first task has been completed, the determination unit 33 ends the process. On the other hand, if the determination result in step S14 indicates that the first task has not been completed, the robot control device 11 according to the embodiment advances the process to the next step S15.

[0102] In step S15, the determination unit 33 determines the progress of the first task based on the work plan (whether the first task can be continued).

[0103] If the determination in step S15 indicates that the first task has been completed, the determination unit 33 ends the series of processing flows.

[0104] On the other hand, if the result of the judgment in step S14 is that the first task has not been completed, in step S15, the judgment unit 33 judges the progress of the first task (whether the first task can be continued).

[0105] If the result of the judgment in step S15 is that the progress of the first task can be continued, the robot control device 11 according to the embodiment returns the processing flow to step S13 and causes the subsequent processing to be performed sequentially.

[0106] On the other hand, if the result of the determination in step S15 is that the progress of the first task is not to be continued, the robot control device 11 according to the embodiment advances the processing flow to the next step S16.

[0107] In step S16, the search unit 35 searches for a partner 17 that can provide specific information for enabling the progress of the first task to be continued.

[0108] If the partner 17 is extracted as a result of the search in step S16, in step S17, the estimation unit 37 estimates the total work cost including the second work to change the progress status of the first work from ``cannot continue'' to ``can continue'' based on the work cost table stored in the memory unit 23b.

[0109] In step S18, the plan creation unit 38 creates a work plan so as to minimize the total work cost estimated in step S17 (the total work cost related to the second work for changing the progress status of the first work from "cannot continue" to "can continue"). Next, the robot control device 11 according to the embodiment returns the process flow to step S13 and sequentially performs the subsequent processes.

[0110] That is, in step S13, the control unit 39 causes the robot 1 to perform the second task based on the task plan created by the plan creation unit 38 in step S18. Here, the second task includes the task of requesting the partner 17 extracted by the search in step S16 to provide specific information, and the task of receiving the specific information from the partner 17, in order to change the progress status of the first task from "cannot continue" to "can continue." Thereafter, the robot control device 11 according to the embodiment causes the following processes to be performed sequentially.

[0111] FIG. 5 is an explanatory diagram showing an example of a display screen of specific information useful for making the progress of the first work in accordance with the work instruction continuable.

[0112] 5, a question inquiring about the work object, "Which bottle is the one to be replenished?", is displayed as specific information using a window corresponding to the display unit 41. In addition, in the example shown in FIG. 5, the content of the work instruction and the data acquired via the sensor 9 are also displayed as specific information.

[0113] In addition, in the task of acquiring specific information from the partner 17, the acquisition unit 31 acquires the specific information via the input device 13 or via the mobile terminal 19 and communication unit 18 carried by the partner 17.

[0114] 5, the input device 13 and the output device 15 are configured as a common touch panel device. Therefore, the robot control device 11 acquires answers to questions displayed on the display unit 61 via operation inputs of the text input unit 51 and the control instruction icon 53. When an instruction to execute control is acquired via operation input of the control instruction icon 53, the robot control device 11 causes the robot 1 to perform work in accordance with a predetermined work plan.

[0115] 5, an example in which an answer to a question is acquired via the text input unit 51 has been described, but the answer may also be acquired via voice, contact information on a touch panel, or an image. For example, when acquiring the answer via an image, an identified gesture can be used as the answer to the question based on the posture estimation result of the partner 17 in the image.

[0116] In addition, the judgment unit 33 may re-judge whether the progress of the first work has become possible to continue through the second work using the acquired specific information, and if it is not possible to continue, may repeat the question and answer work (request for specific information and acquisition of specific information in response to the request) until it becomes possible to continue.

[0117] For example, after the second task is performed using the acquired specific information, the judgment unit 33 may determine whether the progress of the first task, for which it has been determined that the first task cannot be continued, satisfies the start criteria, thereby determining whether the progress of the first task has become one that can be continued.

[0118] Furthermore, the plan creation unit 38 may recreate the work plan based on the acquired specific information. For example, the plan creation unit 38 may be configured to create a new work plan by modifying the content of the first work in accordance with the work instruction based on the acquired specific information.

[0119] In the robot control device 11 according to an embodiment of the present invention, the total work cost, which is the sum of the work costs of the robot 1 and the partner 17, is estimated based on the work content of the partner 17, and a work plan for returning to work is created so that the estimated total work cost is minimized. Therefore, a work plan can be created to return the first work of the robot 1, which has fallen into a state where it cannot be continued, to a state where it can be continued, taking into consideration the respective circumstances (work costs) of the robot 1 and the partner 17.

[0120] Therefore, according to the robot control device 11 of the embodiment of the present invention, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, a decrease in overall work efficiency can be suppressed.

[0121] [Configuration of robot control device 11 according to first modified example of the present invention] Next, the configuration of the robot control device 11 according to the first modified example of the present invention will be described with reference to Fig. 2. The robot control device 11 according to the embodiment of the present invention and the robot control device 11 according to the first modified example have a common basic configuration. Therefore, the following description will focus on the differences between the two, instead of describing the configuration of the robot control device 11 according to the first modified example.

[0122] In the robot control device 11 according to an embodiment of the present invention, if there is a bias in the work costs stored in advance in the memory unit 23b as work cost information, or if changes in the work cost information occur in response to changes in the work environment or the partner 17, there is a concern that a discrepancy will occur between the work cost estimated by the estimation unit 37 and the actual work cost.

[0123] Therefore, in the robot control device 11 according to the first variant, the estimation unit 37 estimates (in the first variant, this includes the concepts of measurement and estimation) the work cost before and after the work (including the first work and the second concept) according to the work plan, and updates the work cost information stored in the memory unit 23b based on the comparison results of the estimated work costs.

[0124] An example of updating work cost information is to obtain (including the concept of measurement) the time spent waiting for the partner 17 to complete the work and the time taken until the work can be continued, and then update the work cost information based on the obtained time.

[0125] Furthermore, the work cost information may be updated based on information about the workload of the partner 17 obtained by an interview during or after the dialogue with the partner 17. For example, if the work cost information indicates a probability distribution of the work cost associated with the work content of the partner 17, the obtained work cost may be used to calculate a posterior probability distribution of the work cost, and the work cost information may be updated using the obtained posterior probability distribution.

[0126] Furthermore, the task cost information (task execution probability) of the partner 17 may be updated as appropriate. For example, the task content of the partner 17 may be obtained (including the concept of recognition) through an interview during or after the dialogue with the partner 17, a posterior probability distribution may be calculated based on the obtained probability of each task, and the task cost information (task execution probability) of the partner 17 may be updated using the obtained posterior probability distribution.

[0127] In the robot control device 11 according to the first variant of the present invention, the estimation unit 37 is configured to estimate (in the first variant, this includes concepts of measurement or estimation) the work costs before and after the execution of work according to the work plan (including the first work and the second concept), and update the work cost information stored in the memory unit 23b based on the comparison results of the estimated work costs (before and after execution).

[0128] According to the robot control device 11 of the first variant of the present invention, the work cost information stored in the memory unit 23b is updated based on the comparison results of the work costs (before and after execution) estimated by the estimation unit 37. Therefore, even if the robot 1 falls into a situation where it is difficult to continue working in a scene where the robot 1 and its partner 17 are working together, in addition to the effect of suppressing a decrease in overall work efficiency, the effect of improving the accuracy of work cost estimation can be expected.

[0129] [Configuration and Effects of Robot Control Device 11 According to Second Modification of the Present Invention] Next, a robot control device 11 according to a second modification of the present invention will be described with reference to Fig. 2. The robot control device 11 according to the embodiment of the present invention and the robot control device 11 according to the second modification have a common basic configuration. Therefore, the following description will focus on the differences between the two, instead of describing the configuration of the robot control device 11 according to the second modification.

[0130] In the robot control device 11 according to the embodiment of the present invention, if the variance of the work costs estimated by the estimation unit 37 is large, there is a concern that a discrepancy will occur between the expected work cost and the actual work cost.

[0131] Therefore, in the robot control device 11 according to the second variant, the plan creation unit 38 creates a work plan that includes the task of requesting the partner to provide actual information that will help reduce the uncertainty of the work cost, in other words, improve the accuracy of estimating the work cost, and the task of acquiring the actual information provided by the partner based on the request.

[0132] The control unit 39 causes the output device 15 to perform an operation of requesting the partner 17 to provide actual status information that is useful for improving the accuracy of estimating the work cost, as an operation in accordance with the work plan created by the plan creation unit 38. The control unit 39 also causes the input device 13 to perform an operation of acquiring actual status information provided by the partner based on the request.

[0133] In addition, the control unit 39 may be configured to cause the robot 1 to perform operations related to the work by sending a control command to the robot 1 to cause the robot 1 to perform work in accordance with the work plan created by the plan creation unit 38 (the work of requesting the partner 17 to provide actual information, and the work of obtaining specific information provided based on the request).

[0134] Here, actual status information is a concept that includes all information that is useful for improving the accuracy of estimating the task cost. The actual status information includes, for example, a question asking whether the partner 17 is currently performing a task, a question asking about the content of the task that the partner 17 is performing, a question asking about the estimated time of completion of the task, confirmation of whether or not communication is possible, and answers to these questions.

[0135] For example, if the output device 15 is a speaker and the input device 13 is a microphone, a question inquiring about the estimated time for completion of the work, such as "When will the work be finished?", is output via the output device 15, and an answer to the question, such as "10 minutes left," is obtained via the input device 13.

[0136] Furthermore, the plan creation unit 38 may recreate a work plan based on the re-estimated work cost. For example, if there is a work in a series of work sequences that is not dependent on the work determined not to be continued, the plan creation unit 38 may recreate a work plan to execute this work during the re-estimated standby time of the robot 1.

[0137] The robot control device 11 according to the second modification of the present invention uses a novel concept of actual situation information that is useful for improving the accuracy of task cost estimation. In order to improve the accuracy of task cost estimation by the estimation unit 37, the plan creation unit 38 creates a task plan that includes an operation of requesting the partner 17 to provide actual situation information that is useful for improving the accuracy of task cost estimation, and an operation of acquiring the actual situation information provided by the partner 17 based on the request. The control unit 39 performs the operations of requesting the partner 17 to provide the actual situation information based on the task plan, and acquiring the actual situation information provided by the partner 17 based on the request. The acquisition unit 31 acquires the actual situation information. The estimation unit 37 is configured to re-estimate the task cost based on the acquired actual situation information.

[0138] According to the robot control device 11 of the second variant of the present invention, the estimation unit 37 re-estimates the work cost based on actual information that is useful for improving the accuracy of work cost estimation. Therefore, even if the robot 1 finds itself in a situation where it is difficult to continue working in a scene where the robot 1 and its partner 17 are working together, in addition to the effect of suppressing a decline in overall work efficiency, the effect of improving the accuracy of work cost estimation can be expected.

[0139] [Configuration and effects of robot control device 11 according to the present invention] A robot control device 11 according to a first aspect is a robot control device 11 that causes a robot 1 to perform a task in accordance with predetermined task instructions, and includes: an acquisition unit 31 that acquires the task instructions; a determination unit 33 that determines the progress status of a first task in accordance with the task instructions; a search unit 35 that searches for a partner 17 that can provide specific information for changing the progress status of the first task from "cannot continue" to "can continue" as a result of the determination by the determination unit 33; an estimation unit 37 that, when the partner 17 is extracted as a result of the search by the search unit 35, estimates a total task cost including a second task for changing the progress status of the first task from "cannot continue" to "can continue"; a plan creation unit 38 that creates a task plan including the second task for the partner 17 based on the total task cost estimated by the estimation unit 37; and a control unit 39 that causes a task including the second task to be performed in accordance with the task plan created by the plan creation unit 38. The plan creation unit 38 creates the work plan so as to minimize the total work cost.

[0140] According to the robot control device 11 based on the first aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to provide a robot control device 11 that can suppress a decrease in overall work efficiency.

[0141] The robot control device 11 based on the second aspect may be the robot control device 11 based on the first aspect, and may be configured such that the estimation unit 37 estimates the total work cost based on the work cost associated with the standby work of the robot 1 and the work cost associated with the workload of the partner 17.

[0142] According to the robot control device 11 based on the second perspective, the estimation unit 37 estimates the total work cost based on the work cost associated with the robot 1 waiting to work and the work cost associated with the workload of the partner 17. Therefore, in addition to the effects of the robot control device 11 based on the first perspective, collaborative work between the robot 1 and the partner 17 can be appropriately and smoothly promoted while taking into consideration the respective circumstances (work costs) of the robot 1 and the partner 17.

[0143] The robot control device 11 based on the third aspect may be the robot control device 11 based on the second aspect, and may be configured such that the acquisition unit 31 further acquires the specific information, and the plan creation unit 38 recreates a work plan for performing work based on the acquired specific information and work including the first work.

[0144] According to the robot control device 11 based on the third perspective, the plan creation unit 38 recreates a work plan for performing work based on the acquired specific information and work including the first work, and therefore, in addition to the effects of the robot control device 11 based on the second perspective, it is possible to promote the creation of an excellent work plan that takes into account work based on the specific information and work including the first work.

[0145] The robot control device 11 based on the fourth aspect may be the robot control device 11 based on the third aspect, and may be configured such that the determination unit 33 re-determines the progress status of the first task in accordance with the re-created task plan.

[0146] According to the robot control device 11 based on the fourth perspective, the judgment unit 33 re-evaluates the progress of the first work in accordance with the recreated work plan, so that in addition to the effects of the robot control device 11 based on the third perspective, it is possible to promote appropriate and smooth work taking into account the progress of the first work.

[0147] A robot control device 11 based on the fifth aspect may be a robot control device 11 based on the fourth aspect, and may be configured such that, if the result of the re-determination by the judgment unit 33 regarding the progress of the first task is that the progress of the first task cannot be continued, the plan creation unit 38 recreates a task plan including an operation of causing the search unit 35 to re-search for another partner 17 that is different from the partner 17 and can provide the specific information.

[0148] According to the robot control device 11 based on the fifth perspective, if the result of the re-evaluation by the judgment unit 33 regarding the progress of the first task is that the progress of the first task cannot be continued, the plan creation unit 38 recreates a work plan that includes an operation of having the search unit 35 re-search for another partner 17 that is different from the partner 17 and can provide the specific information.Therefore, compared to the effect of the robot control device 11 based on the fourth perspective, the effect of appropriate and smooth work promotion taking into account the progress of the first task can be further enhanced.

[0149] The robot control device 11 based on the sixth aspect may be the robot control device 11 based on the second aspect, further comprising a memory unit 23b that stores work cost information that associates a corresponding work cost with each work content of the partner 17, and the estimation unit 37 may acquire the work content of the partner 17 and estimate the work cost related to the workload of the partner 17 based on the acquired work content of the partner 17 and the work cost information.

[0150] According to the robot control device 11 based on the sixth perspective, the estimation unit 37 acquires the work content of the partner 17 and estimates the work cost related to the workload of the partner 17 based on the acquired work content of the partner 17 and the work cost information. Therefore, compared to the effect of the robot control device 11 based on the second perspective, the effect of promoting appropriate and smooth collaborative work that takes into consideration the circumstances of the partner 17 (work cost related to the workload) can be further enhanced.

[0151] The robot control device 11 according to the seventh aspect may be the robot control device 11 according to the sixth aspect, and may employ a configuration in which the task cost information is information in which a probability distribution of the corresponding task cost is associated with each task content of the partner.

[0152] According to the robot control device 11 based on the seventh aspect, it is possible to expect the effect of improving the accuracy of the estimation of the work cost by the estimation unit 37, compared to the effect of the robot control device 11 based on the sixth aspect.

[0153] The robot control device 11 according to the eighth aspect may be the robot control device 11 according to the sixth aspect, and may employ a configuration in which the task cost information is information in which a corresponding task cost is associated with each attribute of the partner 17.

[0154] According to the robot control device 11 based on the eighth aspect, similar to the effect of the robot control device 11 based on the sixth aspect, it is possible to expect the effect of improving the accuracy of the work cost estimation by the estimating unit 37.

[0155] A robot control device 11 according to a ninth aspect may be the robot control device 11 according to the sixth aspect, wherein the task cost information is information including schedule information of the partner 17.

[0156] According to the robot control device 11 based on the ninth aspect, similar to the effect of the robot control device 11 based on the sixth aspect, it is possible to expect the effect of improving the accuracy of the work cost estimation by the estimating unit 37.

[0157] A robot control device 11 based on the tenth aspect may be the robot control device 11 based on the ninth aspect, and the estimation unit 37 may be configured to estimate whether the partner 17 is currently performing work based on the work cost information including schedule information of the partner 17, and to estimate the work cost related to the workload of the partner 17 based on whether the partner 17 is currently performing work.

[0158] According to the robot control device 11 based on the tenth aspect, compared to the effect of the robot control device 11 based on the ninth aspect, it is expected that the accuracy of the work cost estimation by the estimation unit 37 will be further improved.

[0159] The robot control device 11 based on the eleventh aspect may be the robot control device 11 based on the seventh aspect, and may be configured such that the estimation unit 37 estimates the work costs before and after the work according to the work plan is performed, and updates the work cost information stored in the memory unit 23 b based on a comparison result of the estimated work costs.

[0160] According to the robot control device 11 based on the eleventh aspect, it is expected that the accuracy of the work cost estimation by the estimation unit 37 will be further improved compared to the effect of the robot control device 11 based on the seventh aspect.

[0161] A robot control device 11 based on the twelfth aspect may be the robot control device 11 based on the seventh aspect, and may be configured as follows: the plan creation unit 38 creates a work plan including the task of requesting the partner 17 to provide actual information useful for improving the estimation accuracy of the work cost, and the task of acquiring the actual information provided by the partner 17 based on the request; the control unit 39 performs the tasks of requesting the partner 17 to provide the actual information based on the work plan, and acquiring the actual information provided by the partner 17 based on the request; the acquisition unit 31 acquires the actual information; and the estimation unit 37 re-estimates the work cost based on the acquired actual information.

[0162] According to the robot control device 11 based on the twelfth aspect, the control unit 39 performs the tasks of requesting the partner 17 to provide the actual information based on the work plan, and acquiring the actual information provided by the partner 17 based on the request, and the estimation unit 37 re-estimates the work cost based on the acquired actual information, so that it is expected that the accuracy of the work cost estimation by the estimation unit 37 will be further improved compared to the effect of the robot control device 11 based on the seventh aspect.

[0163] A robot control system (10) according to a thirteenth aspect is a robot control system (10) comprising: a robot (1); and a robot control device (11) that causes the robot (1) to perform work in accordance with predetermined work instructions, wherein the robot control device (11) comprises: an acquisition unit (31) that acquires the work instructions; a determination unit (33) that determines a progress status of a first work in accordance with the work instructions; a search unit (35) that searches for a partner (17) that can provide specific information for changing the progress status of the first work from ``cannot be continued'' to ``can be continued'' as a result of the determination by the determination unit (33); an estimation unit (37) that, when the partner (17) is extracted as a result of the search by the search unit (35), estimates a total work cost including a second work for changing the progress status of the first work from ``cannot be continued'' to ``can be continued''; a plan creation unit (38) that creates a work plan including the second work for the partner (17) based on the total work cost estimated by the estimation unit (37); and a control unit (39) that causes the robot to perform work including the second work in accordance with the work plan created by the plan creation unit (38). The plan creation unit 38 creates the work plan so as to minimize the total work cost.

[0164] According to the robot control system 10 based on the thirteenth aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, it is possible to provide a robot control system 10 that can suppress a decrease in overall work efficiency.

[0165] A robot 1 based on the fourteenth aspect is a robot 1 that performs work in accordance with predetermined work instructions, and comprises: an acquisition unit 31 that acquires the work instructions; a determination unit 33 that determines a progress status of a first work in accordance with the work instructions; a search unit 35 that searches for a partner 17 that can provide specific information for changing the progress status of the first work from ``cannot be continued'' to ``cannot be continued'' as a result of the determination by the determination unit 33; an estimation unit 37 that, when the partner 17 is extracted as a result of the search by the search unit 35, estimates a total work cost including a second work for changing the progress status of the first work from ``cannot be continued'' to ``cannot be continued''; a plan creation unit 38 that creates a work plan including the second work for the partner 17 based on the total work cost estimated by the estimation unit 37; and a control unit 39 that causes work including the second work to be performed in accordance with the work plan created by the plan creation unit 38, and the plan creation unit 38 creates the work plan so that the total work cost is minimized.

[0166] According to the robot 1 based on the fourteenth aspect, in a scene where the robot 1 and its partner 17 are working together, even if the robot 1 finds itself in a situation where it is difficult to continue working, a decrease in overall work efficiency can be suppressed.

[0167] A robot control method based on the fifteenth aspect is a robot control method used when making a robot 1 perform work in accordance with predetermined work instructions, comprising: a step of acquiring the work instructions; a step of determining a progress status of a first work in accordance with the work instructions; a step of searching for a partner 17 that can provide specific information for changing the progress status of the first work from ``cannot be continued'' to ``can be continued'' as a result of the determination; a step of estimating a total work cost including a second work for changing the progress status of the first work from ``cannot be continued'' to ``can be continued'' if the partner 17 is extracted as a result of the search; a step of creating a work plan including the second work for the partner 17 based on the estimated total work cost; and a step of having the robot perform work including the second work in accordance with the created work plan, wherein in the step of creating the work plan, the work plan is created so that the total work cost is minimized.

[0168] According to the robot control method based on the fifteenth aspect, in a scene where robot 1 and partner 17 are working together, even if robot 1 finds itself in a situation where it is difficult to continue working, it is possible to suppress a decrease in overall work efficiency.

[0169] [Other Embodiments] The above-described embodiments are merely examples of realizing the present invention. Therefore, the technical scope of the present invention should not be interpreted as being limited by these embodiments. This is because the present invention can be embodied in various forms without departing from the spirit or main characteristics thereof.

[0170] Furthermore, it is possible to replace part of the configuration of the embodiments described herein with the configuration of other embodiments, and furthermore, it is possible to add the configuration of one embodiment to the configuration of another embodiment. It is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations. Furthermore, the control lines and information lines shown are those considered necessary for the explanation, and not all control lines and information lines are necessarily shown in the product. In reality, it can be assumed that almost all of the configurations are interconnected.

[0171] For example, the robot 1 according to the embodiment of the present invention may be, but is not limited to, an orthogonal robot. The robot 1 may also have a built-in robot control device 11. Furthermore, the robot 1 may also be a robot 1 in a fixed form.

[0172] Furthermore, in the description of the robot control device 11 according to the embodiment of the present invention, an example has been given in which the articulated arm unit 5 of the robot 1 is provided with a sensor 9 that monitors and measures the surroundings of the working environment of the robot 1, but the present invention is not limited to this example. The sensor 9 may be built into the robot 1.

[0173] Finally, in the robot control device 11 according to the embodiment of the present invention, some or all of the functional units, processing units, and processing means may be implemented in hardware, such as integrated circuits. The functional units, processing units, and processing means may also be implemented in software, with the processor 21 interpreting and executing programs that implement the respective functions. Information such as programs, tables, and files that implement the respective functions may be stored in a storage device, such as a memory, a hard disk, or an SSD (Solid State Drive), or on a storage medium, such as a flash memory card or a DVD (Digital Versatile Disk).

[0174] REFERENCE SIGNS LIST 1 Robot 3 Mobile cart unit 5 Articulated arm unit 7 Hand unit 9 Sensor 10 Robot control system 11 Robot control device 13 Input device 15 Output device 17 Partner 18 Communication unit 19 Portable terminal 20 Product inventory storage shelf 21 Processor 22a Bottle 23b Memory unit 31 Acquisition unit 33 Determination unit 35 Search unit 37 Estimation unit 38 Plan creation unit 39 Control unit

Claims

1. A robot control device that causes a robot to perform work in accordance with predetermined work instructions, comprising: an acquisition unit that acquires the work instructions; a determination unit that determines the progress status of a first work in accordance with the work instructions; a search unit that searches for a partner that can provide specific information to change the progress status of the first work from ``cannot be continued'' to ``cannot be continued'' as a result of the determination by the determination unit; an estimation unit that, when the partner is extracted as a result of the search by the search unit, estimates a total work cost including a second work to change the progress status of the first work from ``cannot be continued'' to ``cannot be continued''; a plan creation unit that creates a work plan including the second work for the partner based on the total work cost estimated by the estimation unit; and a control unit that causes a robot to perform work including the second work in accordance with the work plan created by the plan creation unit, wherein the plan creation unit creates the work plan so as to minimize the total work cost.

2. A robot control device according to claim 1, characterized in that the estimation unit estimates the total work cost based on the work cost associated with the robot's standby work and the work cost associated with the partner's workload.

3. A robot control device according to claim 2, wherein the acquisition unit further acquires the specific information, and the plan creation unit recreates a work plan for performing work based on the acquired specific information and work including the first work.

4. A robot control device according to claim 3, wherein the determination unit re-determines the progress of the first task in accordance with the recreated task plan.

5. A robot control device as described in claim 4, characterized in that, when the result of the re-determination by the determination unit regarding the progress status of the first task is that the progress status of the first task cannot be continued, the plan creation unit recreates a task plan that includes an operation of having the search unit re-search for another partner that is different from the partner and can provide the specific information.

6. A robot control device as described in claim 2, further comprising a memory unit that stores work cost information in which a corresponding work cost is associated with each of the work contents of the partner, and the estimation unit acquires the work contents of the partner and estimates the work cost related to the work load of the partner based on the acquired work contents of the partner and the work cost information.

7. A robot control device according to claim 6, wherein the task cost information is information in which a probability distribution of the corresponding task cost is associated with each task content of the partner.

8. A robot control device according to claim 6, wherein the task cost information is information that associates a corresponding task cost with each attribute of the partner.

9. A robot control device according to claim 6, wherein the task cost information includes schedule information of the partner.

10. A robot control device as described in claim 9, characterized in that the estimation unit estimates whether or not the partner is currently performing work based on the work cost information including the partner's schedule information, and estimates the work cost related to the partner's workload based on whether or not the partner is currently performing work.

11. A robot control device as described in claim 7, characterized in that the estimation unit estimates the work cost before and after the execution of work according to the work plan, and updates the work cost information stored in the memory unit based on the comparison result of the estimated work costs.

12. A robot control device as described in claim 7, wherein the plan creation unit creates a work plan including an operation of requesting the partner to provide actual information useful for improving the accuracy of the work cost estimation, and an operation of acquiring the actual information provided by the partner based on the request; the control unit performs the operations of requesting the partner to provide the actual information based on the work plan, and acquiring the actual information provided by the partner based on the request; the acquisition unit acquires the actual information; and the estimation unit re-estimates the work cost based on the acquired actual information.

13. A robot control system comprising a robot and the robot control device according to claim 1.

14. A robot that performs work according to predetermined work instructions, characterized in that it is equipped with the robot control device according to claim 1.

15. A robot control method used when making a robot perform work in accordance with specified work instructions, comprising: a step of acquiring the work instructions; a step of determining the progress status of a first work in accordance with the work instructions; a step of searching for a partner who can provide specific information for changing the progress status of the first work from ``cannot be continued'' to ``can be continued'' as a result of the determination; a step of estimating a total work cost including a second work for changing the progress status of the first work from ``cannot be continued'' to ``can be continued'' if the partner is extracted as a result of the search; a step of creating a work plan including the second work for the partner based on the estimated total work cost; and a step of having the robot perform work including the second work in accordance with the created work plan, wherein in the step of creating the work plan, the work plan is created so as to minimize the total work cost.

Images