Operation schedule generation device, method, program, and system

Abstract

To provide, in a system operating in a constant operation cycle, a device for generating an operation schedule capable of continuously operating a robot for performing an operation with a variation in required time.SOLUTION: An operation schedule generating device includes: an operating-state acquisition unit that acquires a current operating state of a robot; a recognition processing unit that performs a recognition process relating to a work target associated with a work; an estimated operation time generating unit that, based on a result of the recognition process, generates an estimated operation time for the robot for each work target; and an operation schedule generating unit that, according to an operation cycle, the operating state and the estimated operation time, generates an operation schedule that enables the robot to operate under the constant operation cycle.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an apparatus for generating an operation schedule for a robot that performs work within a system that operates in a fixed operation cycle. [Background technology]

[0002] 2. Description of the Related Art Robots that perform specific tasks are incorporated into systems that operate in a fixed operating cycle.

[0003] However, among the tasks performed by this type of robot, there are some tasks whose required time varies from cycle to cycle. For example, if the task is picking a workpiece, the required time for the task varies depending on the position, posture, and transport path of the workpiece to be picked up. If such tasks exist in the system, they may affect the operating cycle of the entire system.

[0004] As an example of a robot operating within a system, Patent Document 1 discloses a system in which multiple robots pick up workpieces transported on a transport conveyor. This document discloses a technology that determines whether the workpiece can be grasped based on the results of recognition processing by a visual sensor, and adjusts the transport speed of the transport conveyor accordingly.

[0005] That is, in this example, by determining whether a robot constituting part of the system can grasp a workpiece, the conveying speed, i.e., the conditions on the system side, is changed to ensure that the robot can grasp the workpiece. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2005-111607 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, for example, in the method described in Patent Document 1, it is necessary to change the operation cycle of other devices in the system according to the working time of the robot. As a result, for example, there is a risk that the design of the system will become complicated and production forecasting will become difficult.

[0008] In response to this, it is also assumed that the fluctuation in the required time is absorbed by the robot. In this case, the maximum required time for each robot operation is estimated, and the system is designed to make the operation cycle longer than that. However, such a design results in the robot being stopped during each operation cycle, which reduces work efficiency. Furthermore, if the operation cycle is set close to the maximum required time with priority given to work efficiency, even a small error may prevent the robot from completing the work within the operation cycle, which may result in system shutdown or reduced yield.

[0009] In other words, with the conventional methods, it has been difficult to continuously operate a robot that performs operations with fluctuating required times in a system that operates on a fixed operating cycle.

[0010] The present invention has been made in consideration of the above-mentioned technical background, and its purpose is to generate an operation schedule that enables the continuous operation of a robot that performs operations with fluctuating required times in a system that operates on a constant operating cycle. [Means for solving the problem]

[0011] The above-mentioned technical problems can be solved by an apparatus having the following configuration.

[0012] In other words, the operation schedule generation device of the present invention is an operation schedule generation device that generates an operation schedule for a robot that performs work in a system that operates in a fixed operation cycle, and is equipped with an operation status acquisition unit that acquires the current operation status of the robot, a recognition processing unit that performs recognition processing on a work object related to the work, an estimated operation time generation unit that generates an estimated operation time of the robot for each work object based on the result of the recognition processing, and an operation schedule generation unit that generates an operation schedule that enables the robot to operate under the fixed operation cycle in accordance with the operation cycle, the operation status, and the estimated operation time.

[0013] With this configuration, a robot operation schedule is generated based on the system's operation cycle, operating status, and estimated operation time, making it possible to continuously operate a robot that performs operations that vary in the amount of time required, without changing the operation cycle of the entire system or other devices within the system.

[0014] The operating cycle may be a cycle time obtained by dividing the production time in the system by the number of products produced in the system.

[0015] With this configuration, it is possible to generate an operation schedule that enables the robot to operate within a constant cycle time.

[0016] The operation schedule may be generated such that the completion time of each operation constituting the operation schedule falls once within each work cycle.

[0017] With this configuration, it is possible to ensure that each operation constituting the operation schedule is completed within each operating cycle.

[0018] The operation schedule may be generated so that the completion time of each operation constituting the operation schedule is included once within each operation cycle and is before a predetermined timing within each operation cycle.

[0019] This configuration ensures that each operation constituting the operation schedule is completed within each operation cycle, and also ensures a margin between the completion of an operation and the start of the next operation cycle, allowing the operations to continue more stably.

[0020] The operation schedule may be an operation schedule in which a work order for the work objects is determined according to an estimated operation time for each of the work objects.

[0021] According to this configuration, the work sequence for the work objects is determined according to the estimated operation time, so that the work sequence can be set appropriately to ensure continuous operation.

[0022] The operation schedule may be generated so that a margin of time from the completion time of each of the operations to an end of the operation cycle is equal to or greater than a predetermined value.

[0023] According to such a configuration, since the operations can be completed with ample time within each operation cycle, it is possible to provide an operation schedule that allows the operations to be continued more stably.

[0024] The system may further include an environment recognition processing unit that recognizes the surrounding environment of each of the work objects, and an interruption processing unit that interrupts execution of the operation schedule in response to a change in the surrounding environment.

[0025] According to such a configuration, when the environment changes significantly, the operation schedule being executed can be temporarily interrupted and, for example, rescheduled, thereby providing an operation schedule that allows the operation to continue more stably.

[0026] The operation schedule generating section may further generate the operation schedule based on an execution time of the recognition process.

[0027] With this configuration, an operation schedule can be generated taking into account the execution time of the recognition process, so that the work object can be recognized again during the operation schedule, and an operation schedule can be generated that allows operations to be continued more stably.

[0028] The operation schedule may include the recognition process being executed at the end of the operation schedule.

[0029] According to such a configuration, since the recognition process is performed at the end, it is possible to grasp the state of the work target object etc. at the time of completion of the operation, and thereby it is possible to immediately generate the next operation schedule etc.

[0030] An environmental change prediction processing unit predicts environmental changes that will occur when work is performed on each of the work objects based on the operation status and the results of the recognition processing, and the operation schedule generation unit may further generate the operation schedule in accordance with the prediction results of the environmental changes.

[0031] With this configuration, environmental changes resulting from work are predicted and an operation schedule is generated accordingly, making it possible to formulate an operation schedule that minimizes environmental changes, for example, and allows operations to continue more stably.

[0032] The system may further include a reliability generation unit that generates a reliability of the recognition processing for the work object, and the operation schedule generation unit may generate the operation schedule further based on the reliability.

[0033] With this configuration, it is possible to postpone work on work objects with low recognition accuracy, or to lower the priority of a task when there is not enough time to perform the task and perform the task when there is time to spare, thereby allowing the operation to continue more stably.

[0034] The recognition processing unit may perform recognition processing on a work object related to the work within a predetermined area.

[0035] According to this configuration, the recognition process is performed only on a predetermined area, so that the recognition process can be performed efficiently and quickly.

[0036] The region may be a region where there is a change from the recognition result of a previous recognition process.

[0037] For areas without any change, the previous recognition results can be used, so that highly efficient and high-speed recognition processing can be achieved.

[0038] The information processing device may further include a learning processing unit that performs a predetermined learning process based on the operation schedule or information related to the operation schedule, and a result of execution of the operation schedule.

[0039] According to such a configuration, since a learning process is performed based on the execution results, the operation schedule can be gradually improved.

[0040] The operation schedule generating section may further generate the operation schedule based on a result of the learning process.

[0041] According to such a configuration, since a learning process is performed based on the execution results, the operation schedule can be gradually improved.

[0042] The learning process may be a reinforcement learning process.

[0043] With this configuration, the operation schedule can be gradually improved by repeating trials.

[0044] The evaluation index in the reinforcement learning process may be an index related to stability when the robot is operated under a certain operation cycle.

[0045] With this configuration, reinforcement learning progresses in a way that increases the evaluation, so that it is possible to gradually generate an operation schedule that enables the robot to operate stably under a certain operating cycle.

[0046] The present invention can also be conceived as a method. That is, the method according to the present invention may be an operation schedule generation method for generating an operation schedule for a robot performing work in a system that operates in a fixed operation cycle, the operation schedule generation method including an operation status acquisition step for acquiring a current operation status of the robot, a recognition processing step for performing a recognition processing on a work object related to the work, an estimated operation time generation step for generating an estimated operation time of the robot for each work object based on a result of the recognition processing, and an operation schedule generation step for generating an operation schedule that enables the robot to operate under the fixed operation cycle in accordance with the operation cycle, the operation status, and the estimated operation time.

[0047] The present invention can also be conceived as a computer program. That is, the program according to the present invention is a motion schedule generation program for generating a motion schedule for a robot performing work in a system that operates in a fixed operation cycle, and includes an operation status acquisition step for acquiring a current operation status of the robot, a recognition processing step for performing a recognition process on a work object related to the work, an estimated operation time generation step for generating an estimated operation time of the robot for each work object based on a result of the recognition processing, and an operation schedule generation step for generating an operation schedule that enables the robot to operate under the fixed operation cycle in accordance with the operation cycle, the operation status, and the estimated operation time.

[0048] The present invention can also be conceived as a system. That is, the system according to the present invention is an operation schedule generation system that generates an operation schedule for a robot that performs work in a system that operates in a fixed operation cycle, and includes an operation status acquisition unit that acquires a current operation status of the robot, a recognition processing unit that performs recognition processing on a work object related to the work, an estimated operation time generation unit that generates an estimated operation time of the robot for each work object based on a result of the recognition processing, and an operation schedule generation unit that generates an operation schedule that enables the robot to operate under the fixed operation cycle in accordance with the operation cycle, the operation status, and the estimated operation time. Effect of the Invention

[0049] According to the present invention, it is possible to generate an operation schedule that enables continuous operation of a robot that performs operations with fluctuating required times in a system that operates on a fixed operating cycle. [Brief description of the drawings]

[0050] [Figure 1] FIG. 1 is an overall configuration diagram showing an overview of the system. [Diagram 2]FIG. 2 is a functional block diagram of an information processing device (first embodiment). [Diagram 3] FIG. 3 is a flowchart relating to the operation schedule generation process (first embodiment). [Figure 4] FIG. 4 is a conceptual diagram relating to an example of an operation schedule. [Diagram 5] FIG. 5 is an explanatory diagram of an example of arranging element actions having short required times. [Figure 6] FIG. 6 is an explanatory diagram of a configuration in which the start of each element operation is synchronized with the start of an operation cycle. [Figure 7] FIG. 7 is a functional block diagram of an information processing device (second embodiment). [Figure 8] FIG. 8 is a flowchart relating to the operation schedule generation process (second embodiment). [Figure 9] FIG. 9 is a functional block diagram of an information processing device (third embodiment). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0052] 1. First Embodiment As a first embodiment, an example in which the present invention is applied to a system 100 in a factory will be described. Note that, in this embodiment, work in a factory is illustrated, but the present invention is not limited to such a configuration. Therefore, for example, the present invention can be applied to any system, such as a system in a logistics warehouse.

[0053] (1.1 Configuration) 1 is an overall configuration diagram showing an overview of a system 100 according to this embodiment. As is clear from the figure, in a part of the system 100, a robot arm 1 executes an operation (task) of grasping (picking) an object (workpiece) 41 in a predetermined box 4, removing it, and placing it on a belt conveyor 9. At this time, the system 100 operates in a constant cycle, and the belt conveyor 9 also operates at a constant speed accordingly. This cycle may be, for example, a cycle time obtained by dividing the production time by the number of products produced in the system 100.

[0054] In this embodiment, although the robot arm 1 is used, a robot having other functions and uses may be used. Furthermore, the task is not limited to picking, and includes various other tasks.

[0055] The robot arm 1 is connected to a host control device 8 of the system 100 by wire or wirelessly. For example, a synchronization signal for synchronizing with a production line including a belt conveyor 9 is provided from the host control device 8. The host control device 8 is an information processing device including a control unit such as a CPU, a storage unit such as a ROM, RAM, or flash memory, a communication unit, an input / output unit, and the like.

[0056] In the example shown in the figure, the robot arm 1 is connected by wire or wirelessly to an information processing device 6 that controls the operation of the robot arm 1. The information processing device 6 is an information processing device that includes a control unit such as a CPU, a storage unit such as a ROM, a RAM or a flash memory, a communication unit, an input / output unit, etc., and generates, for example, an operation schedule, which will be described later. The robot arm 1 operates according to the generated operation schedule.

[0057] The information processing device 6 is connected to a three-dimensional sensor 5 capable of acquiring three-dimensional information, and acquires sensor information. The three-dimensional sensor 5 captures an image of the entire inside of the box 4, in which the objects 41 are stored in a disorderly manner, within its angle of view, and generates sensor information. The generated sensor information is transmitted to the information processing device 6.

[0058] The sensor may be any sensor capable of detecting the state of the object 41. Therefore, the sensor may be one that acquires two-dimensional information, or may not involve imaging. A configuration may also be provided in which multiple sensors are provided. Furthermore, the sensor does not need to be one that is fixed to the environment of the object 41, and may be, for example, a hand camera attached to the hand of the robot arm 1.

[0059] 2 is a functional block diagram of the information processing device 6 according to this embodiment. As is clear from the diagram, the information processing device 6 includes a recognition processing unit 61, an action generation unit 62, an action time estimation unit 63, an action schedule generation unit 65, an information acquisition unit 64, and an action execution unit 66.

[0060] The recognition processing unit performs recognition processing on the object 41 based on the sensor information acquired from the three-dimensional sensor 5. The recognition result is provided to the motion generation unit 62, which generates a motion of the robot arm 1 based on the recognition result. In this embodiment, this motion is a motion to grasp one of the objects 41 in the box 4 and place it on the belt conveyor 9.

[0061] The generated actions are provided to the action time estimation unit 63. The action time estimation unit 63 performs a process of estimating the time required for each generated action (estimated action time). The estimated action time is provided to the action schedule generation unit 65.

[0062] Furthermore, the information acquisition unit 64 acquires various information regarding the operation cycle of the entire system and the current state of the robot arm 1 (such as the operation currently being performed by the robot arm 1) based on a synchronization signal or the like provided from the upper control device 8. These pieces of information are also provided to the operation schedule generation unit 65.

[0063] The operation schedule generating unit 65 generates an optimal operation schedule based on information provided from the operation time estimating unit 63, the information acquiring unit 64, etc. The generated operation schedule is provided to the operation executing unit 66. The robot arm 1 operates based on the output from this operation executing unit.

[0064] The above-mentioned system and the configuration of the information processing device 6 are merely examples, and can be modified in various ways. For example, some functions can be executed by other devices to distribute the processing, or functions can be consolidated to be executed by one device.

[0065] (1.2 Operation) Next, the operation of the information processing device 6 that operates in the system 100 having the above-mentioned configuration will be described.

[0066] 3 is a flowchart relating to the operation schedule generation process executed on the information processing device 6. As is clear from the figure, when the process starts, the recognition processing unit 61 acquires 3D sensor information from the 3D sensor 5 capturing images of the entire inside of the box 4, in which the objects 41 are stored in a disorderly manner, within its angle of view (S10).

[0067] After acquiring this three-dimensional sensor information, the recognition processing unit 61 performs object recognition processing based on the three-dimensional sensor information (S12), and recognizes one or more objects 41 in the box 4.

[0068] The motion generation unit 62 performs a process of generating a motion for each object 41 based on the result of the recognition process (S14). That is, in this embodiment, a process of generating a time-series displacement of each joint from a predetermined posture of the robot arm 1 to picking up the object 41 and placing it on the belt conveyor 9 is performed.

[0069] Thereafter, the operation time estimation unit 63 generates an estimated operation time required for each operation based on the information about the hardware of the robot arm 1 and the generated operation for each target object 41 (S16). The estimated operation time may be, for example, a value obtained by dividing the transport distance by the moving speed. This estimated operation time is provided to the operation schedule generation unit 65.

[0070] The information acquisition unit 64 acquires from the upper control device 8 a synchronization signal related to the operation cycle of the entire system 100 or other devices in the system (such as the belt conveyor 9) and operation status information acquired from sensors provided at each joint of the robot arm 1, and provides these to the operation schedule generation unit 65 (S17). Note that the operation status information is not limited to the example of this embodiment, and includes any information that indicates the state of the robot arm 1.

[0071] The operation schedule generating unit, which has acquired the information from the operation time estimating unit 63 and the information acquiring unit 64, generates an operation schedule for the robot arm 1 that satisfies predetermined conditions based on that information (S18).

[0072] The operation execution unit 66 issues commands to the robot arm 1 based on the generated operation schedule (S20). The robot arm 1 operates according to these commands. When a series of processes is completed, the process returns to the process of acquiring 3D sensor information (S10), and thereafter, the process is repeated until an end command or interrupt command is issued.

[0073] The above-mentioned conditions regarding the operation schedule are conditions for ensuring continuous operation of the robot arm 1 performing work with variable required time in the system 100 that operates in a constant cycle, and various conditions may be set. For example, the condition regarding the operation schedule may be that the completion time of each element operation occurs once in each operation cycle.

[0074] FIG. 4 is a conceptual diagram of an example of an operation schedule generated by the operation schedule generating unit 65. In the upper part of the figure, multiple arrows of uniform length indicating a certain operation cycle are shown, and in the lower part of the figure, the generated operation schedule is shown. In the example of the figure, the operation schedule is composed of multiple element operations for picking up objects (works), and specifically, the time required for the operation of picking and placing an object 41 named A is shown by an arrow labeled "Work A", the time required for the operation of picking and placing an object 41 named B is shown by an arrow labeled "Work B", the time required for the operation of picking and placing an object 41 named C is shown by an arrow labeled "Work C", the time required for the operation of picking and placing an object 41 named D is shown by an arrow labeled "Work D", and the time required for recognition processing is shown by "Recognition".

[0075] In this embodiment, the recognition process is included in the elemental actions. However, the recognition process may be performed separately, either constantly or periodically, and may not be included in the elemental actions.

[0076] The element operations in the figure are arranged in an adjusted order so that an operation related to the target object 41 (workpiece) is performed only once during each operation cycle based on the required time for each element operation until a predetermined future time (workpiece A → workpiece B → workpiece C → workpiece D → recognition processing). That is, in this example, the predetermined condition is that the completion time of each element operation exists once within each operation cycle.

[0077] According to this configuration, an operation schedule for the robot arm 1 is generated based on the operation cycle of the system 100, the operating status of the robot arm 1, and the estimated operation time, so that the robot arm 1, which performs operations with fluctuating required times, can be continuously operated without changing the operation cycle of the entire system 100.

[0078] The conditions regarding the operation schedule are not limited to those in this embodiment, but can be modified in various ways.

[0079] For example, the specified condition may be that the completion time of each element operation occurs once within each operation cycle, and the lag time from the completion time of each operation to the end of each operation cycle is greater than or equal to a specified value.

[0080] This configuration ensures that each operation constituting the operation schedule is completed within each operation cycle, and also ensures a margin between the completion of an operation and the start of the next operation cycle, allowing the operations to continue more stably.

[0081] The specified condition may also be a condition that identifies the element action having the smallest required time among the element actions constituting the operation schedule, or one or more element actions having a required time equal to or less than a specified value, and places the element action immediately after an action whose slack time is equal to or less than the specified value.

[0082] FIG. 5 is an explanatory diagram of an example of arranging element actions having a shorter required time when the slack time is equal to or less than a predetermined value.

[0083] As is clear from the upper diagram of the figure, if an operation schedule is generated by determining the order under the condition that an operation related to the object 41 (work) is performed only once during each operation cycle, and if there is a work C that requires a short time, a waiting time will occur between the element operation for work B and the element operation for work C. This will reduce the efficiency of the system.

[0084] In contrast, in the diagram at the bottom of the same figure, the completion time of the element operation for the previous work E is close to the end of the operating cycle and there is little time to spare, so the element operation for work C is placed immediately after it.

[0085] With this configuration, it is possible to generate an operation schedule that makes up for delays. In addition, since the operation schedule can be made more efficient, it is possible to provide a robot arm 1 that can be operated in a shorter operation cycle (see the dashed arrow in the figure).

[0086] Furthermore, in this embodiment, a configuration is adopted in which there is no gap between each element operation, but the present invention is not limited to such a configuration.

[0087] 6 is an explanatory diagram of a configuration in which the start of each element operation is synchronized with the start of the operation cycle. As is clear from the figure, in this case, although there will be a stop time for the robot arm 1, it will be easier to design the operation schedule.

[0088] 2. Second Embodiment Next, a second embodiment of the present invention will be described.

[0089] (2.1 Configuration) The overall configuration of the system according to the second embodiment is substantially the same as the overall configuration of the system 100 according to the first embodiment (FIG. 1), and therefore a description thereof will be omitted.

[0090] 7 is a functional block diagram of the information processing device 6 according to this embodiment. As is clear from the figure, the information processing device 6 includes a recognition processing unit 610, an action generation unit 620, an action time estimation unit 630, an action schedule generation unit 650, an information acquisition unit 640, and an action execution unit 660, but in this embodiment, it also includes an evaluation information generation unit 670. The recognition processing unit 610 is further provided with a recognition area specification processing unit 611. Note that the configuration corresponding to the first embodiment performs approximately the same operation, so detailed description will be omitted below.

[0091] The evaluation information generation unit 670 generates evaluation information (described later) based on the result of the recognition processing, and provides it to the operation schedule generation unit 650. The evaluation information generation unit includes an environmental change prediction processing unit 671 that predicts an environmental change that occurs in the target object 41 due to the operation of the robot arm 1, and a reliability evaluation unit 672 that generates the reliability of the recognition result, etc.

[0092] Furthermore, the recognition area specification processing unit 611 specifies an area to be recognized and provides it to the recognition processing unit.

[0093] (2.2 Operation) Next, the operation of the information processing device 6 according to this embodiment will be described. Fig. 8 is a flowchart relating to the operation schedule generation process executed on the information processing device 6. As is clear from the figure, when the process starts, similarly to the first embodiment, the recognition processing unit 610 performs a process of acquiring 3D sensor information from the 3D sensor 5 (S20).

[0094] After this acquisition process, the recognition area specification processing unit 611 specifies the area to be recognized. For example, this specification process specifies as the recognition area only an area that has changed by a predetermined amount or more since the previous recognition process was performed. Note that the area specification method is not limited to this method, and various other methods known to those skilled in the art can be used.

[0095] According to this configuration, since the recognition process is not performed on areas that do not change, the recognition process can be performed efficiently and quickly.

[0096] After that, the recognition processing unit 610 performs an object recognition process in the recognition area (S21). After the object recognition process, similarly to the first embodiment, a process of generating a motion for each object 41 (S24) and a process of estimating a motion time for each object 41 are performed (S26).

[0097] The environmental change prediction processing unit 671 performs processing for predicting, for each object 41, a change that will occur in the environment around the object 41 due to the element operation (S27). For example, by performing a picking operation on a specific object 41, the environmental change prediction processing unit 671 predicts the change in state and its degree that will occur due to the surrounding objects 41 rolling or a pile of objects 41 collapsing. The prediction processing result is provided to the operation schedule generating unit 650.

[0098] Furthermore, the reliability evaluation unit 672 generates a reliability of the recognition result for each object 41 (S28). The reliability of the recognition result is a value corresponding to the accuracy of the recognition process. Note that the reliability can be generated using various methods known to those skilled in the art. For example, when the recognition process is performed using a trained model or the like, the reliability may be generated based on an index such as a prediction error amount. The generated reliability is provided to the operation schedule generation unit 650.

[0099] The information acquisition unit 64 acquires from the upper control device 8 a synchronization signal related to the operating cycle of the entire system 100 and information acquired from sensors installed at each joint of the robot arm 1, and provides the information to the operation schedule generation unit 650 (S29).

[0100] The operation schedule generation unit 650 acquires information from the operation time estimation unit 630, the information acquisition unit 640, and the evaluation information generation unit 670, and generates an operation schedule for the robot arm 1 that satisfies specified conditions based on that information (S30).

[0101] The operation execution unit 660 issues a command to the robot arm 1 based on the generated operation schedule (S31), similarly to the first embodiment. When a series of processes is completed, the process returns to the process of acquiring three-dimensional sensor information (S20), and thereafter, the process is repeated until an end command or an interrupt command is issued.

[0102] In this embodiment, the predetermined conditions for the operation schedule are that the completion time of each element operation exists once in each operation cycle, that objects 41 predicted to have small environmental fluctuations are preferentially picked, and that the priority of the picking operation for objects 41 with low reliability or below a predetermined value is lowered or no picking is performed. The operation schedule is determined so as to simultaneously satisfy these multiple conditions.

[0103] By setting such conditions, it is possible to reduce the risk of a large environmental change occurring due to the execution of an elemental motion, and therefore the motion can be continued more stably. Also, since the priority of an elemental motion for an object with a low recognition reliability is lowered or the elemental motion is not executed, the motion can be continued more stably.

[0104] The predetermined condition regarding the operation schedule can be modified in various ways, and may be, for example, a condition that an element operation is not performed on an object 41 whose environmental fluctuation is expected to be equal to or greater than a predetermined value.

[0105] 3. Third embodiment Next, a third embodiment will be described.

[0106] The overall configuration of the system according to the third embodiment is substantially the same as the overall configuration of the system 100 according to the first embodiment (FIG. 1), and therefore a description thereof will be omitted.

[0107] 9 is a functional block diagram of the information processing device 6 according to this embodiment. As is clear from the figure, the information processing device 6 includes a recognition processing unit 610, an action generation unit 620, an action time estimation unit 630, an action schedule generation unit 650, an information acquisition unit 640, an action execution unit 660, and an evaluation information generation unit 670, similar to the second embodiment. However, in this embodiment, the information processing device 6 further includes a learning processing unit 680. Note that the configurations corresponding to the second embodiment perform substantially the same operations, and therefore detailed description thereof will be omitted below.

[0108] In this embodiment, the operation is almost the same as in the second embodiment, but in this embodiment, the learning processing unit 680 further performs a machine learning process to learn so that the generated operation schedule brings about more continuity of the operation. The result of the learning process is used for generating the operation schedule in the operation schedule generating unit 650.

[0109] The learning method may be any of various known methods. For example, supervised learning or reinforcement learning may be used. In addition, a neural network (or deep learning) or any of various other known learning algorithms may be used.

[0110] In this embodiment, reinforcement learning is performed based on the determined operation schedule and the actual execution result as an evaluation index for the continuity of the operation. Examples of the evaluation index for the continuity of the operation include the number of recognized objects 41, the estimated operation time, and the leeway time. In this case, the greater the number of recognized objects 41 or the leeway time, or the shorter the estimated operation time, the higher the reward is given as the continuity of the operation is high. In addition, the input data is, for example, various information provided to the operation schedule generating unit 650. In this way, the policy for formulating the operation schedule, that is, the policy for the arrangement of the element operations, is learned. After learning, the learned model is used in the operation schedule generating unit 650, and an appropriate operation schedule is generated according to the predetermined input data.

[0111] With this configuration, reinforcement learning progresses to increase the evaluation index, so that it is possible to gradually generate an operation schedule that enables the robot to operate stably under a certain operating cycle.

[0112] (2. Modifications) In the above embodiment, one target object 41 (workpiece) is targeted for each elemental motion, but the present invention is not limited to such a configuration. Therefore, for example, each elemental motion may target multiple workpieces.

[0113] In the above embodiment, after the operation schedule is generated, the robot arm 1 is continuously operated according to the operation schedule, but the present invention is not limited to such a configuration. Therefore, for example, the operation schedule may be interrupted when a predetermined event occurs, and the recognition process may be performed again to generate the operation schedule. The predetermined event may be, for example, a case where the environmental information acquired by a three-dimensional sensor or the like changes by a predetermined amount or more.

[0114] Although the embodiments of the present invention have been described above, the above embodiments merely show some of the application examples of the present invention, and the technical scope of the present invention is not limited to the specific configurations of the above embodiments. Moreover, the above embodiments can be appropriately combined within a range where no contradiction occurs.

[0115] Furthermore, the order of the steps in the flowcharts shown in this specification may be interchanged as long as no technical contradiction occurs. [Industrial Applicability]

[0116] The present invention can be used in industries that manufacture systems including at least robots. [Explanation of symbols]

[0117] 1. Robot Arm 4 boxes 41 Object 5 3D Sensor 6. Information processing equipment 61 Recognition processing section 62 Motion generator 63 Operation time estimation unit 64 Information Acquisition Department 65 Operation schedule generation unit 66 Operation Execution Unit 8. Upper control device 9. Conveyor Belt 100 Systems

Claims

1. An operation schedule generating device that generates an operation schedule for a robot that performs work in a system that operates in a constant operation cycle, comprising: An operation status acquisition unit that acquires a current operation status of the robot; A recognition processing unit that performs a recognition process on a work target related to the work; an estimated motion time generating unit that generates an estimated motion time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating unit that generates an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time, The operation schedule generating device, wherein the operation schedule is generated so that a margin of time from the completion time of each of the operations to an end point of the operation cycle is equal to or greater than a predetermined value.

2. 2. The operation schedule generating device according to claim 1, wherein the operation cycle is a cycle time obtained by dividing a production time in the system by a production number of products produced in the system.

3. The operation schedule generating device according to claim 1 , wherein the operation schedule is generated such that a completion time of each operation constituting the operation schedule is included once within each operation cycle.

4. 4. The operation schedule generating device according to claim 3, wherein the operation schedule is generated such that the completion time of each operation constituting the operation schedule is included once within each operation cycle and is prior to a predetermined timing within each operation cycle.

5. The operation schedule generating device according to claim 1 , wherein the operation schedule is an operation schedule in which a work order for the work objects is determined according to an estimated operation time for each of the work objects.

6. an environment recognition processing unit that recognizes a surrounding environment of each of the work objects; The operation schedule generating device according to claim 1 , further comprising: an interruption processing unit that interrupts execution of the operation schedule in response to a change in the surrounding environment.

7. An operation schedule generation device that generates an operation schedule for a robot that performs work in a system that operates in a constant operating cycle, comprising: An operation status acquisition unit that acquires a current operation status of the robot; A recognition processing unit that performs a recognition process on a work target related to the work; an estimated motion time generating unit that generates an estimated motion time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating unit that generates an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time, The operation schedule generating device, wherein the operation schedule generating unit further generates the operation schedule based on an execution time of the recognition process.

8. The operation schedule generating device according to claim 7 , wherein the operation schedule includes the recognition process executed at an end of the operation schedule.

9. An operation schedule generation device that generates an operation schedule for a robot that performs work in a system that operates in a constant operating cycle, comprising: An operation status acquisition unit that acquires a current operation status of the robot; A recognition processing unit that performs a recognition process on a work target related to the work; an estimated motion time generating unit that generates an estimated motion time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating unit that generates an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time; an environmental change prediction processing unit that predicts an environmental change when work is performed on each of the work objects based on the operation status and a result of the recognition processing, The operation schedule generating device, wherein the operation schedule generating unit further generates the operation schedule according to a prediction result of the environmental variation.

10. An operation schedule generating device that generates an operation schedule for a robot that performs work in a system that operates in a constant operating cycle, comprising: An operation status acquisition unit that acquires a current operation status of the robot; A recognition processing unit that performs a recognition process on a work target related to the work; an estimated motion time generating unit that generates an estimated motion time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating unit that generates an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time, A reliability generating unit is further provided to generate a reliability of the recognition process for the work object, The operation schedule generating device, wherein the operation schedule generating unit generates the operation schedule further based on the reliability.

11. The operation schedule generating device according to claim 1 , wherein the recognition processing unit performs recognition processing on a work target related to the work within a predetermined area.

12. The operation schedule generating device according to claim 11 , wherein the region is a region in which a change has occurred since the recognition result of a previous recognition process.

13. The operation schedule generating device according to claim 1 , further comprising a learning processing unit that performs a predetermined learning process based on the operation schedule or information related to the operation schedule, and an execution result of the operation schedule.

14. The operation schedule generating device according to claim 13 , wherein the operation schedule generating unit further generates the operation schedule based on a result of the learning process.

15. The operation schedule generating device according to claim 13 , wherein the learning process is a reinforcement learning process.

16. 16. The operation schedule generating device according to claim 15, wherein the evaluation index in the reinforcement learning process is an index related to stability when the robot is operated under a certain operation cycle.

17. 1. An operation schedule generation method for generating an operation schedule for a robot that performs work in a system that operates in a constant operation cycle, comprising the steps of: An operation status acquisition step of acquiring a current operation status of the robot; a recognition processing step of performing a recognition process on a work object related to the work; an estimated operation time generating step of generating an estimated operation time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating step of generating an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time, The operation schedule generating method, wherein the operation schedule is generated so that a margin of time from the completion time of each operation to an end point of the operation cycle is equal to or greater than a predetermined value.

18. An operation schedule generation program for generating an operation schedule for a robot that performs work in a system that operates in a constant operation cycle, comprising: An operation status acquisition step of acquiring a current operation status of the robot; a recognition processing step of performing a recognition process on a work object related to the work; an estimated operation time generating step of generating an estimated operation time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating step of generating an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time, an operation schedule generating program, the operation schedule being generated so that a margin of time from the completion time of each operation to an end point of the operation cycle is equal to or greater than a predetermined value;

19. An operation schedule generation system that generates an operation schedule for a robot that performs work in a system that operates in a constant operating cycle, comprising: An operation status acquisition unit that acquires a current operation status of the robot; A recognition processing unit that performs a recognition process on a work target related to the work; an estimated motion time generating unit that generates an estimated motion time of the robot for each of the work objects based on a result of the recognition processing; an operation schedule generating unit that generates an operation schedule that enables the robot to operate under the certain operation cycle according to the operation cycle, the operation status, and the estimated operation time, An operation schedule generating system, wherein the operation schedule is generated so that a margin of time from the completion time of each operation to an end point of the operation cycle is equal to or greater than a predetermined value.

Images