TEACHING SUPPORT DEVICE, WORKING SYSTEM, TEACHING SUPPORT METHOD AND STORAGE MEDIUM

Abstract

A teaching support device (1) according to the present disclosure includes a robot motion calculation unit (12) that generates a motion path of a robot (4) based on a task description entered by a user and an environment model. The environment model, which is a three-dimensional model, is a modeled environment of the robot (4). The motion path enables the robot (4) to avoid obstructing the environment.

Description

Area

[0001] The present disclosure relates to a teaching support apparatus, a work system, a teaching support method, and a teaching support program for assisting in teaching a work machine. background

[0002] To perform work using a work machine, including an industrial robot, the work machine is taught position information to perform a desired operation, and a robot operation program is created based on the adjustment of teaching points, that is, the taught position information. This adjustment process is performed by a system integration engineer. Such an adjustment process is sometimes performed by a user. Accordingly, it is desirable that the adjustment process can be easily performed even by a person unfamiliar with the programming language. Patent Literature 1 discloses a programming support device that supports the creation of an operation program for a robot.The programming support device described in Patent Literature 1 sets environmental conditions that specify the operating environment of the robot according to an input to a user interface. According to the input to the user interface, the programming support device sets a plurality of work tasks to be executed by the robot. In an execution flow that defines an execution order of the set plurality of work tasks, the programming support device determines whether the work tasks meet the environmental conditions based on this execution order. List of citationsPatent literature

[0003] Patent literature 1: WO 2018 / 194094 A Brief descriptionTechnical problem

[0004] The programming support device described in Patent Literature 1 requires a user to use a user interface not only to set a teaching point and a constraint condition regarding the robot's movement to perform a task job by job, but also to set the sequence of the jobs. This setting process requires time for the user, which requires time for the user to start up the system.

[0005] The present disclosure has been made in view of the foregoing, and an object of the present disclosure is to provide a teaching support device capable of reducing the time required for the user to put a system into operation. Solution to the problem

[0006] To solve the above problem and achieve the object, a teaching support device according to the present disclosure includes: a movement calculation unit for generating a movement path of a work machine based on a task description and an environment model, the movement path enabling the work machine to avoid interference with an environment of the work machine, the task description being input by a user, the environment model being a three-dimensional model obtained by modeling the environment of the work machine. Advantageous effects of the invention

[0007] A teaching support device according to the present disclosure provides an advantage in that the time required for the user to operate the system is reduced. Short description of the drawings Fig. 1 is a diagram illustrating an exemplary configuration of a work system according to a first embodiment. Fig. 2 is a flowchart illustrating an example of a procedure for processing performed in the teaching support device of the first embodiment. Fig. 3 is a diagram schematically illustrating an example of a method for obtaining environmental measurement data in the first embodiment. Fig. 4 is a diagram illustrating an example of a task selection screen of the first embodiment. Fig. 5 is a diagram illustrating an example of information to be received as a task description by an input unit of the first embodiment. Fig. 6 is a diagram illustrating exemplary configurations of a task setting unit and a robot motion calculation unit of the first embodiment. Fig. 7 is a diagram illustrating an example of a cross-step overall operation calculation of the first embodiment. Fig. 8 is a diagram illustrating an example of paths generated by a step-by-step operation calculation and paths generated by a cross-step overall operation calculation. Fig. 9 is a diagram illustrating an exemplary configuration of a computer system implementing the teaching support device of the first embodiment. Fig. 10 is a diagram illustrating an exemplary configuration of a teaching support device according to a second embodiment. Fig. 11 is a diagram illustrating an example of a display screen of the second embodiment. Fig. 12 is a diagram illustrating an exemplary configuration of a work system according to a third embodiment. Fig. 13 is a flowchart illustrating an example of a procedure for processing performed in a teaching support device of the third embodiment. Fig. 14 is a diagram illustrating an exemplary configuration of a work system according to a fourth embodiment. Fig. 15 is a diagram illustrating an exemplary configuration of a work system according to a fifth embodiment. Description of embodiments

[0008] A teaching support apparatus, a work system, a teaching support method, and a teaching support program according to embodiments will be described in detail below with reference to the drawings. First embodiment.

[0009] Fig. 1 is a diagram illustrating an exemplary configuration of a work system according to a first embodiment. A work system 6 of the present embodiment includes a teaching support device 1, an image sensor 2, a robot control device 3, a robot 4, and a sensor 5. The work system 6 is a system that performs tasks using the robot 4. The robot 4 is an example of a work machine and may be an industrial robot that performs, for example, packing, assembling, processing, transportation, packaging, and / or the like, or may be a service robot for assisting humans. Furthermore, the robot 4 may be an outdoor robot that performs farm work, construction work, or the like, or may be an indoor robot.Furthermore, robot 4 is, for example, among other things, a multi-axis vertical articulated robot, but may be a linear robot or a robot that provides a combination of linear motion and rotary motion. There are no restrictions regarding the type of robot. An example is described below in which robot 4 is, for example, an industrial multi-axis vertical articulated robot.

[0010] The robot control device 3 receives a robot movement command from the teaching support device 1. The robot movement command is a command related to a movement of the robot. The robot control device 3 calculates the control amount for controlling a motor of the robot so that the robot 4 follows the received robot movement command. The robot control device 3 outputs the calculated control amount to the robot 4, thereby controlling a movement of the robot 4. The robot control device 3 can modify a movement of the robot 4 using a detection result from the sensor 5. The sensor 5, which is a sensor for detecting the position, attitude, etc., of the target object of a task of the robot 4, is, for example, a two-dimensional or three-dimensional vision sensor.The robot control device 3 controls the movement of the robot 4 by driving a motor (not illustrated) of each shaft in the robot 4. The robot control device 3 can provide control using a common method, and therefore, a detailed description thereof will be omitted. Although illustrated, . Fig. 1 illustrates the robot control device 3 and the robot 4 as separate components, however, the robot control device 3 may be integrated into the robot. Furthermore, the sensor 5 may be provided at a position other than the robot 4 or may be provided on the robot 4.

[0011] The image sensor 2 is a sensor for capturing a three-dimensional image. The image sensor 2 acquires environmental measurement data by measuring an environment around the robot 4. The image sensor 2 acquires the environmental measurement data by imaging the environment. The environment around the robot 4 is, for example, an object, a structure, etc., that exists within a range that includes the movement range of the robot 4. The image sensor 2 is, for example, an RGB depth (D) sensor capable of acquiring a three-dimensional image by acquiring a depth and a color.

[0012] Based on the environmental measurement data from the image sensor 2 and a task description input by the user, the teaching support device 1 automatically generates an operation program corresponding to a task that the robot 4 must perform. The teaching support device 1 generates a robot movement command using the operation program and sends the generated robot movement command to the robot control device 3. In the present embodiment, the teaching support device 1 receives input of the task description itself, for example, natural speech and a selection of videos showing tasks, thereby allowing even a user unfamiliar with programming to easily specify the task description.

[0013] The teaching support device 1 is, for example, a mobile terminal device such as a tablet device, a smartphone, or a personal computer, among others. The teaching support device 1 includes a modeling unit 11, a robot motion calculation unit 12, a task information storage unit 13, a display unit 14, an input unit 15, and a task setting unit 16.

[0014] The modeling unit 11 receives environmental measurement data from the image sensor 2 to thereby acquire the environmental measurement data, generates an environmental model using the acquired environmental measurement data, and outputs the generated environmental model to the display unit 14 and the robot motion calculation unit 12. The environmental model is a three-dimensional model obtained by modeling the environment. The modeling unit 11 generates the environmental model, for example, through three-dimensional restructuring based on environmental measurement data, which is point cloud data. Three-dimensional restructuring is a process in which multiple pieces of three-dimensional data are combined.

[0015] The input unit 15 receives input from the user. For example, the input unit 15 receives input of a task description from the user and outputs the received task description (i.e., information reflecting the task description) to the display unit 14 and the task setting unit 16. The input unit 15 may receive a user's speech via voice and / or may receive a result of a selection made by the user from options displayed on the display unit 14. In the case where the input unit 15 is to receive voice input, the input unit 15 includes a microphone and a speech recognition unit for performing speech recognition processing (each not illustrated).Note that the microphone may be provided separately from the teaching support device 1 to input the voice picked up by the microphone into the input unit 15. In the case where the input unit 15 is to receive a selection result made by the user from options displayed on the display unit 14, the options may be text information representing candidates for the task description, or videos or images. The input unit 15 receives a selection result through an operation such as tapping, a keyboard operation, or a mouse operation. Alternatively, the input unit 15 may receive input of the task description or a selection result by recognizing the image of a gesture or eye contact.In the case where the input unit 15 is to receive an input using a gesture or eye contact, the input unit 15 may have a function for imaging the gesture or eye contact, or the image captured by imaging means separate from the input unit 15 may be input to the input unit 15.

[0016] The display unit 14 displays various types of information, images, and videos. For example, the display unit 14 displays the task description received from the input unit 15. The display unit 14 also displays a movement of the robot through augmented reality (AR) representation based on the environment model received from the modeling unit 11 and a robot model, and a robot movement command (or information representing the movement of the robot) received from the robot movement calculation unit 12. Note that the robot model can be stored in a model storage unit (not illustrated) of the teaching support device 1 and read from the model storage unit by the display unit 14. Likewise, the modeling unit 11 can store the environment model in the model storage unit, and the display unit 14 can read the environment model from the model storage unit.Furthermore, the model storage unit may be provided outside the teaching support device 1. Furthermore, when supporting the use of multiple robot models, the model storage unit may store multiple robot models, the robot motion calculation unit 12 or the task setting unit 16 may specify the robot model to be used, and the display unit 14 may read the specified robot model from the model storage unit.

[0017] Note that the display unit 14 may be implemented using a hardware element capable of receiving input, such as a touch panel. For example, the display unit 14 may include a touch panel and a display data generation unit for providing an AR representation, in which case the touch panel includes part of the functionality of the input unit 15. For example, input from a user may be received when it is detected that a user taps an element, such as an image or text, displayed on the touch panel.

[0018] The task setting unit 16 converts the task description received from the input unit 15 into a programming language and outputs task setting information, including the conversion result, to the robot motion calculation unit 12. Note that the task information storage unit 13 stores, for example, a basic format of each task in a task database (hereinafter referred to as a task DB). The basic format represents a motion of the robot 4 corresponding to each task. The basic format, which defines a main part of the operation program corresponding to the task, includes information representing the motion corresponding to the operation program in natural language or through a video.When multiple basic formats for tasks are prepared and stored in the task DB of the task information storage unit 13, the task setting unit 16 causes the display unit 14 to display information representing the basic formats. The information representing the multiple basic formats may be information representing the movements of the tasks as text, or may be identification information corresponding to each basic format. In addition, a video corresponding to each basic format may be displayed on the display unit 14. Furthermore, the task setting unit 16 may cause the display unit 14 to display an explanatory video, an illustrative diagram, a descriptive text, etc., format by format.For example, the display unit 14 displays the information representing the basic format in the form of a pull-down menu to allow the input unit 15 to receive input representing the selection result. The input unit 15 outputs the selection result to the task setting unit 16. Note that the method for displaying the information representing the basic format is not limited to this example. The task setting unit 16 acquires a task description corresponding to the basic format via the input unit 15 using the selection result of the basic format, recognizes the task description based on the acquired input, and converts the recognized task description into a programming language.

[0019] The basic format describes the outline of a typical task and is generated for each task, such as a packing task, an assembly task, and a packaging task. In the case of a packing task, the robot 4 performs a task in which a target object is grasped and the target object is placed in a designated space, and the basic format is therefore information that represents, for example, the process "Grab X, carry X to Y, and place X in Y," where X is the target object and Y is the location where the target object is to be placed. X and Y are entered by the user to determine the task description. The basic format can also include the number of objects to be placed, such as "Grab X, carry X to Y, and place X in Y."

[0020] The task information storage unit 13 stores, in the task DB, task information associated with the tasks to be performed by the robot 4. The task information includes, for example, the basic format corresponding to each of the above tasks and a restriction condition for each basic format.

[0021] The robot motion calculation unit 12 is a motion calculation unit that generates a motion path of the robot 4 that enables the robot 4 to avoid interference with the environment based on the task description input by the user and the environment model, which is a three-dimensional model obtained by modeling the environment of the work machine. Specifically, the robot motion calculation unit 12 determines a path for the robot 4 (i.e., a robot motion path) to perform a task in accordance with the task description such that the robot 4 does not interfere with the environment using the task setting information received from the task setting unit 16, the environment model received from the modeling unit 11, and the detection result from the sensor 5. Fig. 1, the robot motion calculation unit 12 acquires a detection result of the sensor 5 via the robot control device 3, however, the robot motion calculation unit 12 may acquire the detection result of the sensor 5 from the sensor 5. The present embodiment is based on the assumption that, as described later, the path of the robot 4 is determined to organize a task involving multiple steps into a single operation. The robot motion calculation unit 12 generates a robot motion command, which is a movement command for the robot 4, based on the determined path and sends the generated robot motion command to the robot control device 3.

[0022] Next, an operation in the present embodiment will be described. Fig. 2 is a flowchart illustrating an example of a procedure for processing performed in the teaching support device 1 of the present embodiment. As shown in Fig. As illustrated in Figure 2, the teaching support device 1 determines whether an environment model exists (step S1). Specifically, the modeling unit 11 determines whether an environment model has already been created.

[0023] If no environment model is present (No in step S1), the teaching support device 1 generates an environment model (step S2) and the processing proceeds to step S3. Specifically, in step S2, the modeling unit 11 generates an environment model, for example, by three-dimensional restructuring, using the environment measurement data acquired by the image sensor 2.

[0024] Fig. 3 is a diagram schematically illustrating an example of a method for obtaining environmental measurement data in the present embodiment. Fig. Figure 3 illustrates an example of how the robot 4 is caused to perform a task of “packing fried chicken pieces in a food tray.” As shown in Fig. 3, the user holding the teaching support device 1 moves around the robot 4. In the Fig. In the example illustrated in Figure 3, the image sensor 2 is attached to the teaching support device 1, and the user operates the image sensor 2 via the teaching support device 1 or directly operates the image sensor 2 to image the surroundings of the robot 4 from multiple positions. This acquires multiple three-dimensional images captured from multiple positions. The modeling unit 11 performs three-dimensional restructuring by combining these multiple three-dimensional images to generate an environment model, which is a three-dimensional model of the environment. The environment model is represented by, for example, a mesh, an oriented bounding box (OBB), a polygon, or the like, among others, but the environment model may be represented in any desired shape.

[0025] Three-dimensional restructuring is a technique for extracting feature points and planes from multiple partially captured images of environmental measurement data and combining these feature points and planes to generate an overall three-dimensional model. Note that the following describes an example in which a three-dimensional model is generated using environmental measurement data obtained by image sensor 2, which is an RGB-D sensor. However, the three-dimensional model can be generated using multiple images obtained by image sensor 2, which is a two-dimensional sensor.

[0026] In the Fig. In the example illustrated in Figure 3, the robot 4 performs a task of packing fried chicken pieces 32, which are accumulated in a food container 31, into a target partition 34. The target partition 34, which is one of several partitions of a food tray 33, is the partition in which the fried chicken pieces 32 should be placed. When the robot 4 performs this operation, the robot 4 must recognize not only the fried chicken pieces 32, i.e., the target objects, to be moved, but also the surrounding environment, i.e., the surrounding objects such as the food tray 33, the food container 31, and a work table (not illustrated) on which the food container 31 is placed. For example, in the teaching support device 1 of the present embodiment, the user takes images of the surrounding environment with the image sensor 2, as shown in Figure 3. Fig. 3, to automatically generate the environment model.

[0027] It should be noted that in the Fig. In the example illustrated in FIG. 3, the sensor 5 for detecting the position and attitude of the fried chicken 32, ie, the target objects, is provided separately from the robot 4, but is not limited thereto. The sensor 5 may be included in the robot 4 or may be attached to the robot 4.

[0028] The image sensor 2 in the Fig. The example illustrated in Figure 3 is attached to the teaching support device 1, but may be provided separately from the teaching support device 1 such that the user holding the image sensor 2 can image the environment. The teaching support device 1 may acquire the environmental measurement data through wired or wireless communication with the image sensor 2, or the image sensor 2 may store the environmental measurement data in a recording medium such that the teaching support device 1 reads the environmental measurement data from the recording medium.

[0029] Moreover, without referring to the Fig. 3, the image sensor 2 is mounted, for example, at a front end, etc., of the robot 4, and the robot 4 is operated such that the image sensor 2 captures an image to thereby acquire the environmental measurement data. Furthermore, when the sensor 5 is also usable as the image sensor 2, the sensor 5 is mounted at the front end, etc., of the robot 4, and the robot 4 is operated such that the image sensor 2 captures an image to thereby acquire the environmental measurement data.

[0030] It will be repeated again Fig. 2. If an environment model exists (Yes at step S1), the teaching support device 1 proceeds to step S3. At step S3, the teaching support device 1 receives a result of task selection. More specifically, the task setting unit 16 causes the display unit 14 to display the basic format of each task stored in the task DB of the task information storage unit 13 (i.e., the basic format of each task type), and the input unit 15 receives an input of a selection result made by the user and outputs the received selection result to the task setting unit 16. Note that, when the teaching support device 1 supports only a single type of task, no processing of step S3 needs to be performed.

[0031] As described above, the tasks can be presented as a pull-down menu for selection. Alternatively, a selection screen prompting the user to select a task can be displayed in such a way that the user selects the task from that screen. Fig. Fig. 4 is a diagram illustrating an example of a task selection screen of the present embodiment. In the Fig. In the example illustrated in Figure 4, buttons 21, 22, and 23 are displayed, corresponding to individual tasks, and the user presses a corresponding button to select a task. Note that the Fig. 4 is merely an example, and the content of the task selection screen and the selection process is not limited to this example.

[0032] Next, the teaching support device 1 receives an instruction about the details of the selected task (step S4). Specifically, the task setting unit 16 outputs the basic format corresponding to the result of the selection made in step S3 to the input unit 15. The input unit 15 then receives input from the user, determines the details of the selected task (i.e., a task description) based on the input information and the basic format, and outputs the determined task description to the display unit 14 and to the task setting unit 16.

[0033] The input unit 15 receives the input of the task description via voice, tapping or the like, as described above. Fig. Fig. 5 is a diagram illustrating an example of information to be received as a task description by the input unit 15 of the present embodiment. In the Fig. In the example illustrated in Figure 5, task A is selected, and the operation "Grab X, carry X to Y, and place Z piece(s) in Y" is predetermined as the basic format of task A. In this case, when the user provides a task description, for example, by saying "Place fried chicken pieces in the food tray," the input unit 15 recognizes the captured speech through a speech recognition technique using the basic format. The speech recognition technique may be a general-purpose speech recognition technique or may be processing specifically directed at recognizing a description of a task performed by the robot 4, such as a speech recognition technique using a learned model obtained through machine learning specifically directed at descriptions of tasks using the robot 4.

[0034] If the speech recognition result indicates that one or some of the information X, Y, and Z is not input or recognized, the input unit 15 causes the display unit 14 to display information prompting the user to enter a detail of the information not input or recognized, so that the input unit 15 receives input of the missing information. For example, speech recognition of the voice tone "Place fried chicken pieces in the food tray" allows the input unit 15 to recognize that X is "fried chicken pieces" and Y is "food tray," in which case the information Z is missing. The input unit 15 therefore causes the display unit 14 to display a question inquiring about the number of fried chicken pieces. Note that a loudspeaker is used instead of the display unit 14 to ask the user a spoken question.Alternatively, both the display and the language can be provided to the user. Furthermore, if the food tray is divided into multiple partitions, the user is asked which partition of the "food tray" should contain the fried chicken pieces, and the user is prompted to enter information about the partition.

[0035] The input unit 15 is not limited to the above example using voice recognition. The input unit 15 can cause the display unit 14 to display information representing the X, Y, and Z options in the form of text, images, and / or the like, and receive input of X, Y, and Z by tapping or other selection action.

[0036] It will be repeated again Fig. 2. The teaching support device 1 displays the task description (step S5). More specifically, the input unit 15 instructs the display unit 14 to display the task description received at step S4, and the display unit 14 displays the task description. Using AR technology, the display unit 14 can display the movement of the robot corresponding to the task description. This allows the user to check whether the determined task description is a desired task description. If the task description is not the desired task description, the processing from and after step S3 or the processing of step S4 can be performed again.

[0037] Next, the teaching support device 1 calculates a robot motion based on the task description and the environment model (step S6). Specifically, the task setting unit 16 outputs task setting information to the robot motion calculation unit 12. The task setting information includes a conversion result of the task description to an operation program. The robot motion calculation unit 12 determines a path of the robot using the task setting information received from the task setting unit 16 and the environment model received from the modeling unit 11 or read from a model storage unit (not illustrated), and generates a robot motion command based on the determined path.In the present embodiment, for example, the robot motion calculation unit 12 organizes multiple motions corresponding to multiple steps defining a task into a single overall operation and generates a motion path using the start and end positions of the overall operation and constraint conditions related to the movement of the work machine. Note that the path determination method to be used by the robot motion calculation unit 12 is not limited to the method of this example.

[0038] Fig. Fig. 6 is a diagram illustrating exemplary configurations of the task setting unit 16 and the robot motion calculation unit 12 of the present embodiment. In the Fig. 6, the task setting unit 16 includes a task specification unit 161 and a target information setting unit 162.

[0039] The task specification unit 161 generates task specification information specifying a task. The task specification information is an operation program that matches the task description received by the input unit 15. The target information setting unit 162 generates target object information based on the task description, which includes, for example, information indicating the target object and the delivery method. The target object indicates the type of the target object, such as fried chicken or screw. The delivery method indicates how a target object is delivered, such as whether a pile of target objects is delivered in a food container or target objects are delivered by conveyance using a conveyor belt or the like.The feeding manner is, for example, a predetermined target object for each target object, and correspondence information representing a relationship between a target object and a feeding manner is stored in the task DB. After consulting the correspondence information in the task DB, the target information setting unit 162 determines the feeding manner corresponding to the target object. In addition to the feeding manner, there is information necessary for setting a movement of the robot 4 for each target object. In this case, correspondence information representing a relationship between such information and the target object may be stored in the task DB as with the feeding manner, to enable the target information setting unit 162 to acquire this information using this correspondence information and to incorporate this information into the target information.

[0040] The robot motion calculation unit 12 outputs task setting information to the robot motion calculation unit 12. The task setting information is defined by the task setting information generated by the task setting unit 161 and the target information generated by the target information setting unit 162. Note that although the example in which the task setting unit 16 generates an operation program for operating the robot 4 has been described here, the operation program may be generated by the robot motion calculation unit 12.

[0041] As in Fig. 6, the robot motion calculation unit 12 includes a total operation calculation unit 121. Using the task specification information and the target object information and an environment model and sensor information stored in Fig. 6, the overall operation calculation unit 121 performs a cross-step overall operation calculation to thereby generate a path of the robot 4 that provides a shorter operation time than connected paths calculated individually per step.

[0042] Fig. 7 is a diagram illustrating an example of a cross-step overall operation calculation of the present embodiment. Fig. Fig. 7 illustrates an example in which the robot 4 performs a task to pack the fried chicken pieces 32 accumulated in the food container 31 into the target partition 34 of the food tray 33. As shown in the upper part of Fig. As illustrated in Figure 7, this task includes: Step 1, in which the robot 4 picks up the roast chicken 32 from the food container 31; Step 2, in which the robot 4 carries the roast chicken 32 picked up from the food container 31 to the food tray 33; and Step 3, in which the robot 4 places the roast chicken 32 into the target partition 34 of the food tray 33.

[0043] In general, a path that does not cause obstruction to the environment can be generated by a step-by-step process calculation that gradually optimizes the movement of the robot 4, as shown in the upper illustration of Fig. 7. In contrast, the Fig. 6, the overall operation calculation unit 121 illustrates the steps as a single operation, as shown in the lower illustration of Fig. 7 illustrates that, without dividing the movement of the robot 4 by steps, it generates a path that does not cause interference with the environment and reduces operation time, taking into account constraints on the path (the direction of the path and the movement speed) in the previous and next steps. Note that the robot movement calculation unit 12 can generate a path that does not cause interference with the environment through a step-by-step operation calculation.

[0044] Fig. Figure 8 is a diagram illustrating an example of paths generated by a step-by-step operation calculation and paths generated by a cross-step overall operation calculation. The upper part of Fig. Figure 8 illustrates an example of paths generated by a step-by-step operation calculation, where in this case paths are generated for the following movements: Movements No. 1 to No. 4, which correspond to the Fig. 7, corresponding to step No. 1; movements No. 5 and No. 6, which correspond to the Fig. 7; movement No. 7, which corresponds to the Fig. 7. Motions No. 1 to No. 3 are movements by which the robot 4 starts to move toward the food container 31 such that the robot 4 moves slowly due to movement restrictions near the start and end of the movement of the robot 4 and moves quickly in motion No. 2. Motion No. 4 is a movement of closing a hand serving as a gripping unit provided at the front end of the robot 4. Note that the hand is open at the start of the task. Motion No. 5 is a low-speed movement in which a movement toward the food tray 33 is started, and motion No. 6 is a high-speed movement in which a movement toward the food tray 33 is moved. The opening of the hand in motion No.7 allows the roast chicken 32 to be placed in the target partition 34 of the food tray 33. It should be noted that the movements from . Fig. 8 are only intended as an example and the specific movements do not refer to those of the Fig. 8 illustrated example.

[0045] The target position of the robot 4 is determined based on sensor information. In the step-by-step operation calculation technique, an optimal path is generated for each step based on the target position and the environment model. The term "optimal path" in this context refers, for example, to a path that provides a minimum value of an evaluation function under constraint conditions and does not cause interference with the environment. Note that the optimal path is not limited to a path that provides a minimum value of an evaluation function, but may be a path that provides an evaluation function with a value less than or equal to a threshold value. The evaluation function concerns, for example, at least one of the working time of the robot 4, the energy consumption of the robot 4, and the moving distance of the robot 4.The constraint conditions are restrictions on the speed and direction of movement of the robot and are stored in the task DB.

[0046] When a path is created using the technique for calculating the overall operation across steps, a movement A and a movement B are defined, as shown in the lower part of Fig. 8, the overall operation calculation unit 121 determines the target position of each of the movements using the sensor information. The overall operation calculation unit 121 calculates an optimal path for movement A. That is, the overall operation calculation unit 121 integrates movements No. 1 to No. 3 into movement A and uses the sensor information, the target object information, and a constraint condition on speed to thereby generate a single path that provides the evaluation function with a value less than or equal to a threshold. For example, the overall operation calculation unit 121 generates a path using the constraint condition on speed in view of the remaining distance. The overall operation calculation unit 121 also integrates movements No. 5 and No. 6 into movement B and generates a single path in a similar manner.

[0047] Using the cross-step operation calculation technique makes it possible to provide a more suitable path for the entire task than when using the step-by-step operation calculation technique. For example, the evaluation function has a smaller value for a single path associated with motion A than the sum of the values ​​of the evaluation function for the individual paths associated with motions No. 1 to No. 3 generated by a step-by-step operation calculation. Thus, using the cross-step operation calculation technique makes it possible to generate a more suitable path than when performing optimization step-by-step. This provides advantages such as reducing the operating time and reducing the energy consumption of the robot 4.

[0048] It will be repeated again Fig. 2. After step S6, the teaching support device 1 displays a robot motion (step S7). More specifically, the robot motion calculation unit 12 outputs information representing a robot motion corresponding to the generated path (motion path) to the display unit 14, and the display unit 14 displays the motion of the robot 4 corresponding to the motion path. For example, the display unit 14 can display the robot motion using AR technology by displaying a robot model in a desired posture with an input of joint angle information, which is a motion command at each time point. This allows the user to check whether the determined robot motion is the desired robot motion. If the robot motion is not the desired robot motion, the processing from and after step S3 or the processing of step S4 can be performed again.

[0049] Next, the teaching support device 1 sends the robot movement command to the robot control device 3 (step S8) and ends the process. Specifically, in step S8, the robot movement calculation unit 12 sends the robot movement command generated in step S7 to the robot control device 3.

[0050] Note that when the sensor 5, which is a three-dimensional sensor for use in detecting the target object to be grasped, is attached to the robot 4, the robot motion calculation unit 12 may briefly stop the sensor 5 over the target to be grasped and cause the sensor 5 to image the target object to obtain the sensor information, thereby identifying the position of the target object and / or the like. When the sensor 5 is attached to the robot 4, the sensor 5 moves with the movement of the robot 4, causing motion blur during image capture. By briefly stopping the movement of the robot 4, the motion blur can be prevented.Alternatively, when the sensor 5, which is a three-dimensional sensor, is mounted above the target object, the position of the target object and / or the like can be identified by causing the sensor 5 to image the target object to acquire the sensor information after the robot 4, which has completed grasping the target object, moves past the grasping area without stopping above the object to be grasped.

[0051] A hardware configuration of the teaching support device 1 of the present embodiment will be described next. The teaching support device 1 of the present embodiment is configured such that a computer system functions as the teaching support device 1 by executing a program in the computer system. The program is a computer program that describes the processing to be performed in the teaching support device 1. Fig. 9 is a diagram illustrating an exemplary configuration of the computer system implementing the teaching support device 1 of the present embodiment. As shown in Fig. As illustrated in Figure 9, this computer system includes a control unit 101, an input unit 102, a storage unit 103, a display unit 104, a communication unit 105, and an output unit 106, which are interconnected via a system bus 107.

[0052] In Fig. 9, the control unit 101 is a processor such as a central processing unit (CPU). The control unit 101 executes a program describing processing to be performed in the teaching support device 1 of the present embodiment. Note that part of the control unit 101 may be implemented in a dedicated hardware element such as a graphics processing unit (GPU) or a field-programmable gate array (FPGA). The input unit 102 includes, for example, a keyboard, a mouse, and / or the like, and is used to input various types of information by a user of the computer system. The input unit 102 may include a microphone. The storage unit 103 includes storage devices such as various types of memories, such as a random access memory (RAM) and / or a read-only memory (ROM), and a hard disk.The storage unit 103 stores a program to be executed by the aforementioned control unit 101, necessary data acquired during a process, and the like. The storage unit 103 is also used as a temporary storage area for a program. The display unit 104 includes a display, a liquid crystal display (LCD), or the like for displaying various types of screens to the user of the computer system. Furthermore, the display unit 104 may be a touch panel that has the functionality of the input unit 102. The communication unit 105 is a receiver and a transmitter that perform communication processing. The output unit 106 is a printer, a speaker, and / or the like. Note that. Fig. 9 merely illustrates an example and the configuration of the computer system does not affect the configuration of Fig. 9 is limited.

[0053] Next, an exemplary operation of the computer system for making the program of the present embodiment executable will be described. Regarding the computer system configured as described above, a computer program is installed in the storage unit 103, for example, from a compact disc (CD)-ROM or a digital versatile disc (DVD)-ROM inserted into a CD-ROM drive or a DVD-ROM drive (not illustrated). Then, upon execution of the program, the program read from the storage unit 103 is loaded into a main storage area of ​​the storage unit 103, in which state the control unit 101 performs processing according to the program stored in the storage unit 103 as the processing of the teaching support device 1 of the present embodiment.

[0054] Note that, in the above description, it is assumed that the program describing the processing to be performed in the teaching support device 1 is provided using a recording medium such as a CD-ROM or a DVD-ROM, but the program provision method is not limited to this. A program provided via the communication unit 105 using a transmission medium such as the Internet may be used, for example, depending on factors such as the configuration of the computer system and / or the size of the program to be provided.

[0055] For example, a teaching support program of the present embodiment causes a computer system to perform a step of acquiring a task description input by the user and an environment model that is a three-dimensional model obtained by modeling the environment of the work machine, and a step of generating a movement path of the work machine based on the task description and the environment model, the movement path enabling the work machine to avoid interference with the environment.

[0056] The modeling unit 11, the robot motion calculation unit 12 and the task setting unit 16, which are Fig. 1 are implemented in such a way that the Fig. 9 illustrated memory unit 103 stored computer program by the in Fig. 9 illustrated control unit 101. The modeling unit 11, the robot motion calculation unit 12 and the task setting unit 16, which in Fig. 1 are also shown using the method described in Fig. 9 illustrated memory unit 103. The Fig. The task information storage unit 13 illustrated in Figure 1 represents a part of the Fig. 9 illustrated storage unit 103. The Fig. 1 is in the display unit 104 and in the control unit 101, which in Fig. 9 are implemented. The Fig. 1 illustrated input unit 15 is in the input unit 102 and in the input unit 101, which in Fig. 9. Furthermore, if the input unit 15 is to receive input via speech, the input unit 102 may include a microphone or an external microphone may be used.

[0057] Furthermore, the teaching support device 1 can be implemented in multiple computer systems. For example, the teaching support device 1 can be implemented such that the input unit 15, the display unit 14, and the task setting unit 16, which are Fig. 1 are implemented in one computer system, such as a tablet device, and the modeling unit 11, the robot motion calculation unit 12, and the task information storage unit 13 are implemented in another computer system. Alternatively, the teaching support device 1 may be implemented such that the components other than the robot motion calculation unit 12 are implemented in one computer system, such as a tablet device, and the robot motion calculation unit 12 is implemented in another computer system. When the teaching support device 1 is implemented in multiple computer systems, the combination of functional units implemented in each of the computer systems is not limited to the combinations of these examples. In addition, the robot control device 3 may include one or some of the functional units of the Fig. 1. For example, the teaching support device 1 may be implemented in a cloud computing system.

[0058] It should be noted that in the Fig. 1, the teaching support device 1 generates the environment model based on the environment measurement data acquired by the image sensor 2, however, the environment model may also be input from outside the teaching support device in a case where the environment model is generated in the form of a computer-aided design (CAD) model. For example, the teaching support device 1 receives an environment model from another device (not illustrated), and the environment model is then input to the robot motion calculation unit 12 and the display unit 14. In this case, the work system 6 does not need to include the image sensor 2, and the teaching support device 1 does not need to include the modeling unit 11. Alternatively, the Fig. 1 may further include functionality to input an externally input environment model to the robot motion calculation unit 12 and the display unit 14 and to use both an environment model based on environment measurement data and an environment model generated in advance.

[0059] As described above, the teaching support device 1 of the present embodiment receives input of a task description from a user and automatically generates an operation program based on the received task description. Note that the program for each task is registered as template information in the task DB, and the corresponding person for the task is loaded into the robot motion calculation unit 12. The robot motion calculation unit 12 provides the loaded program with information about the size of an operation area related to the work task and information about the position of the operation area. The programs are configured to automatically set teaching points and generate necessary paths in the program loaded from the template based on the given task-specific area information and the environment model.This means that no user input is required to specify a robot movement. Thus, simply specifying information such as the size of the operating area, the position of the operating area, and the objective of the task makes it possible to provide a robot movement. Furthermore, a sensor-based feedback control system can be configured to continuously adjust its operation based on information from a sensor attached to the robot or the environment. Such a sensor-based feedback control system autonomously modifies an operation depending on the situation. This enables automatic generation of an operation program for operating the robot 4 without the user having to perform various input operations job by job and without having to have knowledge of the programming language. This can reduce the time required for the user to commission a system.

[0060] Moreover, the teaching support device 1 of the present embodiment automatically generates an environment model using the environment measurement data acquired by the image sensor 2 and determines a path of the robot that does not obstruct the environment using the generated environment model, thereby enabling a reduction in the time required to generate an environment model compared to the time required when the user generates the environment model himself.

[0061] Furthermore, the teaching support device 1 of the present embodiment generates a path of the robot 4 that organizes several steps of a task into a single operation. This enables a reduction in the operation time of the robot and a reduction in the time required to adjust the operation program.

[0062] Furthermore, the teaching support device 1 of the present embodiment generates the basic format corresponding to each task in advance, and displays information specified by the format corresponding to the task to a user. This allows even a novice user to easily input a task description. Furthermore, the teaching support device 1 displays a video or explanation corresponding to each basic format to the user, thereby allowing the user to select a more suitable basic format (i.e., task). Second embodiment.

[0063] Fig. Fig. 10 is a diagram illustrating an exemplary configuration of a teaching support device according to a second embodiment. A teaching support device 1a of the present embodiment is similar to the teaching support device 1 of the first embodiment, except that the modeling unit 11 shown in Fig. Fig. 1 is removed, and a simulation unit 17 is added thereto. Components having similar functionality to the functionality of the first embodiment are denoted by the same reference numerals as the reference numerals of the first embodiment, and duplicate descriptions are omitted. The following description primarily describes differences from the first embodiment.

[0064] In the present embodiment, the simulation unit 17, which is a simulator, performs a simulation process in which a movement of the robot 4 is simulated. For example, a simulation result can be used in a preliminary study before the actual installation of the robot 4.

[0065] Although, as in the first embodiment, the robot motion calculation unit 12 performs processing and sets task descriptions when performing a simulation, the robot motion calculation unit 12 receives an environment model generated in the form of a CAD model. The robot motion calculation unit 12 outputs a robot motion command, a robot model, and the environment model to the simulation unit 17.

[0066] The simulation unit 17 performs a simulation process in which the motion of the robot 4 is simulated using the environment model and the motion path determined by the robot motion calculation unit 12. For example, the simulation unit 17 performs a simulation process to simulate the motion of the robot 4 using the robot motion command, the robot model, and the environment model received from the robot motion calculation unit 12, and outputs a simulation result, that is, a result of the simulation, to the display unit 14.

[0067] The display unit 14 displays the simulation result. Furthermore, the display unit 14 can display a visual programming screen and can also display the visual programming screen and the simulation result overlapping each other. The visual programming screen, which is, for example, a screen for specifying an operation by arranging blocks in a sequence, is, for example, a simplified pragmatic language for block-based programming, such as "Scratch." However, the programming language is not limited to that of this example.

[0068] Fig. 11 is a diagram illustrating an example of a display screen of the present embodiment. In the Fig. In the example illustrated in Figure 11, the teaching support device 1a displays a simulation result. For example, when the robot is tapped on the display screen, a visual programming screen is displayed on the display screen, overlapping the simulation result. The display screen of Fig. 11 is only an example, and a specific display screen is not limited to the display screen of the Fig. 11 illustrated example.

[0069] The visual programming screen also allows for input, for example, a change in the type of sensor or robot. The arrangement of the sensor type or robot can also be changed using the screen displaying the simulation results. For example, the area in which the robot is displayed is selected and moved to change the robot's arrangement. These changes are input to the input unit 15, and the input results are output to the task setting unit 16.

[0070] For example, the visual programming screen displays a list of basic formats as a task item list. When a task is selected by the user, a preset of a robot, hand, and sensor corresponding to the selected task is displayed. A robot list (various types of robots) and a sensor list (vision sensor, force sensor, touch sensor, distance measurement sensor, etc.) can also be displayed, so that an item to be added can be tapped and dragged from these lists onto the screen to change or add the item. When a sensor is added, a physical quantity val (e.g., contact force with the environment, the position of the target object, or the distance between specified points) measured by that sensor is defined.The simulation unit 17 uses the defined physical quantity val input thereto to check the feasibility of an operation based on the operating conditions of block-based programming. The simulation unit 17 then causes the display unit 14 to display the simulation result in the form of a video. For example, if the target object is to be transported on a conveyor, it is possible that a vision sensor, which is a sensor for detecting the position of the target object, is mounted too close to the robot to detect the target object in time. The robot's movement is simulated, thereby making it possible to consider an appropriate positional relationship in advance to prevent such a disadvantage from occurring.

[0071] As described above, the display unit 14 can display a screen for block-based programming, the input unit 15 can receive input of a change to the robot in block-based programming, and the robot motion calculation unit 12 can generate a path reflecting the input received by the input unit 15. The input unit 15 can receive input of an addition or change to the sensor for use in determining the movement of the robot in block-based programming, and the robot motion calculation unit 12 can generate the path reflecting the input received by the input unit 15.

[0072] Please note that Fig. 10 illustrates a functional configuration when the simulation is performed, the teaching support device 1a configured for the purpose of performing the simulation as shown in Fig. 10, but can be used in such a way or the simulation unit 17 to the one shown in Fig. 1 such that both the operation described in the first embodiment and the simulation can be performed.

[0073] The teaching support device 1a of the present embodiment has a hardware configuration similar to the hardware configuration of the teaching support device 1 of the first embodiment. For example, the simulation unit 17 is implemented such that the Fig. 9 illustrated memory unit 103 stored computer program by the in Fig. 9 illustrated control unit 101. The simulation unit 17 is also configured using the Fig. 9. Furthermore, the teaching support device 1a can be implemented in multiple computer systems, similar to the case of the first embodiment.

[0074] Moreover, the teaching support device 1a may use the image sensor 2 described in the first embodiment and include the modeling unit 11 that generates an environment model using environment measurement data, and the simulation unit 17 performs simulation using the generated environment model.

[0075] In the present embodiment, the teaching support device 1a performs the simulation process in which the movement of the robot is simulated, thereby enabling the use of the simulation result for a preliminary study. Third embodiment.

[0076] Fig. 12 is a diagram illustrating an exemplary configuration of a work system according to a third embodiment. A work system 6b of the present embodiment is similar to the work system 6 of the first embodiment, except that it includes a teaching support device 1b instead of the teaching support device 1. Components having similar functionality to the functionality of the first embodiment are denoted by the same reference numerals as the reference numerals of the first embodiment, and duplicate descriptions are omitted. In the following description, differences from the first embodiment will be primarily described.

[0077] The teaching support device 1b is similar to the teaching support device 1 of the first embodiment, except that it includes a modeling unit 11a, an input unit 15a, and a task setting unit 16a instead of the modeling unit 11, the input unit 15, and the task setting unit 16. The task setting unit 16a is similar to the task setting unit 16 of the first embodiment, except that a motion modification unit 163 is added.

[0078] In addition to performing an operation similar to the operation of the first embodiment, the teaching support device 1b of the present embodiment receives an instruction to receive at least one of the environment model and the movement of the robot 4, and reflects the modification. Fig. 13 is a flowchart illustrating an example of a processing procedure performed in the teaching support device 1b of the present embodiment. The operations in steps S1 to S7 are similar to the corresponding operations in the first embodiment.

[0079] After step S7, the teaching support device 1b determines whether a modification instruction has been issued (step S9). More specifically, it is determined whether the input unit 15a has received an instruction from the user to modify at least one of the environment model and the motion of the robot 4. If a modification instruction has been issued (Yes at step S9), the teaching support device 1b modifies the robot motion (step S10) and the processing proceeds to step S8. If no modification instruction has been issued (No at step S9), the teaching support device 1b proceeds the processing to step S8. The process at step S8 is similar to the corresponding process of the first embodiment.

[0080] For example, if the path of the robot 4 displayed at step S7 is a path that an operator working with the robot 4 perceives as a threat, the user inputs at least part of a forbidden area and a permitted area using the input unit 15a. If the display unit 14 is, for example, a touch panel having a functionality of part of the input unit 15a, the user inputs a position of at least one of a forbidden area model and the permitted area in the displayed environment by tapping, thereby specifying at least one of the forbidden area model and the permitted area model. The input unit 15a outputs the input of at least one of the forbidden area model and the permitted area model to the modeling unit 11a.Using information received from the input unit 15a, the modeling unit 11a makes a modification to add at least one of the forbidden area model and the permitted area model to the environment model, and outputs the modified environment model to the display unit 14 and the robot motion calculation unit 12. Thus, using the modified environment model, the robot motion calculation unit 12 performs robot motion calculation as in the first embodiment. That is, the robot motion calculation unit 12 generates a path to reflect the modification instruction received by the input unit 15a.Specifically, when a model for a forbidden area is added, the robot motion calculation unit 12 determines a path such that the robot 4 is prevented from passing through the forbidden area and the condition that the robot 4 does not obstruct the surroundings is satisfied. Furthermore, when a model for a permitted area is added, the robot motion calculation unit 12 determines a path under a condition that the robot 4 is allowed to pass through an area in the permitted area even if the robot 4 obstructs the surroundings.

[0081] For example, the user sets as the forbidden area an area that an operator working with the robot 4 perceives as a threat. Furthermore, the entire environment model can be a target obstructed by the robot. With the entire environment as such an obstruction target, it is thus possible that the robot 4 is prevented from even reaching a location near the target object to grasp it. However, if a permitted area is set, the robot 4 is permitted to reach a location near the target object. Moreover, a permitted area near the target object can be automatically set using the sensor information acquired by the sensor 5.For example, the modeling unit 11a may automatically add a model for a permissible area, which is an area around the target object specified by the sensor information, using the sensor information.

[0082] Furthermore, the display unit 14 may, for example, display an element such as a button for enlarging a partial model of the environment model when the partial model is selected. When the input unit 15a detects the pressing of this button, the input unit 15a may instruct the modeling unit 11a to enlarge the selected partial model. Based on an instruction from the input unit 15a, the modeling unit 11a enlarges the size of the partial model selected by the user according to a predetermined rule. The predetermined rule may be a rule to increase the size by a certain amount in a specific direction or in all directions, or to increase the size by multiplying the size by a factor of a real number greater than or equal to 1 in a specific direction or in all directions. The size may be increased by a method other than these methods.This allows the creation of a path that has a margin with respect to the actual distance from the environment.

[0083] Furthermore, the input unit 15a can receive a user's voice indicating a target to be modified and the modification detail, such as "Move P slightly to the right" as the modification detail to the movement of the robot 4, and recognize the modification detail through a voice recognition technique. The input unit 15a outputs the recognition result of the voice through the task setting unit 16a, and the task setting unit 16a modifies the operation program according to the modification detail. Alternatively, options of items of targets to be modified, such as the target position of the movement of the robot 4, can be provided to the user in such a way that the input unit 15a receives an input of the selection result of the options and receives the modification detail to the corresponding item via voice.The input unit 15a can receive an input of a change in a restriction condition of the movement of the robot 4 as the modification detail to the movement of the robot 4. The modification detail to the movement of the robot 4 is output to the robot movement calculation unit 12 via the task setting unit 16a, and the robot movement calculation unit 12 generates the path reflecting the modification.

[0084] Furthermore, the display unit 14 can display elements such as buttons for instructions to change the speed of the AR display, temporarily pause the AR display operation, etc., and the input unit 15a can receive an instruction related to the AR display by detecting the pressing of the corresponding button. In this case, the input unit 15a notifies the display unit 14 of the received instruction, and the display unit 14 controls the AR display according to the notification. As a result, the user who wants to check a specific part of the operation can check that part in detail, thereby reducing the speed or pausing the operation.

[0085] It should be noted that the present embodiment, although the Fig. 1 is an exemplary configuration with the added modification functionality. The modification functionality can also be added when using a previously created environment model. Furthermore, the teaching support device 1b of the present embodiment can additionally include functionality for performing the simulation described in the second embodiment.

[0086] The teaching support device 1b of the present embodiment has a hardware configuration similar to the hardware configuration of the teaching support device 1 of the first embodiment. Furthermore, the teaching support device 1b can be implemented in multiple computer systems, similar to the case of the first embodiment.

[0087] The present embodiment, which can perform the modification after checking the operation based on an automatically generated operation program, makes it possible to simplify the customization reflecting a user's intention in a short time. Fourth embodiment.

[0088] Fig. 14 is a diagram illustrating an exemplary configuration of a work system according to a fourth embodiment. A work system 6c of the present embodiment is similar to the work system 6b of the third embodiment, except that it includes a teaching support device 1c instead of the teaching support device 1b and additionally includes an analysis device 7. Components having similar functionality to the functionality of the third embodiment are denoted by the same reference numerals as the reference numerals of the third embodiment, and duplicate descriptions are omitted. In the following description, differences from the third embodiment will be primarily described.

[0089] The teaching support device 1c of the present embodiment is the teaching support device 1b of the third embodiment with a transmission / reception unit 18 added. While the task setting unit 16a stores the task setting information in the task DB of the task information storage unit 13, setting information associated with the set task, such as the task (i.e., the type of task), the target object, and the target area, are stored in the task DB. The robot motion calculation unit 12 outputs motion information to the transmission / reception unit 18. The motion information is information related to robot motion, such as a robot motion command.The transmission / reception unit 18 reads the setting information from the task DB and sends the read setting information and the motion information received from the robot motion calculation unit 12 to the analysis device 7.

[0090] The analysis device 7 includes a transmission / reception unit 71 and an analysis unit 72. The transmission / reception unit 71 receives the setting information and the movement information from the teaching support device 1c, receives measurement information from a peripheral device 8, receives success or failure information from the teaching support device 1c or another device, and outputs the received information to the analysis unit 72. The success or failure information indicates the success or failure of the task. Note that the success or failure information can be input by a user into the input unit 15a of the teaching support device 1c and transmitted to the analysis device 7 by the transmission / reception unit 18.Alternatively, the success or failure information may be input by a user into another device (not illustrated) and transmitted from the other device to the analysis device 7. Furthermore, the transmission / reception unit 71 transmits an analysis result received from the analysis unit 72 to the teaching support device 1c.

[0091] The analysis unit 72 analyzes an influencing factor that is most likely a cause of failure using the setting information, the motion information, the measurement information, and the success or failure information, and outputs an analysis result to the transmission / reception unit 71. The peripheral device 8, which is, for example, a sensor for detecting the state of the robot 4, is defined by various types of sensors, such as a temperature sensor, a motor speed sensor, and a vibration sensor, which are attached to at least one of the robot 4 and the surrounding environment. Note that Fig. 14 illustrates the analysis device 7 receiving the measurement information directly from the peripheral device 8, but the configuration is not limited to this. The analysis device 7 may receive the measurement information via the robot control device 3, the teaching support device 1c, or another device (not illustrated).

[0092] For example, the analysis unit 72 classifies input information, including setting information, movement information, measurement information, and success or failure information, into groups of the same task descriptions or groups of similar task descriptions. The analysis unit 72 determines, group by group, whether information on each of the pieces of movement information and measurement information is outside a normal range, and stores a correspondence between a normal range determination result and the success or failure information. For example, the analysis unit 72 may reproduce a regression model using Gaussian process regression, etc., for average behavior based on data obtained during the execution of a specific task, in advance, a distribution region in normal operation (ie,normal range) and evaluate the degree to which the information deviates from the region. The normal range may be statistically defined such that, for example, feature quantities (e.g., the degrees of deviation of sensor outputs from a normal value in the case of multiple trials) and a variance thereof result in 3σ or less, which is normal, and 3σ or more, which is abnormal. The analysis unit 72 extracts, group by group, an item having a determination result deviating from a normal range when input information corresponding to this determination result indicates that the success-or-failure information indicates failure, the item having a determination result falling within the normal range when input information corresponding to this determination result indicates that the success-or-failure information indicates success.The analysis unit 72 determines that the extracted item is the information corresponding to the influencing factor. For example, if measurement information from a sensor K deviates from a normal range when the success or failure information indicates a failure, and falls within the normal range when the success or failure information indicates a success, the analysis unit 72 determines that the measurement information from the sensor K is information corresponding to the influencing factor. For example, if the sensor K is the temperature of a part P of the robot 4, it is highly likely that the part P is the influencing factor in the failure.

[0093] Alternatively, if the motion information and the measurement information corresponding to the success or failure information indicating failure include an item whose information deviates from the normal range, the analysis unit 72 may determine that such an item is information corresponding to the influencing factor. Furthermore, since the input information includes the success or failure information indicating success, the analysis unit 72 may determine the normal range of each item using accumulated input information through a statistical method or machine learning such as cluster analysis. Furthermore, the analysis device 7 may perform analysis using information acquired from a plurality of teaching support devices 1c.

[0094] Alternatively, the analysis unit 72 may acquire information indicating the type of failure (i.e., the details of the failure) in the case where the success-or-failure information indicates a failure, and when the operator, etc., identifies the location as a cause of the failure by inspecting the robot 4, perform machine learning using the identified location as labeled data. For example, the analysis unit 72 generates a learned model through supervised learning using multiple data sets each defined by the type of failure and the cause (i.e., the location of the cause), i.e., the corresponding labeled data. The analysis unit 72 may infer the cause by inputting the type of failure to the learned model when analyzing the influencing factor.In supervised learning, the analysis unit 72 may generate the learned model using at least one of the above input information as the input data in addition to the type of failure. Alternatively, in supervised learning, the analysis unit 72 may generate the learned model using at least one of the above input information as the input data. By inputting the input data to the learned model, it becomes possible to similarly infer the cause of a failure. Furthermore, the following description provides another example. The operator preliminarily intentionally induces a specific abnormal state in a real system, after which the operator collects data on an error signal or an output from each device several times.To learn the order and timing of an abnormality occurrence indicated by each device, the analysis unit 72 performs machine learning with the input "time series data for each signal or graphically plotted information on such time series data." Then, when a similar signal change pattern occurs, it is determined that a pre-registered abnormal condition has occurred. Note that the determination of an abnormal condition can be made during or after information collection. In particular, making such a determination during information collection during operation provides an advantage in that the system is proactively stopped before a situation occurs in which a measure of a somewhat similar abnormal condition leads to system failure. The following description provides yet another example.A server collects data acquired in multiple machine systems, and the analysis unit 72 makes the conclusion, performs supervised learning based on a database that includes at least one of a condition monitoring signal, an error signal, and an output from each sensor, etc., all labeled as the abnormal state. This means that since the determination of the abnormality can be made based on data acquired in previous instances of another system without requiring the actually used system to generate learning data, it becomes possible to provide an advantage in reducing the time required for the learning operation and reducing the system startup time for performing anomaly determination and analysis.The above analysis methods for use in the analysis unit 72 are discussed as examples, and the analysis method for use in the analysis unit 72 is not limited to the above examples. Furthermore, information for use in the analysis performed by the analysis unit 72 is not limited to those in the above examples.

[0095] After receiving the analysis result from the analysis device 7, the transmission / reception unit 18 of the teaching support device 1c outputs the analysis result to the display unit 14, and the display unit 14 displays the analysis result.

[0096] As described above, in the present embodiment, the teaching support device 1c sends, for example, at least one of information representing the task description and motion information to the analysis device 7, and the analysis device 7 analyzes the cause of failure of a task using information acquired from the sensor for detecting the state of the robot 4, the information representing the task description, and the success or failure information representing success or failure of the task. The analysis device 7 sends the analysis result to the teaching support device 1c. The display unit 14 of the teaching support device 1c displays the analysis result received from the analysis device 7. This allows the user to check the analysis result displayed on the display unit 14, that is,the derived cause, and attempt to take a specific action at the appropriate location, thereby enabling the acceleration of recovery from a failure.

[0097] The teaching support device 1c of the present embodiment has a hardware configuration similar to that of the teaching support device 1 of the first embodiment. Furthermore, the teaching support device 1c can be implemented in multiple computer systems, similar to the case of the first embodiment.

[0098] The analysis device 7 of the present embodiment is also configured as a computer system such as one used in Fig. 9 of the first embodiment. The analysis device 7 can be set up in a cloud system.

[0099] Moreover, the teaching support device 1 described in the first embodiment can also send information to the analysis device 7, obtain an analysis result from the analysis device 7, and display the analysis result. Fifth embodiment.

[0100] Fig.15 is a diagram illustrating an exemplary configuration of a work system according to a fifth embodiment. A work system 6d of the present embodiment is similar to the work system 6 of the first embodiment, except that it includes a teaching support device 1d instead of the teaching support device 1. Components having similar functionality to the functionality of the first embodiment are denoted by the same reference numerals as the reference numerals of the first embodiment, and duplicate descriptions are omitted. In the following description, differences from the first embodiment will be primarily described.

[0101] The teaching support device 1d of the present embodiment is similar to the teaching support device 1 of the first embodiment, except that it includes a setting information obtaining unit 19 instead of the task setting unit 16 and the input unit 15.

[0102] The setting information acquisition unit 19 of the teaching support device 1d receives, from another device (not illustrated), task setting information similar to the task setting information output by the task setting unit 16 of the first embodiment to the robot motion calculation unit 12, and outputs the received task setting information to the robot motion calculation unit 12. For example, task setting information generated in the past by another support device, which is another teaching support device 1d, is available to the teaching support device 1d, in which case the teaching support device 1d receives this task setting information from the other support device.The teaching support device 1d of the present embodiment can thus reduce the burden on the user.

[0103] The teaching support device 1d of the present embodiment has a hardware configuration similar to the hardware configuration of the teaching support device 1 of the first embodiment. Furthermore, the teaching support device 1d can be implemented in multiple computer systems, similar to the case of the first embodiment.

[0104] The configurations described in the above embodiments are merely examples. These configurations may be combined with other known techniques, and configurations of different embodiments may be combined with each other. Furthermore, some of such configurations may be omitted and / or modified without deviating from their essence. List of reference symbols

[0105] 1, 1a, 1b, 1c, 1d teaching support device; 2 image sensor; 3 robot control device; 4 robot; 5 sensor; 6, 6b, 6c, 6d work system; 7 analysis device; 8 peripheral device; 11, 11a modeling unit; 12 robot motion calculation unit; 13 task information storage unit; 14 display unit; 15, 15a input unit; 16, 16a task setting unit; 17 simulation unit; 18, 19 transmission / reception unit; 19 setting information acquisition unit; 72 analysis unit; 121 overall operation calculation unit; 161 task specification unit; 162 target information setting unit; 163 motion modification unit. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] WO 2018 / 194094 A

[0003]

Claims

[1] Teaching support device comprising: a movement calculation unit for generating a movement path of a work machine based on a task description and an environment model, the movement path enabling the work machine to avoid interference with an environment of the work machine, the task description being input by a user, the environment model being a three-dimensional model obtained by modeling the environment of the work machine. [2] A teaching support device according to claim 1, comprising: a modeling unit for generating the environment model using environment measurement data, wherein the environment measurement data is data obtained by an image sensor for capturing a three-dimensional image by imaging the environment. [3] The teaching support device according to claim 1 or 2, wherein the movement calculation unit organizes a plurality of movements corresponding to a plurality of steps defining a task into a single overall operation and generates the movement path using start and end positions of the overall operation and a restriction condition on a movement of the work machine. [4] Teaching support device according to one of claims 1 to 3, comprising: a display unit to display a movement of the working machine corresponding to the movement path. [5] The teaching support device according to claim 4, wherein the display unit displays the movement of the work machine through augmented reality display. [6] A teaching support device according to claim 4 or 5, comprising: an input unit to receive an instruction to modify at least one of the environmental model or the movement of the work machine, wherein the movement calculation unit generates the movement path that reflects the instruction. [7] The teaching support device according to claim 6, wherein the input unit receives an input of the task description by voice or by tapping. [8] A teaching support device according to claim 7, comprising: a task information storage unit for storing a basic format of each of tasks, the basic format reflecting a movement of the work machine corresponding to the task, wherein the display unit displays information representing the tasks, the basic formats of the tasks being stored in the task information storage unit, and the input unit receives the input of the task description by receiving a result of a user selection from the tasks displayed on the display unit, and determines the task description based on the basic format of the task corresponding to a received result of the user selection and information input by the user. [9] A teaching support device according to claim 1, comprising: a simulation unit for performing a simulation process for simulating a movement of the work machine using the movement path and the environment model; and a display unit to show a result of the simulation. [10] A teaching support device according to claim 9, comprising: an input unit, where the display unit shows a screen for block-based programming, the input unit receives an input of a change of the working machine in the block-based programming and the movement calculation unit generates the movement path that reflects the input received by the input unit. [11] A teaching support device according to claim 10, wherein the input unit receives an input of an addition or a change of a sensor for use in determining the movement of the work machine in the block-based programming and the movement calculation unit generates the movement path that reflects the input received by the input unit. [12] The teaching support device according to any one of claims 1 to 11, wherein the movement calculation unit generates a movement command for controlling the work machine based on the movement path. [13] The teaching support device according to any one of claims 1 to 12, wherein the teaching support device generates an operation program for operating the work machine using the movement path. [14] Work system comprising: an image sensor for capturing a three-dimensional image as environmental measurement data by imaging an environment of a working machine; and a teaching support device, wherein the teaching support device includes: a modeling unit for generating an environment model using the environment measurement data, wherein the environment model is a three-dimensional model obtained by modeling an environment, wherein the environment measurement data is data obtained by the image sensor for capturing a three-dimensional image by imaging the environment, and a movement calculation unit for generating a movement path of the work machine based on a task description input by a user and the environment model, wherein the movement path of the work machine enables to avoid obstruction of the environment. [15] Work system comprising: a teaching support device; and an analysis device, wherein the teaching support device generates a movement path of a work machine based on a task description and an environment model and sends information reflecting the task description to the analysis device, wherein the movement path of the work machine makes it possible to avoid obstruction of an environment of the work machine, wherein the task description is input by a user, wherein the environment model is a three-dimensional model obtained by modeling the environment of the work machine, the analysis device analyzes a cause of failure of a task using information obtained from a sensor for detecting a state of the work machine, information representing the task description, and success or failure information, and sends an analysis result to the teaching support device, wherein the success or failure information represents success or failure of the task, and the teaching support device displays the analysis result received from the analysis device. [16] A work system according to claim 14 or 15, comprising: the working machine; and a control device to control the working machine, wherein the teaching support device generates a movement command for the working machine based on the movement path and sends the movement command to the control device. [17] A teaching support method for use in a teaching support device, the teaching support method comprising: a step of obtaining a task description and an environment model, wherein the task description is input by a user, wherein the environment model is a three-dimensional model obtained by modeling an environment of a work machine; and a step of generating a movement path of the work machine based on the task description and the environment model, wherein the movement path of the work machine makes it possible to avoid obstruction of the environment. [18] Teaching support program that causes a computer system to perform the following: a step of obtaining a task description and an environment model, wherein the task description is input by a user, wherein the environment model is a three-dimensional model obtained by modeling an environment of a work machine; and a step of generating a movement path of the work machine based on the task description and the environment model, wherein the movement path of the work machine makes it possible to avoid obstruction of the environment.

Images