Model generation device, model generation method, control device and control method

Abstract

One aspect of the present invention relates to a model generation device that implements a machine learning model for a decision model using multiple learning datasets, each of which is composed of a combination of training data representing the positional relationship between two objects and correct answer data indicating whether the two objects are in contact with each other in that positional relationship. The machine learning is implemented by training the decision model for each learning dataset so that, for each training dataset, the model outputs an output value appropriate for the corresponding correct answer data, relative to the training data input.

Description

Technical Field

[0001] The present invention relates to a model generating device, a model generating method, a control device and a control method. Background Art

[0002] Production lines that manufacture products utilize robotic devices such as manipulators. The components of a manipulator, such as its mechanism, end effector, and workpiece, vary widely depending on the task being performed. Manually developing a sequence of robot movements that corresponds to all of these components and then teaching the robot the target task is difficult. Therefore, a conventional method involves determining the types of components such as the mechanism, end effector, and workpiece, then manually moving the robot and recording its movements during the series of movements to directly teach the task being performed.

[0003] However, in this method, every time a component such as a mechanism, an end effector, or a workpiece is changed, the robot device must be taught the operation to be performed. Therefore, teaching the robot device the operation to be performed would incur excessive costs. Therefore, in recent years, research has been conducted on methods of using machine learning to enable a robot device to master the operation to be performed. For example, in non-patent document 1, a method is proposed in which a robot device is taught to grasp the action of gripping an object based on image data obtained from a camera through reinforcement learning. According to this method, at least a portion of the series of processes for teaching the robot device the action of gripping an object can be automated. Therefore, the cost of teaching the robot device the operation can be reduced.

[0004] Prior art literature

[0005] Non-patent literature

[0006] Non-patent document 1: Dmitry Kalashnikov, et al. "QT-Opt: Scalable DeepReinforcement Learning for Vision-Based Robotic Manipulation" arXiv preprintarXiv:1806.10293, 2018. Summary of the Invention

[0007] Technical problem to be solved by the invention

[0008] The inventors of the present application have discovered that the following problems exist in the conventional control methods for robotic devices such as those described above. For example, the movement of objects such as end effectors, workpieces, and obstacles is sometimes expressed in a coordinate space. In this case, the boundary of contact between the two objects can be expressed by maintaining a value indicating whether the two objects are in contact with each other at each coordinate point. However, in this expression method, the amount of data required to express the boundary of contact between the two objects increases explosively depending on the combination of objects, the dimensionality of the coordinates, and the resolution (resolving power). For example, assume that the coordinates of an object are expressed in three-dimensional position and three-dimensional posture. In this case, the coordinates of the object are expressed in six dimensions. Therefore, if the resolution (resolving power) of the information expressing the boundary of contact is increased, the amount of data increases by the order of 6. Therefore, in the above expression method, it is difficult to store the information expressing the boundary of contact in systems with relatively small storage capacity, such as embedded systems. In addition, this problem is not limited to scenarios that specify the movement of the robotic device, but occurs in all scenarios that deal with the contact between two objects.

[0009] One aspect of the present invention has been made in view of such actual circumstances, and an object of the present invention is to provide a technique for reducing the amount of data representing information on a boundary where two objects come into contact.

[0010] Technical solutions to solve problems

[0011] The present invention adopts the following configuration in order to solve the above-mentioned technical problems.

[0012] That is, one aspect of the present invention involves a model generation device comprising: a data acquisition unit, which acquires multiple learning data sets, each of which is composed of a combination of training data representing the positional relationship between two objects and correct answer data representing whether the two objects are in contact with each other in the positional relationship; and a machine learning unit, which uses the acquired multiple learning data sets to implement machine learning of a judgment model, and the implementation of the machine learning is constructed by training the judgment model in a manner that outputs an output value suitable for the corresponding correct answer data relative to the input of the training data for each of the learning data sets.

[0013] The model generation device involved in this configuration uses machine learning to generate a determination model for determining whether two objects are in contact within a positional relationship. Based on the learned determination model generated through machine learning, even when the positional relationship of the objects is provided as a continuous value (e.g., coordinates), it is possible to determine whether the two objects are in contact within that positional relationship without significantly increasing the amount of data for the determination model. Therefore, this configuration can significantly reduce the amount of data required to represent the boundary between the two objects in contact.

[0014] In addition, the object may also include all kinds of objects that can be processed in the real space or virtual space. The object may be, for example, an end effector, a workpiece, an obstacle, or other objects that can be associated with the action of the robotic device. In this case, the robotic device may include, for example, an industrial robot such as a manipulator, a mobile body that can move automatically, etc. Industrial robots may include, for example, vertical multi-joint robots, SCARA robots, parallel linkage robots, orthogonal robots, collaborative robots, etc. Mobile bodies that can move automatically may include, for example, drones, vehicles configured to be autonomously driven, mobile robots, etc.

[0015] A single object may also be composed of multiple objects. When there are three or more objects, the multiple objects may be considered as a single object, and a determination model may be constructed to determine whether contact has occurred between the multiple objects and other objects. Alternatively, each object may be considered as a single object, and a determination model may be constructed to determine whether contact has occurred between the individual objects. Multiple determination models may be prepared to determine whether contact has occurred between different objects, respectively. Alternatively, the determination model may be constructed to also accept input of information indicating conditions such as the type of object and determine whether contact has occurred between two objects corresponding to the input conditions.

[0016] The type of machine learning model that constitutes the determination model is not particularly limited and can be appropriately selected depending on the implementation. For example, the determination model can employ machine learning models such as neural networks, support vector machines, regression models, and decision trees. Whether two objects are in contact can be determined in either real space or virtual space.

[0017] In the model generation device involved in the above-mentioned aspect, the positional relationship between the two objects can be expressed by relative coordinates. Relative coordinates are coordinates when observing the other object from one object. Any one of the two objects can be selected as the basis for the relative coordinates. According to this structure, the physical and relative positional relationship between the two objects can be appropriately and clearly expressed. However, the method of expressing the positional relationship is not limited to such an example and can be appropriately selected according to the implementation method. The positional relationship between the two objects can also be expressed, for example, by their respective absolute coordinates. In addition, "coordinates" can include at least one of position and posture. In three-dimensional space, the position can be expressed by the three axes of front and back, left and right, and up and down, and the posture can be expressed by the rotation of each axis (roll, pitch, yaw).

[0018] In the model generation device according to the aforementioned aspect, at least one of the two objects can be moved by the action of the robot. This configuration reduces the amount of data required to represent the contact boundary between the two objects in scenarios that specify the action of the robot. Furthermore, the object moved by the action of the robot can be, for example, a component of the robot, such as an end effector, or the robot itself, or an object other than a component of the robot, such as a workpiece held by the end effector.

[0019] In one aspect, the present invention relates to a control device for controlling the operation of a robot apparatus that performs a task of moving a first object in an environment in which a first object and a second object are present, the control device comprising: a state acquisition unit for acquiring a current task state of the robot apparatus, the task state being defined by a positional relationship between the first object and the second object; an action determination unit for determining a target task state to be subsequently transitioned to, relative to the acquired current task state, so as to approximate a final target task state; a contact determination unit for determining whether the first object and the second object are in contact with each other under the target task state by providing the target task state of the first object and the second object to a learned determination model trained through machine learning to output, in response to input of training data indicating the positional relationship between the first object and the second object, an output value appropriate for correct answer data indicating whether the first object and the second object are in contact with each other under the positional relationship; a command determination unit for determining a control command to be provided to the robot apparatus so as to change the task state of the robot apparatus from the current task state to the target task state; and a drive unit for driving the robot apparatus by providing the determined control command to the robot apparatus.

[0020] In this configuration, by utilizing a learned judgment model generated through machine learning, the amount of data representing the contact boundary between the first and second objects can be significantly reduced. Consequently, even with a relatively low-performance computer, such as an embedded system, this configuration can implement a control device capable of appropriately driving a robotic device while avoiding unnecessary contact between the first and second objects.

[0021] In addition, in the previous control method, the time sequence of control instructions provided to the robot device is directly associated with the task to be performed. Therefore, if at least one of the environment and the object for performing the task changes slightly, the learning result cannot cope with the change, and there is a possibility that the task cannot be properly performed. In contrast, in this structure, the sequence of control instructions provided to the robot device is not directly associated with the task, but is associated with the change in the positional relationship between the objects (task state). That is, the time sequence of control instructions can be generated or taught with respect to the change in the positional relationship without relying on the content of the task. Therefore, according to this structure, the versatility of the ability to perform the mastered task can be improved, thereby reducing the cost of teaching the task to the robot device.

[0022] In addition, the robotic device may include, for example, an industrial robot such as a manipulator, a mobile body capable of automatic movement, and the like. In the case where the robotic device is a manipulator, the first object may be the end effector of the manipulator, and the second object may be a workpiece that is an object held by the end effector. In this case, the task performed by the manipulator may be to hold the workpiece by the end effector. In addition, after the end effector holds the workpiece, the first object may be the workpiece, and the second object may be another workpiece that is used to configure the workpiece. In this case, the task performed by the manipulator may be to configure the workpiece held by the end effector to another workpiece. The types of the end effector, the workpiece, and other workpieces may not be particularly limited, and may be appropriately selected according to the embodiment. The end effector may be, for example, a clamp, a suction device, a screwdriver, and the like. The workpiece may be, for example, a connector, a wedge, and the like. The other workpieces may be, for example, a socket, a hole, and the like.

[0023] A task is an operation that a robot device performs, and its type is not particularly limited and can be appropriately selected according to the implementation method. For example, a task can be part handling, part fitting, screw rotation, etc. A task can also be a simple operation such as holding a workpiece or releasing a workpiece. A task can be, for example, a change in the coordinates of an object workpiece, such as holding the object workpiece and configuring the object workpiece at a specified coordinate (position and posture). A task can be, for example, using a sprayer as an end effector to spray paint onto the workpiece from a specified relative coordinate. In addition, a task can also be, for example, configuring a camera mounted on the end effector at a specified coordinate. The task can be provided in advance or provided by the operator's designation.

[0024] The "final goal" is the final destination point (goal), which is achieved at the time point when the execution of the task is completed and is set according to the task provided. The "goal" includes the final goal and can be set appropriately in order to achieve the execution of the task. The number of goals set can be one (in this case, only the final goal is set) or multiple. Goals other than the final goal are passing points passed from the starting point to the end point. Therefore, the final goal can also be referred to as the "goal", and goals other than the final goal can be called "sub-goals (sub-goals)". Sub-goals can also be called "passing points".

[0025] In the control device according to the above aspect, the positional relationship between the first object and the second object can be expressed using relative coordinates. This configuration allows for an appropriate and clear expression of the physical and relative positional relationship between the first object and the second object. However, the method for expressing the positional relationship is not limited to this example and can be appropriately selected depending on the implementation. The positional relationship between the first object and the second object can also be expressed, for example, using their respective absolute coordinates.

[0026] The control device according to the above aspect further includes a model generation unit that generates an inference model using the determination result of the learned determination model, the inference model being used to infer the task state of the target to be switched to next based on the current task state and the task state of the final target so that the first target does not come into contact with the second target. The action determination unit determines the task state of the target to be switched to next relative to the current task state using the generated inference model.

[0027] Based on the learned judgment model, it is possible to determine whether the first and second objects are in contact with each other. Therefore, by utilizing the judgment results of the learned judgment model, it is possible to construct an inference model that can infer the target task state from the current task state and the final target task state in a manner that avoids unnecessary contact between the first and second objects. This configuration utilizes this inference model when controlling the operation of the robot device, making it possible to determine the target task state so that the first and second objects do not come into contact with each other, even without the computational processing of the judgment model. This reduces the computational cost of the control processing.

[0028] In addition, inferring the task state of the target in a manner that prevents the first object from contacting the second object means: determining the task state of the target in a manner that avoids accidental contact between the first object and the second object. For example, it may also include determining the task state in which the first object, such as the end effector holding the workpiece, is in proper contact with the second object as the task state of the target.

[0029] In the control device involved in the above-mentioned aspect, the model generation unit can also generate the inference model by implementing machine learning. According to this structure, an inference model that can be used to perform a task can be appropriately generated. In addition, the type of machine learning model that constitutes the inference model may not be particularly limited and can be appropriately selected according to the implementation method. The inference model can be expressed, for example, by a functional formula, a data table, etc. In the case of expressing it using a functional formula, the inference model can be composed of, for example, a neural network, a support vector machine, a regression model, a decision tree, etc. In addition, the machine learning method may not be particularly limited and can be appropriately selected according to the structure of the inference model. The machine learning of the inference model can adopt, for example, supervised learning, reinforcement learning, etc.

[0030] In the control device involved in the above-mentioned aspect, the machine learning can be composed of reinforcement learning, and the reinforcement learning sets a reward based on the result of determining whether the first object and the second object are in contact with each other in the task state of the object through the learned judgment model, and the distance between the task state of the object and the task state of the final target. The reward can be that the first object and the second object are not in contact with each other, and the shorter the distance, the greater the reward is set, and the first object and the second object are in contact with each other, or the longer the distance, the less the reward is set. The implementation of the machine learning can include repeatedly correcting the values ​​of the calculation parameters constituting the inference model to obtain more of the reward. According to this structure, an inference model can be generated that can avoid unnecessary contact between the first object and the second object, and can determine the target task state in a manner that the task state of the robot device quickly reaches the task state of the final target. In this way, a control device that can control the robot device in a manner that appropriately performs the task can be realized.

[0031] In the control device involved in the above-mentioned aspect, the inference model can be composed of a potential field, which specifies the potential of each coordinate in the task space that expresses the set of task states. The model generation unit can also use the learned judgment model in the task space to implement path planning in a manner that the first object does not contact the second object, thereby generating a path from each of the multiple task states provided as starting points to the task state of the final target, and can also generate the potential field by setting the potential of each coordinate according to the passing frequency of each generated path. The action decision unit can also determine the task state of the target to be converted next based on the gradient of the potential set at the coordinate corresponding to the current task state in the generated potential field. According to this structure, an inference model that can be used to execute a task can be appropriately generated.

[0032] Furthermore, the potential of each coordinate in the potential field represents the evaluation value of the positional relationship between the first and second objects at each coordinate relative to reaching the final goal. Specifically, a higher potential indicates a higher probability that the positional relationship at that coordinate will reach the final goal, while a lower potential indicates a lower probability that the positional relationship at that coordinate will reach the final goal. Therefore, by shifting toward a higher potential gradient, it is possible to appropriately reach the final goal task state from any starting point.

[0033] As another embodiment of the model generation device and control device involved in each of the above-mentioned embodiments, one aspect of the present invention can be an information processing method for realizing the various components of the above-mentioned model generation device or control device, or a program, or a computer-readable storage medium that stores such a program. A computer-readable storage medium refers to a medium that stores information such as a program through electrical, magnetic, optical, mechanical or chemical effects. In addition, the control system involved in one aspect of the present invention can be composed of the model generation device involved in any of the above-mentioned embodiments and the control device involved in any of the above-mentioned embodiments. In addition, the model generation device involved in one aspect of the present invention can be composed of a portion related to the model generation unit of the above-mentioned control device.

[0034] For example, one aspect of the present invention relates to a model generation method which is an information processing method in which a computer executes the following steps, namely: a step of acquiring multiple learning data sets, wherein the multiple learning data sets are respectively composed of a combination of training data representing the positional relationship between two objects and correct answer data representing whether the two objects are in contact with each other in the positional relationship; and a step of implementing machine learning of a judgment model using the acquired multiple learning data sets, wherein the implementation of the machine learning is constructed by training the judgment model in a manner that outputs an output value suitable for the corresponding correct answer data relative to the input of the training data for each of the learning data sets.

[0035] In addition, for example, one aspect of the present invention involves a model generation program that causes a computer to execute the following steps, namely: a step of acquiring multiple learning data sets, wherein the multiple learning data sets are respectively composed of a combination of training data representing the positional relationship between two objects and correct answer data representing whether the two objects are in contact with each other in the positional relationship; and a step of implementing machine learning of a judgment model using the acquired multiple learning data sets, wherein the implementation of the machine learning is constituted by training the judgment model in a manner that outputs an output value suitable for the corresponding correct answer data relative to the input of the training data for each of the learning data sets.

[0036] For example, one aspect of the present invention relates to a control method that is an information processing method for controlling the operation of a robot apparatus that performs a task of moving a first object in an environment in which a first object and a second object are present. The control method comprises a computer executing the following steps: acquiring a current task state of the first object and the second object, the task state being defined by a positional relationship between the first object and the second object; determining a target task state to be subsequently transitioned to relative to the acquired current task state so as to approximate a final target task state; determining whether the first object and the second object are in contact with each other under the target task state by providing the target task state of the first object and the second object to a learned determination model trained by machine learning to output an output value appropriate for correct answer data indicating whether the first object and the second object are in contact with each other under the target task state, in response to input of training data indicating the positional relationship between the first object and the second object; determining a control instruction to be provided to the robot apparatus so as to change the task state of the robot apparatus from the current task state to the target task state; and controlling the operation of the robot apparatus by providing the determined control instruction to the robot apparatus.

[0037] Furthermore, for example, a control program according to one aspect of the present invention is a program for controlling the operation of a robot apparatus that performs a task of moving a first object in an environment in which a first object and a second object are present, the program causing a computer to execute the following steps: acquiring a current task state of the first object and the second object, the task state being defined by a positional relationship between the first object and the second object; determining a target task state to be subsequently transitioned to relative to the acquired current task state so as to approximate a final target task state; determining whether the first object and the second object are in contact with each other under the target task state by providing the target task state of the first object and the second object to a learned determination model, wherein the learned determination model is trained through machine learning to: output an output value appropriate for correct answer data indicating whether the first object and the second object are in contact with each other under the positional relationship, in response to input of training data indicating the positional relationship between the first object and the second object; determining a control instruction to be provided to the robot apparatus so as to change the task state of the robot apparatus from the current task state to the target task state; and controlling the operation of the robot apparatus by providing the determined control instruction to the robot apparatus.

[0038] Effects of the Invention

[0039] According to the present invention, the amount of data expressing the boundary where two objects come into contact can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 An example of an application scenario of the present invention is schematically illustrated.

[0041] Figure 2A An example of the positional relationship between two objects according to the embodiment is schematically illustrated.

[0042] Figure 2B An example of the positional relationship between two objects according to the embodiment is schematically illustrated.

[0043] Figure 3 An example of a method of storing a value indicating whether two objects are in contact at each coordinate point is schematically illustrated.

[0044] Figure 4 An example of the hardware configuration of the model generation device according to the embodiment is schematically illustrated.

[0045] Figure 5 An example of the hardware configuration of the control device according to the embodiment is schematically illustrated.

[0046] Figure 6 An example of the robot device according to the embodiment is schematically illustrated.

[0047] Figure 7 An example of the software configuration of the model generation device according to the embodiment is schematically illustrated.

[0048] Figure 8 An example of the software configuration of the control device according to the embodiment is schematically illustrated.

[0049] Figure 9 An example of the processing procedure of the model generation device according to the embodiment is illustrated.

[0050] Figure 10 An example of a processing procedure related to generation of an inference model performed by the control device according to the embodiment will be described.

[0051] Figure 11A An example of a task space according to the embodiment is schematically illustrated.

[0052] Figure 11B An example of a task space according to the embodiment is schematically illustrated.

[0053] Figure 11C An example of a task space according to the embodiment is schematically illustrated.

[0054] Figure 12 An example of the configuration and generation method of the inference model according to the embodiment is schematically illustrated.

[0055] Figure 13 An example of the configuration and generation method of the inference model according to the embodiment is schematically illustrated.

[0056] Figure 14A An example of learning data according to the embodiment is schematically illustrated.

[0057] Figure 14B An example of the configuration of an inference model according to the embodiment is schematically illustrated.

[0058] Figure 15A An example of a processing procedure related to the control of the operation of the robot apparatus by the control device according to the embodiment is illustrated.

[0059] Figure 15B An example of a processing procedure related to the control of the operation of the robot apparatus by the control device according to the embodiment is illustrated.

[0060] Figure 16 An example of a calculation process of each element according to the embodiment is illustrated.

[0061] Figure 17 The positional relationship between objects according to the embodiment is schematically illustrated.

[0062] Figure 18A An example of the relationship between each joint and the fingertip when the end effector is not holding a workpiece is schematically shown.

[0063] Figure 18B An example of the relationship between the joints and the fingertips when the end effector holds a workpiece is schematically shown.

[0064] Figure 19 An example of a processing procedure of a subroutine related to target determination performed by a control device according to a modification example is illustrated.

[0065] Figure 20 An example of the software configuration of the model generation device according to the modification is schematically illustrated. DETAILED DESCRIPTION

[0066] Hereinafter, an embodiment of one aspect of the present invention (hereinafter also referred to as "the present embodiment") is described with reference to the accompanying drawings. However, the present embodiment described below is merely an illustration of the present invention in all respects. Of course, various improvements and modifications can be made without departing from the scope of the present invention. That is to say, when implementing the present invention, a specific configuration that conforms to the embodiment may also be appropriately adopted. It should be noted that, in the present embodiment, natural language is used to describe the data that appears, but, more specifically, it is specified using a computer-recognizable simulation language, command, parameter, machine language, etc.

[0067] §1 Application

[0068] First, use Figure 1 An example of an applicable scenario of the present invention is described. Figure 1 An example of an applicable scenario of the present invention is schematically illustrated. Figure 1 As shown, the control system 100 according to this embodiment includes a model generation device 1 and a control device 3. The model generation device 1 and the control device 3 can be connected to each other via a network. The type of network can be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like.

[0069] <Model Generation Device>

[0070] The model generation device 1 according to this embodiment is a computer configured to generate a determination model 50 for determining whether two objects are in contact with each other in a positional relationship. Specifically, the model generation device 1 according to this embodiment acquires a plurality of learning data sets 121, each of which is composed of a combination of training data 122 representing the positional relationship between two objects and correct answer data 123 indicating whether the two objects are in contact with each other in that positional relationship.

[0071] In this embodiment, the positional relationship between two objects is expressed by relative coordinates. Relative coordinates are the coordinates when observing the object on the other side from the object on one side. Any one of the two objects can be selected as the basis for the relative coordinates. "Coordinates" can include at least one of position and posture. In three-dimensional space, position can be expressed by three axes: front and back, left and right, and up and down, and posture can be expressed by the rotation of each axis (roll, pitch, yaw). In this embodiment, relative coordinates can be expressed by six dimensions: three-dimensional relative position and three-dimensional relative posture. In addition, the number of dimensions of relative coordinates is not limited to six dimensions and can be appropriately reduced.

[0072] The model generation device 1 according to this embodiment then performs machine learning on the determination model 50 using the acquired plurality of learning data sets 121. This machine learning is performed by training the determination model 50 for each learning data set 121 so that the determination model 50 outputs an output value appropriate for the corresponding correct answer data 123 in response to the input of the training data 122. This machine learning allows the construction of a learned determination model 50 that has mastered the ability to determine whether two objects are in contact with each other in their positional relationship.

[0073] In this embodiment, the learned judgment model 50 is used to determine whether contact occurs between the workpiece W and the end effector T, and whether contact occurs between the workpiece W and the other workpieces G, in a space where a robot 4 having an end effector T, a workpiece W, and other workpieces G are present. The robot 4 is an example of a "robot device." The end effector T, the workpiece W, and other workpieces G are examples of "objects." The types of the end effector T, the workpiece W, and other workpieces G are not particularly limited and can be appropriately selected according to the task. The end effector T can be, for example, a clamp, a suction device, a screwdriver, etc. The workpiece W can be, for example, a connector, a wedge, etc. The other workpieces G can be, for example, a socket, a hole, etc. The other workpieces G are examples of objects to be assembled by the workpiece W. Holding the workpiece W by the end effector T can be, for example, gripping the workpiece by a clamp, holding the workpiece by suction by a suction device, holding the workpiece at the tip of a screwdriver, etc.

[0074] More specifically, as an example, the robot 4 involved in this embodiment performs a task of holding a workpiece W by means of an end effector T and assembling the held workpiece W onto another workpiece G. This task can be divided into two tasks: a first task of holding the workpiece W by means of the end effector T and a second task of transporting the held workpiece W to another workpiece G. In the scenario where the first task of holding the workpiece W is performed by moving the end effector T, the learned judgment model 50 is used to determine whether unnecessary contact occurs between the workpiece W and the end effector T. In addition, in the scenario where the second task of transporting the held workpiece W to another workpiece G is performed by moving the end effector T after holding the workpiece W, the learned judgment model 50 is used to determine whether unnecessary contact occurs between the workpiece W and the other workpiece G.

[0075] That is, in this embodiment, the learned determination model 50 determines whether at least one of the two objects in contact is an object that is moved by the robot (manipulator 4). Alternatively, only one of the two objects may be moved by the robot, or both may be moved by the robot. However, the application of the present invention is not limited to these examples. The present invention is applicable to all scenarios in which contact between two objects is determined.

[0076] Furthermore, as described above, when there are multiple objects to determine whether contact exists, multiple learned determination models 50 may be prepared, each for determining whether contact exists between different objects. Alternatively, the learned determination model 50 may also be configured to accept input of information indicating object conditions, such as object type and object identifier, and determine whether contact exists between two objects corresponding to the input conditions. Any method may be employed. For ease of explanation, the following description will be based on the learned determination model 50 without distinguishing between the objects to be determined.

[0077] <Control device>

[0078] The control device 3 involved in this embodiment is a computer configured to control the operation of a robot device that performs a task of moving the first object relative to the second object in an environment in which a first object and a second object are present. The first and second tasks described above are examples of this "task of moving the first object relative to the second object." When performing the first task, the end effector T is an example of the first object, and the workpiece W is an example of the second object. Furthermore, when performing the second task, the workpiece W is an example of the first object, and another workpiece G is an example of the second object. As described above, the robot arm 4 is an example of a robot device.

[0079] First, the control device 3 involved in this embodiment obtains information indicating the current task status of the first object and the second object. The task status is determined by the positional relationship between the first object and the second object (i.e., the two objects). In this embodiment, a camera S1 is used as a visual sensor for observing the task status. In addition, as described later, an encoder is used to measure the angle of each joint of the robot 4, and a tactile sensor is used to measure the holding state of the end effector T on the workpiece W. In this embodiment, the current task status can be observed by these sensors.

[0080] Here, further use Figure 2A and Figure 2B, a specific example of a method of defining a task state based on a positional relationship between a first object and a second object is described. Figure 2A An example of the positional relationship between the end effector T and the workpiece W in a scenario where the first task is performed is schematically illustrated. Figure 2B An example of the positional relationship between the workpiece W and another workpiece G in the scenario of executing the second task is schematically illustrated. As described above, in this embodiment, the positional relationship between two objects is expressed by relative coordinates.

[0081] In this embodiment, if Figure 2A As shown, during a period when the end effector T is not holding the workpiece W, such as in a scenario where the first task is being executed, the focus point T0 of the end effector T is treated as the fingertip of the robot 4. In the first task, the workpiece W is the target of movement of the end effector T. The positional relationship between the end effector T and the workpiece W is expressed by the relative coordinates RC1 of the workpiece W relative to the end effector T. The relative coordinates RC1 represent the local coordinate system CW with the focus point W0 of the workpiece W as its origin, as viewed from the local coordinate system CT with the focus point T0 of the end effector T as its origin. In this embodiment, the task state of the robot 4 in the scenario where the first task is being executed is defined by the relative coordinates RC1.

[0082] On the other hand, Figure 2B As shown, while the end effector T is holding the workpiece W, such as in a scenario where the second task is being executed, the focus point W0 of the workpiece W is treated as the fingertip of the robot 4. In the second task, the other workpiece G is the target of movement of the end effector T. The positional relationship between the workpiece W and the other workpiece G is expressed by the relative coordinates RC2 of the other workpiece G relative to the workpiece W. The relative coordinates RC2 represent the local coordinate system CG with the focus point G0 of the other workpiece G as its origin, as viewed from the local coordinate system CW with the focus point W0 of the workpiece W as its origin. In this embodiment, the task state of the robot 4 in the scenario where the second task is being executed is specified by the relative coordinates RC2.

[0083] That is, in this embodiment, in scenarios where both the first and second tasks are being executed, the task state is defined by the positional relationship (in this embodiment, relative coordinates) between the fingertips of manipulator 4 and the target object. The fingertips of manipulator 4 correspond to the first target object, and the target object corresponds to the second target object. Thus, both the first and second tasks can be understood as tasks that involve moving the fingertips of manipulator 4 relative to the target object. Therefore, according to this embodiment, the control process for manipulator 4 can be simplified, thereby reducing the cost of generating or teaching the movements of manipulator 4.

[0084] In addition, each focus point (T0, W0, G0) can be set arbitrarily. In addition, the method of assigning relative coordinates may not be limited to the above-mentioned examples, and may be appropriately determined according to the embodiment. For example, the relative coordinate RC1 may also be expressed as a local coordinate system CT with the focus point T0 of the end effector T as the origin, as observed from the local coordinate system CW with the focus point W0 of the workpiece W as the origin, and the relationship between the relative coordinates (RC1, RC2) may be exchanged. In addition, moving the fingertip may not be limited to bringing the fingertip closer to the target object, and may be appropriately determined according to the embodiment. Moving the fingertip may, for example, be moving the fingertip away from the target object, moving the fingertip to a specified position based on the target object, and so on.

[0085] Next, the control device 3 according to this embodiment determines the next target task state to transition to, relative to the current task state indicated by the acquired information, so as to approximate the target task state. In the context of executing the first task, the state in which the workpiece W is held by the end effector T is an example of the target task state. Furthermore, in the context of executing the second task, the state in which the workpiece W is positioned at a desired position relative to another workpiece G is an example of the target task state.

[0086] Furthermore, the control device 3 according to this embodiment determines whether the first and second objects are in contact with each other under the task state of the object by providing information indicating the task state of the first and second objects to the learned determination model 50. The learned determination model 50 is trained through the aforementioned machine learning to output, in response to input of training data (training data 122) indicating the positional relationship between the first and second objects, correct answer data (correct answer data 123) appropriate for indicating whether the first and second objects are in contact under the positional relationship.

[0087] The control device 3 according to this embodiment determines a control command to be provided to the robot (manipulator 4) so ​​as to change the task state of the robot (manipulator 4) from the current task state to the target task state. The control device 3 then drives the robot by providing the determined control command. Thus, the control device 3 according to this embodiment can control the operation of the robot to perform the task of moving a first object relative to a second object.

[0088] Effects

[0089] As described above, the model generation device 1 according to this embodiment uses machine learning to generate a determination model 50 for determining whether two objects are in contact within a positional relationship. Based on the learned determination model 50 generated through machine learning, even when the positional relationship of the objects is provided as continuous values ​​(relative coordinates in this embodiment), it is possible to determine whether the two objects are in contact within that positional relationship without significantly increasing the amount of data in the determination model 50. Therefore, according to this embodiment, the amount of data representing the boundary between the two objects in contact can be significantly reduced.

[0090] Here, further use Figure 3 A specific example of this action and effect will be described. Figure 3 This diagram schematically illustrates an example of a method for storing a value indicating whether two objects are in contact with each other at each coordinate point. A white circle indicates that the two objects are not in contact with each other in the positional relationship corresponding to the coordinate, and a black circle indicates that the two objects are in contact with each other in the positional relationship corresponding to the coordinate. Figure 3 In the 6D space, each coordinate point is expressed in two dimensions. However, in the 6D relative coordinate space described above, each coordinate point is expressed in six dimensions. In this case, increasing the spatial resolution (resolving power) increases the amount of data by the order of 6. For example, if the coordinate points are set at a resolution that can be used in real-world space, the amount of data for this information can easily be measured in gigabytes.

[0091] In contrast, in this embodiment, the learned judgment model 50 maintains information indicating whether two objects are in contact with each other in the positional relationship of the objects. Although the number of operation parameters of the learned judgment model 50 depends on the dimension of the relative coordinates, continuous values ​​can be processed without increasing the number of operation parameters. Therefore, for example, as described later, when the judgment model 50 is composed of a three-layer neural network, the amount of data of the learned judgment model 50 can be suppressed to about several megabytes. Therefore, according to this embodiment, the amount of data expressing the boundary of contact between two objects can be greatly reduced. As a result, even a computer with relatively low performance, such as an embedded system, can realize a control device 3 that can appropriately drive the robot device in a manner that avoids unnecessary contact between the first object and the second object.

[0092] Furthermore, conventional control methods for robotic devices directly associate the time-series control instructions provided to the robotic device with the task to be performed. Specifically, in conventional control methods, the task to be performed is directly described by the sequence of control instructions. Consequently, if at least one of the task's environment or the object being performed changes slightly, the learning results may not reflect this change, and the task may not be properly performed.

[0093] For example, consider a scenario where a robot is taught to hold a workpiece using an end effector. In this scenario, as long as the workpiece is accurately positioned at the intended location, the robot can hold the workpiece using the end effector based on the learning results. On the other hand, if the workpiece's posture differs from that during learning, or if the workpiece is positioned differently than during learning, the coordinates used to hold the workpiece by the end effector change. Consequently, the content of the task the robot is supposed to perform in this scenario changes substantially. Consequently, under the sequence of control instructions derived from the learning results, the robot may not be able to properly hold the workpiece using the end effector.

[0094] Thus, conventional control methods have the following problem: if at least one of the task environment or the object being performed changes slightly, the learning results may not be able to adapt to this change. Without relearning the task, the robot may not be able to properly perform the task. For this reason, in order to ensure universal operation of the robot, control instructions must be learned for different conditions even for the same task, which in turn increases the cost of teaching the robot the task.

[0095] In contrast, in this embodiment, the state of the task performed by the robot 4 is expressed by the relative relationship between objects, such as the end effector T, the workpiece W, and other workpieces G, specifically the positional relationship between the objects. Thus, the control instructions provided to the robot 4 are not directly associated with the task, but rather with the amount of change in the relative positional relationship between the objects. That is, regardless of the content of the task, a time-series of control instructions provided to the robot 4 can be generated or taught to change the relative positional relationship of the objects. For example, in the above example, even if the coordinates of the workpiece change, the change in the coordinates of the workpiece is taken into account when learning the positional relationship between the end effector and the workpiece. Therefore, the robot device can appropriately hold the workpiece based on the learning results. Therefore, according to this embodiment, the versatility of the ability to perform learned tasks can be improved, thereby reducing the cost of teaching the task to the robot 4.

[0096] §2 Example of construction

[0097] [Hardware composition]

[0098] <Model Generation Device>

[0099] Next, use Figure 4 An example of the hardware configuration of the model generation device 1 according to this embodiment will be described. Figure 4 An example of the hardware configuration of the model generation device 1 according to the present embodiment is schematically illustrated.

[0100] like Figure 4As shown, the model generation device 1 according to this embodiment is a computer including a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a driver 17 electrically connected. Figure 4 In the description, the communication interface and the external interface are described as "communication I / F" and "external I / F".

[0101] The control unit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory), serving as hardware processors, and is configured to execute information processing based on programs and various data. The storage unit 12 is an example of a memory device and may be comprised of, for example, a hard disk drive or solid-state drive. In this embodiment, the storage unit 12 stores various types of information, including the model generation program 81, CAD (computer-aided design) data 120, multiple learning data sets 121, and learning result data 125.

[0102] The model generation program 81 is used to cause the model generation device 1 to execute information processing (described later) related to machine learning of the determination model 50. Figure 9 ) program. The model generation program 81 includes a series of commands for processing this information. The CAD data 120 includes configuration information representing the geometric configuration of a model (e.g., a three-dimensional model) of each object (end effector T, workpiece W, other workpieces G). The CAD data 120 can be generated by publicly known software. A plurality of learning data sets 121 are used for machine learning of the judgment model 50. The learning result data 125 represents information related to the learned judgment model 50 generated by machine learning. The learning result data 125 is obtained as a result of executing the model generation program 81. This will be described in detail later.

[0103] The communication interface 13 is an interface for performing wired or wireless communication via a network, such as a wired LAN (Local Area Network) module or a wireless LAN module. By utilizing the communication interface 13, the model generation device 1 can perform data communication with other information processing devices (e.g., the control device 3) via the network.

[0104] The external interface 14 is an interface for connecting to external devices, such as a USB (Universal Serial Bus) port or a dedicated port. The type and number of external interfaces 14 can be appropriately selected based on the type and number of external devices to be connected. The model generation device 1 can also be connected to the robot 4 and the camera S1 via the external interface 14 to determine whether an object is in contact in real space.

[0105] The input device 15 is a device for inputting information, such as a mouse or keyboard. The output device 16 is a device for outputting information, such as a display or a speaker. The operator can operate the model generating device 1 using the input device 15 and the output device 16 .

[0106] The drive 17 is, for example, a CD drive or a DVD drive, and is a drive device for reading the program stored in the storage medium 91. The type of the drive 17 can be appropriately selected according to the type of the storage medium 91. At least one of the model generation program 81, the CAD data 120, and the plurality of learning data sets 121 may also be stored in the storage medium 91.

[0107] Storage medium 91 is a medium that stores information such as a program, etc., through electrical, magnetic, optical, mechanical, or chemical processes, in a manner that enables a computer, other device, or machine to read the stored information. Model generation device 1 can also obtain at least one of the model generation program 81, CAD data 120, and multiple learning data sets 121 from storage medium 91.

[0108] Here, in Figure 4 In the embodiment, a disk-type storage medium such as a CD or DVD is shown as an example of the storage medium 91. However, the type of storage medium 91 is not limited to a disk-type storage medium and may be a type other than a disk-type storage medium. Examples of storage media other than a disk-type storage medium include semiconductor memories such as flash memory.

[0109] In addition, regarding the specific hardware configuration of the model generation device 1, components may be omitted, replaced, or added as appropriate depending on the implementation method. For example, the control unit 11 may also include multiple hardware processors. The hardware processor may be composed of a microprocessor, an FPGA (field-programmable gate array), a DSP (digital signal processor), etc. The storage unit 12 may also be composed of the RAM and ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may also be omitted. The model generation device 1 may also be composed of multiple computers. In this case, the hardware configuration of each computer may be consistent or inconsistent. In addition, in addition to being an information processing device designed specifically for the service provided, the model generation device 1 may also be a general-purpose server device, a PC (Personal Computer), etc.

[0110] <Control device>

[0111] Next, use Figure 5 An example of the hardware configuration of the control device 3 according to this embodiment will be described. Figure 5 An example of the hardware configuration of the control device 3 according to the present embodiment is schematically illustrated.

[0112] like Figure 5 As shown, the control device 3 according to this embodiment is a computer to which a control unit 31, a storage unit 32, a communication interface 33, an external interface 34, an input device 35, an output device 36, and a driver 37 are electrically connected. Figure 5 In, with Figure 4 Likewise, a communication interface and an external interface are described as “communication I / F” and “external I / F”.

[0113] The control unit 31 through the driver 37 of the control device 3 can be configured similarly to the control unit 11 through the driver 17 of the aforementioned model generation device 1. Specifically, the control unit 31 includes a CPU, RAM, ROM, and other hardware processors, and is configured to perform various information processing operations based on programs and data. The storage unit 32 is comprised of, for example, a hard disk drive or solid-state drive. The storage unit 32 stores various types of information, including the control program 83, CAD data 320, robot data 321, learning result data 125, learning data 323, and inference model data 325.

[0114] The control program 83 is for causing the control device 3 to execute information processing (described later) related to the generation of an inference model for inferring the target task state. Figure 10) and the information processing ( Figure 15A and Figure 15B ) program. The control program 83 includes a series of commands for processing this information. Like the aforementioned CAD data 120, the CAD data 320 includes configuration information representing the geometric configuration of the models of each object (end effector T, workpiece W, other workpieces G). The robot data 321 includes configuration information representing the configuration of the manipulator 4, such as parameters of each joint. The learning result data 125 is used to set the learned judgment model 50. The learning data 323 is used to generate an inference model. The inference model data 325 represents information related to the generated inference model. This will be described in detail later.

[0115] The communication interface 33 is, for example, a wired LAN module or a wireless LAN module, and is an interface for wired or wireless communication via a network. By using the communication interface 33, the control device 3 can communicate data with other information processing devices (eg, the model generation device 1) via the network.

[0116] The external interface 34 is an interface for connecting to external devices, such as a USB port or a dedicated port. The type and number of external interfaces 34 can be appropriately selected based on the type and number of external devices to be connected. The control device 3 can be connected to the camera S1 and the manipulator 4 via the external interface 34. In this embodiment, the manipulator 4 includes an encoder S2 for measuring the angle of each joint and a tactile sensor S3 for measuring the force acting on the end effector T.

[0117] The types of camera S1, encoder S2, and tactile sensor S3 are not particularly limited and can be appropriately determined according to the implementation. Camera S1 may be, for example, a conventional digital camera configured to capture RGB images, a depth camera configured to capture depth images, or an infrared camera configured to image infrared radiation. Tactile sensor S3 may be, for example, a contact sensor.

[0118] The control device 3 can obtain sensing data (image data, angle data, pressure distribution data) from each sensor (camera S1, encoders S2, tactile sensor S3) via the external interface 34. Furthermore, the connection method with the camera S1 and the manipulator 4 is not limited to this example. For example, if the camera S1 and the manipulator 4 have communication interfaces, the control device 3 can also be connected to the camera S1 and the manipulator 4 via the communication interface 33.

[0119] The input device 35 is a device for inputting information, such as a mouse or keyboard. The output device 36 is a device for outputting information, such as a display or a speaker. The operator can operate the control device 3 using the input device 35 and the output device 36.

[0120] The drive 37 is, for example, a CD drive or a DVD drive, and is a drive device for reading the program stored in the storage medium 93. The storage medium 93, like the storage medium 91, may be a disk-type device or a device other than a disk-type device. At least one of the control program 83, CAD data 320, robot data 321, learning result data 125, and learning data 323 may also be stored in the storage medium 93. Furthermore, the control device 3 may also obtain at least one of the control program 83, CAD data 320, robot data 321, learning result data 125, and learning data 323 from the storage medium 93.

[0121] In addition, regarding the specific hardware composition of the control device 3, the components can be appropriately omitted, replaced, and added according to the implementation method. For example, the control unit 31 may also include multiple hardware processors. The hardware processor may be composed of a microprocessor, an FPGA, a DSP, etc. The storage unit 32 may also be composed of the RAM and ROM contained in the control unit 31. At least any one of the communication interface 33, the external interface 34, the input device 35, the output device 36, and the driver 37 may also be omitted. The control device 3 may also be composed of multiple computers. In this case, the hardware composition of each computer may be consistent or inconsistent. In addition, in addition to being an information processing device designed to be dedicated to the service provided, the control device 3 may also be a general-purpose server device, a general-purpose PC, a PLC (programmable logic controller), etc.

[0122] <Robot>

[0123] Next, use Figure 6 An example of the hardware configuration of the robot 4 according to this embodiment will be described. Figure 6 An example of the hardware configuration of the robot 4 according to this embodiment is schematically illustrated.

[0124] The manipulator 4 involved in this embodiment is a six-axis vertical multi-joint industrial robot, comprising a base 40 and six joints 41 to 46. Each joint 41 to 46 is configured to be rotatable around each axis by a built-in servo motor (not shown). The first joint 41 is connected to the base 40 so that the front end portion rotates around the axis of the base. The second joint 42 is connected to the first joint 41 so that the front end portion rotates in the front-to-back direction. The third joint 43 is connected to the second joint 42 via a connecting rod 491 so that the front end portion rotates in the up-down direction. The fourth joint 44 is connected to the third joint 43 via a connecting rod 492 so that the front end portion rotates around the axis of the connecting rod 492. The fifth joint 45 is connected to the fourth joint 44 via a connecting rod 493 so that the front end portion rotates in the up-down direction. The sixth joint 46 is connected to the fifth joint 45 via a connecting rod 494 so that the front end portion rotates around the axis of the connecting rod 494. An end effector T is attached to the distal end side of the sixth joint 46 together with the tactile sensor S3 .

[0125] Each joint 41 to 46 also has an encoder S2 built in. Each encoder S2 is configured to measure the angle (control amount) of each joint 41 to 46. The measurement data (angle data) of each encoder S2 can be used to control the angle of each joint 41 to 46. In addition, the tactile sensor S3 is configured to detect the force acting on the end effector T. The measurement data (pressure distribution data) of the tactile sensor S3 can be used to infer the position and posture of the workpiece W held by the end effector T, or to detect whether an abnormal force is acting on the end effector T.

[0126] In addition, the hardware structure of the manipulator 4 may not be limited to such an example. Regarding the specific hardware structure of the manipulator 4, the components can be appropriately omitted, replaced, and added according to the implementation method. For example, the manipulator 4 may also have sensors other than the encoder S2 and the tactile sensor S3 to observe the control amount or other properties. For example, the manipulator 4 may also have a torque sensor. In this case, the manipulator 4 can measure the force acting on the end effector T through the torque sensor, and control it according to the measured value of the torque sensor to prevent excessive force from acting on the end effector T. In addition, the number of axes of the manipulator 4 may not be limited to six axes. The manipulator 4 can adopt a well-known industrial robot.

[0127] [Software composition]

[0128] <Model Generation Device>

[0129] Next, use Figure 7 An example of the software configuration of the model generation device 1 according to this embodiment will be described. Figure 7An example of the software configuration of the model generation device 1 according to the present embodiment is schematically illustrated.

[0130] The control unit 11 of the model generation device 1 loads the model generation program 81 stored in the storage unit 12 into the RAM. Then, the control unit 11 interprets and executes the commands contained in the model generation program 81 loaded into the RAM through the CPU, thereby controlling each component. Figure 7 As shown, the model generation device 1 according to this embodiment operates as a computer having a data acquisition unit 111, a machine learning unit 112, and a storage processing unit 113 as software modules. Specifically, in this embodiment, each software module of the model generation device 1 is implemented by the control unit 11 (CPU).

[0131] The data acquisition unit 111 acquires a plurality of learning data sets 121. Each learning data set 121 is composed of a combination of training data 122 representing the positional relationship between two objects and correct answer data 123 representing whether the two objects are in contact with each other in the positional relationship. The training data 122 is used as input data for machine learning. The correct answer data 123 is used as a teacher signal (label) for machine learning. The forms of the training data 122 and the correct answer data 123 may also not be particularly limited and may be appropriately selected according to the implementation method. For example, the training data 122 may directly utilize the relative coordinates between the two objects, or may utilize the values ​​obtained by converting the relative coordinates into feature quantities. Based on the CAD data 120, it can be determined whether the two objects of the object are in contact with each other in the positional relationship of the object. Therefore, by utilizing the CAD data 120, each learning data set 121 can be generated.

[0132] The machine learning unit 112 uses the acquired multiple learning data sets 121 to perform machine learning on the determination model 50. This machine learning is performed by training the determination model 50 for each learning data set 121 so that the model outputs an output value appropriate for the corresponding correct answer data 123, relative to the input of the training data 122. This machine learning constructs a learned determination model 50 capable of determining whether two objects are in contact with each other. The storage processing unit 113 generates information related to the constructed learned determination model 50 as learning result data 125 and stores the generated learning result data 125 in a predetermined storage area.

[0133] (Composition of Judgment Model)

[0134] Next, an example of the structure of the determination model 50 is described. The determination model 50 according to this embodiment is composed of a multi-layered neural network for deep learning. Figure 7In the example, the decision model 50 is composed of a three-layer fully connected neural network. The decision model 50 includes an input layer 501, an intermediate (hidden) layer 502, and an output layer 503. However, the structure of the decision model 50 is not limited to this example and can be appropriately determined according to the implementation method. For example, the number of intermediate layers in the decision model 50 is not limited to one and can also be two or more. Alternatively, the intermediate layer 502 can be omitted.

[0135] The number of neurons (nodes) included in each layer 501 to 503 can be appropriately determined according to the embodiment. For example, the number of neurons in the input layer 501 can be determined according to the dimension of the relative coordinates that express the positional relationship between the two objects. In addition, the number of neurons in the output layer 503 can be determined according to the method of expressing whether the two objects are in contact with each other. For example, in the case where a numerical value is used to express whether the two objects are in contact with each other (for example, a numerical value in the range of [0, 1]), the number of neurons in the output layer 503 can be one. In addition, for example, in the case where two objects are in contact with each other by two numerical values, a first numerical value representing the probability of contact and a second numerical value representing the probability of non-contact, the number of neurons in the output layer 503 can be two.

[0136] The neurons of adjacent layers are appropriately connected to each other. In the present embodiment, each neuron is connected to all neurons of the adjacent layer. However, the connection relationship between each neuron may not be limited to such an example and may be appropriately set according to the embodiment. A weight (connection weight) is set in each connection. A threshold is set in each neuron. Basically, the output of each neuron is determined by whether the sum of the product of each input and each weight exceeds the threshold. The threshold can also be expressed by an activation function. In this case, the output of each neuron is determined by inputting the sum of the product of each input and each weight into the activation function and performing the operation of the activation function. The type of activation function may also not be particularly limited and may be appropriately selected according to the embodiment. The weight of the connection between each neuron contained in each layer 501 to 503 and the threshold of each neuron are examples of the calculation parameters of the judgment model 50.

[0137] In this embodiment, the machine learning unit 112 performs machine learning on the determination model 50, which is constructed using the aforementioned neural network, using multiple training data sets 121. Specifically, the machine learning unit 112 adjusts the values ​​of the determination model 50's operational parameters to train the determination model 50 for each training data set 121. This allows the output layer 503 to output an output value appropriate for the correct answer data 123 when the training data 122 is input to the input layer 501. This allows the machine learning unit 112 to generate a learned determination model 50 that has mastered the ability to determine whether two objects are in contact with each other.

[0138] The storage processing unit 113 generates information representing the structure and computational parameters of the constructed and learned decision model 50 as learning result data 125. Furthermore, the storage processing unit 113 stores the generated learning result data 125 in a predetermined storage area. Furthermore, the content of the learning result data 125 is not limited to this example, as long as it can reproduce the learned decision model 50. For example, if the structure of the decision model 50 is to be shared between various devices, information representing the structure of the decision model 50 may be omitted from the learning result data 125.

[0139] <Control device>

[0140] Next, use Figure 8 An example of the software configuration of the control device 3 according to the present embodiment will be described. Figure 8 An example of the software configuration of the control device 3 according to the present embodiment is schematically illustrated.

[0141] The control unit 31 of the control device 3 loads the control program 83 stored in the storage unit 32 into the RAM. Then, the control unit 31 interprets and executes the commands contained in the control program 83 loaded into the RAM through the CPU, thereby controlling each component. Figure 8 As shown, the control device 3 according to this embodiment operates as a computer having a target setting unit 311, a state acquisition unit 312, an action determination unit 313, a contact determination unit 314, a command determination unit 315, a driver unit 316, a data collection unit 317, a model generation unit 318, and a storage processing unit 319 as software modules. Specifically, in this embodiment, as in the aforementioned model generation device 1, each software module of the control device 3 is implemented by the control unit 31 (CPU).

[0142] The target setting unit 311 sets the task state of the final target according to the task to be performed. The task state is determined by the positional relationship between the first object and the second object in the task to be performed. In this embodiment, the positional relationship is expressed by the above-mentioned relative coordinates. The "final target" is the final arrival point (target), which is achieved at the time point when the execution of the task is completed. The state acquisition unit 312 acquires information representing the current task state of the robot device (manipulator 4). "Current" is the time point when the action of the robot device is controlled, and is the time point before the control instruction to be provided to the robot device is determined. The first object and the second object can be appropriately set according to the task to be performed. As an example, in the scenario of performing the first task, the first object is the end effector T, and the second object is the workpiece W. In the scenario of performing the second task, the first object is the workpiece W held by the end effector T, and the second object is another workpiece G.

[0143] The action decision unit 313 determines the task state of the target to be converted next relative to the current task state represented by the acquired information in a manner close to the task state of the final goal. "Goal" includes the final goal, which can be set appropriately in order to achieve the execution of the task. The number of goals set up to the final goal can be one (in this case, only the final goal is set) or multiple. Goals other than the final goal are passing points passed from the starting point of the task to the end point. Therefore, the final goal can also be referred to as "goal", and goals other than the final goal can be referred to as "lower-level goals (sub-goals)". Lower-level goals can also be called "passing points". "The next target to be converted to" is the task state that will be the next target from the current task state (if it is a target other than the final goal, it is a tentative task state), for example, the target that is closest to the current task state toward the final goal.

[0144] The contact determination unit 314 stores the learned determination model 50 by storing the learning result data 125. The contact determination unit 314 sets the learned determination model 50 by referring to the learning result data 125. The learned determination model 50 has acquired the ability to determine whether the first and second objects are in contact with each other through the aforementioned machine learning. The contact determination unit 314 provides the learned determination model 50 with information indicating the task status of the first and second objects, thereby determining whether the first and second objects are in contact with each other under the task status of the objects.

[0145] The command decision unit 315 determines the control command to be provided to the robot (manipulator 4) so ​​as to change the task state of the robot (manipulator 4) from the current task state to the target task state. The driving unit 316 drives the robot by providing the determined control command to the robot. In this embodiment, the operation of the manipulator 4 is controlled through this series of processes.

[0146] The contact determination results of the learned decision model 50 can also be directly utilized in controlling the robot's movements. Alternatively, the contact determination results of the learned decision model 50 can be used to generate an inference model 55, which infers the next target task state from the current task state and the final target task state. Thus, the determination results of the learned decision model 50 can be indirectly utilized in controlling the robot's movements. In this embodiment, the latter method is selected as the method for utilizing the determination results of the learned decision model 50.

[0147] Specifically, the data collection unit 317 and the model generation unit 318 use the judgment results of the learned judgment model 50 to generate an inference model 55. This inference model 55 is configured to determine the next target task state to be transitioned to so that the first object does not come into contact with the second object. The data collection unit 317 uses the judgment results of the learned judgment model 50 to collect learning data 323 for generating the inference model 55. The collection of learning data 323 can further utilize CAD data 320. The model generation unit 318 uses the collected learning data 323 to generate the inference model 55. Details of the learning data 323 and the inference model 55 will be described later.

[0148] The storage processing unit 319 generates information related to the generated inference model 55 as inference model data 325 and stores the generated inference model data 325 in a predetermined storage area. In this embodiment, the action determination unit 313 stores the generated inference model 55 by storing the inference model data 325. The action determination unit 313 uses the generated inference model 55 to determine the next target task state for the current task state.

[0149] <Other>

[0150] The software modules of the model generation device 1 and the control device 3 will be described in detail in the operational examples described below. Furthermore, in this embodiment, an example is described in which the software modules of the model generation device 1 and the control device 3 are implemented by a general-purpose CPU. However, some or all of the above software modules may be implemented by one or more dedicated processors. Furthermore, software modules may be omitted, replaced, or added to the software configurations of the model generation device 1 and the control device 3, depending on the implementation.

[0151] §3 Action Examples

[0152] [Model generation device]

[0153] Next, use Figure 9 An operation example of the model generation device 1 will be described. Figure 9 This is a flowchart illustrating an example of a processing sequence related to machine learning of the determination model 50 performed by the model generation device 1 according to this embodiment. The processing sequence described below is an example of the "model generation method" of the present invention. However, each processing sequence described below is merely an example, and each step can be modified within a reasonable range. Furthermore, steps in each processing sequence described below may be omitted, replaced, or added as appropriate depending on the embodiment.

[0154] (Step S101)

[0155] In step S101, the control unit 11 operates as the data acquisition unit 111 and acquires a plurality of learning data sets 121 used in the machine learning of the determination model 50. Each learning data set 121 is composed of a combination of training data 122 indicating the positional relationship between two objects and correct answer data 123 indicating whether the two objects are in contact with each other in the positional relationship.

[0156] The method for generating each learning dataset 121 is not particularly limited and can be appropriately selected depending on the implementation. For example, using CAD data 120, two objects are arranged in various positional relationships in virtual space. In this embodiment, the positional relationship is expressed using relative coordinates. In addition, in this embodiment, at least one of the two objects is an object moved by the action of the robot device (manipulator 4). In the scenario assuming the execution of the first task described above, the end effector T and the workpiece W are each an example of an object. In the scenario assuming the execution of the second task described above, the workpiece W held by the end effector T and the other workpiece G are each an example of an object. One of the two objects is the fingertip of the manipulator 4, and the other is the target object. The arrangement of the objects can be specified by the operator or determined randomly. Alternatively, various positional relationships can be achieved by fixing the position of one object and changing the position of the other object according to a rule. The rule for arranging the other object can be appropriately set. In this way, relative coordinates in each positional relationship can be obtained as training data 122 for each learning dataset 121. Furthermore, CAD data 120 includes models of each object. Therefore, based on CAD data 120, it is possible to determine whether two objects are in contact with each other in their positional relationship. Using CAD data 120, the results of the determination of whether two objects are in contact with each other in each positional relationship are associated with the corresponding training data 122 as correct answer data 123. This allows the generation of each learning dataset 121. Furthermore, the method for generating each learning dataset 121 is not limited to this example. Each learning dataset 121 can also be generated using the actual objects in real space.

[0157] Each learning data set 121 can be automatically generated by the action of the computer or manually generated by at least partially including the operation of the operator. In addition, the generation of each learning data set 121 can be performed by the model generation device 1 or by another computer other than the model generation device 1. In the case where each learning data set 121 is generated by the model generation device 1, the control unit 11 automatically or manually performs the above series of processes by the operator through the input device 15 to obtain multiple learning data sets 121. On the other hand, in the case where each learning data set 121 is generated by another computer, the control unit 11 obtains multiple learning data sets 121 generated by another computer, for example, via a network, storage medium 91, etc. In this case, the CAD data 120 can also be omitted from the model generation device 1. It is also possible that a part of the learning data set 121 is generated by the model generation device 1, and the other learning data sets 121 are generated by one or more other computers.

[0158] The number of acquired learning data sets 121 is not particularly limited and may be appropriately selected according to the embodiment. When a plurality of learning data sets 121 are acquired, the control unit 11 advances the process to the next step S102.

[0159] (Step S102)

[0160] In step S102, the control unit 11 functions as the machine learning unit 112, performing machine learning on the determination model 50 using the acquired plurality of training data sets 121. In this embodiment, the control unit 11 trains the determination model 50 using machine learning for each training data set 121, so that when the training data 122 is input to the input layer 501, the output layer 503 outputs an output value appropriate for the corresponding correct answer data 123. Thus, the control unit 11 constructs a learned determination model 50 that has mastered the ability to determine whether two objects are in contact with each other in a positional relationship.

[0161] The processing order of machine learning can be appropriately determined according to the implementation method. As an example, the control unit 11 first prepares the determination model 50 as the processing object. The structure of the prepared determination model 50 (for example, the number of layers, the number of neurons contained in each layer, the connection relationship between neurons in adjacent layers, etc.), the initial value of the weight of the connection between each neuron, and the initial value of the threshold of each neuron can be provided by a template or by operator input. In addition, when relearning, the control unit 11 can also prepare the determination model 50 based on the learning result data obtained by performing past machine learning.

[0162] Next, the control unit 11 uses the training data 122 included in each learning data set 121 as input data and the correct answer data 123 as a teacher signal to perform a learning process for the decision model 50 (neural network). This learning process can use batch gradient descent, stochastic gradient descent, mini-batch gradient descent, or the like.

[0163] For example, in the first step, the control unit 11 inputs the training data 122 for each learning data set 121 into the decision model 50 and executes the computational processing of the decision model 50. Specifically, the control unit 11 inputs the training data 122 into the input layer 501 and sequentially determines the firing of each neuron contained in each layer 501 to 503 (i.e., performs forward propagation). Through this computational processing, the control unit 11 obtains an output value from the output layer 503 of the decision model 50 corresponding to the determination result of whether two objects are in contact in the positional relationship represented by the training data 122.

[0164] In the second step, the control unit 11 calculates the error (loss) between the output value obtained from the output layer 503 and the correct answer data 123 using a loss function. The loss function evaluates the difference (i.e., the degree of difference) between the output of the learning model and the correct answer. The greater the difference between the output value obtained from the output layer 503 and the correct answer data 123, the greater the error calculated by the loss function. The type of loss function used to calculate the error is not particularly limited and can be appropriately selected depending on the embodiment.

[0165] In the third step, the control unit 11 calculates the error in the values ​​of the various computational parameters of the judgment model 50 (such as the weights of the connections between neurons and the thresholds of each neuron) using the gradient of the calculated output value error through the back propagation method. In the fourth step, the control unit 11 updates the values ​​of the computational parameters of the judgment model 50 based on the calculated errors. The degree of updating the computational parameter values ​​can be adjusted using a learning rate. The learning rate can be provided by the operator or as a set value within the program.

[0166] The control unit 11 adjusts the values ​​of the operational parameters of the judgment model 50 for each learning data set 121 by repeatedly performing the above-mentioned first to fourth steps so as to reduce the sum of the errors between the output value output from the output layer 503 and the correct answer data 123. For example, the control unit 11 may also repeatedly perform the processing of the above-mentioned first to fourth steps until the sum of the errors becomes below a threshold value. The threshold value may be appropriately set according to the embodiment. Based on the results of the machine learning, the control unit 11 can construct a learned judgment model 50 for each learning data set 121 that is trained to output an output value suitable for the corresponding correct answer data 123 from the output layer 503 when the training data 122 is input to the input layer 501. This "suitability" may also include a difference between the output value of the output layer 503 and the teacher signal (correct answer data 123) that is permissible according to a threshold value or the like. When the machine learning of the judgment model 50 is completed, the control unit 11 causes the processing to proceed to the next step S103.

[0167] (Step S103)

[0168] In step S103, the control unit 11 operates as the storage processing unit 113 and stores information related to the learned decision model 50 constructed through machine learning as learning result data 125 in a predetermined storage area. In this embodiment, the control unit 11 generates information indicating the structure and calculation parameters of the learned decision model 50 constructed in step S102 as learning result data 125. The control unit 11 then stores the generated learning result data 125 in a predetermined storage area.

[0169] The prescribed storage area may be, for example, the RAM within the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination thereof. The storage medium may be, for example, a CD, a DVD, or the like, and the control unit 11 may store the learning result data 125 in the storage medium via the drive 17. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). In this case, the control unit 11 may also store the learning result data 125 in the data server via the network using the communication interface 13. Furthermore, the external storage device may be, for example, an external storage device connected to the model generation device 1.

[0170] Thus, when the storage of the learning result data 125 is completed, the control unit 11 ends a series of processes related to the generation of the learned determination model 50 .

[0171] Furthermore, the generated learning result data 125 can be provided to the control device 3 at any time. For example, the control unit 11 may transmit the learning result data 125 to the control device 3 as part of the processing of step S103 or separately from the processing of step S103. The control device 3 may also obtain the learning result data 125 by receiving the transmission. Alternatively, for example, the control device 3 may access the model generation device 1 or a data server via a network using the communication interface 33 to obtain the learning result data 125. Alternatively, for example, the control device 3 may obtain the learning result data 125 via the storage medium 93. Alternatively, for example, the learning result data 125 may be pre-embedded in the control device 3.

[0172] Furthermore, the control unit 11 may also update or newly generate the learning result data 125 by periodically or irregularly repeating the processing of steps S101 to S103. During this repetition, at least a portion of the plurality of learning data sets 121 may be appropriately modified, corrected, added, or deleted. Furthermore, the control unit 11 may also update the learning result data 125 held by the control device 3 by providing the updated or newly generated learning result data 125 to the control device 3 each time the learning process is executed.

[0173] [Control device]

[0174] (A) Generation of Inference Model

[0175] Next, use Figure 10 An operation example related to the generation of the inference model 55 by the control device 3 will be described. Figure 10 This is a flowchart illustrating an example of a processing sequence for generating an inference model 55 by the control device 3 according to this embodiment. The processing sequences described below are merely examples, and each step can be modified within a reasonable range. Furthermore, steps in the processing sequences described below may be omitted, replaced, or added as appropriate depending on the embodiment.

[0176] (Step S201)

[0177] In step S201, the control unit 31 receives a designation of a final target task state for the task to be executed by the robot 4. The task state is expressed by the positional relationship between the first object and the second object. In this embodiment, the positional relationship is expressed by relative coordinates.

[0178] The method for specifying the relative coordinates in the final task state may also not be particularly limited and may be appropriately selected according to the implementation method. For example, the relative coordinates in the final task state may also be directly specified by the operator through input via the input device 35. In addition, for example, the task to be performed may be selected by the operator's input, and the relative coordinates in the final task state may be specified based on the selected task. In addition, for example, the CAD data 320 may be used to configure the models of each object in the virtual space to the positional relationship of the final target, thereby specifying the relative coordinates in the final target. The configuration of the models of each object may be performed automatically by the simulator or manually by the operator's input. When the task state of the final target is specified, the control unit 31 causes the processing to proceed to the next step S202.

[0179] (Steps S202 to S204)

[0180] In step S202, the control unit 31 sets an arbitrary task state as the starting point. The task state set as the starting point is equivalent to the task state at the time when the task is started. The task state as the starting point can be set randomly or specified by the operator's input. The method by which the operator specifies the starting point can be the same as the method for specifying the final target mentioned above. In addition, the task state as the starting point can be determined by an arbitrary algorithm. As an example, the physical objects of each object can be arranged in an actual space, and each object can be photographed by a camera to obtain image data reflecting each object. Then, the task state as the starting point can be determined by performing image processing on the obtained image data (for example, based on matching of CAD data 320). In addition, the task state as the starting point can also be appropriately determined using CAD data 320.

[0181] In step S203, the control unit 31 functions as the contact determination unit 314, using the learned determination model 50 to determine whether the two objects are in contact with each other in the task state set as the starting point. Specifically, the control unit 31 sets the learned determination model 50 by referring to the learning result data 125. Next, the control unit 31 inputs the relative coordinates of the task state set in step S202 into the input layer 501 of the learned determination model 50. Then, as computational processing for the learned determination model 50, the control unit 31 sequentially determines the firing of each neuron contained in each layer 501 to 503, starting from the input side. Consequently, the control unit 31 obtains an output value from the output layer 503 of the learned determination model 50 corresponding to the result of determining whether the two objects are in contact with each other in the task state set as the starting point.

[0182] In step S204, the control unit 31 determines the branch destination of the process based on the determination result of step S203. If it is determined in step S203 that the two objects are in contact with each other in the task state set as the starting point, the control unit 31 returns the process to step S202 and resets the task state of the starting point. On the other hand, if it is determined that the two objects are not in contact with each other in the task state set as the starting point, the control unit 31 identifies the task state set as the starting point as the current task state of the robot 4 and advances the process to the next step S205.

[0183] Figure 11A An example of a scenario in which the starting point and the final target are set in the task space SP by processing the above steps S201 to S204 is schematically illustrated. The task space SP expresses a set of relative coordinates that specify the task state. The information representing the task space SP may be maintained in the control device 3 or not. Each node (point) belonging to the task space SP corresponds to the relative coordinates between two objects. Figure 11A In the example, node Ns corresponds to the relative coordinates of the starting point in the task state, and node Ng corresponds to the relative coordinates of the final goal in the task state. In this embodiment, the boundary surface (contact boundary surface) that determines whether two objects in the task space SP are in contact is derived based on the judgment results of the learned judgment model 50.

[0184] (Steps S205 to S207)

[0185] In step S205 , the control unit 31 determines the target task state to be transitioned next with respect to the current task state so as to approach the target task state.

[0186] The method for determining the task state of the target may also not be particularly limited and may be appropriately selected according to the implementation method. For example, the relative coordinates of the task state of the target may also be determined by the input of the operator. Similar to the setting of the task state as the starting point, the relative coordinates of the task state of the target may be determined by an arbitrary algorithm or may be appropriately determined using the CAD data 320. In addition, for example, the control unit 31 may also determine the relative coordinates of the task state of the target by randomly changing the relative coordinates of the task state of the starting point. In addition, for example, the control unit 31 may also select a node that is separated from the node Ns by a specified distance in a manner close to the node Ng in the task space SP. The control unit 31 may also obtain the task state corresponding to the selected node as the task state of the target. In addition, for example, in the case where the inference model 55 is generated by the reinforcement learning described later, the inference model 55 in the reinforcement learning process may be used to determine the task state of the target.

[0187] In addition, for example, a well-known method such as path planning can be used to determine the target task state. As an example, the control unit 31 can also set the candidate nodes of the target task state in the task space SP. The setting of the nodes can be performed automatically by methods such as random sampling, or manually by the operator's input. It is also possible to set a part of the nodes automatically and set the remaining nodes manually. After setting the candidate nodes of the target task state, the control unit 31 can appropriately select a combination of nodes that can be converted. The method for selecting the combination of nodes that can be converted can be, for example, the nearest neighbor method. In the task space SP, the combination of nodes that can be converted can be expressed by the edges connecting the nodes. Then, the control unit 31 searches for the path from the starting point node Ns to the final target node Ng. The path search method can be Dijkstra's method, etc. The control unit 31 can also obtain the task state corresponding to the node contained in the path obtained by the search as the target task state.

[0188] In step S206, the control unit 31 functions as the contact determination unit 314, utilizing the learned determination model 50 to determine whether the two objects are in contact with each other under the determined target task state. The control unit 31 performs the same processing as step S203 described above, except that the task state to be determined is replaced with the target task state instead of the starting point. Specifically, the control unit 31 inputs the relative coordinates of the target task state into the learned determination model 50 and executes the calculations of the learned determination model 50. Consequently, the control unit 31 obtains an output value from the learned determination model 50 corresponding to the determination result of whether the two objects are in contact with each other under the target task state.

[0189] In step S207, the control unit 31 determines the branch destination of the process based on the determination result of step S206. If it is determined in step S206 that the two objects are in contact with each other in the target task state, the control unit 31 returns the process to step S205 and determines the target task state again. On the other hand, if it is determined that the two objects are not in contact with each other in the target task state, the control unit 31 advances the process to the next step S208.

[0190] In addition, the branch target in step S207 may not be limited to such an example. For example, when it is determined that the two objects are in contact with each other in the task state of the target, the control unit 31 may also return the processing to step S202 and re-process from the setting of the starting point. In addition, for example, after the task state of the target is determined multiple times, when it is determined that the two objects are in contact with each other in the task state of the last determined target, the control unit 31 may also return the processing to step S205 and re-determine the task state of the next target to be converted from the starting point. The sequence of the task state of the target determined before the contact can also be collected as a failure case of not being able to reach the task state of the final target.

[0191] Figure 11B An example of a scenario in which the target task status is determined by the processing of steps S205 to S207 in the task space SP is schematically illustrated. Figure 11B In the example, the node N1 corresponds to the relative coordinates of the task state determined to be the target task state for the next transition following the task state (node ​​Ns) at the starting point. Figure 11B In the example, it is assumed that the task state of the target for one conversion is determined in step S205. However, the number of target task states determined in step S205 is not limited to one. In step S205, the control unit 31 may determine the task states of the targets for multiple conversions (a sequence of target task states) toward the final target task state.

[0192] (Step S208)

[0193] In step S208, the control unit 31 switches the current task state of the manipulator 4 to the target task state determined in step S205. Then, the control unit 31 determines whether the task state of the manipulator 4 has reached the task state of the final target, that is, whether the task state of the conversion destination is the task state of the final target. The task state transition can be performed in a virtual space through simulation. When it is determined that the task state of the final target has been reached, the control unit 31 advances the processing to the next step S209. On the other hand, when it is determined that the task state of the final target has not been reached, the control unit 31 returns the processing to step S205 to determine a further target task state.

[0194] Figure 11CSchematically illustrates an example of a scenario in which a sequence of task states transitioning from the task state of the starting point to the task state of the final target is determined by the processing up to the above step S208 in the task space SP. Each node N1 to N4 corresponds to the relative coordinates of the task state determined as the target task state before reaching the final target node Ng from the starting point node Ns. Node N(k+1) represents the task state of the target to be transitioned next after node N(k) (k is 1 to 3). Figure 11C As illustrated, through the processing up to step S208 , the control unit 31 can obtain a sequence of task states of targets that transition from the starting point to the final target.

[0195] (Step S209)

[0196] In step S209, the control unit 31 determines whether to repeat the process of step S202 to step S208. The criterion for repeating the process can be appropriately determined according to the embodiment.

[0197] For example, a predetermined number of repetitions of the process may be set. The predetermined number may be provided, for example, as a set value or by an operator's designation. In this case, the control unit 31 determines whether the process of steps S202 to S208 has been executed the predetermined number of times. If it is determined that the execution has not reached the predetermined number of times, the control unit 31 returns the process to step S202 and repeats the process of steps S202 to S208. On the other hand, if it is determined that the execution has reached the predetermined number of times, the control unit 31 proceeds to the next step S210.

[0198] Alternatively, for example, the control unit 31 may ask the operator whether to repeat the process. In this case, the control unit 31 determines whether to repeat the process from step S202 to step S208 based on the operator's answer. If the operator replies that the process is to be repeated, the control unit 31 returns the process to step S202 and repeats the process from step S202 to step S208. On the other hand, if the operator replies that the process is not to be repeated, the control unit 31 proceeds to the next step, step S210.

[0199] Through the processing up to step S209, it is possible to obtain Figure 11COne or more sequences of task states of the target transitioning from the starting point to the final target are shown as an example. The control unit 31 operates as a data collection unit 317 to collect one or more sequences of task states of the target transitioning from the starting point to the final target. The control unit 31 then generates learning data 323 based on the collected sequences. The control unit 31 can directly obtain the collected sequences as learning data 323, or generate learning data 323 by performing certain information processing on the collected sequences. The structure of the learning data 323 can be appropriately determined based on the method of generating the inference model 55. The structure of the learning data 323 will be described later.

[0200] (Step S210 and Step S211)

[0201] In step S210, the control unit 31 operates as the model generation unit 318. Specifically, the control unit 31 uses the learning data 323 obtained from the judgment results of the learned judgment model 50 to generate an inference model 55. This inference model 55 is used to infer the next target task state from the current task state and the final target task state so that the first object does not come into contact with the second object. The method for generating the inference model 55 will be described later.

[0202] In step S211, the control unit 31 acts as a storage processing unit 319. That is, the control unit 31 generates information related to the generated inference model 55 as inference model data 325, and saves the generated inference model data 325 to a specified storage area. The specified storage area can be, for example, the RAM in the control unit 31, the storage unit 32, the external storage device, the storage medium, or a combination thereof. The storage medium can be, for example, a CD, a DVD, etc., and the control unit 31 can also store the inference model data 325 in the storage medium via the drive 37. The external storage device can be, for example, a data server such as NAS. In this case, the control unit 31 can also use the communication interface 33 to store the inference model data 325 in the data server via the network. In addition, the external storage device can also be, for example, an external storage device connected to the control device 3.

[0203] Thus, when the storage of the inference model data 325 is completed, the control unit 31 ends a series of processes related to the generation of the inference model 55 .

[0204] <Inference Model Generation Method>

[0205] Next, a specific example of the method for generating the inference model 55 in step S210 will be described. In this embodiment, the control unit 31 can generate the inference model 55 by at least one of the following two methods.

[0206] (1) First method

[0207] In the first method, the control unit 31 generates an inference model 55 by implementing machine learning. In this case, the inference model 55 is composed of a machine learning model. The type of machine learning model may not be particularly limited and may be appropriately selected according to the implementation method. The inference model 55 may be expressed, for example, by a functional expression, a data table, etc. When expressed using a functional expression, the inference model 55 may be composed, for example, of a neural network, a support vector machine, a regression model, a decision tree, etc. In addition, the machine learning method may not be particularly limited and may be appropriately selected according to the composition of the inference model 55. The machine learning method of the inference model 55 may adopt, for example, supervised learning, reinforcement learning, etc. Below, two examples of the machine learning model and the machine learning method that constitute the inference model 55 are described.

[0208] (1-1)First example

[0209] Figure 12 The first example of the machine learning model and the machine learning method constituting the inference model 55 is schematically shown. In the first example, the inference model 55 adopts a neural network and the machine learning method adopts supervised learning. Figure 12 In the example, for convenience of explanation, an example of the inference model 55, the learning data 323, and the inference model data 325 are each expressed as an inference model 551, the learning data 3231, and the inference model data 3251.

[0210] (1-1-1) Example of Reasoning Model Configuration

[0211] In the first example, the inference model 551 is composed of a three-layer recursive neural network. Specifically, the inference model 551 includes an input layer N51, an LSTM (Long Short-Term Memory) block N52, and an output layer N53. The LSTM block N52 corresponds to the middle layer.

[0212] The LSTM block 52 is a block that includes an input gate and an output gate and is configured to learn the timing of information storage and output (S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, 9(8): 1735-1780, November 15, 1997). The LSTM block 52 may also include a forget gate that adjusts the timing of information forgetting (Felix A. Gers, Jürgen Schmidhuber and Fred Cummins, "Learning to Forget: Continual Prediction with LSTM," Neural Computation, pages 2451-2471, October 2000). The configuration of the LSTM block N52 can be appropriately set depending on the implementation.

[0213] In addition, the structure of the inference model 551 may not be limited to such an example, and may be appropriately determined according to the implementation method. The inference model 551 may also be composed of a recursive neural network with a different structure. Alternatively, the inference model 551 may not be recursive, but may be composed of a fully connected neural network or a convolutional neural network like the above-mentioned determination model 50. Alternatively, the inference model 551 may also be composed of a combination of multiple neural networks. For example, the number of intermediate layers possessed by the inference model 551 may not be limited to one, but may be two or more. Alternatively, the intermediate layer may be omitted. In addition to this, the structure of the inference model 551 may also be the same as the above-mentioned determination model 50.

[0214] (1-1-2) Example of Learning Data Structure

[0215] The learning data 3231 used in the supervised learning of the inference model 551 is composed of a plurality of learning data sets L30 including a combination of training data (input data) and correct answer data (teacher signal). The training data can be composed of relative coordinates under the current task state L31 used for training and relative coordinates under the task state L32 of the final target used for training. The correct answer data can be composed of relative coordinates in the task state L33 of the target used for training. In addition, the form of the training data and the correct answer data is not particularly limited and can be appropriately selected according to the embodiment. For example, the relative coordinates can be used directly in the training data, or the values ​​obtained by converting the relative coordinates into feature quantities can be used.

[0216] The control unit 31 can generate each learning data set L30 based on one or more sequences of the target task states obtained through the processing up to step S209. For example, the task state of the final target represented by node Ng can be used as the task state L32 of the final target for training. In addition, when the control unit 31 sets the task state of the starting point represented by node Ns to the current task state L31 for training, the task state represented by node N1 can be set to the task state L33 of the target for training in the corresponding correct answer data. Similarly, when the control unit 31 sets the task state represented by node N(k) to the current task state L31 for training, the task state represented by node N(k+1) can be set to the task state L33 of the target for training in the corresponding correct answer data. When the control unit 31 sets the task state represented by node N4 to the current task state L31 for training, the task state of the final target represented by node Ng can be set to the task state L33 of the target for training in the corresponding correct answer data. In this way, each learning data set L30 can be generated based on one or more sequences of the target task states obtained.

[0217] (1-1-3) Regarding Step S210

[0218] In step S210, the control unit 31 uses the acquired multiple learning data sets L30 to perform machine learning (supervised learning) on ​​the inference model 551. In the first example, the control unit 31 trains the inference model 551 through machine learning for each learning data set L30, so that when the training data is input to the input layer N51, the output layer N53 outputs an output value appropriate for the correct answer data. This generates a learned inference model 551 that has acquired the ability to infer the next target task state from the current task state and the final target task state.

[0219] The machine learning method for the inference model 551 can be the same as the machine learning method for the decision model 50 described above. Specifically, in the first step, the control unit 31 inputs training data for each learning data set L30 into the input layer N51 of the inference model 551 and executes the computational processing of the inference model 551. Consequently, the control unit 31 obtains an output value from the output layer L53 of the inference model 551 corresponding to the result of inferring the next target task state to be transitioned to, relative to the current task state. In the second step, the control unit 31 calculates the error between the output value of the output layer L53 and the correct answer data using a loss function.

[0220] Next, in the third step, the control unit 31 calculates the error in the value of each operation parameter of the inference model 551 by the error back propagation method and using the gradient of the error of the calculated output value. The control unit 31 uses the gradient of the calculated error to calculate the error in the value of each operation parameter of the inference model 551 (for example, the weight of the connection between each neuron, the threshold of each neuron, etc.). In the fourth step, the control unit 31 updates the value of the operation parameter of the inference model 551 based on the calculated errors. The degree of update can be adjusted according to the learning rate. The learning rate can be provided either by the operator's specification or as a set value within the program.

[0221] The control unit 31 repeatedly performs the first to fourth steps above, thereby adjusting the values ​​of the operational parameters of the inference model 551 for each learning data set L30 so that the sum of the errors between the output value output from the output layer N53 and the correct answer data becomes smaller. Alternatively, the control unit 31 may repeatedly perform the first to fourth steps above until the sum of the errors becomes below a threshold. The threshold can be appropriately set according to the embodiment. Alternatively, the control unit 31 may repeat the first to fourth steps above a predetermined number of times. The number of repeated adjustments can be specified, for example, using a set value within the program or by input from the operator.

[0222] Based on the results of this machine learning (supervised learning), the control unit 31 can construct, for each learning data set L30, a learned inference model 551 that is trained to output an output value appropriate for the corresponding correct answer data from the output layer N53 when the training data is input to the input layer N51. In other words, a learned inference model 551 can be constructed that has acquired the ability to infer the next target task state from the current task state and the final target task state.

[0223] In step S211, the control unit 31 generates information representing the structure and operational parameters of the learned inference model 551 constructed through supervised learning as inference model data 3251. The control unit 31 then stores the generated inference model data 3251 in a predetermined storage area. Furthermore, the content of the inference model data 3251 is not limited to this example, as long as it can reproduce the learned inference model 551. For example, if the structure of the inference model 551 is shared between various devices, information representing the structure of the inference model 551 may be omitted from the inference model data 3251.

[0224] (1-1-4) Others

[0225] Furthermore, when supervised learning is employed as the machine learning method, the configuration of inference model 551 is not limited to a neural network. Machine learning models other than neural networks may also be employed as inference model 551. Machine learning models constituting inference model 551 may also employ, for example, support vector machines, regression models, decision trees, and the like. The supervised learning method is not limited to the aforementioned examples and may be appropriately selected depending on the configuration of the machine learning model.

[0226] (1-2) Second Example

[0227] Figure 13 A second example of a machine learning model and a machine learning method constituting the inference model 55 is schematically shown. In the second example, the machine learning method adopts reinforcement learning. Figure 13 In the example, for convenience of explanation, an example of the inference model 55, the learning data 323, and the inference model data 325 are each expressed as an inference model 552, the learning data 3232, and the inference model data 3252.

[0228] (1-2-1) Example of Reasoning Model Configuration

[0229] In the second example, the inference model 552 can adopt a value basis, a policy basis, or both. In the case of adopting a value basis, the inference model 552 can be composed of value functions such as a state value function and an action value function (Q function). The state value function is configured to output the value of the provided state. The action value function is configured to output the value of each action relative to the provided state. In the case of adopting a policy basis, the inference model 552 can be composed of a policy function, for example. The policy function is configured to output the probability of selecting each action relative to the provided state. In the case of adopting both, the inference model 552 can be composed of a value function (Critic) and a policy function (Actor). Each function can be expressed, for example, by a data table, a functional formula, etc. In the case of expressing it by a functional formula, each function can be composed of a neural network, a linear function, a decision tree, etc. In addition, deep reinforcement learning can also be implemented by configuring each function with a multi-layer neural network having multiple intermediate (hidden) layers.

[0230] (1-2-2) Example of Learning Data Structure

[0231] In reinforcement learning, an intelligent agent is basically assumed to interact with the learning environment by acting according to a strategy. The entity of the intelligent agent is, for example, a CPU. The inference model 552 acts as a strategy for determining actions through the above-mentioned structure. The intelligent agent observes the state related to the reinforced action in the provided learning environment. In this embodiment, the state as the observation object is the task state specified by the relative coordinates, and the action to be performed is the transition from the current task state to the target task state. The strategy is configured to determine (infer) the target task state to be converted next from the current task state and the final target task state.

[0232] The agent can provide the observed current task state (input data) to the inference model 552, thereby inferring the target task state to be transformed next. The agent can also determine the target task state based on the result of the inference. Alternatively, the target task state can be determined randomly. Thus, the agent can determine the action to be taken. When the agent performs the action to transform to the determined target task state, the observed task state is transformed to the next task state. Depending on the situation, the agent can receive immediate rewards from the learning environment.

[0233] While repeatedly deciding and executing these actions through trial and error, the agent updates its inference model 552 to maximize the sum of immediate rewards (i.e., value). This reinforces the optimal actions, those with the potential to generate high value, and yields a strategy capable of selecting such actions (learned inference model 552).

[0234] Therefore, in reinforcement learning, learning data 3232 is composed of state transition data, which is obtained through trial and error and represents a state transition from the current task state to the next task state through the action being performed, and can obtain immediate rewards depending on the situation. A piece of state transition data can be composed of data representing the trajectory of all state transitions in an episode, or it can be composed of data representing a specified number of state transitions (one or more). During the processing of steps S202 to S209, the control unit 31 can obtain the above state transition data by performing the above trial and error using the inference model 552 being trained.

[0235] Furthermore, a reward function can be used to calculate immediate rewards based on state transitions. This function can be expressed using a data table, a functional form, or a rule. When expressed using a functional form, the reward function can be constructed using neural networks, linear functions, decision trees, and the like. The reward function can also be manually configured by an operator.

[0236] Alternatively, the reward function can be configured to provide an immediate reward based on the result of determining, using the learned judgment model 50, whether the first object and the second object are in contact with each other in the task state of the transition object, and the distance between the task state of the object and the task state of the final target. Specifically, if the first object and the second object are not in contact with each other and the distance between the task state of the object and the task state of the final target is shorter, the immediate reward is set to a larger value; if the first object and the second object are in contact with each other or the distance is longer, the immediate reward is set to a smaller value. The following formula 1 illustrates an example of a reward function that provides such an immediate reward.

[0237] [Formula 1]

[0238]

[0239] s c Indicates the task status of the target determined by the policy. g Indicates the task status of the final goal. F(s c ) indicates that the learned judgment model 50 determines that the task state s c The result of whether the first object and the second object are in contact with each other can be set to F(s) when it is determined that they are in contact with each other. c ) value becomes smaller (e.g., 0), and when it is determined that they are not in contact with each other, F(s c ) value becomes larger (for example, 1). If the output value of the learned judgment model 50 corresponds to this setting, the output value of the learned judgment model 50 can also be directly used as F(s c ).

[0240] Alternatively, the reward function can be inferred from case data obtained by an expert through inverse reinforcement learning. The case data can be composed of data representing the expert's demonstration (trajectory). In this embodiment, the case data can be composed of data representing the path of the first object actually moved in a manner such that the task state from an arbitrary starting point reaches the task state of the final goal. The method for generating case data is also not particularly limited and can be appropriately selected according to the embodiment. For example, the case data can be generated by recording the trajectory of the expert's demonstration using a sensor or the like.

[0241] The method of inverse reinforcement learning is not particularly limited and can be appropriately selected according to the implementation method. In inverse reinforcement learning, for example, methods based on the maximum entropy principle, methods based on minimization of relative entropy, and methods using generative adversarial networks (e.g., Justin Fu, et al., "Learning Robust Rewards with Adversarial Inverse Reinforcement Learning", arXiv: 1710.11248, 2018) can be used. When the reward function is obtained through inverse reinforcement learning, the learning data 3232 can also include case data for inverse reinforcement learning.

[0242] (1-2-3) Regarding Step S210

[0243] In step S210, control unit 31 updates the values ​​of the computational parameters of inference model 552 based on the obtained state transition data to maximize the value. The method for adjusting the values ​​of the computational parameters of inference model 552 can be appropriately selected depending on the configuration of inference model 552. For example, if inference model 552 is configured as a neural network, the values ​​of the computational parameters of inference model 552 can be adjusted using a method similar to the first example described above, such as the error backpropagation method.

[0244] The control unit 31 adjusts the values ​​of the computational parameters of the inference model 552 to maximize the resulting value (expected value) (e.g., until the update amount falls below a threshold). Specifically, training the inference model 552 involves repeatedly modifying the values ​​of the computational parameters that comprise the inference model 552 to obtain a high reward until a predetermined condition is satisfied (e.g., until the update amount falls below a threshold). This allows the control unit 31 to generate a fully learned inference model 552 that has acquired the ability to infer the next target task state from the current task state and the final target task state.

[0245] Furthermore, the control unit 31 may adjust the values ​​of the computational parameters of the inference model 552 after collecting the learning data 3232 through the processing of steps S202 to S209. Alternatively, the control unit 31 may adjust the values ​​of the computational parameters of the inference model 552 while repeatedly performing the processing of steps S202 to S210.

[0246] When inference model 552 is value-based, the aforementioned reinforcement learning methods can include TD (temporal difference) methods, TD(λ) methods, Monte Carlo methods, dynamic programming methods, and the like. Actions can be determined within or across different strategies during trial and error. Specifically, Q-learning and Sarsa can be used as reinforcement learning methods. During trial and error, random actions can also be taken based on probability ε (ε-greedy algorithm).

[0247] Furthermore, when the inference model 552 is policy-based, the aforementioned reinforcement learning method can employ a policy gradient method, TRPO (trust region policy optimization), PPO (proximal policy optimization), or the like. In this case, the control unit 31 calculates the gradient of the policy function's operational parameters in the direction of increasing value and updates the values ​​of the policy function's operational parameters based on the calculated gradient. For example, the REINFORCE algorithm can be used to calculate the policy function's gradient.

[0248] In addition, when the inference model 55 is composed of two parties, the above-mentioned reinforcement learning method can use the actor-critic method, A2C (Advantage Actor Critic), A3C (Asynchronous Advantage Actor Critic), etc.

[0249] Furthermore, when implementing inverse reinforcement learning, the control unit 31 also obtains case data before executing the above-mentioned reinforcement learning process. The case data can be generated by the control device 3 or by other computers. When generated by other computers, the control unit 31 can also obtain case data generated by other computers via a network, storage medium 93, etc. Next, the control unit 31 performs inverse reinforcement learning by using the acquired case data, thereby setting a reward function. Then, the control unit 31 performs the above-mentioned reinforcement learning process using the reward function set by inverse reinforcement learning. Thus, the control unit 31 can use the reward function set by inverse reinforcement learning to generate a learned inference model 552 that has acquired the ability to infer the next target task state from the current task state and the final target task state.

[0250] In step S211, the control unit 31 generates information representing the learned inference model 552 constructed through reinforcement learning as inference model data 3252. The information representing the learned inference model 552 may include, for example, values ​​of various items in a data table, information representing operational parameters such as coefficients of a functional expression, and so on. The control unit 31 then stores the generated inference model data 3252 in a predetermined storage area. According to the second example, an inference model 55 can be generated that can avoid unnecessary contact between a first object and a second object and can determine a target task state so that the robot device's task state quickly reaches the final target task state.

[0251] (1-3) Summary

[0252] In this embodiment, when constructing inference model 55 using a machine learning model, inference model 55 can also be constructed using at least one of the two examples described above. By employing at least one of the two machine learning methods described above, control unit 31 can generate a learned inference model 55 capable of inferring the next target task state from the current task state and the final target task state, so that the first object and the second object do not come into contact. Therefore, using the first method, an inference model 55 that can be appropriately used to execute a task can be generated.

[0253] (2) Second method

[0254] Figure 14A An example of the learning data 323 in the second method is schematically illustrated. Figure 14B An example of the configuration of the inference model 55 in the second method is schematically illustrated. In the second method, the inference model 55 is composed of a potential field that defines the potential of each coordinate in the task space SP that expresses a set of task states. Figure 14A and Figure 14B In FIG. 5 , for convenience of explanation, an example of the inference model 55 , the learning data 323 , and the inference model data 325 are each expressed as an inference model 553 , the learning data 3233 , and the inference model data 3253 .

[0255] The control unit 31 performs path planning in the task space SP by using the learned judgment model 50 through the processing of the above steps S202 to S209 so that the first object does not come into contact with the second object. Figure 14A As illustrated, the control unit 31 can generate learning data 3233 indicating a path Hb from each of a plurality of task states provided as a starting point (node ​​Ns) to the task state of the final target. Each starting point (node ​​Ns) can be provided randomly.

[0256] In the above step S210, the control unit 31 sets the potential of each coordinate according to the passing frequency of each path Hb represented by the generated learning data 3233, thereby generating a potential field. The method of deriving the potential field is not particularly limited and can be appropriately selected according to the implementation method. The control unit 31 can also derive the potential field from the learning data 3233 by, for example, kernel density estimation or estimation using a Gaussian mixture model (GMM). Thus, it can be obtained Figure 14B The illustrated potential field (inference model 553).

[0257] The potential of each coordinate in the potential field represents an evaluation value of the positional relationship between the first and second objects at each coordinate relative to reaching the final goal. Specifically, a higher potential indicates a higher probability that the positional relationship at that coordinate will reach the final goal, while a lower potential indicates a lower probability that the positional relationship at that coordinate will reach the final goal. Therefore, by shifting toward a higher potential gradient, it is possible to appropriately reach the final goal task state from any starting point. Therefore, according to the second method, an inference model 55 suitable for executing a task can be appropriately generated.

[0258] In step S211, the control unit 31 generates information representing the generated potential field as inference model data 3253. The potential field can be expressed by a data table, a functional expression, etc. Then, the control unit 31 stores the generated inference model data 3253 in a predetermined storage area.

[0259] (3) Summary

[0260] In this embodiment, at least one of the two methods described above can be used as a method for generating the inference model 55. By using at least one of the two methods described above, the control unit 31 can generate an inference model 55 configured to infer the task state of the next target to be transitioned to from the current task state and the task state of the final target so that the first object does not come into contact with the second object.

[0261] Furthermore, inferring the target task state so that the first object does not contact the second object means determining the target task state to avoid unintended contact between the first and second objects. For example, this may include determining the target task state as a task state in which the first object, such as the end effector T holding the workpiece W, and the second object are in proper contact. Specifically, the "contact" state to be avoided includes, for example, an excessive force acting between the first and second objects, or inappropriate contact, such as one object being assembled to the other in an incorrect position. Therefore, "the first object does not contact the second object" can be replaced with "avoiding inappropriate contact between the first and second objects."

[0262] (B) Motion Control of Robotic Devices

[0263] Next, use Figure 15A 、 Figure 15B as well as Figure 16 Next, an operation example related to the motion control of the robot device (manipulator 4) by the control device 3 according to this embodiment will be described. Figure 15A and Figure 15B This is a flowchart showing an example of a processing procedure related to the motion control of the robot device (manipulator 4) by the control device 3 according to the present embodiment. Figure 16 An example of a flow of calculation processing of each element in the process of motion control is shown. Including the processing sequence related to the generation of the above-mentioned inference model 55, the processing sequence described below is an example of the "control method" of the present invention. However, each processing sequence described below is only an example, and each step can be changed within a possible range. Furthermore, for each processing sequence described below, steps can be appropriately omitted, replaced, and added according to the implementation method. In addition, the control of the motion of the robot device (manipulator 4) described below can be implemented in real space or in virtual space.

[0264] (Step S301 and Step S302)

[0265] In step S301, the control unit 31 accepts the designation of a task to be executed. The method for accepting the designation of a task is not particularly limited and may be appropriately selected depending on the implementation. For example, the control unit 31 may accept the designation of a task to be executed by inputting the name of the task via the input device 35. Alternatively, the control unit 31 may accept the designation of a task to be executed by outputting a list of candidate tasks to be executed to the output device 36 and allowing the operator to select the task to be executed from the list.

[0266] In the present embodiment, the control unit 31 accepts the execution of a task of moving a first object relative to a second object in an environment in which a first object and a second object are present. Specifically, a series of operations of driving the robot 4, holding the workpiece W through the end effector T, and assembling the held workpiece W to another workpiece G is an example of a designated task. In the present embodiment, in the process of moving to the first task of holding the workpiece W, the focus point T0 of the end effector T is treated as the fingertip of the robot 4, and the workpiece W is the target object of the movement of the fingertip. On the other hand, in the process of assembling the workpiece W to another workpiece G after holding the workpiece W, the focus point W0 of the workpiece W held by the end effector T is treated as the fingertip of the robot 4, and the other workpiece W that is the assembly target of the workpiece W is the target object of the movement of the fingertip. In each task, the fingertip of the robot 4 is equivalent to the first object, and the target object is equivalent to the second object.

[0267] In step S302, the control unit 31 operates as the target setting unit 311 and sets the task state s of the final target according to the designated task. g As described above, in this embodiment, the task state is defined by the positional relationship between the fingertips of the manipulator 4 and the target object. In addition, the positional relationship is expressed by relative coordinates. The task state s of the final target g The relative coordinates in can be provided by a simulator such as CAD or by the operator's designation. The final target mission state s g The relative coordinates in can be set by the same method as in step S201 above. g , the control unit 31 advances the processing to the next step S303.

[0268] (Step S303)

[0269] In step S303 , the control unit 31 acquires sensing data from each sensor that observes the environment in which the task is performed.

[0270] In this embodiment, a camera S1 is used as a visual sensor for observing the environment in which the task is performed. In addition, the manipulator 4 is equipped with an encoder S2 for measuring the angle of each joint (joints 41 to 46) and a tactile sensor S3 for measuring the force acting on the end effector T. Therefore, the control unit 31 can obtain image data reflecting the environment in which the task is performed from the camera S1 as sensing data. In addition, the control unit 31 can obtain the current value q of the angle of each joint in the manipulator 4 from each encoder S2. (j)(i.e., the current measured value) is used as the sensing data. In addition, for the sake of convenience in the following description, symbols such as (j) indicating the time are used to specifically distinguish the time such as "current", and the symbols are omitted otherwise. Furthermore, the control unit 31 can obtain the measurement data of the force acting on the end effector T from the tactile sensor S3 as the sensing data. The control unit 31 can directly obtain the sensing data from each sensor (camera S1, encoder S2, tactile sensor S3), or can also obtain the sensing data indirectly, for example, through other computers. The camera S1 and the tactile sensor S3 are each an example of a sensor that observes the state of the workpiece W relative to the end effector T. When the sensing data is obtained, the control unit 31 causes the processing to proceed to the next step S304.

[0271] (Step S304)

[0272] In step S304 , the control unit 31 determines based on the acquired sensing data whether the end effector T holds the workpiece W. The determination method is not particularly limited and may be appropriately determined based on the sensing data.

[0273] For example, in this embodiment, image data representing the environment where the task is being performed can be acquired from camera S1 as sensor data. Therefore, the control unit 31 can use CAD data 320 to match the models of the end effector T and workpiece W with the acquired image data. The control unit 31 can then determine whether the end effector T is holding the workpiece W based on the positional relationship between the end effector T and workpiece W determined based on the matching results. This matching method can utilize known image processing techniques.

[0274] Furthermore, for example, in this embodiment, measurement data of the force acting on the end effector T can be acquired as sensing data. Therefore, the control unit 31 can also determine whether the end effector T is holding the workpiece W based on the force distribution indicated by the measurement data. If, based on the measurement data, it is inferred that a force is acting on the end effector T that indicates that the end effector T is holding the workpiece W, the control unit 31 can determine that the end effector T is holding the workpiece W. Otherwise, the control unit 31 can determine that the end effector T is not holding the workpiece W.

[0275] When the determination of whether the end effector T holds the workpiece W is completed based on the sensing data, the control portion 31 advances the process to the next step S305 .

[0276] (Step S305)

[0277] In step S305, the control unit 31 sets the motion mode of the robot 4 based on the determination result of step S304. Specifically, if the end effector T is determined not to be holding a workpiece W, the control unit 31 sets the focus point T0 of the end effector T to the fingertip of the robot 4 and sets the motion mode to a mode for performing the first task of holding the workpiece W by the end effector T. On the other hand, if the end effector T is determined to be holding a workpiece W, the control unit 31 sets the focus point W0 of the workpiece W to the fingertip of the robot 4 and sets the motion mode to a mode for performing the second task of assembling the workpiece W held by the end effector T onto another workpiece G. Once the motion mode setting is complete, the control unit 31 proceeds to the next step, S306.

[0278] (Step S306)

[0279] In step S306, the control unit 31 operates as the state acquisition unit 312 to acquire the current task state s of the robot 4. (j) .

[0280] As described above, in this embodiment, when the end effector T does not hold the workpiece W, the task state s is defined by the relative coordinates of the workpiece W relative to the end effector T. On the other hand, when the end effector T holds the workpiece W, the task state s is defined by the relative coordinates of another workpiece G relative to the workpiece W. In this embodiment, the control unit 31 uses the CAD data 320 to match each object with the image data obtained by the camera S1. The control unit 31 can obtain the current task state s based on the result of this matching. (j) .

[0281] Here, further use Figure 17 , to get the current task status s (j) An example of the method is described below. Figure 17 An example of the positional relationship between the objects is schematically shown. Figure 17 In the example, the origin of the observation space is set to the base 40 of the manipulator 4. However, the position of the origin is not limited to this example and can be appropriately determined according to the embodiment. The homogeneous coordinates (T C ) can be expressed by the following formula 2.

[0282] [Formula 2]

[0283]

[0284] R rc represents the rotation component of the coordinate system of camera S1 observed from the coordinate system of the origin, t rc In the following, for the sake of convenience, it is assumed that the homogeneous coordinates of the origin (TR ) Calibrate the camera S1 in a manner that satisfies the following equation 3.

[0285] [Formula 3]

[0286]

[0287] I represents the identity matrix. Figure 17 In the example, the relative coordinates of the end effector T’s focus point T0 relative to the origin are the coordinates of the end effector T (T t The relative coordinates of the focus point W0 of the workpiece W relative to the origin are the coordinates of the workpiece W (T w The relative coordinates of the focus point G0 of the other workpiece G with respect to the origin are the coordinates of the other workpiece G (T g The control unit 31 can use the CAD data 320 to match the model of each object with respect to the image data, thereby obtaining each coordinate (T t 、T w 、T g ) value.

[0288] [Formula 4]

[0289] s=T w -1 ·T t …(Equation 4)

[0290] When the end effector T does not hold the workpiece W, the task state s and the coordinates of the end effector T and the workpiece W (T t 、T w ) can be expressed by the above formula 4. Therefore, the control unit 31 can obtain the current value of each coordinate of the end effector T and the workpiece W (T t(j) 、T w(j) ) is substituted into the above formula 4, and the calculation process of the above formula 4 is performed to calculate the current task state s (j) .

[0291] [Formula 5]

[0292] s=T g -1·T w …(Equation 5)

[0293] On the other hand, when the end effector T holds the workpiece W, the task state s and the coordinates of the workpiece W and other workpieces G (T w 、T g ) can be expressed by the above formula 5. Therefore, the control unit 31 can use the current value (T w(j) 、T g(j)) is substituted into the above formula 5, and the calculation process of the above formula 5 is performed to calculate the current task state s (j) In addition, each coordinate (T t 、T w 、T g The expression of ) can be selected appropriately. t 、T w 、T g ) can be expressed using, for example, a homogeneous coordinate system. The same applies to the following.

[0294] When the camera S1 is not calibrated, the control unit 31 may further calculate the coordinates of the origin (T R ). A mark such as a marker can be used to detect the origin. That is, the coordinates of the origin can be calculated by matching the mark in the image data (T R The control unit 31 can calculate the coordinates of the origin (T R ) is applied to the above operations to calculate the current task status s (j) In subsequent steps, the case of performing matching based on the CAD data 320 can also be processed in the same way.

[0295] When the current task status s is obtained (j) , the control unit 31 advances the process to the next step S307. Furthermore, the timing for executing the process of step S306 is not limited to this example. The process of step S306 can be executed at any time before executing step S308, which will be described later. For example, if matching is also performed based on the CAD data 320 in step S304, the process of step S306 can be executed together with the process of step S304.

[0296] (Step S307).

[0297] In step S307, the control unit 31 calculates the current value x of the coordinate of the fingertip of the robot 4 based on the acquired sensing data. (j) In this embodiment, the current value x of the fingertip coordinates (j) The calculation of can use the sensing data obtained by at least any one of the camera S1, each encoder S2 and the tactile sensor S3. The control unit 31 can calculate the current value x of the fingertip coordinates by at least one of the following two methods: (j) .

[0298] (1) First method

[0299] like Figure 16As shown in the first method, the control unit 31 calculates the current value q of the angle of each joint of the manipulator 4 in the joint space obtained by each encoder S2 through forward kinematics. (j) The current value x of the coordinate of the fingertip of the robot 4 in the observation space is derived (j) The following description will be divided into a case where the end effector T does not hold the workpiece W and a case where the end effector T holds the workpiece W.

[0300] (1-1) When the workpiece W is not held

[0301] When the end effector T does not hold the workpiece W, the focus point T0 of the end effector T is set to the fingertip. In this case, the control unit 31 calculates the current value q of the angle of each joint by forward kinematics. (j) Calculate the current value x of the set fingertip coordinates (j) , where the forward kinematics calculation uses the first transformation matrix group derived from the first homogeneous transformation matrix of each joint as a transformation function.

[0302] [Formula 6]

[0303] x t =φ(q) …(Equation 6)

[0304] [Formula 7]

[0305] φ(q)=0T 1 ·…· n-1 T n …(Equation 7)

[0306] Specifically, through forward kinematics, the coordinates (x t The relationship between ) and the angle (q) of each joint can be expressed by the above formula 6. The angle (q) is a variable with a dimension corresponding to the number of joints. In addition, the first homogeneous transformation matrix of each joint ( m-1 T m ) and the first transformation matrix group The relationship is provided by the above formula 7 (m is 0 to n. n is the number of joints). The first homogeneous transformation matrix represents the relative coordinates of the coordinate system of the object's joints observed from a coordinate system closer to the heel side than the object's joints, and is used to transform the coordinates from the heel side coordinate system to the coordinate system of the object's joints.

[0307] The values ​​of the parameters of the first homogeneous transformation matrix of each joint are known except for the angles of each joint. In this embodiment, they are included in the robot data 321. These parameters can be set using a well-known method such as the DH (Denavit-Hartenberg) notation or the modified DH notation. The control unit 31 derives the first transformation matrix group shown in the above equation 7 by referring to the robot data 321. Then, the control unit 31 converts the current value q of the angle of each joint into (j) Substitute the derived first transformation matrix group Thus performing the first transformation matrix group Based on the result of the forward kinematics calculation, the control unit 31 can calculate the current value x of the coordinates of the end effector T (the focus point T0). t(j) The control unit 31 obtains the calculated current value x t(j) As the current value x of the fingertip coordinate (j) .

[0308] (1-2) When the workpiece W is held

[0309] On the other hand, when the end effector T holds the workpiece W, the focus point W0 of the workpiece W is set to the fingertip. In this case, the control unit 31 first obtains a second homogeneous transformation matrix ( t T w ).

[0310] Get the second homogeneous transformation matrix ( t T w ) is not particularly limited and can be appropriately selected according to the embodiment. For example, when the end effector T holds the workpiece W, there is a case where the position and posture of the workpiece W relative to the end effector T are constant. Therefore, the second homogeneous transformation matrix ( t T w ) can also be provided using constants.

[0311] Alternatively, the control unit 31 may also derive the second homogeneous transformation matrix ( t T w As an example of the derivation method, the control unit 31 may also use the CAD data 320 to match the models of the end effector T and the workpiece W with respect to the image data obtained by the camera S1. The control unit 31 may obtain the coordinates (T t ) and the coordinates of the workpiece W (T w). As described above, assuming that the camera S1 is calibrated, the control unit 31 can calculate the coordinates of the end effector T (T t ) and the coordinates of the workpiece W (T w ) calculates the second homogeneous transformation matrix ( t T w ).

[0312] [Formula 8]

[0313] t T w =T t -1 ·T w …(Equation 8)

[0314] The coordinates (x t ) is performed near the end effector. If the matching and the above-mentioned forward kinematics calculation are correct, the coordinates of the end effector T obtained by matching (T t ) and the coordinates (x t ) are consistent. Therefore, the control unit 31 may also calculate the coordinates (x t ) is used as the coordinate (T t ). Thus, the control unit 31 can derive the second homogeneous transformation matrix ( t T w ). In addition, in the calculation process of the above formula 4 in the above step S306, the current value of the coordinate of the end effector T obtained by matching (T t(j) ) and use the current value of the coordinates calculated by forward kinematics (x t(j) ).

[0315] In addition, as another example of the derivation method, the distribution of the force acting on the end effector T measured by the tactile sensor S3 may depend on the position and posture of the workpiece W relative to the end effector T. Therefore, the control unit 31 can estimate the relative coordinates (relative position and relative posture) of the workpiece W relative to the end effector T based on the measurement data obtained by the tactile sensor S3. The control unit 31 can derive the second homogeneous transformation matrix ( t T w ).

[0316] In addition, a second homogeneous transformation matrix ( t T w ) method is also not limited to the above analytical method. In the second homogeneous transformation matrix ( t T w), for example, the second homogeneous transformation matrix ( t T w ) ability. In this case, the control unit 31 provides the acquired sensing data to the learned machine learning model and performs the computational processing of the learned machine learning model. Thus, the control unit 31 can obtain the second homogeneous transformation matrix ( t T w ) is the output value corresponding to the inference result.

[0317] Next, the control unit 31 transforms the obtained second homogeneous transformation matrix ( t T w ) multiplied by the first transformation matrix group And calculate the second transformation matrix group The second transformation matrix group can be expressed by the following formula 9. In addition, the first homogeneous transformation matrix is ​​an example of the first transformation formula, and the second homogeneous transformation matrix is ​​an example of the second transformation formula. is an example of the first transformation matrix group, and the second transformation matrix group This is an example of the second group of transformation equations. The form of each transformation equation is not particularly limited as long as it can be used for fingertip coordinate calculations. For example, each transformation equation can be expressed using a transformation matrix in a format other than a homogeneous coordinate system, or using a mathematical expression in a format other than a matrix.

[0318] [Formula 9]

[0319] φ(q)· t T w =x t · t T w …(Equation 9)

[0320] The control unit 31 uses the calculated second transformation matrix group as the forward kinematics calculation of the transformation function to obtain the current value q of the angle of each joint. (j) Calculate the current value x of the set fingertip coordinates (j) That is, the control unit 31 sets the current value q of the angle of each joint (j) Substitute into the second transformation matrix group Execute the second transformation matrix group Based on the result of the forward kinematics calculation, the control unit 31 can calculate the current value of the coordinates of the workpiece W (the focus point W0). The control unit 31 obtains the calculated current value of the coordinates of the workpiece W as the current value x of the fingertip coordinates. (j) .

[0321] Furthermore, in the above, by deriving the second homogeneous transformation matrix ( t T w ), even if the holding state of the workpiece W in the end effector T changes, the second homogeneous transformation matrix ( t T w Thus, even when the holding state of the workpiece W in the end effector T may vary, the current coordinate values ​​of the workpiece W, that is, the current coordinate values ​​of the fingertip of the robot 4, can be appropriately calculated.

[0322] (2) Second method

[0323] In the second method, the control unit 31 uses the CAD data 320 to match the model of each object with the image data obtained by the camera S1, thereby deriving the current value x of the coordinate of the fingertip of the robot 4. (j) In this case, the control unit 31 may also use the current task state s obtained in the task space in step S306 to determine the task state. (j) The current value x of the coordinate of the fingertip of the robot 4 is derived (j) .

[0324] [Equation 10]

[0325] x=φ(s)=T w ·s …(Equation 10)

[0326] according to Figure 17 The positional relationship of each object shown in FIG1 and the relationship between the task state s and the coordinate x of the fingertip when the end effector T does not hold the workpiece W can be expressed by the above formula 10. In this case, the transformation function (ψ) from the task space to the observation space is expressed by the coordinates of the workpiece W (T w The control unit 31 can obtain the current task status s obtained in step S306. (j) and the current value of the coordinates of the workpiece W obtained by matching (T w(j) ) is substituted into equation 10, and the calculation process of equation 10 is performed to calculate the current value x of the coordinate of the fingertip of the manipulator 4. (j) .

[0327] [Equation 11]

[0328] x=φ(s)=T g ·s …(Equation 11)

[0329] Similarly, the relationship between the task state s and the coordinate x of the fingertip when the end effector T holds the workpiece W can be expressed by the above formula 11. In this case, the transformation function (ψ) from the task space to the observation space is obtained by the coordinates (T gThe control unit 31 can obtain the current task status s obtained in step S306. (j) and the current value of the coordinates of the other workpiece G obtained by matching (T g(j) ) is substituted into equation 11, and the calculation process of equation 11 is performed to calculate the current value x of the coordinate of the fingertip of the manipulator 4. (j) .

[0330] Furthermore, the current value x of the coordinates of the fingertip of the robot 4 is derived by matching using the CAD data 320. (j) The method is not limited to this example. When the end effector T does not hold the workpiece W, the control unit 31 can also calculate the current value of the coordinate of the end effector T (T t(j) ) and obtain the calculated current value (T t(j) ) as the current value x of the coordinate of the fingertip (j) Similarly, when the end effector T holds the workpiece W, the control unit 31 can also calculate the current value of the coordinates of the workpiece W (T w(j) ) and obtain the calculated current value (T w(j) ) as the current value x of the coordinate of the fingertip (j) That is, the control unit 31 can also directly derive the current value x of the coordinate of the fingertip of the robot 4 through the above matching. (j) .

[0331] (3) Summary

[0332] In this embodiment, the control unit 31 can calculate the current value x of the coordinate of the fingertip of the robot 4 by using at least one of the above two methods. (j) When the current value x of the coordinate of the fingertip of the robot 4 is calculated (j) , the control unit 31 advances the process to the next step S308. Furthermore, the timing for executing the process of step S307 is not limited to this example. The process of step S307 can be executed at any time before executing the process of step S310 described later. For example, the process of step S307 can be executed before the above-mentioned step S306. Furthermore, for example, in the case of matching using CAD data 320, the process of step S307 can be executed together with the above-mentioned process of step S306 or step S304.

[0333] (Step S308)

[0334] In step S308, the control unit 31 acts as the action decision unit 313 to approach the task state s of the final target. g In this way, relative to obtaining the current task status s (j) The task state s that determines the target of the next transitions(j) In this embodiment, the control unit 31 refers to the inference model data 325 and uses the inference model 55 generated by the process of step S210 to calculate the current task state s. (j) To determine the next target task state s s(j) .

[0335] The task state s used to reason about the target of the next transition s(j) The calculation processing of the inference model 55 can be appropriately performed according to the structure of the inference model 55. When the inference model 55 is generated by the first method described above and the inference model 55 is composed of a functional expression, the control unit 31 sets the current task state s (j) And the task status s of the final goal g Substitute the function and execute the calculation process of the function. When the inference model 55 is composed of a neural network, the control unit 31 (j) And the task status s of the final goal g The control unit 31 inputs the current task state s into the input layer and performs firing judgment of each neuron contained in each layer in sequence from the input side. (j) And the task status s of the final goal g The control unit 31 thus obtains the task state s of the target to be converted next as the output of the inference model 55. s(j) The control unit 31 can determine the next target task state s according to the inference result. s(j) .

[0336] In addition, when the inference model 55 is generated by the second method, that is, when the inference model 55 is composed of a potential field, the control unit 31 refers to the generated potential field and the corresponding task state s (j) Then, the control unit 31 sets the potential value at the corresponding coordinate. (j) The gradient of the potential set by the corresponding coordinate determines the task state s of the target to be converted next s(j) Specifically, the control unit 31 determines the target task state s so as to shift to the side with a higher potential gradient (for example, shift to the side with the highest gradient by a predetermined distance). s(j) .

[0337] The number of task states of the target to be determined may not be limited to one. In step S308, the control unit 31 may use the task state of the determined target as the current task state to determine the task state of the target to be further converted. The control unit 31 may also determine the task state of the target multiple times by repeating this process. When the task state of the target to be converted is determined, s(j) , the control unit 31 advances the processing to the next step S309.

[0338] (Step S309)

[0339] In step S309, the control unit 31 operates as the instruction determination unit 315 and determines the target task state s. s(j) Calculate the target value x of the fingertip coordinates s(j) .like Figure 16 As shown, the control unit 31 can use the above-mentioned transformation function (ψ) to transform the task state s of the target in the task space into s(j) The target value x is transformed into the coordinates of the fingertip in the observation space s(j) .

[0340] That is, the transformation function (ψ) from the task space to the observation space when the end effector T does not hold the workpiece W is given by the above equation 10. The control unit 31 can determine the target task state s s(j) Substituting into the above equation 10 and executing the calculation process of the above equation 10, the target value x of the coordinate of the fingertip is calculated. s(j) On the other hand, the transformation function (ψ) from the task space to the observation space when the end effector T holds the workpiece W is given by the above equation 11. The control unit 31 can determine the target task state s s(j) Substituting into the above equation 11 and executing the calculation process of the above equation 11, the target value x of the coordinate of the fingertip is calculated. s(j) When the target value x of the fingertip coordinate is calculated s(j) , the control unit 31 advances the processing to the next step S310.

[0341] (Step S310)

[0342] In step S310, the control unit 31 operates as the instruction determination unit 315 and determines the fingertip coordinates based on the current value x (j) and the target value x of the fingertip coordinates s(j) Determine the change in fingertip coordinates (Δx (j) ). Specifically, Figure 16 As shown, the control unit 31 calculates the current value of the fingertip coordinate (x (j) ) and the target value (x s(j) ) determines the change in fingertip coordinates (Δx (j)For example, the deviation between the current value of the fingertip coordinate and the target value (x s The relationship between the change (Δx) and the fingertip coordinates can be given by the following formula 12. The change (Δx) of the fingertip coordinates is an example of the difference between the current value and the target value of the fingertip coordinates.

[0343] [Equation 12]

[0344] Δx=α×(x s -x) …(Equation 12)

[0345] α is an arbitrary coefficient. For example, the value of α can be appropriately determined within the range of less than 1 and greater than 0. α can also be omitted. The control unit 31 can set the current value x of the fingertip coordinates obtained in step S307 and step S309 to (j) and the target value x of the fingertip coordinates s(j) Substituting into the above equation 12 and performing the calculation of the above equation 12, the change amount of the fingertip coordinates (Δx (j) When the change in the fingertip coordinates (Δx (j) ), the control unit 31 advances the processing to the next step S311.

[0346] (Step S311)

[0347] In step S311, the control unit 31 operates as the command determination unit 315, and calculates the fingertip coordinates according to the determined change amount (Δx (j) ) calculate the change in angle of each joint (Δq (j) Specifically, the amount of change in the fingertip coordinates (Δx) and the amount of change in the angle of each joint (Δq) can be expressed by the following formula 13.

[0348] [Equation 13]

[0349]

[0350] J is the Jacobian matrix derived from the transformation function in the forward kinematics calculation above. i represents the matrix component of the i-th joint, Δq i Represents the change of the i-th joint.

[0351] Here, further use Figure 18A and Figure 18B , an example of the method for calculating the Jacobian matrix is ​​described. Figure 18A An example of the relationship between the joints and the fingertips when the end effector T does not hold the workpiece W is schematically illustrated. Figure 18BAn example of the relationship between the joints and the fingertips when the end effector T holds the workpiece W is schematically illustrated.

[0352] like Figure 18A As shown in , when the end effector T does not hold the workpiece W, the components of each joint in the Jacobian matrix are calculated based on the positional relationship between each joint and the end effector T. For example, the control unit 31 can calculate the components of each joint using the following formula 14. On the other hand, Figure 18B As shown, when the end effector T holds the workpiece W, the component of each joint in the Jacobian matrix is ​​calculated based on the positional relationship between each joint and the workpiece W. For example, the control unit 31 can calculate the component of each joint using the following equation 15.

[0353] [Equation 14]

[0354]

[0355] [Equation 15]

[0356]

[0357] z i represents the component of the rotation axis in the homogeneous coordinates of the i-th joint, a i represents the parallel translation component of the homogeneous coordinates of the i-th joint. i and a i Extracted from the first homogeneous transformation matrix of the i-th joint. a t represents the parallel translation component of the end effector T in homogeneous coordinates. w Represents the parallel translation component of the workpiece W in the homogeneous coordinates. t From the coordinates of the end effector T (T t ) extraction. w From the coordinates of the workpiece W (T w ) Extract. The components j of the Jacobian matrix i Represents the differential component of the first homogeneous transformation matrix of each joint.

[0358] The control unit 31 calculates the Jacobian matrix according to the above equations 14 and 15 according to the motion mode. In addition, in this embodiment, between the case where the end effector T does not hold the workpiece W and the case where the end effector T holds the workpiece W, the component of the end effector T (a t ) and the component of the workpiece W (a w Therefore, the control unit 31 can calculate the Jacobian matrix in each case through simple calculation processing.

[0359] Next, the control unit 31 calculates the inverse matrix (J -1The control unit 31 uses the calculated inverse matrix (J -1 ) performs inverse kinematics calculation. Specifically, each change (Δx, Δq) and the inverse matrix (J -1 ) is derived from the above formula 13 as shown in the following formula 16.

[0360] [Equation 16]

[0361] Δq=J -1 ·Δx …(Equation 160

[0362] The control unit 31 can calculate the inverse matrix (J -1 ) and the change in fingertip coordinates (Δx (j) ) is substituted into Equation 16, and the calculation process of Equation 16 is performed to calculate the change in the angle of each joint (Δq (j) ). When the change in angle of each joint is calculated (Δq (j) ), the control unit 31 advances the processing to the next step S312.

[0363] (Step S312)

[0364] In step S312, the control unit 31 acts as the instruction determination unit 315, and determines the instruction value relative to each joint based on the calculated change in the angle of each joint. The method for determining the instruction value can adopt a well-known method such as PID (Proportional-Integral-Differential) control and PI control. The instruction value relative to each joint is an example of a control instruction provided to the robot device. In this embodiment, the control unit 31 can determine the instruction to be provided to the manipulator 4 through the processing of steps S309 to S312 so that the task state of the manipulator 4 changes from the current task state s (j) Change to the target task state s s(j) When the control instruction is determined, the control unit 31 advances the processing to the next step S313.

[0365] (Step S313)

[0366] In step S313, the control unit 31 acts as the driving unit 316, and drives the manipulator 4 by providing the determined control instructions to the manipulator 4. In the present embodiment, the control unit 31 drives the joints of the manipulator 4 according to the determined instruction values. In addition, the driving method may not be particularly limited and may be appropriately selected according to the embodiment. For example, the control unit 31 may also directly drive the joints of the manipulator 4. Alternatively, the manipulator 4 may also have a controller (not shown). In this case, the control unit 31 may also indirectly drive the joints of the manipulator 4 by providing instruction values ​​relative to the joints to the controller. When the manipulator 4 is driven according to the determined control instructions, the control unit 31 causes the processing to proceed to the next step S314.

[0367] (Steps S314 to S316)

[0368] The processing of steps S314 to S316 is the same as the processing of steps S303, S306, and S307 above, except that the loop goes from (j) to (j+1). That is, in step S314, the control unit 31 obtains sensing data from each sensor. In step S315, the control unit 31 operates as the state acquisition unit 312 and obtains the current task state s of the robot 4. (j+1) In step S316, the control unit 31 calculates the current value x of the coordinate of the fingertip of the robot 4 based on the acquired sensing data. (j+1) When the current value x of the fingertip coordinate is calculated (j+1) , the control unit 31 advances the processing to the next step S317.

[0369] (Step S317)

[0370] In step S317, the control unit 31 determines whether the driving in step S313 has caused the task state of the robot 4 to transition to the target task state s. s(j) .

[0371] The determination method is also not particularly limited and can be appropriately selected according to the embodiment. Figure 16 As shown, the angles of each joint after driving (q (j+1) ) and the angles of each joint before driving (q (j) ) can be expressed by the following formula 17.

[0372] [Equation 17]

[0373] q(j+1)=q (j) +Δq (j) …(Equation 17)

[0374] Therefore, the control unit 31 may also determine whether the angle value of each joint obtained by each encoder S2 in step S314 is the same as the angle value of each joint obtained by each encoder S2 before driving (q (j) ) and the change amount (Δq (j) ) is consistent with the sum of the angles of each joint after driving and the angles of each joint before driving and the calculated changes (q (j) +Δq (j) ) are consistent, the control unit 31 can determine that the task state of the robot 4 is converted to the target task state s s(j) Otherwise, the control unit 31 may determine that the task state of the robot 4 has not been converted to the target task state s s(j) .

[0375] In addition, for example, similar to the transformation function in the forward kinematics calculation, the transformation function (ψ) can also be used to derive the Jacobian matrix J ψ . Jacobian matrix J ψ Represents the differential component of the transformation function (ψ). The Jacobian matrix J can be derived ψ Calculate the inverse matrix (J ψ -1 The change in fingertip coordinates (Δx) and the change in task state (Δs) are related to the inverse matrix (J ψ -1 ) can be expressed by the following formula 18.

[0376] [Equation 18]

[0377] Δs=J φ -1 Δx (Equation 18)

[0378] The control unit 31 can calculate the inverse matrix (J ψ -1 ) and the change in fingertip coordinates (Δx (j) ) is substituted into Equation 18, and the calculation process of Equation 18 is performed to calculate the change in task state (Δs (j) ). Task status after driving s (j+1) The task status s before driving (j) The relationship can be expressed by the following formula 19 in the same way as the above formula 17.

[0379] [Equation 19]

[0380] s (j+1) =s (j) +Δs (j) …(Equation 19)

[0381] Therefore, the control unit 31 may also determine whether the current task state obtained after the driving in step S315 is the same as the current task state s obtained before the driving in step S306. (j) And the change (Δs (j) ) is consistent with the sum of the current task state obtained after driving and the current task state obtained before driving and the calculated change amount (s (j) +Δs (j) ) are consistent, the control unit 31 can determine that the task state of the robot 4 is converted to the target task state s s(j) Otherwise, the control unit 31 may determine that the task state of the robot 4 has not been converted to the target task state s s(j) In addition, in this embodiment, the task space is defined by the relative coordinates between two objects, so the task space and the observation space can be expressed in mutually shared dimensions. Therefore, depending on the situation, the inverse matrix (J) of Equation 18 can also be expressed as ψ -1 ) is replaced by the unit matrix, and the change in the fingertip coordinates (Δx) is directly treated as the change in the task state (Δs). As an example, when the task state is specified based on the relative coordinates of the workpiece W observed from another workpiece G, the inverse matrix (J ψ -1 ) can be replaced by the identity matrix.

[0382] Alternatively, the control unit 31 may determine whether the current task state obtained in step S315 is consistent with the target task state s determined in step S308. s(j) When the current task state is consistent with the target task state s s(j) If the task state of the robot 4 is consistent, the control unit 31 can determine that the task state of the robot 4 has been transferred to the target task state s s(j) Otherwise, the control unit 31 may determine that the task state of the robot 4 has not been converted to the target task state s s(j) .

[0383] In addition, for example, the current value of the fingertip coordinates after driving (x (j+1) ) and the current value of the fingertip coordinates before driving (x (j) ) can be expressed by the following formula 20 in the same way as the above formula 17.

[0384] [Equation 20]

[0385] x (j+1) =x (j) +Δx (j) ···(Formula 20)

[0386] Therefore, the control unit 31 may determine whether the current value of the fingertip coordinate after driving calculated in step S316 is the same as the current value of the fingertip coordinate before driving calculated in step S307 (x (j) ) and the change (Δx (j) ) is consistent with the sum of the current value of the fingertip coordinates after driving and the current value of the fingertip coordinates before driving and the calculated change (x (j) +Δx (j) ) are consistent, the control unit 31 can determine that the task state of the robot 4 is converted to the target task state s s(j) Otherwise, the control unit 31 may determine that the task state of the robot 4 has not been converted to the target task state s s(j) .

[0387] Alternatively, the control unit 31 may determine whether the current value of the fingertip coordinates calculated in step S316 is equal to the target value (x s(j) ) is consistent. When the current value of the fingertip coordinate after driving is consistent with the target value of the fingertip coordinate calculated before driving (x s(j) ) are consistent, the control unit 31 can determine that the task state of the robot 4 is converted to the target task state s s(j) Otherwise, the control unit 31 may determine that the task state of the robot 4 has not been converted to the target task state s s(j) .

[0388] The control unit 31 can use any of the above methods to determine whether the task state of the robot 4 has been converted to the target task state s s(j) In addition, in the above judgments, “consistency” includes not only the complete consistency of the two values, but also the case where the difference between the two values ​​is below the threshold (allowable error). When it is determined that the task state of the robot 4 has been converted to the target task state s s(j) If the value x is not the same as the value x, the control unit 31 proceeds to the next step S318. Otherwise, the control unit 31 returns to step S310 and drives the robot 4 again. At this time, the control unit 31 may use the current value of the fingertip coordinates calculated in step S316 as the current value x. (j) , execute step S310 and subsequent processing.

[0389] (Step S318)

[0390] In step S318, the control unit 31 determines whether the task state s of the final goal can be achieved. g .

[0391] The determination method is not particularly limited and can be appropriately selected according to the implementation. For example, the control unit 31 can also determine the current task state s obtained in step S315. (j+1) Is it consistent with the task status s of the final goal? g When the current task status s (j+1) The task status s with the final goal g If they match, the control unit 31 determines that the task state s can achieve the final goal. g Otherwise, the control unit 31 determines that the task state s cannot achieve the final goal. g As described above, “match” in this determination includes not only a complete match between the two values ​​but also a case where the difference between the two values ​​is less than a threshold value (allowable error).

[0392] When it is determined that the task state s can achieve the final goal g When the control unit 31 ends the series of processing related to the motion control of the robot 4. On the other hand, when it is determined that the task state s cannot achieve the final goal g When the control unit 31 returns the process to step S308. Then, the control unit 31 uses the results of steps S315 and S316 to execute the process of steps S308 to S313 again. The control unit 31 repeatedly performs the above series of processes to achieve the final target task state s g Thus, the control device 3 according to this embodiment can control the operation of the robot 4 so as to perform a designated task.

[0393] In addition, the task state s is determined to be unable to achieve the final goal. g The branch target when the robot 4 is executed may not be limited to the above step S308. For example, when the robot 4 is caused to execute a series of tasks consisting of multiple tasks, the task state s of the final target may be s308. g In this embodiment, when executing a task of holding a workpiece W by the end effector T and assembling the held workpiece W to another workpiece G, the task state of the final target s is set. g The state where the workpiece W is assembled to the other workpiece G can be adopted. In this case, the execution of a series of tasks can be started from the starting point of the first task. gThe branch target in this case may not be step S308, but step S303. Thus, the control unit 31 can drive the robot 4 while confirming the operation mode through the processing of steps S304 and S305. As a result, a series of tasks can be executed while smoothly switching between tasks. In this embodiment, while the end effector T is holding a workpiece W, the operation mode can be smoothly switched to a task of transporting the workpiece W to another workpiece G.

[0394] [feature]

[0395] As described above, the model generation device 1 according to this embodiment implements machine learning through the processing of steps S101 and S102, thereby generating a determination model 50 for determining whether two objects are in contact within a positional relationship. Based on the learned determination model 50 generated through machine learning, even when the positional relationship of the objects is presented as a continuous value, it is possible to determine whether two objects are in contact within that positional relationship without significantly increasing the amount of data in the determination model 50. Therefore, according to this embodiment, the amount of data representing the boundary of contact between two objects can be significantly reduced.

[0396] Furthermore, in this embodiment, the learned determination model 50 is used to determine whether at least one of the two objects in contact is an object that is moved by the manipulator 4. Therefore, in scenarios that define the manipulator 4's movements, the amount of data required to express the boundary between the two objects in contact can be significantly reduced. Even with relatively small RAM, ROM, and storage unit 32 capacities in the control device 3, the learned determination model 50 can be utilized, allowing the manipulator 4 to be appropriately driven without wasting fingertips contacting the target object.

[0397] Furthermore, in this embodiment, through the processing of steps S201 to S210, the learned judgment model 50 is used to generate an inference model 55 for determining the target task state so that the first object does not come into contact with the second object. In the context of controlling the movement of the robot 4, the generated inference model 55 is used to determine the target task state in step S308. Thus, even in the context of controlling the movement of the robot 4, the target task state can be determined so that the first object does not come into contact with the second object, that is, the fingertips of the robot 4 do not wastefully contact the target object, even without the computational processing of the learned judgment model 50. This reduces the computational cost of controlling the movement of the robot 4.

[0398] Furthermore, in this embodiment, the state of the task executed by the robot 4 is expressed by the relative positional relationship between objects, such as the end effector T (end effector), the workpiece W, and other workpieces G. Thus, control instructions are not directly associated with the task, but rather with the amount of change in the relative positional relationship between the objects. That is, a time-series of control instructions provided to the robot 4 can be generated or taught to change the relative positional relationship of the objects, regardless of the content of the task. For example, even if the coordinates of the workpiece W change, the change in the coordinates of the workpiece W is taken into account when the positional relationship (task state) between the end effector T and the workpiece W is learned in steps S306 and S315. Therefore, the robot 4 can appropriately hold the workpiece W using the end effector T based on the learning results. Therefore, according to this embodiment, the versatility of the ability to execute learned tasks can be improved, thereby reducing the cost of teaching the task to the robot 4.

[0399] In addition, in this embodiment, the positional relationship between objects is expressed using relative coordinates. This allows the positional relationship between two objects to be appropriately and clearly expressed. Therefore, the positional relationship between the two objects (or task status in the controlled scenario) can be easily grasped.

[0400] In addition, in this embodiment, as a method for calculating the coordinates of the fingertips of the robot 4 in the above-mentioned steps S307 and S316, a first method based on forward kinematics calculation can be adopted. In the first method, when the end effector T does not hold the workpiece W, the end effector T is set as a fingertip, and the first transformation matrix group derived from the first homogeneous transformation matrix of each joint (joint parts 41 to 46) is used in the forward kinematics calculation. As a transformation function. On the other hand, when the end effector T holds the workpiece W, the workpiece W is set as the fingertip, and the transformation function used in the forward kinematics calculation is expanded. Specifically, in the forward kinematics calculation, the second homogeneous transformation matrix ( t T w ) multiplied by the first transformation matrix group The second transformation matrix group obtained That is, in the present embodiment, when the workpiece W is held by the end effector T, the kinematic reference point is changed from the end effector T to the workpiece W.

[0401] Thus, the forward kinematic calculations in steps S307 and S316, and the inverse kinematic calculations in step S311, can be processed in roughly the same manner whether the end effector T is not holding a workpiece W or when it is holding a workpiece W. In other words, the first task of holding a workpiece W by the end effector T and the second task of assembling the workpiece W held by the end effector T onto another workpiece G can be processed as a common task of "moving the fingertips of the robot 4 relative to a target object." Therefore, according to this embodiment, the control process can be universally and uniformly specified, without distinguishing between the cases where the end effector T is not holding a workpiece W and the cases where the end effector T is holding a workpiece W. This simplifies the control process, thereby reducing the cost of generating or teaching the movements of the robot 4.

[0402] §4 Variations

[0403] The embodiments of the present invention have been described in detail above, but the above description is merely illustrative of the present invention in all aspects. Needless to say, various improvements or modifications can be made without departing from the scope of the present invention. For example, the following changes can be made. In addition, the same reference numerals are used below for the same components as the above embodiment, and descriptions of the same points as the above embodiment are appropriately omitted. The following modifications can be appropriately combined.

[0404] <4.1>

[0405] In the above embodiment, the end effector T, workpiece W, and other workpieces G are each an example of an object. However, the objects for which contact determination is made are not limited to these examples. Objects may include any type of object that can be handled in real or virtual space. In addition to the end effector T, workpiece W, and other workpieces G described above, objects such as obstacles that may be relevant to the robot's movements may also be included.

[0406] Furthermore, an object may be composed of one object or multiple objects. When there are three or more objects, the determination model 50 may be configured to treat the multiple objects as a single object and determine whether contact occurs between the multiple objects and other objects. Alternatively, the determination model 50 may be configured to treat the individual objects as a single object and determine whether contact occurs between the individual objects.

[0407] Furthermore, in the above-described embodiment, at least one of the two objects is an object that is moved by the operation of the robot. The object that is moved by the operation of the robot may be, for example, a component of the robot such as an end effector, or the robot itself, or an object other than a component of the robot such as a workpiece held by the end effector. However, the type of object is not limited to this example. Both objects may also be objects other than the object that is moved by the operation of the robot.

[0408] In addition, in the above embodiment, the manipulator 4 is an example of a robot device. However, the type of robot device may not be limited to such an example. In addition to industrial robots such as the manipulator 4, the robot device may also be, for example, a mobile body capable of automatic movement, an autonomous robot, etc. Mobile bodies capable of automatic movement may include, for example, drones, vehicles configured to be automatically driven, mobile robots, etc. In addition, in the above embodiment, the manipulator 4 is a vertical multi-joint robot. However, the type of industrial robot may not be limited to such an example. In addition to the above-mentioned vertical multi-joint robot, industrial robots may also include SCARA robots, parallel linkage robots, orthogonal robots, collaborative robots, etc.

[0409] In addition, in the above-mentioned embodiment, the operation of holding the workpiece W by the end effector T and the operation of assembling the held workpiece W to other workpieces G are each an example of a task performed by the robot device. A task is an operation that the robot device performs, and its type may not be particularly limited and may be appropriately selected according to the embodiment. In addition to the above-mentioned holding of the workpiece W and the transportation of the workpiece W, tasks may also be, for example, component fitting, screw rotation, etc. Tasks may also be, for example, simple operations such as holding a workpiece and releasing a workpiece. A task may, for example, be a change in the coordinates of an object workpiece, such as holding the object workpiece and configuring the object workpiece at specified coordinates (position and posture). A task may, for example, be using a sprayer as an end effector to spray paint onto the workpiece from specified relative coordinates. In addition, a task may, for example, be configuring a camera mounted on the end effector at specified coordinates. Tasks may be provided in advance or provided by the operator's designation.

[0410] Furthermore, in the above embodiment, a camera S1, an encoder S2, and a tactile sensor S3 are used as sensors for observing the environment in which the task is performed. The camera S1 and the tactile sensor S3 are examples of sensors for observing the position of the workpiece W relative to the end effector T. However, the types of sensors that can be used are not limited to these examples. In addition to cameras, encoders, and tactile sensors, sensors such as proximity sensors, force sensors, torque sensors, and pressure sensors can be used. The proximity sensor can be placed within a range capable of observing the perimeter of the end effector T to detect the presence of objects approaching the end effector T. Similarly, as with the tactile sensor S3, the force sensor, torque sensor, and pressure sensor can be placed within a range capable of measuring the force acting on the end effector T to observe the force acting on the end effector T. At least one of the proximity sensor, force sensor, torque sensor, and pressure sensor can be used as a sensor for observing the position of the workpiece W relative to the end effector T. Furthermore, the camera S1 can be configured to be freely movable by the manipulator 4 or other robotic device. In this case, the coordinates of the camera S1 can be appropriately calibrated. Thereby, the range observed by the camera S1 can be arbitrarily controlled.

[0411] In the above embodiment, the setting of the fingertips of the robot 4 according to whether or not the workpiece W is held can be omitted. In this case, the fingertips of the robot 4 can be set appropriately. For example, the end effector T can be set as the fingertips of the robot 4 regardless of whether or not the workpiece W is held.

[0412] <4.2>

[0413] In the above embodiment, the control device 3 utilizes the learned judgment model 50 when generating the inference model 55. However, the use of the learned judgment model 50 is not limited to this example. The control device 3 may also utilize the learned judgment model 50 when controlling the operation of the robot 4.

[0414] Figure 19 This is an example of the processing sequence of the subroutine related to the determination of the task status of the target involved in this modification. The determination process of the task status of the target in step S308 can also be replaced by Figure 19 Processing of the illustrated subroutine.

[0415] In step S501, the control unit 31 functions as the action decision unit 313 to determine the next target task state to be transitioned to relative to the acquired current task state so as to approach the final target task state. Step S501 can be processed similarly to step S308 described above.

[0416] In step S502, the control unit 31 operates as the contact determination unit 314 and determines whether two objects are in contact with each other in the determined target task state using the learned determination model 50. Step S502 can be processed in the same manner as steps S203 and S206 described above.

[0417] In step S503, the control unit 31 determines the branch target based on the determination result of step S502. If step S502 determines that the two objects are in contact with each other under the target task state, the control unit 31 returns the process to step S501 and determines the target task state again. On the other hand, if it is determined that the two objects are not in contact with each other under the target task state, the control unit 31 proceeds to the next step S309. Thus, when controlling the movement of the manipulator 4, the control device 3 can utilize the learned determination model 50 to determine the movement of the manipulator 4, thereby preventing the first and second objects from unnecessarily coming into contact with each other.

[0418] <4.3>

[0419] In the above embodiment, the control device 3 determines the task state of the target using the inference model 55 in the above step S308. However, the method for determining the task state of the target may not be limited to such an example. The inference model 55 may not be used in determining the task state of the target. For example, in the above step S308, the task state of the target may be determined in the same manner as in the above step S205. As an example, the control unit 31 may also determine the task state of the target by a well-known method such as path planning. In addition, for example, a sequence of the task states of the target may also be provided in advance. In this case, in the above step S308, the control unit 31 may also refer to the data representing the sequence to determine the task state of the target to be converted next. The same is true for the above step S501.

[0420] Furthermore, in the above embodiment, the judgment results of the learned judgment model 50 are used to collect the learning data 223. However, the collection of the learning data 223 is not limited to this example. For example, the learning data 223 can be collected by using physical objects, etc., without using the learned judgment model 50. In this way, the inference model 55 can be generated without using the learned judgment model 50.

[0421] Furthermore, in the above-described embodiment, the series of processes for generating the inference model 55 (steps S201 to S211) can be omitted from the processing sequence of the control device 3. In this case, the data collection unit 317, the model generation unit 318, and the storage processing unit 319 can be omitted from the software configuration of the control device 3. Furthermore, the information processing related to the generation of the inference model 55 and the information processing related to the motion control of the manipulator 4 can be performed by different computers. Alternatively, the software module related to the motion control of the manipulator 4 can be omitted, and a new model generation device can be constructed by extracting the software module related to the generation of the inference model 55.

[0422] Figure 20 An example of the software configuration of the model generation device 3A according to this embodiment is schematically illustrated. The hardware configuration of the model generation device 3A can be the same as that of the control device 3 according to the above-described embodiment. The model generation device 3A operates as a computer including a contact determination unit 314, a data collection unit 317, a model generation unit 318, and a storage processing unit 319 as software modules. Thus, the model generation device 3A can execute the processing of steps S201 to S211 described above to generate the inference model 55. The portion of the control program 83 that provides commands related to the generation of the inference model 55 can also be referred to as a model generation program.

[0423] Furthermore, in the above-described embodiment and this variation, the processing related to the collection of learning data 323 from steps S201 to S209 may be performed by another computer. In this case, the control device 3 and model generation device 3A according to the above-described embodiment may acquire learning data 323 generated by another computer and execute steps S210 and S211 using the acquired learning data 323.

[0424] <4.4>

[0425] In the above embodiment, the positional relationship between two objects is expressed using relative coordinates. However, the method of expressing the positional relationship is not limited to this example. For example, the positional relationship can also be expressed using the absolute coordinates of each of the two objects. In this case, the absolute coordinates can be converted into relative coordinates, and the above information processing can be performed.

[0426] <4.5>

[0427] Furthermore, in the above embodiment, the control device 3 analyzes the target values ​​of the fingertip coordinates of the manipulator 4 in steps S307 and S309 to S312 to determine the command values ​​(control commands) to be provided to each joint. However, the method for determining control commands is not limited to this example. The control commands provided to the robot device can be appropriately determined so that the robot device's task state changes from the current task state to the target task state.

[0428] For example, the control instruction can be directly determined based on the current task state and the target task state. As an example, reference data such as a data table can be used to determine the control instruction. In this case, the control unit 31 can obtain the control instruction from the reference data by comparing the current task state and the target task state with the reference data. As another example, for example, a learned machine learning model that has mastered the ability to determine the control instruction based on the current task state and the target task state through machine learning can be used in the same way as the judgment model 50, the inference model 551, etc. In this case, the control unit 31 provides the current task state and the target task state to the learned machine learning model and executes the calculation processing of the learned machine learning model. Thus, the control unit 31 can obtain an output value corresponding to the determination result of the control instruction from the learned machine learning model. In addition, in the above embodiment, the control instruction is composed of the instruction value relative to the angle of each joint. However, the composition of the control instruction is not limited to such an example and can be appropriately determined according to the type of robot device, etc.

[0429] In the above embodiment, for example, if the task state of the final target is set in advance or if the task state of the final target is set by another method, the processing of steps S301 and S302 can be omitted from the processing sequence of the control device 3. In this case, the target setting unit 311 can be omitted from the software configuration of the control device 3.

[0430] <4.6>

[0431] In the above embodiment, the decision model 50 is composed of a fully connected neural network. However, the type of neural network that constitutes the decision model 50 is not limited to this example. In addition to the fully connected neural network, the decision model 50 may also be composed of, for example, a convolutional neural network, a recurrent neural network, etc. Furthermore, the decision model 50 may also be composed of a combination of multiple neural networks.

[0432] Furthermore, the type of machine learning model that constitutes determination model 50 is not limited to a neural network and may be appropriately selected depending on the implementation. In addition to neural networks, machine learning models such as support vector machines, regression models, and decision trees may also be used in determination model 50. Whether two objects are in contact with each other can be determined in either real space or virtual space.

[0433] In the above embodiment, the inference models 55 can be prepared according to the type of task to be performed by the robot 4. That is, multiple inference models 55 can be prepared, each trained to infer the task state of a target in different tasks. In this case, the control unit 31 of the control device 3 can select an inference model 55 to be used for inference from the multiple prepared inference models 55 based on the action mode set in step S305. Thus, the control unit 31 can switch the inference model 55 based on the action mode. Alternatively, the inference model 55 can be configured to further accept input of information representing task conditions, such as the type of object, the object identifier, the task identifier, and the type of task, and infer the task state of the target in the task corresponding to the input conditions. In this case, when determining the task state of the target to be switched to, the control unit 31 can further input information representing the action mode set in step S305 into the inference model 55 and perform the calculation processing of step S308.

[0434] In addition, in the above-mentioned embodiment, the form of the input and output of the judgment model 50 and the inference model 55 may not be particularly limited and may be appropriately determined according to the embodiment. For example, the judgment model 50 may be configured to accept input of information other than information indicating the task state. Similarly, the inference model 55 may be configured to accept input of information other than the current task state and the task state of the final target. In the case where the task state of the final target is constant, the information indicating the task state of the final target may be omitted from the input of the inference model 55. The output form of the judgment model 50 and the inference model 55 may also be either recognition or regression.

[0435] Description of Reference Numerals

[0436] 1…Model generation device,

[0437] 11...control unit, 12...storage unit, 13...communication interface,

[0438] 14…External interface,

[0439] 15…input device, 16…output device, 17…driver,

[0440] 91…storage medium, 81…model generation program,

[0441] 111…Data Acquisition Department, 112…Machine Learning Department,

[0442] 113…Storage Processing Unit,

[0443] 120…CAD data,

[0444] 121…Learning Dataset,

[0445] 122…training data, 123…correct answer data,

[0446] 125…Learning result data,

[0447] 3…control device,

[0448] 31 ...control unit, 32 ...storage unit, 33 ...communication interface,

[0449] 34…External interface,

[0450] 35…input device, 36…output device, 37…driver,

[0451] 93…Storage medium, 83…Control program,

[0452] 311 ...target setting unit, 312 ...status acquisition unit,

[0453] 313…Action Decision Unit, 314…Contact Decision Unit,

[0454] 315 ...command determination unit, 316 ...driving unit,

[0455] 317…Data Collection Department, 318…Model Generation Department,

[0456] 319…Storage Processing Department,

[0457] 320…CAD data, 321…Robot data,

[0458] 323…Learning data, 325…Inference model data,

[0459] 4…Manipulator (Robot device),

[0460] 40…base,

[0461] 41-46...joints, 491-494...connecting rods,

[0462] T…end effector,

[0463] TO…point of interest, CT…local coordinate system,

[0464] W...workpiece,

[0465] WO…focus point, CW…local coordinate system,

[0466] G...other workpieces, CG...local coordinate system,

[0467] RC1, RC2... relative coordinates,

[0468] S1...camera, S2...encoder, S3...tactile sensor,

[0469] 50…determination model,

[0470] 501…input layer, 502…middle (hidden) layer,

[0471] 503…output layer,

[0472] 55…Inference Model

Claims

1. A control device for controlling the operation of a robot apparatus that performs a task of moving a first object in an environment in which a first object and a second object are present, the control device comprising: a state acquisition unit configured to acquire a current task state of the robot apparatus, wherein the task state is defined by a positional relationship between the first object and the second object; an action decision unit for deciding a task state of a target to be converted next with respect to the acquired current task state in such a manner as to approach the task state of the final target; a contact determination unit that determines whether the first object and the second object are in contact with each other under the task state of the object by providing the task state of the object of the first object and the second object to a learned determination model trained by machine learning to output, in response to input of training data representing a positional relationship between the first object and the second object, an output value suitable for correct answer data indicating whether the first object and the second object are in contact with each other under the positional relationship; an instruction determination unit that determines a control instruction to be provided to the robot apparatus so as to change the task state of the robot apparatus from the current task state to the target task state; as well as a driving unit that drives the robot device by providing the determined control command to the robot device, The control device further includes a model generation unit, which generates an inference model using the judgment result of the learned judgment model, and the inference model is used to infer the task state of the target to be converted next based on the current task state and the task state of the final target in such a way that the first object does not contact the second object. The action decision unit determines the target task state to be converted next relative to the current task state using the generated inference model. The inference model is composed of a potential field that specifies the potential of each coordinate in the task space expressing the set of task states, The model generation unit generates a path from each of a plurality of task states provided as starting points to the final target task state by performing path planning in the task space using the learned judgment model so that the first object does not come into contact with the second object. The potential field is generated by setting the potential of each coordinate according to the passing frequency of each generated path. The action decision unit decides the task state of the target to be switched next based on the gradient of the potential set at the coordinates corresponding to the current task state in the generated potential field.

2. The control device according to claim 1, wherein: The positional relationship between the first object and the second object is expressed by relative coordinates.

3. The control device according to claim 1, wherein: The model generation unit generates the inference model by performing machine learning.

4. The control device according to claim 3, wherein: The machine learning is composed of reinforcement learning, wherein the reinforcement learning is rewarded based on the result of determining whether the first object and the second object are in contact with each other in the task state of the object using the learned judgment model and the distance between the task state of the object and the task state of the final goal. The reward is set to be greater when the first object and the second object are not in contact with each other and the distance is shorter, and the reward is set to be less when the first object and the second object are in contact with each other or the distance is longer. The implementation of the machine learning includes repeatedly modifying the values ​​of the operation parameters constituting the inference model to obtain more of the rewards.

5. A control method for controlling the operation of a robot device that performs a task of moving a first object in an environment where a first object and a second object exist. The control method is performed by a computer to perform the following steps: obtaining a current task status of the first object and the second object, wherein the task status is determined by a positional relationship between the first object and the second object; Determining a task state of a target to be converted next relative to the acquired current task state in a manner close to the task state of the final target; The learned determination model is provided with the task state of the object of the first object and the second object, thereby determining whether the first object and the second object are in contact with each other under the task state of the object, wherein the learned determination model is trained through machine learning to output an output value suitable for correct answer data indicating whether the first object and the second object are in contact with each other under the positional relationship, in response to input of training data indicating the positional relationship between the first object and the second object; determining a control instruction to be provided to the robot apparatus so as to change the task state of the robot apparatus from the current task state to the target task state; and By providing the determined control instructions to the robot device, the action of the robot device is controlled. The computer also performs the following steps: An inference model is generated using the judgment result of the learned judgment model, wherein the inference model is used to infer the task state of the target to be converted next based on the current task state and the task state of the final target in such a manner that the first object does not come into contact with the second object. In the step of determining the task state of the target, the generated inference model is used to determine the task state of the target to be converted next relative to the current task state. The inference model is composed of a potential field that specifies the potential of each coordinate in the task space expressing the set of task states, In the step of generating the inference model, a path is generated from each of a plurality of task states provided as starting points to the task state of the final target by performing path planning in the task space in a manner such that the first object does not come into contact with the second object. The potential field is generated by setting the potential of each coordinate according to the passing frequency of each generated path. In the step of determining the task state of the target, the task state of the target to be switched next is determined based on the gradient of the potential set at the coordinates corresponding to the current task state in the generated potential field.

Images