Construction method of trained model

Abstract

To efficiently obtain a high-performance trained model.SOLUTION: A construction method of a trained model includes six steps. In a first step, a computer collects data for performing machine learning for operation of a control object machine executed by a human. In a second step, the computer evaluates collection data which is the collected data, and re-collects the data when it does not satisfy a predetermined evaluation criterion. In a third step, the computer selects training data from the collection data that satisfies the evaluation criterion. In a fourth step, the computer evaluates the training data, and re-selects the training data when it does not satisfy the predetermined evaluation criterion. In a fifth step, the computer constructs the trained model, by machine learning using the training data which satisfies the evaluation criterion. In a sixth step, the computer evaluates the trained model, and re-trains the trained model when it does not satisfy the predetermined evaluation criterion.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to building trained models using machine learning. [Background technology]

[0002] Conventionally, systems have been known that use machine learning to control the behavior of robots, which automatically find laws and rules through repeated learning from collected data and realize functions similar to the natural learning ability of humans. Patent Document 1 discloses this type of system.

[0003] In the motion prediction system of Patent Document 1, a motion prediction model is constructed by machine learning data obtained when an operator manually and remotely controls a robot arm to perform a task. The robot arm is automatically operated based on the output of the motion prediction model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-206286 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, there are cases where it is necessary to evaluate the performance of the motion prediction model (trained model) constructed in Patent Document 1. However, the autonomous motion of a robot arm reflects a variety of aspects in a complex manner, such as the quality of the remote operation performed by the user during learning, the quality of the acquired training data, the quality of the constructed trained model, and the quality of the automatic motion based on the output of the trained model. Furthermore, each quality often mutually influences the other qualities. Therefore, it has been difficult to evaluate each quality independently. As a result, even if there is a need to improve the trained model, it is unclear what should be improved, requiring inefficient trial and error.

[0006] The present disclosure has been made in consideration of the above circumstances, and its purpose is to efficiently obtain a high-performance trained model. [Means for solving the problem]

[0007] The problem to be solved by the present disclosure is as described above. Next, the means for solving this problem and the effects thereof will be described.

[0008] According to an aspect of the present disclosure, there is provided the following trained model construction method. That is, the trained model construction method includes a first step, a second step, a third step, a fourth step, a fifth step, and a sixth step. In the first step, a computer collects data for machine learning of human operation of a controlled machine. In the second step, the collected data is evaluated based on a first evaluation criterion, and if the first evaluation criterion is not satisfied, the computer re-collects the data. In the third step, the computer selects training data from the collected data that satisfies the first evaluation criterion. In the fourth step, the training data is evaluated based on a second evaluation criterion, and if the second evaluation criterion is not satisfied, the computer re-selects the training data. In the fifth step, the computer constructs a trained model by machine learning using the training data that satisfies the second evaluation criterion. In the sixth step, the trained model is evaluated based on a third evaluation criterion, and if the third evaluation criterion is not satisfied, the computer re-trains the trained model.

[0009] In this way, by proceeding step by step until the trained model is built and evaluating it at each stage, it becomes easier to narrow down the cause if a problem is found at any stage, and the trained model can be built smoothly. [Effects of the Invention]

[0010] According to the present disclosure, a high-performance trained model can be efficiently obtained. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of a robot operation system according to an embodiment of the present disclosure. [Figure 2] FIG. 10 is a conceptual diagram illustrating operation information. [Figure 3] 1A and 1B are diagrams showing an example of a series of tasks performed by a robot and each task state. [Figure 4] FIG. 10 is a diagram showing an example of information presented to a user when selecting training data. [Figure 5] FIG. 10 is a diagram showing an example of selecting training data from collected data. [Figure 6] A figure showing an example of a display when performing an operation to delete part of the robot's behavior based on the output of a trained model. [Figure 7] A diagram showing an example of a display when the robot's movement based on the output of a trained model stalls midway. [Figure 8] FIG. 10 is a diagram showing a display example in which training data is added to resolve stagnation in the robot's operation. [Figure 9] Schematic diagram illustrating verification of learning elements of a trained model. [Figure 10] A flowchart showing the workflow from building a trained model to operating a robot. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, the disclosed embodiments will be described with reference to the drawings. First, a robot operation system 100 that uses a learning model constructed by the method of this embodiment will be briefly described with reference to FIG. 1 etc. FIG. 1 is a schematic diagram illustrating the robot operation system 100. FIG. 2 is a conceptual diagram illustrating operation information. FIG. 3 is a diagram illustrating an example of a series of tasks performed by a robot 11 and each task state.

[0013] The robot operation system 100 is a system that constructs a learning model 31 and operates the robot system 1 based on the output of this learning model 31. As a result of the robot system 1 operating according to the learning model 31, the robot 11 performs work autonomously.

[0014] The work performed by the robot 11 is arbitrary, but examples of the work include welding, assembly, processing, handling, painting, cleaning, polishing, and the like.

[0015] 1, the robot system 1 includes a robot control device 10, a robot (machine to be controlled) 11, and an operating device 12. The devices are connected to each other via a wired or wireless network and can exchange signals (data).

[0016] The robot control device 10 is configured with a known computer. It includes an arithmetic processing unit such as a microcontroller, CPU, MPU, PLC, DSP, ASIC, or FPGA, a robot memory unit such as ROM, RAM, or HDD, and a communication unit capable of communicating with external devices. The robot memory unit stores control applications for controlling the arm unit, etc.

[0017] The robot control device 10 can switch the operation mode of the robot 11 between a manual operation mode and an autonomous operation mode.

[0018] In the manual operation mode, the user manually operates the operation device 12 described below to operate the robot 11.

[0019] In the autonomous operation mode, the robot 11 operates automatically based on the results of prior machine learning of the behavior of the robot 11 under manual operation.

[0020] The robot 11 is configured as, for example, a vertical articulated robot with six degrees of freedom of movement. The robot 11 has an arm attached to a base. The arm has a plurality of joints. Each joint is provided with an actuator (for example, an electric motor) (not shown) for driving the arm around the joint. An end effector according to the work content is attached to the tip of the arm.

[0021] The arm and end effector of the robot 11 operate based on operation commands for operating the robot 11. These operation commands include, for example, commands for linear velocity, commands for angular velocity, and the like.

[0022] The robot 11 is equipped with sensors for detecting the motion and surrounding environment of the robot 11. In this embodiment, the robot 11 is equipped with a motion sensor 11a, a force sensor 11b, and a camera 11c.

[0023] The motion sensor 11a is configured by, for example, an encoder. The motion sensor 11a is provided for each joint of the arm portion of the robot 11, and detects the rotation angle or angular velocity of each joint.

[0024] The force sensor 11b detects the force acting on each joint of the arm section of the robot 11 or on the end effector attached to the tip of the arm section when the robot 11 is in operation. The force sensor 11b may be configured to detect a moment instead of or in addition to a force.

[0025] The camera 11c detects an image of the workpiece 81, which is the work target (the progress of the work being performed on the workpiece 81). In order to detect the progress of the work, a sound sensor that detects sound and / or a vibration sensor that detects vibration may be provided instead of or in addition to the camera 11c. Furthermore, the robot 11 may be provided with a sensor that collects distance information, such as a laser scan sensor or an infrared scan sensor.

[0026] The data detected by the motion sensor 11a is motion data indicating the motion of the robot 11, and the data detected by the force sensor 11b and the camera 11c is ambient environment data indicating the state of the environment around the robot 11. The ambient environment data is a status value indicating the state of progress of the work of the robot 11 at the time the sensor detects the data. The data detected by the motion sensor 11a, the force sensor 11b, and the camera 11c is collected as status information by the management device 20, which will be described later.

[0027] The operating device 12 is a member that the user operates to operate the robot 11. The operating device 12 varies depending on the work content, but is, for example, a lever that the user operates with their hands or a pedal that the user operates with their feet. The operating device 12 is configured, for example, as a remote operating device that is located in a location physically separated from the robot 11.

[0028] The operation device 12 is provided with an operation force detection sensor 13. The operation force detection sensor 13 detects a user operation force, which is a force applied to the operation device 12 by a user. When the operation device 12 is configured to be movable in various directions, the user operation force may be a value including the direction and magnitude of the force, for example, a vector. The user operation force may be detected not only as the force applied by the user, but also in the form of acceleration linked to the force (i.e., the value obtained by dividing the force applied by the user by the mass of the operation device 12).

[0029] 2, the user's operating force detected by the operating force detection sensor 13 includes, for example, force and velocity components (force x and velocity x) on the x-axis and force and velocity components (force y and velocity y) on the y-axis in the coordinate system of the robot 11. Data related to the user's operating force detected by the operating force detection sensor 13 is collected by the management device 20 as operation information.

[0030] 1, the robot operation system 100 includes a learning model 31. In the robot operation system 100, for example, the learning model 31 used to make the robot 11 execute a series of operations for inserting a workpiece 81 into a recess 82 of a member can be constructed by machine learning.

[0031] Specifically, the user operates the operating device 12 to operate the robot 11, for example, as shown below. That is, in operation OA shown in FIG. 3, with the robot 11 holding the workpiece, the workpiece 81 is positioned above a member and the workpiece 81 is brought close to the surface of the member. In operation OB, the workpiece 81 is moved in this state and brought into contact with the surface of the member. In operation OC, the workpiece 81 is moved toward the position of the recess 82. Note that while the workpiece 81 is moving, the state in which the workpiece 81 is in contact with the surface of the member is maintained. In operation OD, the end of the workpiece 81 is brought into contact with the inner wall of the recess 82. In operation OE, the workpiece 81 is inserted into the recess 82.

[0032] In this way, the user operates the robot 11 so that the robot 11 performs the actions in the order of OA to OE. By learning the relationship between the state information and the user's operating force during this process, the robot operation system 100 can construct a learning model 31 that can cause the robot 11 to perform autonomous operations in the order of OA to OE.

[0033] As shown in FIG. 1, the robot operation system 100 of this embodiment includes a management device 20 in addition to the robot system 1.

[0034] The management device 20 is configured, for example, by a known computer, and includes an arithmetic processing unit such as a microcontroller, CPU, MPU, PLC, DSP, ASIC or FPGA, a robot memory unit such as ROM, RAM or HDD, and a communication unit capable of communicating with external devices.

[0035] The robot system 1 and the management device 20 are connected to each other via a wired or wireless network and can exchange signals (data). The management device 20 may be configured with the same physical hardware as the robot control device 10 included in the robot system 1.

[0036] The management device 20 includes a data collection unit 21, a training data selection unit 22, a model construction unit 23, and an action data recording unit 24.

[0037] The data collection unit 21 collects data from the robot system 1. As described above, the data collected by the data collection unit 21 from the robot system 1 includes state information indicating ambient environment data of the robot 11 and operation information reflecting a user operation force corresponding to the ambient environment data of the robot 11. Hereinafter, the data collected by the data collection unit 21 may be referred to as collected data.

[0038] The collected data is time-series data of a series of status information and operation information obtained when the user continuously operates the operation device 12 to make the robot 11 perform a certain task (or part of a task). That is, the data collection unit 21 collects each piece of status information and each piece of operation information in relation to time. For example, one piece of collected data is obtained when the user continuously operates the operation device 12 to make the robot 11 perform a series of tasks including the five actions OA to OE described in FIG. 3 once. The status information and operation information include measurement values ​​based on detection values ​​obtained by the camera 11c, the operating force detection sensor 13, etc.

[0039] The data collection unit 21 has an evaluation function for determining whether the collected data satisfies a predetermined evaluation standard.

[0040] The conditions required for the collected data are arbitrarily determined in consideration of the characteristics of the learning model 31, the contents of preprocessing performed on the collected data, etc. In this embodiment, the conditions that the collected data must satisfy are that the collected data has a length equal to or greater than a predetermined number in a time series, and that data indicating a predetermined operation appears at both the beginning and end of the time series.

[0041] The data collection unit 21 determines the length of the collected data according to the above-mentioned predetermined evaluation criteria. This determination is made based on rules, rather than complex methods using machine learning, for example. This simplifies the process and improves the real-time nature of the determination.

[0042] The data collection unit 21 may calculate a value indicating the degree to which the collected data is suitable for use as training data, and present the calculation result to the user as the effectiveness of the collected data. The calculation of the effectiveness may be performed, for example, as follows: That is, a score is defined in advance for each condition that the collected data must satisfy. The data collection unit 21 determines whether the collected data satisfies each condition, and calculates the total score according to the satisfied conditions as the effectiveness. Based on the calculation result of the effectiveness, it may be determined whether the collected data satisfies the evaluation criteria.

[0043] The effectiveness can be presented to the user by, for example, displaying a numerical value of the effectiveness on an output device such as a display (not shown) provided in the management device 20. Instead of displaying a numerical value on the display, the effectiveness may be expressed by, for example, the color of a displayed figure.

[0044] If the collected data does not satisfy the required conditions or if the effectiveness value is not satisfactory, the data collection unit 21 can suggest to the user how to improve the data. The suggestion of the improvement method can be realized, for example, by displaying a message on the display. The content of the message is arbitrary, but it can be, for example, a text message such as "The operation time of the device is too short." The suggestion of the improvement method is not limited to a text message, and can also be, for example, an icon, audio, video, or the like.

[0045] By repeating the operation while referring to these suggestions for improvement, the user can easily become proficient in operating the operating device 12 even if he or she is initially unfamiliar with the operation, and can operate the operating device 12 in a way that will allow the user to obtain collected data that is suitable for use as training data.

[0046] If the collected data meets a predetermined evaluation standard, the data collection unit 21 outputs the collected data to the training data selection unit 22.

[0047] The training data selection unit 22 selects the collected data input from the data collection unit 21 to obtain training data.

[0048] The management device 20 includes an input device (not shown), which may be, for example, a keyboard, a mouse, or a touch panel.

[0049] The user instructs, via the input device, whether or not to use the collected data collected by the data collection unit 21 when the user previously operated the operation device 12 as training data for machine learning. As a result, a selected portion of the collected data is adopted as training data.

[0050] In this embodiment, the selection of training data from the collected data is determined by user input. This allows the collected data used as training data to be limited to the data intended by the user. However, the training data selection unit 22 may also select the collected data automatically.

[0051] For example, collected data can be automatically selected as follows. The training data selection unit 22 includes a machine learning model constructed by machine learning to evaluate the collected data. This machine learning model is constructed separately from the learning model 31 and at an earlier timing than the learning model 31. Hereinafter, the machine learning model constructed in the training data selection unit 22 may be referred to as a selection model 41.

[0052] The selection model 41 learns the collected data collected by the data collection unit 21. In the training phase of the selection model 41, the collected data is classified into multiple groups using an appropriate clustering method. If the collected data includes multiple work states, the data is divided into each work state and classified. The work states will be described later.

[0053] Clustering is a technique for automatically obtaining multiple clusters, which are groups of data with similar characteristics, by learning distribution laws from a large amount of data. Clustering can be performed using a clustering technique such as the well-known NN method, K-Means method, or self-organizing map. The number of clusters into which the task states included in the collected data are classified can be determined as appropriate. Classification may also be performed using an automatic classification technique other than clustering.

[0054] The work states will now be described. In this embodiment, for example, collected data relating to a series of tasks collected by the data collection unit 21 is classified for each user operation (reference operation) corresponding to the work state. For example, as shown in FIG. 3, when the robot 11 is made to perform a series of tasks of placing a workpiece 81 into a recess 82, the work states can be classified into four states: in the air, in contact, inserted, and completed.

[0055] Working state SA (air) is a state in which the robot 11 holds the workpiece 81 and positions it above the recess 82. Working state SB (contact) is a state in which the workpiece 81 held by the robot 11 is in contact with the surface on which the recess 82 is formed. Working state SC (insertion) is a state in which the workpiece 81 held by the robot 11 is being inserted into the recess 82. Working state SD (complete) is a state in which the workpiece 81 held by the robot 11 is completely inserted into the recess 82.

[0056] In this way, the four work states are a classification of each step of a series of work performed by the robot 11. When the work of the robot 11 progresses correctly, the work state transitions in the order of work state SA (air), work state SB (contact), work state SC (insertion), and work state SD (completed).

[0057] The data learned by the selection model 41 can be, for example, a combination of any one task state and the next task state associated with that task state (i.e., the task state to which the task transitions next), and at least one set of state information and a user operation force associated with this state information. This allows the selection model 41 to learn the order of task states and the order of the corresponding operation forces. The machine learning of the selection model 41 can also be described as data clustering.

[0058] The above-mentioned working states SA, SB, SC, and SD are representative, and in reality, many different working states may exist. Let us consider a case where an operator operates the robot 11 to perform the same work several times, and, for example, a working state SA1 corresponding to one set of state information and operating force, a working state SA2 corresponding to another set of state information and operating force, and a working state SA3 corresponding to yet another set of state information and operating force are collected. Due to variations in the operator's operation and variations in the situation, these working states SA1, SA2, and SA3 are, in detail, different from one another. However, because working states SA1, SA2, and SA3 share common characteristics, they are classified into the same cluster (the cluster of working states SA).

[0059] As described above, the sorting model 41 performs machine learning to reflect the time sequence of the output of the operating forces. Simply put, the sorting model 41 learns at least one set of state information and operating force combination corresponding to each of the working states SA, SB, SC, and SD, and also learns the working order, such as the working state SA being followed by the working state SB. This makes it possible to use the sorting model 41 to perform classification that reflects the time series information of the operating forces. That is, it is possible to reflect each of the operating forces associated with each working state in the working order.

[0060] As described above, this status information is sensor information (e.g., work status such as position, speed, force, moment, and image) detected by the motion sensor 11a, force sensor 11b, and camera 11c. This status information may also include information calculated based on the sensor information (e.g., a value indicating a change over time in the sensor information from the past to the present).

[0061] In the inference phase, the selection model 41, which has completed the training phase, can estimate and output a reference operation corresponding to state information associated with the time-series information of the input collected data. Instead of the estimated reference operation, the selection model 41 may output, for example, information on the cluster to which the reference operation belongs. In this case, the selection model 41 can output a similarity between the operation information of the input collected data and the estimated reference operation. The similarity can be defined, for example, using the well-known Euclidean distance. The similarity output by the selection model 41 can be used as an evaluation value for evaluating the collected data.

[0062] As described in FIG. 3, in a series of tasks performed by the robot 11 indicated by one piece of collected data, the task state transitions sequentially. Taking this into consideration, the selection model 41 outputs similarities for operation information in each predetermined time range, as shown in FIG. 4. Specifically, if the evaluation value output by the selection model 41 for a predetermined time range of the collected data is equal to or greater than a predetermined threshold, the training data selection unit 22 assigns a label (correspondence information) indicating information about the cluster to which the reference operation output by the selection model 41 belongs to the collected data for that time range. On the other hand, if the evaluation value output by the selection model 41 is below the predetermined threshold, the training data selection unit 22 does not assign a label to that time range.

[0063] 4 shows an example in which operation information within a predetermined time range in the collected data is similar to the reference operation corresponding to work state SA, and therefore the operation information is labeled with the numerical value "1." Similarly, time ranges labeled with the numerical values ​​"2," "3," and "4" in FIG. 4 indicate that the operation information in the time range is similar to the reference operations corresponding to work states SB, SC, and SD. FIG. 4 also shows time ranges to which no labels were assigned.

[0064] For time ranges of the collected data that are not assigned labels, the situation is significantly different from any of the work states learned by the selection model 41, and it may be inappropriate to use this collected data as training data. Therefore, the training data selection unit 22 does not adopt this time range as training data. On the other hand, for time ranges of the collected data that are assigned labels, the training data selection unit 22 adopts them as training data.

[0065] The labels output by the selection model 41 do not have to be used for automatically determining whether or not to adopt the collected data as training data. For example, the labels may be presented as reference information when a user manually decides whether or not to adopt the collected data as training data. FIG. 4 shows an example of a screen presented to the user. In this example, the operation information (e.g., operation force) included in the collected data is visually displayed in the form of a graph, and data portions that are continuous in time series and assigned labels with the same numerical value are displayed as a single block. This allows the user to more easily determine whether or not to adopt the collected data as training data.

[0066] In this example, clustering is employed in the selection model 41. Therefore, it becomes easy to extract only data from a portion of the time range (block) in which the operation is valid from the collected data obtained by the user performing a series of operations, and employ it as training data. FIG. 5 shows an example of data employed as training data. In the example of FIG. 5, the entirety of one of the five collected data is employed as training data, and for the remaining three, only a portion of the labeled blocks is employed as training data.

[0067] The training data selection unit 22 has a function of evaluating the selected training data. The evaluation criteria are arbitrary. For example, the evaluation may be performed on whether the number of training data is sufficient, whether the training data corresponding to a certain work state is too small, etc.

[0068] If the training data meets the evaluation criteria, the training data selection unit 22 outputs the training data to the model construction unit 23.

[0069] The model construction unit 23 constructs, by machine learning (for example, supervised learning), a learning model 31 used in the robot system 1. Hereinafter, a learning model for which learning has been completed may be referred to as a trained model.

[0070] The model construction unit 23 uses the training data output from the training data selection unit 22 to construct the learning model 31. As described above, the training data corresponds to the collected data that has been selected. Therefore, like the collected data, the training data includes at least surrounding environment data (i.e., state information) that reflects the working state of the robot 11 and a user operation force (i.e., operation information) associated with the surrounding environment data.

[0071] The learning model 31 is, for example, a neural network of a general configuration having an input layer, a hidden layer, and an output layer. A plurality of units simulating brain cells are arranged in each layer. The hidden layer is provided between the input layer and the output layer and is composed of an appropriate number of intermediate units. The state information (training data) input to the learning model 31 by the model construction unit 23 flows in the order of the input layer, hidden layer, and output layer. The number of hidden layers is determined appropriately. However, this is not limited to this, and the format of the learning model 31 is arbitrary.

[0072] In this learning model 31, the data input to the input layer is state information that reflects the above-mentioned surrounding environment data. The data output by the output layer is the estimated result of the detection value of the operating force detection sensor 13. This essentially means the estimated user operating force. Therefore, the data output by the output layer indicates the user operation estimated by the learning model 31.

[0073] Each input unit and each intermediate unit are connected by a path through which information flows, and each intermediate unit and each output unit are connected by a path through which information flows. In each path, the influence (weight) of the information of the upstream unit on the information of the downstream unit is set.

[0074] In the training phase of the learning model 31, the model construction unit 23 inputs state information to the learning model 31 and compares the operating force output from the learning model 31 with the operating force applied by the user. The model construction unit 23 updates the learning model 31 by updating the weights using, for example, an error backpropagation method, which is a well-known algorithm, so as to reduce the error obtained by this comparison.

[0075] Since the learning model 31 is not limited to a neural network, the update of the learning model 31 is not limited to the backpropagation method. For example, the learning model 31 can also be updated by a known algorithm, SOM (Self-organizing maps). Learning is achieved by continuously performing such processing.

[0076] The model construction unit 23 has a function of evaluating the constructed learning model 31. The criteria for this evaluation are arbitrary. For example, when specific state information is input to the learning model 31, an evaluation is made as to whether or not an expected user operation force is output. The working time, power consumption, etc. may also be evaluated by a simulation using a three-dimensional model of the robot 11, etc.

[0077] Regarding the evaluation of the learning model 31, when the model construction unit 23 operates the learning model 31 in the inference phase, it can also be configured to present training data that serves as the basis for the inference output by the learning model 31. This makes it easier for humans to evaluate the learning model 31.

[0078] For example, consider a case where the learning model 31 is constructed not by a neural network but by clustering, similar to the above-mentioned selection model 41. In the training phase, each training data is plotted as a point in a multidimensional feature space. Each training data is assigned identification information that can uniquely identify the training data.

[0079] Once all the training data has been plotted, multiple clusters are determined using the appropriate clustering method described above, as in the selection model 41. Next, the model construction unit 23 determines data representative of each cluster. Hereinafter, this data may be referred to as a node. For example, a node may be data corresponding to the center of gravity of each cluster in a multidimensional space. When nodes have been determined for all clusters, the training phase is complete.

[0080] In the inference phase of the learning model 31, state information at a certain point in time is input to the learning model 31. The learning model 31 finds one or more nodes that have characteristics similar to the state information. The similarity can be defined using, for example, Euclidean distance.

[0081] When a node having characteristics similar to the state information is found, the learning model 31 finds the user's operating force (in other words, the detection value of the operating force detection sensor 13) included in the data of that node. If multiple nodes with similar state information are found, the user's operating forces of the multiple nodes are appropriately combined. The learning model 31 outputs the obtained user's operating force as the above-mentioned estimated operating force. At this time, the learning model 31 outputs identification information that identifies the training data corresponding to that node. This makes it possible to identify the training data that served as the basis for the inference of the learning model 31.

[0082] For example, suppose that in the above simulation, some of the actions of the robot 11 based on the learning model 31 are found to be undesirable, and the user decides not to allow the robot 11 to perform those actions. FIG. 6 shows an example of a simulation screen displayed on the display of the management device 20. The simulation screen displays a timeline, and labels corresponding to the aforementioned task states, which classify a series of actions performed by the robot 11 in the simulation, are displayed along the time axis. The user operates an appropriate input device to select a time range (in other words, task states) corresponding to the actions to be deleted, in units of the above blocks. Then, the display highlights the portion (block) of the task state corresponding to the action specified for deletion in the training data. The highlighting method can be arbitrary. As a result, the user can intuitively grasp the magnitude of the impact of the deletion and determine whether or not to actually delete the action.

[0083] When the user instructs the deletion of an action, the plot of the training data corresponding to the portion instructed to be deleted is deleted from the clustering result, which corresponds to the learning result of the learning model 31. When the learning model 31 subsequently operates in the inference phase, it will output an operating force based on other training data instead of the operating force that caused the undesired action. In this way, the learning result of the learning model 31 can be partially corrected.

[0084] For example, suppose that in the above simulation, a situation occurs in which the operation of the robot 11 based on the learning model 31 stagnates. To give a specific example, in operation OD in which the end of the workpiece 81 is brought into contact with the inner wall of the recess 82 in Fig. 3, the position of the workpiece 81 does not match the position of the recess 82, so the workpiece 81 cannot be brought into contact with the inner wall of the recess 82, and as a result, the robot 11 stops without performing any further operations. Fig. 7 shows an example of a simulation result in which the operation of the robot 11 stagnates.

[0085] To resolve this situation, suppose the user wants to add one or more new training data. This training data will perform a series of operations, just like the other training data. However, the new training data includes, before the above-mentioned operation OD, an operation in which the workpiece 81 is brought into contact with a surface near the recess 82 and is then slightly moved in various directions in a horizontal plane so that the center of the workpiece 81 is aligned with the center of the recess 82.

[0086] When the user instructs the addition of new training data, a plot of the training data is added to the clustering results in the multidimensional space described above. This corresponds to additional learning being performed in the learning model 31.

[0087] After additional learning of the learning model 31, the user instructs to perform a simulation again under the same conditions as above. In this simulation, the robot 11 performs a new action that was not present in the previous simulation, and as a result, the series of actions is successful. FIG. 8 shows an example of the simulation results in this case. On the simulation result display screen, the user operates the input device to select the part of the timeline that corresponds to the newly performed action. If the part corresponding to the newly added training data is highlighted in response to this selection, the user can determine that the contribution of the newly added training data has resolved the stagnation in the actions in the simulation.

[0088] In this way, in the above configuration, the user is presented with the impact of deleting or adding learning content to the learning model 31. As a result, the user can edit the learning content of the learning model 31 with peace of mind.

[0089] If it is determined that the learning model 31 satisfies the evaluation criteria, the model construction unit 23 outputs information indicating this to the action data recording unit 24.

[0090] The operation data recording unit 24 transmits the output of the learning model 31 to the robot system 1 to cause the robot 11 to operate autonomously, and records the operation data. This operation data is used, for example, to verify the autonomous operation of the robot 11. As will be described in detail later, the operation data recording unit 24 can also be used in subsequent actual operation situations.

[0091] State information indicating the surrounding environment data of the robot 11 is input to the operation data recording unit 24. The operation data recording unit 24 outputs the input state information to the learning model 31 of the model construction unit 23. The learning model 31 operates in the inference phase, and its output is input to the robot control device 10. This allows the robot 11 to operate autonomously.

[0092] When the learning model 31 is operated in the inference phase, the action data recording unit 24 can also record, for each action, whether or not the action has been verified. For this recording, an appropriate storage unit provided in the management device 20 is used.

[0093] This will be explained in detail below. Due to the nature of machine learning, it is difficult to reproduce all of the behavior of a machine, such as the robot 11, based on a machine learning model through repeated testing in advance. Therefore, for example, when the learning model 31 is constructed by clustering as described above, the operation data recording unit 24 is configured to store each plot in the multidimensional space in association with whether it has been verified. Each plot in the clustering may be adopted as a node. As described above, a node is data that represents a cluster. Each piece of data (plot) and node corresponds to an individual learning element in the learning model 31.

[0094] FIG. 9 shows a schematic diagram of changes in the contents stored in the motion data recording unit 24 in relation to a plurality of stages that are performed after the learning model 31 is constructed.

[0095] First, the operation test stage will be described. After the learning model 31 is constructed by the model construction unit 23, an operation test is carried out using the actual robot 11, involving a certain number of trials.

[0096] As described above, in the inference phase of the learning model 31, state information at a certain point in time is input to the learning model 31. When the learning model 31 is constructed by clustering, the learning model 31 finds nodes that have characteristics similar to the state information.

[0097] The learning model 31 has a large number of nodes that may be the basis for outputting operation information in the inference phase. In FIG. 9, the nodes are represented schematically by small ellipses. The storage unit of the management device 20 can record in table format whether or not verification has already been performed for each node. However, the recording may be performed in a format other than a table. Hereinafter, a node for which verification has been recorded in the table may be referred to as a verified node, and a node for which verification has not been recorded may be referred to as an unverified node.

[0098] In the operation test stage, when the learning model 31 outputs the operation information included in the data of the node as an inference result, it outputs information identifying the node to the operation data recording unit 24. Hereinafter, this information may be referred to as node identification information. The node identification information may be, for example, an identification number of the training data corresponding to the node, but is not limited to this.

[0099] Before the operation test begins, all nodes are unverified. The user monitors the trial operation of the robot 11 based on the output of the learning model 31. The user determines whether there is a problem with the operation of the robot 11. In making this determination, the user can refer to the operation data recorded by the operation data recording unit 24 in an appropriate manner.

[0100] If it is determined that there is no problem with the operation of the robot 11, the user appropriately operates the input device of the management device 20 to instruct the completion of the verification. As a result, the operation data recording unit 24 updates the above-mentioned table and records that the verification has been performed for the node that performed the output in the test operation. In Figure 9, the verified nodes are shown with hatching.

[0101] As the operational test of the robot 11 is repeated, the number of unverified nodes decreases and the proportion of verified nodes gradually increases. However, it is practically impossible to make all nodes verified nodes at the operational test stage, and it is inevitable that the transition to the operational stage will occur with unverified nodes remaining.

[0102] Next, the operation phase will be described. During the operation phase of the robot 11, as in the above-mentioned operation test phase, the learning model 31 operates in the inference phase. When state information at a certain point in time is input to the learning model 31, the learning model 31 searches for nodes having characteristics similar to the state information.

[0103] The learning model 31 outputs the above-mentioned node identification information that identifies the determined node to the operation data recording unit 24. The output of this node identification information is performed before outputting the operation information included in the data of the node to the robot 11 as an inference result.

[0104] Based on the node identification information input from the learning model 31, the operation data recording unit 24 refers to the table described above and determines whether the node about to output operation information has been verified.

[0105] If the identified node is a verified node, the operation data recording unit 24 controls the robot 11 to operate according to the output of the learning model 31 from that node.

[0106] If the identified node is an unverified node, the operation data recording unit 24 searches for a verified node similar to the unverified node. This search corresponds to searching for a verified node within a predetermined distance from the unverified node in the multidimensional feature space in which the training data was plotted during clustering. This distance corresponds to the similarity, and can be, for example, the Euclidean distance.

[0107] When a verified node having a similarity equal to or greater than a predetermined value is found, the operation information corresponding to the output of the verified node, which is the search result, is compared with the operation information corresponding to the output of the unverified node. This comparison can also be performed using, for example, the Euclidean distance described above.

[0108] If the comparison determines that the similarity between the two outputs is equal to or greater than a predetermined value, it is considered that even if the robot 11 is operated based on the output of the unverified node, the operation will not be significantly different from the previously verified operation. Therefore, the operation data recording unit 24 controls the robot 11 to operate based on the output of the unverified node. It is preferable that the above-mentioned processes such as determining whether or not the node has been verified, searching for the node, and comparing the outputs are performed within the control cycle of the robot 11.

[0109] The operation of the robot 11 based on the output of the unverified node is monitored by the user as appropriate. If the user determines that no problems have occurred, the user operates an appropriate input device to instruct the management device 20 to complete the verification, as in the operational test phase. In response, the operational data recording unit 24 updates the above-mentioned table to record that verification has been performed on the unverified node. In this way, in this embodiment, an unverified node can be changed to a verified node not only in the operational test phase but also in the operation phase.

[0110] In the operation phase, the robot 11 does not have to operate based on the output of the unverified node itself. For example, the operation data recording unit 24 may operate the robot 11 using an output that is a combination of an unverified node and a similar verified node. The output can be combined by, for example, calculating the average and median of the outputs. When it is important to prevent unexpected behavior, the operation data recording unit 24 may also perform control so that the output of the unverified node is completely replaced by the output of a similar verified node.

[0111] In the above-described search process, it may be possible that a verified node similar to the unverified node is not found. Even if a similar verified node is found, the output of the unverified node may not be similar to the output of the verified node. In either of the above cases, it is preferable that the operation data recording unit 24 forcibly change the output of the learning model 31 to a predetermined output for stable operation of the robot 11.

[0112] For example, consider a case where the learning model 31 is made to learn the operation of a master robot in master-slave control that operates to eliminate the difference in force between the master robot and the slave robot. In this configuration, when the learning model 31 attempts to output operation information based on an unverified node and is unable to find a similar verified node, the operation data recording unit 24 forcibly and continuously changes the user operation force output by the learning model 31 thereafter to zero. As a result, the operation output of the master robot becomes zero, and the slave robot operates in a direction that reduces the external force to zero. Therefore, the slave robot can transition to a stable state where the external force becomes zero.

[0113] The operation data recording unit 24 can also be configured to output a warning when it is detected that the learning model 31 is attempting to output operation information based on an unverified node. The warning can be issued, for example, by displaying a message on a display screen, but can also be issued by other means, such as audio. This allows the user to quickly grasp the situation.

[0114] In addition to information on whether or not the node has been verified, additional information may be stored in the storage unit of the management device 20 in association with each node that may be the basis for the learning model 31 to output operation information in the inference phase. This additional information may include, for example, whether or not the node is a node newly added in additional learning, whether or not the node has an output that causes the robot 11 to operate with a force greater than a predetermined value, whether or not damage to the workpiece 81 has occurred in the past, etc. This information can be registered, for example, by the user appropriately operating the input device of the management device 20.

[0115] The operation data recording unit 24 presents additional information to the user, for example, when outputting the above-mentioned warning. The information is presented to the user, for example, by displaying it on a display. This allows the user to obtain useful information about unverified nodes, making it easier for them to make appropriate decisions regarding the operation of the learning model 31.

[0116] As described above, the robot 11's behavior based on the machine learning model is diverse, making it difficult to verify all behaviors. In particular, when the robot 11 performs a task involving force contact as shown in FIG. 3 , it is impossible to test all possible situations in advance. Taking this into consideration, when the learning model 31 outputs a behavior that has not been verified, the behavior data recording unit 24 forcibly changes the output of the learning model 31, for example, to a similar behavior that has been verified. In this way, in the robot operation system 100 of this embodiment, the behavior data recording unit 24 essentially functions as a model output control unit and performs interfering control on the output of the learning model 31. This makes it possible to prevent unexpected behavior of the robot 11.

[0117] Additional learning of the learning model 31 may be required in the operation phase. In this case, the user operates the operation device 12 to add new training data. As a result of clustering being performed again, a new node is added to the learning model 31. In FIG. 9, the added node is indicated by a dashed line. The added node is recorded in the table described above as an unverified node. Once the additional learning work is completed, the process returns to the operational test phase or operation phase. Once verification of the added node is completed in the same manner as above, the record of the node in the table changes from unverified to verified.

[0118] Next, the flow of building the machine learning model shown above will be explained.

[0119] In this embodiment, as shown in Fig. 10, the workflow from constructing a learning model 31 to operating it is divided into four stages: (A) acquiring collected data, (B) selecting training data, (C) constructing a trained model, and (D) acquiring behavioral data. In Fig. 10, numbers [1] to [8] indicate the first to eighth steps. Evaluation is performed at each stage, and if the evaluation criteria are not met, the work at that stage is redone, as indicated by the dashed arrow in Fig. 10. Only when it is determined that the evaluation criteria are met can the work at the next stage be progressed to.

[0120] Therefore, evaluation at each stage is based on the assumption that the evaluation criteria have been met in the work upstream. Therefore, for example, if the trained model constructed in stage (C) does not meet the evaluation criteria, it can be basically assumed that there is no problem with the work in the upstream stages, and that there is a problem with the work of constructing the trained model.

[0121] When there is a problem with the autonomous operation of a machine using a machine learning model, it is often extremely difficult to determine whether the cause is a problem with the training data or the construction of the trained model. In this regard, in this embodiment, the work is carried out while performing evaluations at each stage, making it easy to narrow down the cause when a problem is found. Therefore, it is possible to smoothly construct and operate a machine learning model.

[0122] Since the work at each stage is independent, it is easy to divide the work of four stages among four people, for example, and the boundaries of each person's responsibilities can be clearly defined.

[0123] Of course, for example, if the trained model constructed in stage (C) does not meet the evaluation criteria, it may be discovered that the cause is a problem with the selection of training data that was overlooked in the evaluation in stage (B). In that case, as shown by the dashed arrow in Figure 10, the process returns to the previous stage and the work and evaluation are redone. This rule prevents major rework and improves work efficiency.

[0124] As described above, in this embodiment, a trained model is constructed by a method including the following six steps. In the first step, data for machine learning of the user's operation of the robot 11 is collected. In the second step, the collected data is evaluated, and if the collected data does not satisfy a predetermined evaluation criterion, the data is re-collected. In the third step, training data is selected from the collected data that satisfies the evaluation criterion. In the fourth step, the training data is evaluated, and if the training data does not satisfy the predetermined evaluation criterion, the training data is re-selected. In the fifth step, a trained model is constructed by machine learning using training data that satisfies the evaluation criterion. In the sixth step, the trained model is evaluated, and if the predetermined evaluation criterion is not satisfied, the trained model is re-trained.

[0125] By proceeding with the work in stages and evaluating each stage in this way, it becomes easier to narrow down the cause of any problems found in any process, and therefore the learning model 31 can be constructed smoothly.

[0126] In this embodiment, if the training data does not satisfy the evaluation criteria in step 4 and there is a problem with the collected data, the process returns to step 1. If the trained model does not satisfy the evaluation criteria in step 6 and there is a problem with the training data, the process returns to step 3. However, the process may return to step 2 instead of step 1, or to step 4 instead of step 3.

[0127] If a problem is found in the work at a previous stage, the problem can be properly resolved by redoing the work at that stage.

[0128] In this embodiment, in a first step, when a user operates the robot 11, information including the operation is collected as data. In a second step, it is determined whether the collected data is appropriate as training data based on a predetermined rule, and the result of the determination is presented to the user.

[0129] This allows the user to easily understand whether the collected data is suitable as training data or not, and also simplifies the processing.

[0130] In this embodiment, when the learning model 31 constructed in the fifth step operates in the inference phase, the training data used to construct the learning model 31, which serves as the basis for the output of the learning model 31, is identified and output.

[0131] This allows the user to grasp to a certain extent the extent of the impact of editing, for example, when editing part of the learning results of the learning model 31. Therefore, partial changes and deletions of the learning contents of the learning model 31 can be made accurately.

[0132] In this embodiment, in the seventh step, the robot 11 is operated based on the output of the learning model 31 that satisfies the evaluation criteria, and the operation data is recorded. In the eighth step, the operation data is evaluated, and if the predetermined evaluation criteria are not satisfied, the robot 11 is operated again and the operation data is recorded again.

[0133] By carrying out evaluations at each stage up to the actual operation of the robot 11, it becomes easier to narrow down the cause if a problem is found in any process.

[0134] In this embodiment, if the motion data does not satisfy the evaluation criteria in step 8 and there is a problem with the learning model 31, the process returns to step 5. However, the process may return to step 6 instead of step 5.

[0135] If a problem is found in the work at a previous stage, the problem can be properly resolved by redoing the work at that stage.

[0136] In this embodiment, the learning model 31 includes multiple learning elements. If the learning model 31 is constructed by clustering, for example, a node, which is data representing a cluster, corresponds to a learning element. In step 7, if the behavior of the robot 11 based on the output of the learned model is verified, the learning element that was the basis for the behavior is recorded as having been verified. When the learned model is put into actual use, the output of the learned model based on an unverified learning element can be changed to a predetermined output or to an output based on a similar verified learning element.

[0137] This makes it possible to prevent the robot 11 from performing unexpected actions due to unverified learning elements.

[0138] In this embodiment, the robot 11 is the subject of autonomous operation by the learning model 31.

[0139] This allows the learning model 31 for the autonomous operation of the robot 11 to be smoothly constructed.

[0140] Although the preferred embodiment of the present disclosure has been described above, the above configuration can be modified, for example, as follows. A single modification may be made, or multiple modifications may be made in any combination.

[0141] The seventh and eighth steps in FIG. 10 may be omitted, and the constructed learning model 31 may be immediately used for practical operation.

[0142] The evaluation of the collected data shown in the second step may be performed based on machine learning rather than on a so-called rule basis.

[0143] In steps 2, 4, 6 and 8, the evaluation may be performed by a computer or by a human.

[0144] The output device used by the management device 20 for various displays may be, for example, a liquid crystal display, but a projector, a head-mounted display, etc. may also be used. For example, when a head-mounted display is used, a display using known augmented reality (AR) may be performed.

[0145] The operation information estimated by the trained model may be, instead of the user operation force, for example, the operation speed or the amount of change in position at which the user operates the operation device 12. The trained model may estimate the relationship between the state information and the control signal to the robot 11, instead of estimating the relationship between the state information and the operation information.

[0146] The constructed learning model 31 can also be applied to machines other than the robot 11 as the control target. [Explanation of symbols]

[0147] 1. Robot System 10 Robot control device 11 Robots (machines to be controlled) 20 Management device 31 Learning model (trained model) 100 Robot Operation System

Claims

1. A first step in which a computer collects data for machine learning of human operation of a robot; a second step of evaluating the collected data based on a first evaluation criterion, and if the first evaluation criterion is not satisfied, the computer recollects the data; a third step in which the computer selects training data from the collected data that meets the first evaluation criterion; a fourth step of evaluating the training data based on a second evaluation criterion, and re-screening the training data by the computer if the second evaluation criterion is not satisfied; A fifth step in which the computer constructs a trained model by machine learning using the training data that satisfies the second evaluation criterion; Including, A method for constructing a trained model, in which the selection in the third step is performed by a selection model, which is a machine learning model constructed earlier than the trained model by learning the collected data.

2. A method for constructing a trained model according to claim 1, comprising: the collected data includes status information indicating ambient environment data of the robot when a human operates the robot to perform a series of tasks, and time-series data of operation information reflecting an operating force of the human corresponding to the ambient environment data of the robot; the selection model is capable of estimating and outputting a reference operation, which is a human operation corresponding to the state information of the data input to the selection model, and outputting a similarity between the reference operation and the operation information of the input data; A method for constructing a trained model, in which the selection in the third step is performed based on the similarity output by the selection model when the collected data is input into the selection model.

3. A method for constructing a trained model according to claim 2, comprising: In the third step, the output of the similarity by the selection model is performed for the operation information in each predetermined time range included in the collected data that satisfies the first evaluation criterion, A method for constructing a trained model, wherein in the third step, a time range in which the similarity output by the selection model is greater than or equal to a predetermined threshold is adopted as the training data.

4. A method for constructing a trained model according to claim 3, comprising: A method for constructing a trained model, in which the selection model is constructed by classifying the collected data by clustering.

Images