Processing device, robot control system and machine learning method

Abstract

To provide a processing device, a robot control system, and a machine learning method, which can reduce temporal variations in teaching data that impede motion learning.SOLUTION: A processor (CPU (23)) of a processing device (computer 20) segments teaching data for a robot (robot device 1) into the same kinds of motion of the robot (motion segmenting unit 41). The processor performs correction for equalizing the speed or timing of each of the same kind of motion of the robot to the plurality of segmented teaching data (speed adjusting unit 42, timing adjusting unit 43). The processor combines the plurality of corrected teaching data (data processing unit 45). The processor performs machine learning by using the combined teaching data (machine learning device 34).SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a processing device, a robot control system, and a machine learning method. [Background technology]

[0002] High-mix, low-volume production systems are gaining attention as a way to respond to rapid changes in the business environment and diversifying people's needs. While robotic automation is considered a way to increase production efficiency, controlling robots requires enormous programming costs and specialized knowledge, making the implementation process challenging. To reduce the amount of work required to implement robots, machine learning is being applied to all robot functions. For example, in object recognition, numerous studies have been reported that use machine learning to quantitatively calculate object features from contact information with the robot and image information in order to estimate the type, position, and orientation of an object.

[0003] Meanwhile, in the area of ​​motion generation, many studies have been reported that apply machine learning to the autonomous control of robots in order to realize movements that are difficult to describe in programs or that respond to environmental changes. Reinforcement learning is one example of an autonomous control technology. Robots acquire the optimal movements to perform a task through trial and error, so there is no need to explicitly teach them movements. However, acquiring the optimal movements requires a huge amount of trial and error (learning time).

[0004] Imitation learning is an autonomous control technology with a short learning time. Imitation learning consists of three phases: teaching, learning, and execution. In the teaching phase, the instructor operates the robot to teach it the desired behavior, and collects and records time-series information output by the robot's onboard encoders, temperature sensors, vision sensors, ultrasonic sensors, and other sensors. Teaching a robot to perform a behavior can be done by remote control using a controller, direct teaching, where the robot is directly touched, or by using a control program.

[0005] The time-series information obtained in the teaching phase is called teaching data. The type of teaching data depends on the movement generation method. For example, when a movement is generated according to the position of the robot or an object, the teaching data includes the robot's position, joint angle information, etc. Furthermore, when a movement is generated according to visual information, the teaching data may include images of the robot and the work environment. Furthermore, when a movement is generated according to the state of the object, the teaching data may include the object's position and posture information. Furthermore, when a movement is generated according to the robot's speed, the teaching data includes the robot's speed, joint angular velocity information, etc. Furthermore, when a movement is generated according to the robot's force, the teaching data includes the robot's tactile information.

[0006] In the learning phase, training data is created based on the teaching data collected in the teaching phase, and this training data is used to train the movement generation model. The training data may be used as is. Alternatively, the training data may be created by converting the teaching data to remove noise, missing values, error values, etc. contained in the teaching data. Converting teaching data into training data is called preprocessing. Preprocessing includes data cleaning, which removes data containing missing values ​​and outliers, and normalization, which converts data values ​​so that they fall within a specified range. The training data is used to update the weights of the machine learning model so that when a certain value is input to the model, the desired value is output. For example, when joint angle information and an image at a certain time are input, the model is trained to predict joint angle information and an image at the next time.

[0007] In the execution phase, the motion generation model obtained in the learning phase and the robot's sensor data are used to generate motion and execute the task. For example, in the learning phase, a machine learning model is trained to predict the joint angle information and image at the next time point when joint angle information and an image at a certain time point are input. By using this machine learning model in the execution phase, joint angle information and images at each time point are input, and the output joint angle information is used as a control command value for the robot, it is possible to generate motion autonomously.

[0008] In order to improve the accuracy of motion generation in imitation learning, the following imitation learning is described in Patent Document 1. Patent Document 1 states that "imitation learning is performed based on observation information grasped by the operator during the model operation." [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Patent Publication No. 2021-10984 Summary of the Invention [Problem to be solved by the invention]

[0010] One of the challenges of imitation learning is the difficulty of achieving both reduced implementation time (the time required for the teaching and learning phases) and the acquisition of generalized behavior. For example, with program-based teaching, the robot is operated by computer control, which allows for more balanced teaching data than when operated by a human, making it easier to learn the relationships between the teaching data and improving generalized behavior performance. However, since the robot's movements must be programmed one by one, the implementation time is high.

[0011] Teaching methods that require less man-hours to implement than program-based methods include remote control of the robot by a human using a controller or direct touch. These teaching methods require less man-hours to implement because they do not require programming each and every robot movement.

[0012] However, compared to program-based teaching methods, when a robot is operated by a human, variations in the speed and timing of each piece of teaching data occur over time, making it difficult to learn the relationships between pieces of teaching data. Ignoring the effects of these variations over time requires a huge amount of training data, which increases the number of teaching sessions.

[0013] Therefore, an object of the present invention is to provide a processing device, a robot control system, and a machine learning method that can reduce the variation in the time direction of teaching data that hinders motion learning. Note that the technology described in Patent Document 1 is similar in that it manipulates learning data to improve the accuracy of motion generation, but does not mention reducing the variation in the time direction. [Means for solving the problem]

[0014] In order to achieve the above-mentioned object, one example of a processing device of the present invention includes a processor that segments teaching data for a robot into groups of similar movements of the robot, corrects the segmented teaching data to align the speed or timing of similar movements of the robot, combines the corrected teaching data, and performs machine learning using the combined teaching data. [Effects of the Invention]

[0015] According to the present invention, it is possible to reduce the variation in the time direction of teaching data that hinders motion learning. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a schematic diagram showing an example of the configuration of a robot control system to which the present invention is applied; [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer that realizes a motion planning unit of the robot control system. [Figure 3] FIG. 2 is a block diagram showing an example of the functional configuration of a computer included in an operation planning unit of the robot control system. [Figure 4] 2 is a block diagram showing an example of the functional configuration of a consistency verification device provided in an operation planning unit in the first exemplary embodiment of the present invention. FIG. [Figure 5] FIG. 2 is an explanatory diagram of an example of a method for implementing speed adjustment in the first embodiment of the present invention. [Figure 6]2 is a block diagram showing an example of the functional configuration of a machine learning device included in an action planning unit in the first exemplary embodiment of the present invention. FIG. [Figure 7] 5 is a flowchart showing an example of the operation of a consistency verification device provided in an operation planning unit in the first exemplary embodiment of the present invention. [Figure 8A] FIG. 2 is an explanatory diagram of a reaching task, showing an example of the effect of a consistency verification device provided in an operation planning unit in the first embodiment of the present invention. [Figure 8B] FIG. 10 is a diagram showing the relationship between time and the hand position in the Y-axis direction during a teaching operation. [Figure 8C] FIG. 10 is a diagram showing the state of a reaching task when there is variation in the teaching data in the time direction. [Figure 8D] FIG. 10 is a diagram showing the relationship between time and the hand position in the Y-axis direction during a teaching operation with time-dependent variations. [Figure 9] FIG. 8C is an explanatory diagram when a consistency verification device is applied to the reaching task shown in FIGS. 8C and 8D. [Figure 10] FIG. 10 is a block diagram showing an example of the internal configuration of a consistency verification device provided in an operation planning unit in a second exemplary embodiment of the present invention. [Figure 11] 10 is a flowchart showing an example of the operation of a consistency verification device provided in an operation planning unit in the second exemplary embodiment of the present invention. [Figure 12] FIG. 10 is a schematic diagram showing an example of the configuration of a robot control system according to a third embodiment of the present invention. [Figure 13] FIG. 11 is a block diagram showing an example of the functional configuration of a computer included in an operation planning unit of a robot control system according to a third embodiment of the present invention. [Figure 14] FIG. 11 is a block diagram showing an example of the hardware configuration of a computer that realizes an operation planning unit of a robot control system according to a third embodiment of the present invention. [Figure 15] FIG. 11 is a block diagram showing an example of the functional configuration of a screening device provided in an operation planning unit of a robot control system according to a third embodiment of the present invention. [Figure 16]FIG. 11 is a block diagram showing an example of the functional configuration of a screen operation unit included in a robot control system according to a third embodiment of the present invention. [Figure 17] FIG. 10 is a block diagram showing an example of the functional configuration of an operation planning unit included in a robot control system according to a fourth embodiment of the present invention. [Figure 18] FIG. 10 is a block diagram showing an example of the functional configuration of an operation parameter adjustment device provided in an operation planning unit of a robot control system according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, first to fourth embodiments will be described with reference to the drawings. The present embodiments relate to a robot motion generation technology that learns time-series data of a robot and autonomously generates motion. The present embodiments aim to achieve both a reduction in implementation man-hours and the acquisition of generalized motion by reducing the time-dependent variation in teaching data that inhibits motion learning.

[0018] (First embodiment) First, a robot control system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 3. FIG.

[0019] Figure 1 shows an example of the configuration of a robot control system to which the present invention is applied. The robot control system 100 shown in Figure 1 comprises a robot device 1, a sensor data acquisition unit 2, a motion planning unit 3, and a control unit 4. Although a single-arm robot arm is shown as a specific example of the robot device 1, the configuration of the robot device 1 is not critical and may be, for example, a double-arm arm. Furthermore, the robot device may be a moving device such as legs, crawlers, wheels, or propellers.

[0020] The sensor data acquisition unit 2 acquires sensor data output from the robot apparatus 1. The sensor data includes sensor data of the robot apparatus 1 and the environment acquired during work.

[0021] The motion planning unit 3 uses the data obtained by the sensor data acquisition unit 2 as teaching data and learns and generates motions in the following three steps. First, it creates learning data by performing preprocessing to reduce variability in the teaching data. Next, it constructs a motion generation model by performing motion learning based on the learning data. Finally, it generates motion command values ​​using the constructed motion generation model.

[0022] The control unit 4 gives the command values ​​calculated by the motion planning unit 3 to the robot apparatus 1, thereby driving the actuators mounted on the robot apparatus 1.

[0023] [Hardware configuration of the motion planning unit] Next, the hardware configuration of the motion planning unit 3 included in the robot control system 100 will be described with reference to Fig. 2. Here, the hardware configuration of a computer that realizes the motion planning unit 3 will be described.

[0024] 2 is a block diagram showing an example of the hardware configuration of a computer that realizes the action planning unit 3. The illustrated computer 20 is an example of hardware that constitutes a computer used in the sensor data acquiring unit 2, the action planning unit 3, and the control unit 4. The computer 20 can be, for example, a personal computer.

[0025] The computer 20 includes a ROM (Read Only Memory) 22, a CPU (Central Processing Unit) 23, a RAM (Random Access Memory) 24, a non-volatile storage 25, an input / output interface 26, and a network interface 27, all connected to a bus 21.

[0026] The ROM 22 stores program code of software that realizes the functions of the sensor data acquisition unit 2, the motion planning unit 3, and the control unit 4 according to this embodiment.

[0027] The CPU 23 functions as an arithmetic processing device that reads out from the ROM 22 the program code of software that realizes the functions of the sensor data acquisition unit 2, the action planning unit 3, and the control unit 4 according to this embodiment, and loads and executes the program into the RAM 24. Values ​​of variables, parameters, etc. that are generated during the arithmetic processing of the CPU 23 are temporarily written to the RAM 24. The values ​​of variables, parameters, etc. written to the RAM 24 are read out by the CPU 23 as appropriate. Although a CPU is used as the arithmetic processing device, other processors such as an MPU (Micro Processing Unit) may also be used.

[0028] The nonvolatile storage 25 is an example of a recording medium, and can store data used by a program, data obtained by executing a program, etc. For example, the nonvolatile storage 25 stores learning data and a behavior generation model, which will be described later. The nonvolatile storage 25 may also store an OS (Operating System) and a program executed by the CPU 23. The nonvolatile storage 25 can be a magnetic recording medium, an optical recording medium, a semiconductor recording medium, or the like.

[0029] The input / output interface 26 is an interface that communicates signals and data from each sensor and each actuator included in the robot control system 100. The input / output interface 26 may also have the functionality of an A / D (Analog / Digital) converter and / or a D / A converter (not shown) that processes input or output signals. The sensor data in this specification includes information obtained from not only each sensor but also each actuator.

[0030] The network interface 27 may be, for example, a network interface card (NIC) or a modem. The network interface 27 is configured to be able to transmit and receive various types of data to and from external devices via a communication network such as a LAN or the Internet, or a dedicated line, to which a terminal is connected. For example, by using the network interface 27, various types of data can be transmitted and received to and from the robot device 1.

[0031] [Functional configuration of the motion planning unit] Next, the functional configuration of the motion planning unit 3 included in the robot control system 100 will be described with reference to FIG. 3. FIG. 3 shows an example of the functional configuration of the motion planning unit 3 embodying the present invention. In the motion planning unit 3 shown in FIG. 3, the sensor data storage device 31 is a device that records the sensor data acquired by the sensor data acquisition unit 2 as time-series data. The consistency verification device 32 is a device that converts the time-series data recorded in the sensor data storage device 31 and outputs learning data. The learning data storage device 33 is a device that records the learning data output from the consistency verification device 32. The machine learning device 34 is a device that learns the learning data recorded in the learning data storage device 33 and outputs a command value to the control unit 4.

[0032] Next, the configuration of the consistency verification device 32 of the motion planning unit 3 will be described with reference to Fig. 4. In the consistency verification device 32, the motion segmentation unit 41 is a device that divides sensor data into groups of motions. Methods for realizing motion segmentation include segmentation based on speed changes, segmentation by clustering, and segmentation by deep learning.

[0033] Segmentation based on speed change is a method based on the robot's movement speed and can be used when the data characteristics are simple. Segmentation by clustering can be used when the teaching data characteristics are complex, but the number of segments must be set in advance. Segmentation by deep learning can be used when the teaching data characteristics are complex and does not require the number of segments to be set in advance. However, compared to segmentation based on speed change and segmentation by clustering, it has higher implementation and calculation costs.

[0034] The speed adjustment unit 42 adjusts the movement speed to reduce variations in movement speed between teaching data, which hinders the acquisition of generalized movements. As an example of a method for realizing the speed adjustment unit, an algorithm will be described that quantizes the teaching data and removes noise to make the speeds between the teaching data uniform.

[0035] Figure 5 shows the procedure and effect of this algorithm. Graph g5A shows data of different types with different movement speeds, with the horizontal axis representing time and the vertical axis representing the hand position along a certain axis in the world coordinate system (hereafter referred to as the x-axis). Graph g5B shows the results of quantization, which converts the hand position values ​​into discrete approximate values, for the two teaching data shown in graph g5A. As shown in graph g5B, quantization makes the amount of change in the teaching data per time constant.

[0036] Note that if the change in the teaching data per unit time is larger than the quantization width, the change after quantization will not necessarily be a constant value. Therefore, the size of the quantization width must be the same as or larger than the change in the teaching data per unit time. In other words, after speed adjustment, the speed will be the same as or larger than the original teaching data.

[0037] However, by performing upsampling, as described in Modification 2, it is possible to adjust the speed to a slower speed than the original teaching data. Enlarged view 5C shows that quantization in one direction of the hand results in periods of time when the hand position does not change, i.e., periods of time when the speed is zero. Graph g5D shows the results of noise removal from the quantized teaching data shown in graph g5B. Noise removal refers to removing the periods of time when the speed becomes zero after quantization, treating them as noise. As shown in graph g5D, it can be seen that the hand speeds of the two teaching data are consistent. It can also be seen that the final hand position is unchanged from before the speed adjustment. From the above, it can be seen that speed adjustment can be achieved by performing quantization and noise removal.

[0038] The timing adjustment unit 43 shown in Fig. 4 adjusts the timing of the start and end of a movement in order to reduce variations in the timing of the start and end of a movement between pieces of teaching data, which may hinder the acquisition of a generalized movement. One example of a method for implementing timing adjustment is to increase or decrease the rest time of the robot device 1. For example, in order to align the start times of the movements of the robot device 1 in all pieces of teaching data, it is possible to calculate the average value of the start times of the movements of all pieces of teaching data, and increase or decrease the rest time so that the start times of the movements of each piece of teaching data become equal to the average value.

[0039] The dynamics adjustment unit 44 adjusts the dynamics of the acceleration, jerk, torque, etc. in the sensor data to prevent the magnitude or change of the dynamics of the acceleration, jerk, torque, etc. in the teaching data after speed adjustment from becoming excessive, thereby making the generated motion unstable. One example of a method for implementing dynamics adjustment is to apply a moving average filter to the hand position data.

[0040] The data processing unit 45 converts the data output from the dynamics adjustment unit 44 into learning data so that it can be learned by a deep learning model. Specifically, for each piece of teaching data, the learning data is constructed by combining each adjusted segment.

[0041] Next, the configuration of the machine learning device 34 will be described with reference to Fig. 6. In the machine learning device 34, a machine learning model definition unit 61 defines the structure and parameters of the machine learning model.

[0042] The learning unit 62 uses the learning data stored in the learning data storage device 33 to update the weights of the machine learning model defined by the machine learning model definition unit 61, so that when a certain value is input to the model, a desired value is output. For example, when the joint angle of the robot at a certain time is input, a predicted value of the joint angle of the robot at the next time is output.

[0043] The trained weight storage unit 63 stores the weight parameters of trained models.

[0044] The inference unit 64 constructs a behavior generation model by using the machine learning model read from the machine learning model definition unit 61 and the weight parameters of the model read from the learned weight accumulation unit 63, and generates behavior by inputting the same sensor data as during learning into the model.

[0045] For example, suppose that during learning, a model is constructed so that when the joint angles of a robot at a certain time are input, the model outputs a predicted value of the joint angles of the robot at the next time. In this case, when generating motion, the joint angles at each time are input into the model to obtain the joint angles at the next time, and the robot's motion can be generated by using the obtained joint angles as command values.

[0046] [Example of operation of motion planning unit 3] Next, we will explain the order in which the functions of the motion planning unit 3 explained so far operate. The operation order can be broadly divided into three phases: teaching, learning, and execution. In the teaching phase, the robot device 1 is taught an operation by operating it, and the sensor data obtained during this process is recorded in the sensor data storage device 31 using the sensor data acquisition unit 2.

[0047] In the learning phase, the consistency verification device 32 converts the sensor data in the sensor data storage device 31 into learning data, and the data is trained using the machine learning model defined by the machine learning model definition unit 61 to obtain learned weights. The consistency verification device 32 converts the sensor data through three steps: motion segmentation, speed adjustment, and timing adjustment. These three steps will be described with reference to FIG. 7.

[0048] 7 is a flowchart showing an example of the operation of the consistency verification device 32. After preprocessing starts (S1), all teaching data is first segmented into motions (S2). In the motion segmentation, the motion segmentation unit 41 divides the sensor data in the time direction to separate it into groups of motions. After motion segmentation, each segment is subjected to speed adjustment (S5), timing adjustment (S6), and dynamics adjustment (S7). The data processed by the timing adjustment unit 43 is converted into learning data using the data processing unit 45 (S8), and preprocessing ends (S9).

[0049] As described above, by processing the teaching data with the consistency verification device 32, learning data can be obtained in which the variation in the time direction between the teaching data is reduced. Note that it is not necessary to perform both speed adjustment and timing adjustment; in some cases, it is sufficient to perform only one of them. For example, if only the speed of the teaching motion differs, it is only necessary to adjust the speed, and timing adjustment is not necessary. Also, if only the timing of the teaching motion differs, it is only necessary to adjust the timing, and speed adjustment is not necessary.

[0050] The training data constructed by the consistency verification device 32 is input to the training unit 62 of the machine learning device 34. The training unit 62 calls up a machine learning model from the machine learning model definition unit 61 and performs training based on the training data. The behavior generation model obtained as a result of training is stored in the trained weight storage unit 63.

[0051] In the execution phase, the robot performs a task by generating an action using the action generation model obtained in the learning phase. The inference unit 64 of the machine learning device 34 is used to calculate a command value for the robot apparatus 1.

[0052] [Example of the effect of the motion planning section] To verify the effectiveness of the motion planning unit 3, we will explain how the operation of the motion planning unit 3 reduces the variation in the time direction between teaching data, with reference to Figs. 8 (8A to 8D) and 9. Fig. 8 (8A to 8D) shows a reaching task in which the hand of a robot arm is moved linearly to a target hand position. We will use Figs. 8 (8A to 8D) to explain the effect of variation in the time direction in the teaching data on the reaching task.

[0053] First, using Figures 8A and 8B (graph g8B), we will explain why a task can be successful if there is no variation in the teaching data in the time direction. The hand depicted in Figure 8A indicates the position of the robot hand at the initial time. Also, p, q, and r depicted in Figure 8A respectively represent the target hand positions. X and Y in Figure 8A are the names of the coordinate axes set in the task environment, and here the X and Y axes indicate position [cm]. Now, let's assume that a movement for the target hand positions p and r is taught. In Figure 8A, the arrows (solid lines) pointing from the hand to p and r represent the trajectory of the hand during the teaching movement.

[0054] Figure 8B (graph g8B) shows the relationship between time and hand position in the Y-axis direction during teaching. As shown in graph g8B, the teaching motions for target hand positions p and r have the same hand velocity and reaching start and end times, and there is no variation in the time direction. By learning the teaching motions for target hand positions p and r in the motion learning phase, a motion generation model is acquired that inputs current sensor data and outputs command values ​​for the robot arm at the next time.

[0055] Next, referring to Figure 8A and graph g8B, consider the movement generated for the target hand position q in the movement execution phase. In the movement execution phase, the target hand position q is generalized by point-to-point learning in the movement generation model constructed in the movement learning phase, and is positioned exactly in the middle of the target hand positions p and r. At this time, the hand position generated for q passes exactly in the middle of the hand positions of the taught movements for p and r at each time point. Figure 8A and graph g8B show the movement generated for q with a dashed line, and the hand is stable with no wobble in the Y-axis direction. This shows that the reaching task will be successful if there is no variation in the teaching data over time.

[0056] Next, we will use Figures 8C and 8D (graphs g8D) to explain why variability causes task failure. Figure 8C shows the reaching task when there is temporal variability in the instruction data. The symbols in the figure have the same meaning as in Figure 8A.

[0057] Figure 8D (graph g8D) shows the relationship between time and hand position in the Y-axis direction during a teaching operation. There is variation in the time direction between the teaching operations for target hand positions p and r. Specifically, the movement speed is faster for the teaching operation for target hand position r than for target hand position p. The reaching start time (two-dot chain line) and completion time (single-dot chain line) are also earlier.

[0058] Next, with reference to Figure 8C and graph g8D, consider the motion generated for the target hand position q. As mentioned above, based on the positional relationship between p, q, and r, the motion generated for q passes exactly halfway between the taught motions for p and r. In Figure 8C and graph g8D, the motion generated for q is shown by the dashed line. Looking at the motion generated for q, the hand is significantly deviated in the Y-axis direction. This shows that if there is temporal variation in the taught data, the reaching task will fail.

[0059] Next, using Fig. 9, it will be explained that even when there is variation in the time direction in the teaching data, generalized behavior can be acquired by using the consistency verification device 32. Fig. 9 is an explanatory diagram when the consistency verification device 32 is applied to the reaching task shown in Figs. 8C and 8D. The same reference numerals in Fig. 9 indicate the same parts as in Figs. 8C and 8D, and therefore repeated explanations will be omitted.

[0060] Graph g9A is the teaching data for the target hand positions p, r shown in graph g8D. As described above, the teaching data for the target hand positions p, r varies over time.

[0061] Graph g9B shows the segmented movements of the taught movements for the target hand positions p and r shown in graph g9A. The dashed dotted lines in graph g9B represent the segmented times. As shown in graph g9B, the movements are divided into the reaching start time and the reaching end time. For example, as shown in graph g9B, in order to detect the reaching start time, it is advisable to find the time when the hand starts moving from a stationary state by performing segmentation based on speed change in the movement segmentation unit 41.

[0062] Graph g9C is an explanatory diagram for aligning the hand speed for the hand movement shown in graph g9B. As shown in graph g9C, the hand speed of the arm from the reaching start time to the reaching completion time is adjusted to be equal to the average value of the hand speeds in the two teaching data + α (constant). More preferably, the hand speed of the arm from the reaching start time to the reaching completion time is made equal to the average value.

[0063] Graph g9D is an explanatory diagram for aligning the timing of the hand movements shown in graph g9C. As shown in graph g9D, the reaching start time and reaching end time are made equal between the teaching data.

[0064] Graph g9E shows the motion generated for the target hand position q in Fig. 8A in the execution phase. Since the target hand position q is located exactly halfway between the target hand positions p and r, the motion generated for q also passes exactly halfway between the taught motions for p and r. Looking at the motion generated for q, the hand does not wobble in the Y-axis direction and is stable.

[0065] <Variation 1> In this embodiment, the motion segmentation unit 41 divides the sensor data into groups of motions. However, if the data characteristics are complex, segmentation may not work well. To address this issue, it is possible to use Recurrent Dropout. Recurrent Dropout is a method of performing learning while randomly deactivating nodes in a recurrent neural network. Using Recurrent Dropout is expected to improve generalization performance with respect to variations in the training data over time.

[0066] <Variation 2> In this embodiment, the sensor data is quantized by the speed adjustment unit 42. However, if the amount of change per unit time of the sensor data is larger than the quantization width (that is, if the speed after conversion is desired to be increased), the amount of change per unit time of the quantized sensor data exceeds the quantization width, and therefore the amount of change per unit time after quantization is not equal, failing to equalize the speeds.

[0067] To avoid this, it is necessary to reduce the amount of change per unit time of the sensor by performing upsampling before quantization in the speed adjustment unit 42. Upsampling means increasing the sampling frequency of time-series data. Methods for achieving upsampling include linear interpolation, nearest neighbor interpolation, and spline interpolation. However, it is known that these interpolation methods cannot be applied to image data, and a separate technique called frame interpolation is required. Details will be explained in the second embodiment.

[0068] The main features of this embodiment can be summarized as follows.

[0069] The processor (CPU 23) of the processing device (computer 20) segments teaching data for the robot (robot device 1) into groups of similar robot movements (movement segmentation unit 41). The processor (CPU 23) corrects the segmented teaching data to align the speed or timing of similar robot movements (speed adjustment unit 42, timing adjustment unit 43). The processor (CPU 23) combines the corrected teaching data (data processing unit 45). The processor (CPU 23) performs machine learning using the combined teaching data (machine learning device 34).

[0070] This reduces the variation in the time direction of the teaching data, which in turn reduces the implementation effort (the effort required for the teaching and learning phases) and makes it easier to learn the relationships between the teaching data, improving generalization performance.

[0071] The processor (CPU 23) applies correction to the segmented teaching data to smooth the dynamics values ​​of the same type of robot movement (dynamics adjustment unit 44). For example, the dynamics values ​​(acceleration, jerk, torque, etc.) are smoothed by taking a moving average. This makes it possible to stabilize the robot's movement.

[0072] The teaching data includes, for example, at least one of position information indicating the position of the robot, joint angle information indicating the joint angles, speed information indicating the speed of the robot, joint angular velocity information indicating the joint angular velocity of the robot, and tactile information indicating the sensor value of a tactile sensor provided on the robot. In this embodiment, the teaching data includes position information indicating the position of the robot (the position of the gripper). This allows machine learning to be performed using the state of the robot as teaching data.

[0073] The teaching data may also include at least one of position information indicating the position of an object on which the robot is to work and posture information indicating the posture of the object, thereby enabling machine learning to be performed using the state of the object as teaching data.

[0074] The dynamics includes, for example, at least one of acceleration, acceleration change, jerk, jerk change, torque, and torque change, which can smooth the motion of the robot.

[0075] The processor (CPU 23) quantizes the segmented multiple pieces of teaching data and removes noise to make the speed of similar robot movements uniform (Fig. 5). This makes it possible to quickly reduce the variation in the teaching data over time.

[0076] The processor (CPU 23) corrects the segmented teaching data by increasing or decreasing the robot's rest time to align the timing of similar robot movements (Fig. 9). This makes it possible to easily reduce the variation in the teaching data in the time direction.

[0077] The robot control system 100 includes a processing device (computer 20) and a robot (robot device 1). A processor (CPU 23) uses a trained machine learning model to generate command values ​​for the robot's movements (machine learning device 34). The robot moves according to the command values. This allows the robot to be controlled autonomously.

[0078] (Second embodiment) Next, a second embodiment of the present invention will be described with reference to FIGS. 10 and 11, in which the sensor data includes an image.

[0079] [Example of functional configuration] FIG. 10 is a block diagram showing an example of the internal configuration of a consistency verification device provided in a motion planning unit according to a second embodiment of the present invention. In FIG. 10, the same reference numerals as in FIG. 4 denote the same components, and therefore a repeated description will be omitted. As shown in FIG. 10, the second embodiment is the same as the first embodiment in that sensor data is divided by a motion segmentation unit 41, variation in the time direction is reduced by using a speed adjustment unit 42 and / or a timing adjustment unit 43, dynamics is adjusted by using a dynamics adjustment unit 44, and the data is converted into training data by a data processing unit 45. The second embodiment differs from the first embodiment in that the consistency verification device 32 includes a frame interpolation unit 101. The frame interpolation unit 101 increases the frame rate of image time-series data.

[0080] Methods for realizing frame interpolation include frame interpolation based on optical flow and frame interpolation using deep learning. Frame interpolation based on optical flow consists of three steps: feature extraction, movement change calculation, and interpolated image generation. For example, consider two consecutive images from a time series of images. In feature extraction, the features of the two images are extracted. In movement change calculation, the same feature calculated in feature extraction is focused on and its movement change amount is calculated. This movement change of the feature is called optical flow. In interpolated image generation, the image between the two images is estimated by moving pixels in the two original images based on the movement change amount.

[0081] Examples of frame interpolation using deep learning include models such as FLAVR and FILM. FLAVR is a deep learning model that enables highly accurate frame interpolation by performing optical flow and 3D convolution calculations internally. FILM is a deep learning model that takes into account the scale of the image (enlargement and reduction). By inputting time-series image data into FLAVR (or FILM), it is possible to obtain time-series image data after frame interpolation.

[0082] [Example of operation] FIG. 11 is a flowchart showing an example of the operation of the consistency verification device 32 in the second embodiment of the present invention. As shown in FIG. 11, the second embodiment is the same as the first embodiment in that it segments the motion of sensor data (S2), adjusts speed (S6) and timing (S7) for each segment (S3, S4), and synthesizes the adjusted segments for each teaching data (S8, S9) to construct learning data. Note that in FIG. 11, the same reference numerals as in FIG. 7 indicate the same parts, and therefore a repeated description will be omitted. The difference from the first embodiment is that in the second embodiment, frame interpolation (S13) is performed before speed adjustment (S6).

[0083] The main features of this embodiment can be summarized as follows.

[0084] The teaching data includes an image of the robot (robot device 1) or the working environment.

[0085] The processor (CPU 23) performs frame interpolation of images included in the segmented teaching data for the plurality of pieces of teaching data, and then performs correction to align the speeds of the same type of robot movements (frame interpolation unit 101). This makes it possible to easily reduce the variation in the time direction of the teaching data even if the teaching data includes images of the robot (robot device 1) or the working environment.

[0086] (Third embodiment) Next, as a third embodiment of the present invention, a case where only teaching data effective for improving generalization performance is extracted will be described with reference to FIGS.

[0087] FIG. 12 is a block diagram showing an example of the configuration of a robot control system 100 according to a third embodiment of the present invention. In FIG. 12, the same reference numerals as in FIG. 1 indicate the same components, and therefore a repeated description will be omitted. The difference from the first embodiment is that the robot control system 100 according to the third embodiment is that it includes a screen operation unit 5. The screen operation unit 5 can display the processing results of the motion planning unit. Furthermore, it determines various parameters in the motion planning unit 3 based on operation input received from the user.

[0088] 13 is a block diagram showing an example of the configuration of the motion planning unit 3 in the third embodiment of the present invention. In FIG. 13, the same reference numerals as in FIG. 3 indicate the same components, and therefore a repeated description will be omitted. The difference from the first embodiment is that the motion planning unit 3 in the third embodiment is provided with a screening device 131. The screening device 131 extracts only sensor data that is effective for improving generalization performance from the sensor data stored in the sensor data storage device 31. Thereafter, the extracted teaching data is output to the consistency verification device 32 and the screen operation unit 5.

[0089] [Hardware configuration example] Next, the hardware configuration of the motion planning unit 3 included in the robot control system 100 according to the third embodiment of the present invention will be described with reference to Fig. 14. In Fig. 14, the same reference numerals as in Fig. 3 indicate the same components, and therefore a repeated description will be omitted. The difference from the first embodiment is that the third embodiment includes a video output interface 141.

[0090] For example, a VGA (Video Graphics Array), a DVI (Digital Visual interface), an HDMI (High-Definition Multimedia Interface, registered trademark), or a Display Port is used as the video output interface 141. The video output interface 141 is configured to be able to transmit video to a display via a dedicated line or the like.

[0091] [Example of functional configuration] Next, an example of the functional configuration of the screening device 131 of the motion planning unit 3 included in the robot control system 100 according to the third embodiment of the present invention will be described with reference to FIG.

[0092] Fig. 15 shows an example of the functional configuration of a screening device 131 embodying the present invention. In Fig. 15, the same reference numerals as in Fig. 13 denote the same parts, and therefore repeated explanations will be omitted.

[0093] 15, the grouping unit 151 groups the sensor data stored in the sensor data storage device 31 so that data with similar motions are grouped together. As a method for realizing the grouping unit, for example, it is possible to group the sensor data for the same target hand position.

[0094] The representative data calculation unit 152 calculates time series data (referred to as representative data) that represents each group obtained by the grouping unit 151. One possible method for realizing the representative data calculation unit 152 is to use, as the representative data, time series data obtained by calculating the median for each time point for all time series data in the group, for example.

[0095] The outlier data detection unit 153 detects time series data (referred to as outlier data) that are dissimilar to the representative data calculated by the representative data calculation unit 152 for each group obtained by the grouping unit 151. The outlier data detection unit can be implemented using, for example, DTW (Dynamic Time Warping) and IQR (Interquartile Range). DTW is an index of similarity between time series data. Possible values ​​of DTW are 0 or greater, with values ​​closer to 0 indicating greater similarity between the time series data. First, a DTW series is obtained by calculating the DTW between the representative data and all time series data in the group. Next, the IQR of the calculated DTW series is calculated. The IQR is an index representing the degree of dispersion of data and is calculated as (third quartile) - (first quartile). Finally, time series data with a DTW relative to the representative data greater than (third quartile + α × IQR) is considered outlier data. The value of α is initially set to 1.5. In addition, the user can change the value by using the screen operation unit 5.

[0096] The calculation result output unit 154 outputs the sensor data other than the outlier data to the consistency verification device 32. Also, it outputs information about the outlier data to the screen operation unit 5.

[0097] Next, an example of the functional configuration of the screen operation unit 5 included in the robot control system 100 according to the third embodiment of the present invention will be described with reference to FIG.

[0098] Fig. 16 shows an example of the functional configuration of the screen operation unit 5 according to the present invention. In Fig. 16, the same reference numerals as in Fig. 1 indicate the same components, and therefore a repeated explanation will be omitted.

[0099] In the screen operation unit 5 shown in FIG. 16, an operation input unit 161 is configured with input devices such as a mouse and a keyboard, and receives mouse input, keyboard input, and the like from the user.

[0100] The screen display unit 162 is configured by, for example, a display, and visualizes information obtained from the operation input unit 161 and the operation planning unit 3.

[0101] The screen control unit 163 receives information from the operation planning unit 3 and the operation input unit 161. It also outputs information to the operation planning unit 3 and the screen display unit 162.

[0102] The main features of this embodiment can be summarized as follows.

[0103] The processor (CPU 23) performs screening by removing outlier data from the teaching data (screening device 131), thereby further improving the generalization performance.

[0104] (Fourth embodiment) Next, as a fourth embodiment of the present invention, a case where an arbitrary motion speed and / or an arbitrary magnitude of force is realized when generating a motion will be described with reference to FIG.

[0105] Fig. 17 is a block diagram showing an example of the configuration of a motion planning unit in the fourth embodiment of the present invention. In Fig. 17, the same reference numerals as in Fig. 3 indicate the same parts, and therefore repeated explanation will be omitted. The difference from the first embodiment is that in the fourth embodiment, the motion planning unit 3 is provided with a motion parameter adjustment device 171. The motion parameter adjustment device 171 realizes an arbitrary motion speed and / or an arbitrary magnitude of force from the sensor data stored in the sensor data storage device 31 when generating a motion.

[0106] [Hardware configuration of the operating parameter adjustment device] The hardware configuration of the operation planning unit 3 included in the robot control system 100 in the fourth embodiment of the present invention is the same as that in the third embodiment of the present invention, and therefore will not be described again.

[0107] [Example of functional configuration of operation parameter adjustment device] Next, an example of the functional configuration of the operation parameter adjusting device 171 of the operation planning unit 3 included in the robot control system 100 according to the fourth embodiment of the present invention will be described with reference to FIG.

[0108] Fig. 18 shows an example of the functional configuration of an operation parameter adjustment device 171 embodying the present invention. In Fig. 18, the same reference numerals as in Figs. 12 and 13 denote the same parts, and therefore repeated explanations will be omitted.

[0109] 18, an operation parameter storage unit 181 stores parameter values ​​necessary for operation parameter adjustment. The types of parameters include, for example, the control period of the operation generation model, the range of current values, the range of torque sensor values, etc. The user can change the parameter values ​​via the screen operation unit 5.

[0110] The motion parameter adjustment calculation unit 182 adjusts the motion speed and / or the magnitude of force when generating a motion based on the parameter values ​​stored in the motion parameter storage unit 181. A specific method for changing the motion speed in the motion parameter adjustment calculation unit is, for example, changing the control period when generating a motion. For example, if you want to increase the motion speed, you can make the control period when generating a motion shorter than the sampling period of the teaching data. A specific method for changing the magnitude of force in the motion parameter adjustment calculation unit is, for example, changing the current value or torque sensor value. For example, if you want to decrease the magnitude of the force, you can make the upper limit of the current value or torque sensor value smaller than the teaching data.

[0111] The main features of this embodiment can be summarized as follows.

[0112] The processing device (computer 20) includes a storage device (operation parameter storage unit 181) that stores operation parameters indicating parameters for adjusting the operation of the robot (robot device 1) after machine learning. The storage device is configured with, for example, a RAM 24, a non-volatile storage 25, etc.

[0113] The processor (CPU 23) uses the trained machine learning model to generate command values ​​for the movement of the robot (robot device 1) based on the movement parameters (machine learning device 34). This makes it possible to adjust the movement of the robot without having to repeat teaching and learning.

[0114] The operation parameter is, for example, the sampling period of the teaching data. The processor (CPU 23) increases the operation speed of the robot (robot device 1) by making the control period of the robot shorter than the sampling period in the command value, or decreases the operation speed of the robot by making the control period of the robot longer than the sampling period in the command value (operation parameter adjustment device 171). This makes it possible to adjust the operation speed of the robot without repeating teaching and learning.

[0115] Furthermore, the operation parameter is, for example, the maximum value M of the robot torque or a value correlated thereto in the teaching data. The processor (CPU 23) increases the force of the robot (robot device 1) by making the upper limit of the torque of the robot or a value correlated thereto (for example, the value of the drive current of the actuator) greater than the maximum value M in the command value, or decreases the force of the robot by making the upper limit of the torque of the robot or a value correlated thereto less than the maximum value M in the command value. This makes it possible to adjust the force of the robot without having to repeat teaching and learning.

[0116] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0117] Furthermore, the above-described configurations, functions, processing units, etc. may be partially or entirely realized in hardware, for example, by designing them as integrated circuits, etc. As the hardware, a broad processor device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) may be used.

[0118] Furthermore, each component of the operation planning unit 3 according to each of the above-described embodiments may be implemented in the control unit 4. Furthermore, the processing performed by a certain processing unit of the operation planning unit 3 may be realized by a single piece of hardware, or may be realized by distributed processing using multiple pieces of hardware.

[0119] In each of the above-described embodiments, the sensor data acquiring unit 2, the action planning unit 3, and the control unit 4 are realized by, for example, a single computer 20, but they may also be realized by separate computers. By realizing the action planning unit 3, which has a large calculation load, by a high-performance computer, it is possible to improve the overall throughput. The computers are connected to each other via a communication network such as a LAN or the Internet.

[0120] The embodiment of the present invention may have the following aspects.

[0121] (1) A machine learning system that performs machine learning to generate robot movements, comprising: an acquisition unit that acquires teaching data (such as time-series sensor information) for the robot; a movement segmentation unit that segments the teaching data based on the robot's movements; a speed adjustment unit that corrects the teaching data to align the speeds of similar movements among the segmented movements when the speeds of the same movements differ between different teaching data that share a common task; a timing adjustment unit that corrects the teaching data to align the timing of the same movements among the segmented movements when the timings of the same movements differ between different teaching data that share a common task; and a dynamics adjustment unit that corrects the robot's dynamics in the teaching data, and the teaching data with at least one of the speed, timing, and dynamics corrected is used as learning data in the machine learning system.

[0122] (2) In (1), the machine learning system is a machine learning system equipped with an acquisition unit that acquires the robot's position, joint angle information, etc. as teaching data for the robot.

[0123] (3) In (2), the machine learning system is a machine learning system that has an acquisition unit that acquires images of the robot and the working environment in addition to the robot's position and joint angle information.

[0124] (4) In (2), the machine learning system is a machine learning system that has an acquisition unit that acquires the position and posture information of the target object in addition to the robot's position and joint angle information.

[0125] (5) In (1), the machine learning system is a machine learning system having an acquisition unit that acquires the robot's speed, joint angular velocity information, etc. as teaching data for the robot.

[0126] (6). In (1), the machine learning system is a machine learning system having an acquisition unit that acquires tactile information of the robot as teaching data for the robot.

[0127] (7). In (1), the machine learning system is a machine learning system having a dynamics adjustment unit that adjusts acceleration and / or acceleration changes among the dynamics of the robot.

[0128] (8). In (1), the machine learning system is a machine learning system having a dynamics adjustment unit that adjusts the jerk and / or jerk change of the robot's dynamics.

[0129] (9). In (1), the machine learning system is a machine learning system having a dynamics adjustment unit that adjusts torque and / or torque changes among the dynamics of the robot.

[0130] (10) In (1), the machine learning system is characterized in that, when the teaching data for the robot includes camera images, the speed adjustment unit performs frame interpolation between the teaching data and then corrects the teaching data so as to align the speeds of the same type of movements.

[0131] (11). In (1), the machine learning system is equipped with a screening device that extracts teaching data that is effective in improving behavior generalization performance from the acquired teaching data.

[0132] (12) In (1), the machine learning system is a machine learning system equipped with a motion parameter adjustment device that adjusts the control period of the robot to achieve an arbitrary motion speed when generating motion using a trained machine learning model.

[0133] (13) In (1), the machine learning system is equipped with a motion parameter adjustment device that adjusts the torque of the robot to achieve an arbitrary force magnitude when generating motion using a trained machine learning model.

[0134] According to (1) to (13), by reducing the time-dependent variation between teaching data that hinders motion learning, it is expected that both the reduction of implementation man-hours and the acquisition of generalization ability can be achieved. [Explanation of symbols]

[0135] 1...Robot device, 2...Sensor data acquisition unit, 3...Movement planning unit, 4...Control unit, 5...Screen operation unit, 22...ROM, 23...CPU, 24...RAM, 25...Non-volatile storage, 26...Input / output interface, 27...Network interface, 31...Sensor data storage device, 32...Integrity verification device, 33...Learning data storage device, 34...Machine learning device, 41...Movement segmentation unit, 42...Speed ​​adjustment unit, 43...Timing adjustment unit, 44...Dynamics adjustment unit, 45...Data Processing unit, 61...machine learning model definition unit, 62...learning unit, 63...learned weight accumulation unit, 64...inference unit, 101...frame interpolation unit, 131...screening device, 161...operation input unit, 151...grouping unit, 152...representative data calculation unit, 153...outlier data detection unit, 154...calculation result output unit, 161...operation input unit, 162...screen display unit, 163...screen control unit, 171...operation parameter adjustment device, 181...operation parameter storage unit, 182...operation parameter adjustment calculation unit

Claims

1. Segmenting the teaching data for the robot into groups of similar movements of the robot; correcting the segmented teaching data to align the speed or timing of the same type of motion of the robot; Combining the corrected teaching data; A processing device comprising a processor that performs machine learning using the synthesized teaching data.

2. 2. The processing device according to claim 1, The processor: A correction is performed on the segmented teaching data to smooth the dynamics values ​​of the same type of motion of the robot. A processing device characterized by:

3. 3. The processing device according to claim 2, The teaching data is position information indicating the position of the robot, joint angle information indicating joint angles, Velocity information indicating the velocity of the robot, joint angular velocity information indicating the joint angular velocity of the robot, and tactile information indicating a sensor value of a tactile sensor provided on the robot A processing device characterized by:

4. 4. The processing device according to claim 3, The teaching data is an image of the robot or the working environment; A processing device characterized by:

5. 4. The processing device according to claim 3, The teaching data is The robot includes at least one of position information indicating the position of an object on which the robot is to perform work and posture information indicating the posture of the object. A processing device characterized by:

6. 3. The processing device according to claim 2, The dynamics are Acceleration, change in acceleration, Jerk, jerk change, torque and torque change A processing device characterized by:

7. 5. The processing device according to claim 4, The processor: For the segmented plurality of pieces of teaching data, frame interpolation of images included in the teaching data is performed, and then correction is performed to align the speeds of the same type of movements of the robot. A processing device characterized by:

8. 2. The processing device according to claim 1, The processor: Screening is performed by removing data that deviates from the teaching data. A processing device characterized by:

9. 2. The processing device according to claim 1, a storage device that stores operation parameters indicating parameters for adjusting the operation of the robot after machine learning; The processor: Using a trained machine learning model, a command value for the robot's operation is generated based on the operation parameters. A processing device characterized by:

10. 10. The processing device according to claim 9, the operation parameter is a sampling period of the teaching data, The processor: In the command value, the control period of the robot is set to be shorter than the sampling period to increase the speed of the robot's movement; or In the command value, the control period of the robot is made longer than the sampling period to reduce the speed of the robot's movement. A processing device characterized by:

11. 10. The processing device according to claim 9, the operation parameter is a maximum value of a torque of the robot or a value correlated therewith in the teaching data, The processor: In the command value, the upper limit of the torque of the robot or a value correlated thereto is set to be greater than the maximum value, thereby increasing the force of the robot; or In the command value, the upper limit of the torque of the robot or a value correlated thereto is set smaller than the maximum value, thereby reducing the force of the robot. A processing device characterized by:

12. 2. The processing device according to claim 1, The processor: The segmented teaching data is quantized and noise is removed to make the speeds of the same type of movements of the robot uniform. A processing device characterized by:

13. 2. The processing device according to claim 1, The processor: Correction is performed for the plurality of segmented teaching data by increasing or decreasing the rest time of the robot to align the timing of the same type of movement of the robot. A processing device characterized by:

14. A robot control system including the processing device according to claim 1 and a robot, The processor generates a command value for the robot's operation using a trained machine learning model; The robot operates according to the command value. A robot control system comprising:

15. A step of segmenting teaching data for a robot into groups of similar operations of the robot; a step of correcting the segmented teaching data to align the speed or timing of the same type of motion of the robot; a step of synthesizing the corrected teaching data; performing machine learning using the synthesized training data; Machine learning methods, including

Images