Robot control system and picking method

Abstract

To provide a robot control system, which controls a robot arm accurately while reducing control loads in a picking robot.SOLUTION: A robot control system inputs data which include deviation-information showing amounts of deviations between a position (a first position) of a picking mechanism at the time of off-line control of a robot arm, which is based on an off-line control command value generated by off-line teaching and a position of the picking mechanism at the time of on-line control of the robot arm, the off-line control command value and information on characteristics of the robot arm, and executes deep reinforcement machine learning so that the amounts of deviations become minimum, so as to generate a learning control command value at the time of on-line control. The robot control system inputs, as portion of input data, updated deviation-information which is information showing amounts of deviations between the first position and a position (a third position) of the picking mechanism at the time of on-line control based on the learning control command value, and executes the deep reinforcement machine learning so that the amounts of deviations between the first position and the third position become minimum, so as to update the learning control command value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a robot control system and a picking method. [Background technology]

[0002] Patent Document 1 discloses a technology in which a robot hand for grasping an object is attached to a robot arm, and a control value is generated by offline teaching using a simulation for an arm robot that performs picking. In the technology described in Patent Document 1, the robot arm is actually controlled online based on the generated control value, and correction control is performed based on the actual position of the robot arm detected by a sensor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6905147 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 requires correction control based on the actual position of the robot arm detected by the sensor each time, which increases the control load. Therefore, in a picking robot, it is desirable to develop a technology that enables precise control of the robot arm while reducing the control load. [Means for solving the problem]

[0005] A robot control system according to the present disclosure includes, for a picking robot that picks up an item using a robot arm having a picking mechanism, an acquisition unit that acquires deviation information indicating an amount of deviation between a first position, which is a position of the picking mechanism when the robot arm is controlled offline based on an offline control command value, which is a control command value for controlling the robot arm generated by offline teaching through simulation, and a second position, which is a position of the picking mechanism when the robot arm is actually controlled online based on the offline control command value; and an acquisition unit that acquires deviation information indicating an amount of deviation between the first position and the second position, which is a position of the picking mechanism when the robot arm is actually controlled online based on the offline control command value, and an acquisition unit that acquires deviation information indicating an amount of deviation between the offline control command value, the deviation information, and an operating characteristic of the robot arm. and a generation unit that uses as input data data including characteristic information indicating the first position and the second position, and executes deep reinforced machine learning so as to minimize the amount of deviation between the first position and the second position, thereby generating a learning control command value that is a control command value for actually controlling the robot arm online, wherein the acquisition unit acquires updated deviation information that is information indicating the amount of deviation between the first position and a third position that is the position of the picking mechanism when the robot arm is actually controlled online based on the learning control command value, and the generation unit inputs the updated deviation information as part of the input data, and executes the deep reinforced machine learning so as to minimize the amount of deviation between the first position and the third position, thereby updating the learning control command value.

[0006] The picking method according to the present disclosure is a picking method in which the robot control system controls the picking robot to pick up the item. Effect of the Invention

[0007] According to the present disclosure, in a picking robot, the robot arm can be accurately controlled while reducing the control load. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of a configuration of a robot control system according to an embodiment. [Diagram 2] FIG. 2 is a diagram for explaining an overview of processing in the robot control system of FIG. [Diagram 3] 3 is a diagram for explaining a positional deviation of the picking mechanism between the offline picking robot and the actual picking robot in FIG. 2. FIG. [Figure 4] 2 is a flow diagram for explaining a processing example in the robot control system of FIG. 1. [Diagram 5] 1. FIG. 4 is a flowchart illustrating another example of the process in the robot control system of FIG. [Figure 6] 1. FIG. 4 is a flowchart illustrating still another example of processing in the robot control system of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Also, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems.

[0010] (Embodiment) First, an example of a robot control system according to the present embodiment (hereinafter, referred to as the present system) will be described with reference to Figs. 1 to 3. Fig. 1 is a block diagram showing an example of the configuration of the present system. Fig. 2 is a diagram for explaining an outline of the processing in the present system shown in Fig. 1. Fig. 3 is a diagram for explaining the positional deviation of the picking mechanism between the offline picking robot and the actual picking robot in Fig. 2.

[0011] As shown in FIG. 1, the system 1 includes, for example, a picking robot 10 that picks up an item with a robot arm, and an information processing device 20. Hereinafter, the picking robot will be simply referred to as a robot, the robot arm will be simply referred to as an arm, and the item will be referred to as a work. The robot 10 includes, for example, a control unit 11, a drive control unit 12, a drive unit 13, a sensor group 14, an input unit 15, and a storage unit 16. As illustrated in FIG. 2, the robot 10 is a robot that picks up a work W placed on a table M or the ground with an arm 150. The arm 150 includes a picking mechanism 151 such as a gripping mechanism, and includes a plurality of joint mechanisms that respectively constitute one or a plurality of joint parts. Each joint mechanism is movable by rotation, extension, or the like. Of course, the gripping mechanism can also include a joint mechanism. The robot 10 can pick up the work W with the picking mechanism 151 and place the work W at any location.

[0012] 2 shows an example of the appearance of the robot 10 in which the arm 150 rotates on six axes, but the shape, number, position, shape, and type of the joint mechanisms, and the shape and type of the picking mechanism 151 are not limited. The picking mechanism 151 may be a gripping mechanism that grips the workpiece W, as exemplified below. However, if the workpiece is made of metal, the picking mechanism 151 may be a mechanism that attracts the workpiece by a permanent magnet or an electromagnetic force, and in that case, the surface of the picking mechanism 151 that attracts the workpiece may be a flat surface, etc.

[0013] The control unit 11 controls the drive control unit 12, the sensor group 14, the input unit 15, and the storage unit 16. The control unit 11 can be realized by a computer configured with, for example, a processor such as a CPU (Central Processing Unit), a working memory, and a non-volatile storage device. The control program executed by the processor is stored in this storage device, and the processor reads the control program into the working memory and executes it, thereby fulfilling the functions of the control unit 11. The storage device can also be the storage unit 16. Of course, the control unit 11 may be configured as a dedicated control circuit.

[0014] 1, an example will be described in which the control unit 11, the input unit 15, and the storage unit 16 are built into the main body of the robot 10. However, the control unit 11, the input unit 15, and the storage unit 16 may be provided in a controller such as a computer disposed at a position separated from the main body of the robot 10. In that case, if the main body of the robot 10 and the controller are capable of communicating with each other via wire or wirelessly, the robot 10 can be controlled from the controller.

[0015] The drive control unit 12 controls the drive unit 13 according to the control from the control unit 11. The drive unit 13 can be composed of an actuator such as a motor that drives the joint mechanism to rotate a plurality of parts connected via the joint mechanism relative to each other, expand and contract the parts, etc. In Figs. 2 and 3, an example is given in which the robot can rotate about the rotation axis of six joint mechanisms as shown by the arrows in the direction of rotation, and the picking mechanism 151 can grip the workpiece W with the gripping units 152 and 153. One or more drive units 13 are provided for each joint mechanism, but for the sake of simplicity of explanation, an example is given in which one drive unit 13 is provided for each joint mechanism. Note that, when the robot 10 is provided with a mechanism other than the joint mechanism, the drive unit 13 is also provided for driving the mechanism.

[0016] The sensor group 14 is a plurality of sensors provided in each drive unit 13 or each joint mechanism. The sensor group 14 includes, for example, a sensor 14a such as a camera provided in the picking mechanism 151 or in its vicinity. The sensor 14a is disposed to recognize the workpiece W.

[0017] The input unit 15 is a part that inputs information from the information processing device 20 or the like. The storage unit 16 can be a storage device that can read and write information from the control unit 11, and can store a learning model 17 (described later) and the like. The control unit 11 may include a program for executing learning using the learning model 17 as a part of the above control program.

[0018] The information processing device 20 includes, for example, an input unit 22, an output unit 23, and a storage unit 24, and a control unit 21 that controls them. The control unit 21 can be realized by a computer that includes, for example, a processor, a working memory, and a non-volatile storage device. The control program executed by the processor is stored in this storage device, and the processor reads the control program into the working memory and executes it, thereby fulfilling the function of the control unit 21. The storage device can also utilize the storage unit 24.

[0019] The input unit 22 inputs information required for a simulation by a simulation program 25 described later from an external device, and stores the information in the memory unit 24 in a state that can be read out during the simulation. Hereinafter, the simulation program 25 will be simply referred to as the program 25. The output unit 23 outputs result information indicating the result of the simulation to the robot 10 via the network. Of course, the result information may be output to a portable recording medium and input to the robot 10 via the portable recording medium. The memory unit 24 can store the program 25 and the like. The control unit 21 may include a program for executing a simulation by the program 25 as part of the above control program.

[0020] In the present system 1, deep reinforcement learning is used to suppress positional deviation of the picking mechanism 151 in the actual robot 10 after offline teaching. A specific description will be given with reference to FIG. 2 and FIG. 3. Hereinafter, models corresponding to the robot 10, the arm 150, and the picking mechanism 151 in the program 25 will be described as the robot 25a, the arm 250, and the picking mechanism 251, respectively. Similarly, models corresponding to the gripping units 152 and 153, the sensor 14a, the base M, and the workpiece W in the program 25 will be described as the gripping units 252 and 253, the sensor 254, the base Ms, and the workpiece Ws, respectively. The robot 25a is a model on the program 25 before the robot 10 is manufactured, but may be a model after the robot 10 is manufactured. Note that in FIG. 2, the learning model 17 and the control unit 11 are drawn outside the robot 10 for convenience, but in reality, they are as shown in FIG. 1.

[0021] First, the control unit 21 of the information processing device 20 executes a simulation by offline teaching for the robot 25a by executing the program 25, and generates an offline control command value that is a control command value for controlling the arm 250. The offline control command value is generated as an optimal control command value. It can be said that the control unit 21 is equipped with a generating unit that generates an offline control command value. The offline control command value may include a control command value for driving a plurality of driving units 13 for driving each joint mechanism. The offline control command value may actually refer to an offline control command value for specifying a target position where a certain workpiece Ws exists and controlling the workpiece Ws to move toward the target position. The control unit 21 may generate a plurality of offline control command values ​​for each of a plurality of possible target positions, that is, for each of a plurality of workpieces Ws with different installation positions.

[0022] The control unit 21 also outputs the first position by the program 25. The first position is the position of the picking mechanism 251 when the arm 250 is controlled offline based on the offline control command value, and is, for example, the coordinates indicated by the midpoint H2 of the gripping units 252 and 253 in FIG. 3. The position of the picking mechanism 251 may be, for example, the picking position when the arm 250 is controlled to pick the workpiece Ws. More specifically, the position of the picking mechanism 251 may be the position when the distance or positional relationship with the workpiece Ws becomes a predetermined distance or positional relationship just before gripping is started, or may be the position where the workpiece Ws is gripped. The control unit 21 instructs the output unit 23 to output the offline control value and the first position, which are result information, to the robot 10.

[0023] An example of calculation of result information by the program 25 will be given. By reading the program 25, the control unit 21 configures, for example, the following object recognition processing unit 211, gripping point search unit 212, and path generation unit 213. In other words, the program 25 can include a program for causing the control unit 21 to realize the functions of each unit 211 to 213. The object recognition processing unit 211 performs processing for the recognition of the workpiece Ws by the sensor 254, and when the workpiece Ws is recognized, passes the position of the workpiece Ws to the gripping point search unit 212. The gripping point search unit 212 searches for a gripping point, which is a gripping position of the workpiece Ws, based on the received position of the workpiece Ws and the current position of the picking mechanism 251, and passes the searched gripping point to the path generation unit 213. The gripping position of the workpiece Ws may be defined as, for example, the center of the position where the workpiece Ws abuts against the picking mechanism 251 when gripped. The path generating unit 213 generates a movement path of the picking mechanism 251 from the gripping point and passes it to the gripping point searching unit 212. The gripping point searching unit 212 and the path generating unit 213 repeatedly generate movement paths to the searched gripping points while exchanging information as described above, and determine optimal gripping points and movement paths. The optimal gripping points and movement paths are determined as gripping points and movement paths that can grip the workpiece Ws at the fastest speed, for example.

[0024] The path generating unit 213 generates offline control command values ​​indicating the drive angle, speed, acceleration, and torque for a plurality of drive units corresponding to each joint mechanism of the arm 250 in order to move the picking mechanism 251 to the gripping point determined along the determined moving path. The drive angle for these drive units indicates the angle of the joint, and the drive speed, acceleration, and torque indicate the speed, acceleration, and torque that can move the joint, respectively. The same applies to the drive angle of the drive unit 13 described later. The offline control command value may be a time-series control command value from the current position to the gripping point. The path generating unit 213 instructs the output unit 23 to output the determined optimal gripping point and optimal offline control command value to the robot 10 as the first position and offline control value, respectively, for the position of each workpiece Ws, that is, each target position. The path generation unit 213 may instruct the output unit 214 to include in the result information the first position and the offline control value, as well as the target position from which they were calculated, and the current position of the picking mechanism 251, which is the position before the picking mechanism 251 was driven.

[0025] A configuration example of the control unit 11 will be described. The control unit 11 includes, for example, an object recognition processing unit 111, a gripping point search unit 112, and a path generation unit 113 that perform processing corresponding to the object recognition processing unit 211, the gripping point search unit 212, and the path generation unit 213, respectively. The functions of each unit 111 to 113 can be realized by, for example, including a program that executes the processing of each unit 111 to 113 in the above control program executed by the processor.

[0026] The object recognition processing unit 111 recognizes the workpiece W by the sensor 14a. The gripping point search unit 112 searches for a gripping point, which is a gripping position on the workpiece W, based on the recognized position of the workpiece W and the current position of the picking mechanism 151. The path generation unit 113 generates a movement path of the picking mechanism 151 from the searched gripping point. The gripping point search unit 112 and the path generation unit 113 repeatedly generate a movement path to the searched gripping point while exchanging information, and determine an optimal gripping point and movement path as, for example, a gripping point and movement path that can grip the workpiece W at the fastest. The path generation unit 113 generates control command values ​​indicating the drive angle, speed, acceleration, and torque for the drive unit 13 corresponding to each joint mechanism of the arm 150 in order to move the picking mechanism 151 to the gripping point determined on the determined movement path. The generated control command value, which is an online control command value, may be a time-series control command value from the current position to the gripping point. The path generating unit 113 sends an instruction to the drive control unit 12 to control the drive unit 13 of the arm 150 based on the online control command value generated for the workpiece W.

[0027] The control unit 11 of the robot 10 has such a configuration and also acquires information indicating the amount of deviation between the first position and the second position. In other words, it can be said that the robot 10 has an acquisition unit that acquires deviation information. The deviation information may be acquired as information indicating the result of calculating the amount of deviation indicating the distance or the distance and direction of the deviation between the first position and the second position, but may also be acquired as information indicating the first position and information indicating the second position.

[0028] The first position is received by the input unit 15 as a part of the result information. The control unit 11 may temporarily store the result information received by the input unit 15 from the information processing device 20 in the storage unit 16. The second position is the position of the picking mechanism 151 when the control unit 11 actually controls the arm 150 online based on the offline control command value received by the input unit 15 from the information processing device 20. The second position may be obtained as a sensor value detected by the sensor group 14. The second position is the position of the picking mechanism 151 of the robot 10 corresponding to the picking mechanism 251 of the robot 25a, and is, for example, the coordinate indicated by the midpoint H1 of the grippers 152 and 153 in FIG. 3. However, the midpoint H1 may deviate from the midpoint H2. However, in this embodiment, this deviation can be reduced by deep reinforcement machine learning, which will be described later.

[0029] An example of control based on an offline control command value and an example of detection of the second position will be described. The object recognition processing unit 111 performs processing for the recognition of the workpiece W by the sensor 14a. The gripping point search unit 112 searches for a gripping point, which is a gripping position on the workpiece W, based on the recognized position of the workpiece W and the current position of the picking mechanism 151, and passes the searched gripping point and the current position of the picking mechanism 151 to the path generation unit 113. The path generation unit 113 determines an optimal gripping point and a moving path while exchanging information with the gripping point search unit 112, and searches for an offline control command value corresponding to the gripping point and the current position of the picking mechanism 151 from among the offline control command values ​​received from the information processing device 20 and stored in the storage unit 16. The path generation unit 113 sends an instruction to the drive control unit 12 to control the drive unit 13 of the arm 150 based on the offline control command value obtained by this search. The second position is detected by the sensor group 14 as the position of the picking mechanism 151 after the drive unit 13 is driven according to this instruction. Similarly, for the positions of other target works W, i.e., other target positions, the drive unit 13 is controlled by the offline control command value selected based on the current position of the picking mechanism 151, and the second positions are detected by the sensor group 14. In this manner, the control unit 11 obtains the second positions for the multiple target positions from the sensor group 14.

[0030] Next, the control unit 11 inputs data including the offline control command value, the deviation information, and characteristic information (described later) acquired as described above, as input data to the learning model 17. The control unit 11 executes deep reinforcement machine learning using such input data in the learning model 17 so as to minimize the deviation amount between the first position and the second position.

[0031] The control unit 11 generates a control command value that actually controls the arm 150 online by this deep reinforcement machine learning. The control command value generated here is called a learning control command value. The learning model 17 can be a model that performs reinforcement learning using DQN (Deep Q-Network), but as long as it is a deep reinforcement learning model, its algorithm and number of layers are not important. The learning control command value may include a control command value that drives a plurality of drive units 13 for driving each joint mechanism. It can be said that the control unit 11 is equipped with a generation unit that generates such learning control command values ​​using the learning model 17. The learning control command value may also be generated for a plurality of target positions in the same way as the offline control command value. In other words, the control unit 11 may generate a plurality of learning control command values ​​for each of a plurality of workpieces W that are installed at different positions, for example.

[0032] The characteristic information input to the learning model 17 as a part of the input data is information indicating the operating characteristics of the arm 150. The characteristic information can be, for example, information indicating the motor characteristics of a motor provided as the drive unit 13 on a rotating shaft or the like, backlash of a gear included in the joint mechanism or drive unit 13, the rigidity of the arm 150, etc. If the robot 10 is provided with an actuator other than a motor, it is advisable to input information indicating the actuator characteristics instead of the motor characteristics as the characteristic information. The rigidity of the arm 150 may be, for example, the rigidity of the entire arm 150, or the rigidity of each arm member constituting the arm 150.

[0033] Furthermore, it is preferable that at least a part of the characteristic information indicates the current operation characteristics. Specifically, it is preferable that at least a part of the characteristic information is a sensor value detected by one or more sensors provided in the arm 150 for detecting the operation characteristics when the arm 150 is actually controlled online. The one or more sensors are included in the sensor group 14. In this way, by including the sensor value resulting from detection of the characteristic information by the sensor group 14 in the input data, it is possible to generate a learning control command value that reflects up to the current characteristics of the arm 150 while also taking into account aging and the like of the robot 10.

[0034] Furthermore, the input data may include sensor values ​​detected by one or more sensors provided in the arm 150 for detecting an operating state when the arm 150 is actually controlled online. The one or more sensors are included in the sensor group 14. The sensor group 14 detects, for example, information indicating the angle, speed, acceleration, and torque of the joint mechanism as the operating state. In this way, by including the sensor values ​​resulting from detection of the operating state by the sensor group 14 in the input data, it is possible to generate a learning control command value that reflects even the operating state.

[0035] The control unit 11 also acquires updated deviation information, which is information indicating the amount of deviation between the first position and the third position. The third position is the position of the picking mechanism 151 when the arm 150 is actually controlled online based on the learning control command value. It can be said that the control unit 11 includes an acquisition unit that acquires such updated deviation information in order to input it to the learning model 17. The updated deviation information may be acquired as information indicating the result of calculating the amount of deviation indicating the distance or the distance and direction of the deviation between the first position and the third position, but since the first information has already been input to the learning model 17, the updated deviation information may be acquired as information indicating the third position.

[0036] The detection process of the third position is basically performed in the same procedure as the detection process of the second position, except that the third position is detected as the position of the picking mechanism 151 by the sensor group 14 after the drive unit 13 is controlled based on the learning control command value instead of the offline control command value.

[0037] Furthermore, the control unit 11 further inputs the update deviation information as part of the input data, and executes deep reinforcement machine learning so as to minimize the deviation amount between the first position and the third position, thereby updating the learning control command value. In this re-learning, the learning control command value obtained during learning may also be input as part of the input data. It can be said that the control unit 11 includes a generation unit that generates such an update value of the learning control command value by using the learning model 17.

[0038] Next, a robot control method according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flow diagram for explaining a processing example in the present system shown in Fig. 1. This robot control method includes, for example, a first generating step, a first acquiring step, a second generating step, a second acquiring step, and an updating step as follows. The first generating step is a step executed by the information processing device 20, and the other steps are steps executed by the robot 10.

[0039] The first generation step is a step of generating an offline control command value for controlling the arm 250 of the robot 25a by offline teaching through simulation. Specifically, the control unit 21 of the information processing device 20 performs offline teaching through simulation on the robot 25a equipped with the arm 250 (step S1). Through this teaching, the control unit 21 generates offline control command values ​​indicating the angle, speed, acceleration, and torque for controlling the driving unit of the arm 250, transmits them to the robot 10, and the robot 10 acquires the offline control command value (step S2). For example, the robot 10 acquires an offline control command value Ia for a certain driving unit of the arm 250 to be input to the learning model 17, and similarly acquires offline control command values ​​for the other driving units.

[0040] In the first acquisition step, deviation information indicating the deviation between the first position and the second position is acquired. Specifically, the control unit 11 acquires deviation information indicating the deviation between when the actual device itself is operated with the offline control command value and when it is operated offline (step S3). For example, the control unit 11 acquires a first position Ib for a certain drive unit of the arm 250 for input to the learning model 17, and similarly acquires first positions for the other drive units. Furthermore, the control unit 11 acquires a second position when the drive unit 13 corresponding to a certain drive unit of the arm 250 is controlled with the offline control command value by the sensor group 14, and similarly acquires second positions for the other drive units 13.

[0041] Furthermore, in step S3, the control unit 11 detects the angle, speed, acceleration, and torque of each drive unit 13 corresponding to each joint mechanism when the drive unit 13 is controlled by the offline control command value using the sensor group 14, and acquires joint information that is the detection result.

[0042] In the second generation step, data including an offline control command value, deviation information, and characteristic information is used as input data, and deep reinforcement machine learning is performed so that the deviation between the first position and the second position is minimized. Through this learning, a learned control command value that actually controls the arm 150 online is generated. Specifically, the control unit 11 acquires characteristic information indicating the motor characteristics of the motors of each rotating shaft, the backlash of the gears, and the rigidity of each arm member of the arm 150 by the sensor group 14 (step S4). The characteristic information and the joint information may be acquired simultaneously. Next, the control unit 11 inputs all the acquired data into the learning model 17, which is a deep reinforcement learning model, and executes machine learning (step S5). Through this machine learning, the deviation amount and its cause are learned, and a learned control command value that minimizes the deviation amount is generated.

[0043] In the second acquisition step, updated deviation information indicating the deviation between the first position and the third position is acquired. Specifically, the control unit 11 controls the robot 10 including the arm 150 based on the learning control command value generated in step S5 (step S6), and acquires updated deviation information indicating the deviation from the offline position and each joint information when the robot 10 operates under the control (step S7). For example, the control unit 11 acquires the third position Id as the updated deviation information for input to the learning model 17. The control unit 11 may further hold the learning control command value Ic for the driving unit 13 of the arm 150 in the storage unit 16 for input to the learning model 17, and may also hold learning control command values ​​for the other driving units 13. Each joint information is acquired in the same manner as in step S3.

[0044] In the update step, the updated deviation information is input as a part of the input data, and the learning control command value is updated by executing deep reinforcement machine learning so that the deviation amount between the first position and the third position is minimized. Specifically, the control unit 11 inputs all data including the updated deviation information and each joint information acquired in step S7 to the learning model 17 and executes re-learning (step S8). All data to be re-learned may include data already input in step S5. By this machine learning, the deviation amount and its cause are re-learned, and an updated learning control command value Oa that minimizes the updated deviation amount is generated. Then, the control unit 11 uses the updated learning control command value Oa generated in step S8 corresponding to the optimal gripping point for the target workpiece W and the current position of the picking mechanism 151 as an online control command value to control and operate the robot 10 including the arm 150 (step S9), and ends the process.

[0045] As described above, according to this embodiment, in the robot 10, it is possible to suppress the difference between the results of offline teaching and the results of the actual machine, i.e., it is possible to accurately control the arm 150, thereby reducing the adjustment work by a person at the installation position of the robot 10. Furthermore, according to this embodiment, the drive control of the drive unit 13 is based on data using the updated learning control command value, so there is no need to perform dynamic correction control for the amount of difference as in rule-based drive control, and therefore the control load of the robot 10 can be reduced.

[0046] Furthermore, by controlling the robot 10 using the robot control method described above and adopting a picking method for picking the workpiece W, i.e., by picking the workpiece W using the robot 10 described above, the workpiece W can be picked accurately even if there is deterioration due to aging, etc.

[0047] As a comparative example, it is assumed that only online teaching is used without using offline teaching. However, in this comparative example, teaching can be performed only after an actual machine such as the robot 10 is actually completed and the usage environment of the actual machine, including the target workpiece W and the positional relationship with each workpiece W, is set up. Therefore, in this comparative example, online teaching is required every time the usage environment changes, which increases the lead time and labor, resulting in high costs. In addition, in the technology of the comparative example, online teaching is required every time the state of the robot's hardware changes due to aging or other reasons, for example, every time characteristic information such as the motor characteristics of each axis, the backlash of the gears, and the rigidity of the arm changes. In contrast, in this embodiment, it is possible to solve the problems of these comparative examples.

[0048] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. For example, when the influence of the backlash of the actuators and gears of each rotating shaft is small and the rigidity of each arm member constituting the arm 150 is the main cause of the deviation, the following processing may be performed. That is, the characteristic information input to the learning model 17 may include, for example, information indicating the rigidity of the arm 150, and may not include information indicating the motor characteristics or the backlash of the gears. In this case, the rigidity of each arm member is learned as the cause of the deviation amount by machine learning, and an updated learning control command value taking into account the rigidity of each arm member is generated so that the deviation amount is minimized. On the other hand, when each arm member constituting the arm 150 has high rigidity but the motor and gears are inexpensive and the variation in the motor characteristics and the backlash of the gears is large, the following processing may be performed. That is, the characteristic information input to the learning model 17 may include, for example, information indicating the motor characteristics and the backlash of the gears, and may not include information indicating the rigidity of the arm 150. In this case, machine learning will learn motor characteristics and gear backlash as the causes of the deviation, and an updated learning control command value will be generated that takes into account the motor attributes and gear backlash so as to minimize the deviation.

[0049] Re-learning is not limited to one time. Both of FIG. 5 and FIG. 6 are flow charts for explaining other processing examples in the present system 1 shown in FIG. 1. As shown in FIG. 5, the control unit 11 judges whether the arm 150 has been operated for a predetermined period or a predetermined number of times after the processing of step S9 (step S10), and when the result is YES, the process returns to step S4, and re-learning is performed based on the current characteristic information, etc. This allows the learning control command value to be updated every time the operation is performed for a predetermined period or a predetermined number of times. Also, as shown in FIG. 6, the control unit 11 judges whether a picking error has occurred after the processing of step S9 (step S11), and when the result is YES, the process returns to step S4, and re-learning is performed based on the current characteristic information, etc. This allows the learning control command value to be updated when a picking error occurs. The occurrence of a picking error can be detected by a sensor such as a camera by detecting that the workpiece W could not be picked, or by detecting that the picking mechanism 151 has come into contact with a workpiece W that is not intended to be picked. Also, the determinations in both steps S10 and S11 may be incorporated, and the process may return to step S4 when either event occurs. [Explanation of symbols]

[0050] 1 Robot control system, 10 Picking robot, 11, 21 Control unit, 12 Drive control unit, 13 Drive unit, 14 Sensor group, 15, 22 Input unit, 16, 24 Memory unit, 17 Learning model, 23 Output unit, 25 Simulation program

Claims

1. an acquisition unit that acquires deviation information indicating an amount of deviation between a first position, which is a position of the picking mechanism when the robot arm is controlled offline based on an offline control command value, which is a control command value for controlling the robot arm generated by offline teaching through simulation, and a second position, which is a position of the picking mechanism when the robot arm is actually controlled online based on the offline control command value, for a picking robot that picks up an item using a robot arm having a picking mechanism; and A generation unit that generates a learning control command value, which is a control command value for actually controlling the robot arm online, by performing deep reinforcement machine learning using data including the offline control command value, the deviation information, and characteristic information indicating operating characteristics of the robot arm as input data so as to minimize the deviation amount between the first position and the second position; and Equipped with the acquisition unit acquires updated deviation information which is information indicating an amount of deviation between the first position and a third position which is a position of the picking mechanism when the robot arm is actually controlled online based on the learned control command value; The generation unit inputs the update deviation information as part of the input data, and executes the deep reinforcement machine learning so as to minimize the deviation amount between the first position and the third position, thereby updating the learning control command value. Robot control system.

2. At least a part of the characteristic information is a sensor value detected by a sensor provided in the robot arm for detecting the operation characteristic when the robot arm is actually controlled online. The robot control system of claim 1 .

3. The input data includes sensor values ​​detected by sensors provided on the robot arm to detect an operating state when the robot arm is actually controlled online.

3. The robot control system according to claim 1 or 2.

4. A picking method for controlling the picking robot with the robot control system according to claim 1 or 2 to pick up the item.

Images