Control device and control method

Abstract

To provide a control device that can derive a grasping point of each work-piece at which each work-piece can be appropriately grasped.SOLUTION: A control device (5) according to one embodiment described in the disclosure comprises a reinforcement learning part (5b) that derives a gravity center point (G) of a work-piece (W) on the basis of detected information by a force sensor (3) provided in a robot arm (2), and learns a grasping point (P) of the work-piece (W) at which deviations of the grasping point from the gravity center point (G) of the work-piece (W) becomes minimum, using a deep reinforcement learning model in which deviations of the grasping point (P) of the work-piece (W) from the gravity center point (G) of the work-piece (W) and information on joints of a robot arm (2) are inputted and the grasping point (P) of the work-piece (W) is outputted. The reinforcement learning part (5b) re-learns the grasping point (P) of the work-piece (W), using the deep reinforcement learning model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a control device and a control method, for example, a control device and a control method for a robot arm that picks up and moves a workpiece. [Background technology]

[0002] For example, the control device in Patent Document 1 calculates the center of gravity of a workpiece when it is picked up by a robot arm based on detection information from a force sensor installed on the robot arm, and determines the type of workpiece by referring to the calculated center of gravity. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-67995 Summary of the Invention [Problem to be solved by the invention]

[0004] The present applicant has found the following problem: The control device of Patent Document 1 can distinguish the type of workpiece, but has a problem in that it cannot derive a gripping point that can properly pick up each workpiece.

[0005] The present disclosure has been made in consideration of such problems, and realizes a control device and control method that can derive a gripping point that can pick up each workpiece well for each workpiece. [Means for solving the problem]

[0006] A control device according to one aspect of the present disclosure is a control device for a robot arm that picks up and moves a workpiece, a reinforcement learning unit that derives the center of gravity of the workpiece when the robot arm picks up the workpiece based on detection information from a force sensor provided on the robot arm, and learns the gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece by a deep reinforcement learning model that receives as inputs the amount of deviation between the center of gravity of the workpiece when the robot arm picks up the workpiece and a gripping point of the workpiece and joint information of the robot arm when the robot arm picks up the workpiece, and outputs the gripping point of the workpiece; A path generation unit that derives joint information of the robot arm so as to pick up the workpiece at the gripping point of the workpiece learned by the deep reinforcement learning model; a control unit for controlling the robot arm based on the derived joint information of the robot arm; Equipped with The reinforcement learning unit uses the deep reinforcement learning model to control the robot arm based on the derived joint information of the robot arm, and re-learn the grasping point of the workpiece using the amount of deviation between the center of gravity of the workpiece and the grasping point of the workpiece when the robot arm picks up the workpiece and the derived joint information of the robot arm.

[0007] The above-mentioned control device acquires information on the type of work to be picked up by the robot arm, the order of the work to be picked up, and information on the type of work that has been learned, and based on the information on the type of work to be picked up by the robot arm, the order of the work to be picked up, and the information on the type of work that has been learned, a determination unit that determines whether the work to be picked up this time has been learned, It is preferable that if the work to be picked up this time has already been learned, learning using the deep reinforcement learning model is omitted, and if the work to be picked up this time has not already been learned, learning using the deep reinforcement learning model is performed.

[0008] In the above-described control device, it is preferable that the joint information of the robot arm includes, in addition to angle information of each joint of the robot arm, at least one of angular velocity information, angular acceleration information, and torque information of a motor that drives the joint of the robot arm.

[0009] A control method according to one aspect of the present disclosure is a control method for a robot arm that picks up and moves a workpiece, comprising: deriving a center of gravity of the workpiece when the workpiece is picked up by the robot arm based on detection information of a force sensor provided on the robot arm; A step of learning a gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece by a deep reinforcement learning model that uses as inputs the amount of deviation between the center of gravity of the workpiece and a gripping point of the workpiece when the robot arm picks up the workpiece and joint information of the robot arm when the robot arm picks up the workpiece, and outputs the gripping point of the workpiece; Deriving joint information of the robot arm so as to pick up the workpiece at the gripping point of the workpiece learned by the deep reinforcement learning model; controlling the robot arm based on the derived robot arm joint information; A step of re-learning the gripping point of the workpiece using the derived joint information of the robot arm and a deviation amount between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm is controlled based on the derived joint information of the robot arm by the deep reinforcement learning model and the derived joint information of the robot arm; Equipped with.

[0010] The above-mentioned control method includes a step of acquiring information on the type of work to be picked up by the robot arm, the order of picking up the work, and information on the type of work that has been learned, and determining whether the work to be picked up this time has been learned based on the information on the type of work to be picked up by the robot arm, the order of picking up the work, and the information on the type of work that has been learned, It is preferable that if the work to be picked up this time has already been learned, learning using the deep reinforcement learning model is omitted, and if the work to be picked up this time has not already been learned, learning using the deep reinforcement learning model is performed. Effect of the Invention

[0011] According to the present disclosure, it is possible to realize a control device and a control method that can derive a gripping point that can properly grip each workpiece. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of a robot arm system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing a state in which a workpiece is gripped by a robot arm of the robot arm system according to the first embodiment. [Diagram 3] FIG. 4 is a flowchart for explaining a flow of controlling the robot arm of the robot arm system according to the first embodiment. [Figure 4] 13 is a diagram showing a state in which a workpiece picked up by the robot arm is stored in a storage box. FIG. [Diagram 5] FIG. 11 is a block diagram showing a configuration of a robot arm system according to a second embodiment. [Figure 6] FIG. 11 is a flowchart for explaining a flow of controlling a robot arm of the robot arm system according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, specific embodiments to which the present disclosure is applied will be described in detail with reference to the drawings. However, the present disclosure is not limited to the following embodiments. In addition, the following description and drawings are appropriately simplified.

[0014] <Embodiment 1> First, the configuration of the robot arm system of the present embodiment will be described. Fig. 1 is a block diagram showing the configuration of the robot arm system of the present embodiment. Fig. 2 is a diagram showing a state in which a workpiece is gripped by a robot arm of the robot arm system of the present embodiment.

[0015] 1, a robot arm system 1 according to the present embodiment includes a robot arm 2, a force sensor 3, a camera 4, a control device 5, and a database (DB) 6. The robot arm 2, the force sensor 3, the camera 4, and the DB 6 are communicatively connected to the control device 5 by wire or wirelessly.

[0016] The robot arm 2 is a multi-joint robot arm like a general robot arm, and includes a hand unit 2a and an arm unit 2b as shown in Fig. 2. The hand unit 2a can be configured, for example, as a two-fingered hand. The arm unit 2b can be configured, for example, as a six-axis robot. Each joint of the hand unit 2a and the arm unit 2b operates based on the driving force of a motor 2c.

[0017] 2, the force sensor 3 is disposed between the hand unit 2a and the arm unit 2b of the robot arm 2. In other words, the hand unit 2a is connected to the arm unit 2b via the force sensor 3. The camera 4 can be configured, for example, by an RGBD camera.

[0018] The camera 4, for example, photographs the workpiece W before it is gripped by the robot arm 2. The camera 4 may be provided on the robot arm 2, but may be disposed at a location where it can photograph the workpiece W before it is gripped by the robot arm 2.

[0019] 1, the control device 5 includes a shape recognition unit 5a, a reinforcement learning unit 5b, a path generation unit 5c, and a control unit 5d. The shape recognition unit 5a recognizes the shape of the workpiece W based on image information captured by the camera 4. The shape of the workpiece W can be recognized by a general method.

[0020] The reinforcement learning unit 5b, details of which will be described later, derives the center of gravity G of the work W when it is picked up by the robot arm 2 based on the detection information from the force sensor 3, and learns the grasping point P of the work W that minimizes the deviation from the center of gravity G of the work W using a deep reinforcement learning model.

[0021] The deep reinforcement learning model is a model that receives as input the amount of deviation between the center of gravity G of the work W and the gripping point P of the work W when the work W is picked up by the robot arm 2, and joint information of the robot arm 2 when the work W is picked up by the robot arm 2, and outputs the gripping point P of the work W. Note that the center of gravity G, the gripping point P, and the amount of deviation may be represented, for example, by three-dimensional coordinate information with a predetermined position as the origin.

[0022] The path generation unit 5c, details of which will be described later, generates a path for the robot arm 2 to approach the grasping point P of the work W derived based on the recognized shape of the work W or the grasping point P of the work W learned by a deep reinforcement learning model and pick up the work W, and derives the angles of each joint as joint information of the robot arm 2 for realizing the generated path.

[0023] In addition, the path generation unit 5c generates a path for moving the workpiece W from the gripping point P of the workpiece W learned by the deep reinforcement learning model to a predetermined location, and derives the angles of each joint as joint information of the robot arm 2 for realizing the generated path.

[0024] The control unit 5d controls each motor 2c of the robot arm 2 based on the derived angle information of each joint of the robot arm 2. At this time, the control unit 5d may control the motor 2c based on detection information of an encoder 2d provided on the motor 2c. The DB6 stores the deep reinforcement learning model, result information on whether or not the gripping of the workpiece W was successful for each gripping point P of the workpiece W, and the like.

[0025] Next, a flow of controlling the robot arm 2 of the robot arm system 1 of this embodiment will be described. Fig. 3 is a flow chart for explaining the flow of controlling the robot arm of the robot arm system of this embodiment.

[0026] 4 shows a state in which the workpiece W to be picked up by the robot arm 2 is stored in the storage box 11, and in the following description, it is assumed that the workpiece W is picked up. In the example of FIG. 4, the plate-shaped workpiece W having an eccentric shape in a side view as shown in FIG. 2 is stored inside the storage box 11 and leaned against a partition 11a provided in the storage box 11.

[0027] In this state, first, the camera 4 photographs the workpiece W from above. Then, the shape recognition unit 5a of the control device 5 recognizes the shape of the workpiece W based on the image information of the workpiece W photographed by the camera 4 (S1). At this time, the shape recognition unit 5a may recognize the shape of the workpiece W viewed from above. In the example of Fig. 4, the shape recognition unit 5a recognizes that the workpiece W viewed from above is a thin plate-like shape.

[0028] Next, the path generating unit 5c of the control device 5 derives a gripping point P of the workpiece W based on the recognized shape of the workpiece W (S2). For example, if the workpiece W is a thin plate when viewed from above, the path generating unit 5c determines the center of the width dimension (left and right direction in FIG. 2) of the workpiece W as the gripping point P, as shown in FIG. However, the path generating unit 5c may determine, for example, approximately the center of the workpiece W when viewed from above as the gripping point P, based on the recognized shape of the workpiece W.

[0029] Next, the path generation unit 5c of the control device 5 generates a path for the robot arm 2 to approach the derived gripping point P of the workpiece W and pick up the workpiece W, and derives the angles of each joint as joint information of the robot arm 2 to realize the generated path.

[0030] Then, the control unit 5d of the control device 5 controls each motor 2c of the robot arm 2 based on the derived angle information of each joint of the robot arm 2, and picks up the workpiece W (S3). Here, the picking up of the workpiece W may be performed by, for example, lifting the workpiece W vertically to a predetermined height.

[0031] Next, the reinforcement learning unit 5b of the control device 5 derives the center of gravity G of the work W when it is picked up by the robot arm 2 based on the detection information of the force sensor 3, and acquires the amount of deviation between the center of gravity G of the work W when it is picked up by the robot arm 2 and the gripping point P of the work W, and the joint information of the robot arm 2 when it picks up the work W (S4).

[0032] Here, a general method can be used as a method for deriving the center of gravity G of the workpiece W using the force sensor 3. The amount of deviation between the center of gravity G of the workpiece W and the gripping point P of the workpiece W may be the amount of deviation in the horizontal direction.

[0033] In addition, the joint information of the robot arm 2 may include at least one of angular velocity information, angular acceleration information, and torque information of the motor 2c, in addition to angle information of each joint when the robot arm 2 grasps the workpiece W to pick it up.

[0034] Next, the reinforcement learning unit 5b of the control device 5 inputs into the deep reinforcement learning model the amount of deviation between the center of gravity G of the work W and the gripping point P of the work W obtained in step S4, as well as the joint information of the robot arm 2 when the robot arm 2 picks up the work W, and learns the gripping point P of the work W that has the smallest amount of deviation from the center of gravity G of the work W (S5).

[0035] At this time, if the gripping point P of the workpiece W is in a local range, it is assumed that the robot arm 2 may not be able to grip the gripping point P of the workpiece W due to factors such as the shape of the workpiece W or interference with the storage box 11. For this reason, the reinforcement learning unit 5b of the control device 5 may distribute the gripping points P of the workpiece W within a predetermined range with respect to the center of gravity G of the workpiece W (in the example of FIG. 2, within a predetermined range in the width direction of the workpiece W with respect to the center of gravity G of the workpiece W).

[0036] Next, the path generation unit 5c of the control device 5 generates a path for the robot arm 2 to approach the learned gripping point P of the workpiece W and pick up the workpiece W, and derives the angles of each joint as joint information of the robot arm 2 to realize the generated path.

[0037] Then, the control unit 5d of the control device 5 controls each motor 2c of the robot arm 2 based on the derived angle information of each joint of the robot arm 2, to pick up the workpiece W (S6).

[0038] Next, the reinforcement learning unit 5b of the control device 5 acquires the center of gravity G of the workpiece W when the robot arm 2 grasps the grasp point P of the workpiece W learned and picks up the workpiece W based on the detection information of the force sensor 3, and acquires the amount of deviation between the center of gravity G of the workpiece W and the grasp point P of the workpiece W when the robot arm 2 picks up the workpiece W, and the joint information of the robot arm 2 when the robot arm 2 picks up the workpiece W (S7).

[0039] Next, the reinforcement learning unit 5b of the control device 5 inputs into the deep reinforcement learning model the amount of deviation between the center of gravity G of the work W and the gripping point P of the work W obtained in step S7, as well as the joint information of the robot arm 2 when the robot arm 2 picks up the work W, and re-learns the gripping point P of the work W that has the smallest amount of deviation from the center of gravity G of the work W (S8).

[0040] At this time, in step S5, if the reinforcement learning unit 5b distributes the gripping points P of the work W within a predetermined range relative to the center of gravity G of the work W, it is preferable to also input result information on whether or not the gripping of the work W was successful into the deep reinforcement learning model, and re-learn the gripping point P of the work W that has the smallest deviation from the center of gravity G of the work W.

[0041] This allows the workpiece W to be reliably gripped at the gripping point P of the workpiece W, which has the smallest amount of deviation from the center of gravity G of the workpiece W. Therefore, even if the workpiece W is large, for example, the moment of the workpiece W generated when gripping the workpiece W can be reduced, and the workpiece W can be picked up well.

[0042] Next, the path generation unit 5c of the control device 5 generates a path for the robot arm 2 to approach the re-learned gripping point P of the workpiece W, pick up the workpiece W, and move it to a specified location, and derives the angles of each joint as joint information of the robot arm 2 to realize the generated path.

[0043] Then, the control unit 5d of the control device 5 controls each motor 2c of the robot arm 2 based on the derived angle information of each joint of the robot arm 2, to pick up the workpiece W and move it to a predetermined location (S9).

[0044] In this manner, the control device 5 and control method of the present embodiment use a deep reinforcement learning model to learn the gripping point P of the workpiece W that minimizes the amount of deviation from the center of gravity G of the workpiece W. Therefore, the control device 5 and control method of the present embodiment can easily derive the gripping point P that can satisfactorily grip each workpiece W.

[0045] Furthermore, there is no need for an operator to perform a teaching process to input the gripping point P of each workpiece W, and compared to the control device of Patent Document 1, the number of workpieces W that can be gripped can be easily increased. In addition, the workpieces W can be picked up regardless of their shape or packaging.

[0046] <Embodiment 2> 5 is a block diagram showing the configuration of a robot arm system according to the present embodiment. As shown in FIG. 5, a robot arm system 21 according to the present embodiment is substantially the same as the robot arm system 1 according to the first embodiment.

[0047] Therefore, redundant explanations will be omitted, but as shown in Figure 5, the control device 22 is equipped with a judgment unit 22a, which judges whether the work W to be picked up this time has already been learned, and judges whether or not to learn the gripping point P of the work W based on the judgment result.

[0048] DB23 stores information regarding the type of work W to be picked up by the robot arm 2, the order in which the work W will be picked up, information regarding the type of work W that has been trained, information regarding the gripping point P of the trained work W, and joint information of the trained work W.

[0049] 6 is a flow chart for explaining the flow of controlling the robot arm of the robot arm system of the present embodiment. As shown in FIG. 6, the flow of controlling the robot arm 2 of the robot arm system 21 of the present embodiment is substantially the same as the flow of controlling the robot arm 2 of the robot arm system 1 of the first embodiment.

[0050] Therefore, without repeating the explanation, before step S1, the judgment unit 22a of the control device 22 reads out information from DB23 regarding the type of work W to be picked up by the robot arm 2, the order in which the work W will be picked up, and information regarding the types of work W that have been learned, and judges whether the work W to be picked up this time has been learned or not based on the information regarding the type of work W to be picked up by the robot arm 2, the order in which the work W will be picked up, and information regarding the types of work W that have been learned (S11).

[0051] If the workpiece W to be picked up this time has not been learned (NO in S11), the steps S1 to S9 are executed, and the robot arm 2 is controlled to pick up and move the workpiece W. In other words, the control device 22 learns the gripping point of the workpiece W to be picked up this time.

[0052] On the other hand, if the workpiece W to be picked up this time has already been learned (YES in S11), the control unit 5d of the control device 22 reads out the joint information of the workpiece W from the DB 23, and based on the read joint information of the workpiece W, controls each motor 2c of the robot arm 2 to attempt to grasp the workpiece W, and determines whether or not the workpiece W has been successfully grasped (S12).

[0053] If gripping of the workpiece W fails (NO in S12), steps S1 to S9 are executed, and the robot arm 2 is controlled to pick up and move the workpiece W. At this time, even if the workpiece W has already been learned, the cause of the failure to grip the workpiece W may be, for example, a change in the packaging form of the workpiece W.

[0054] On the other hand, if the workpiece W is successfully gripped (YES in S12), the motors 2c of the robot arm 2 are controlled based on the read joint information of the workpiece W to continue moving the workpiece W (S13). In other words, the control device 22 omits learning the gripping point of the workpiece W to be picked up this time.

[0055] In this manner, the control device 22 and the control method of the present embodiment also use a deep reinforcement learning model to learn the gripping point P of the workpiece W that minimizes the amount of deviation from the center of gravity G of the workpiece W. Therefore, the control device 22 and the control method of the present embodiment can easily derive the gripping point P that can satisfactorily grip each workpiece W.

[0056] In particular, the control device 22 and the control method of this embodiment determine whether the workpiece W to be picked up this time has already been learned, and based on the determination result, omits learning using the deep reinforcement learning model. Therefore, it is not necessary to learn the gripping point P of the workpiece W every time the workpiece W is picked up, and the calculation load of the control device 22 can be reduced.

[0057] In this embodiment, the judgment unit 22a of the control device 22 reads out information regarding the type of work W to be picked up by the robot arm 2, the order in which the work W will be picked up, and the like, as well as information regarding types of work W that have been learned, from the DB 23, and judges whether or not the work W to be picked up this time has been learned based on the information regarding the type of work W to be picked up by the robot arm 2, the order in which the work W will be picked up, and the information regarding types of work W that have been learned. However, it may also be determined whether or not the work W to be picked up this time has been learned based on the shape of the work W recognized based on image information obtained by photographing the work W to be picked up this time.

[0058] Furthermore, in this embodiment, step S12 is executed, but step S12 may be omitted.

[0059] <Other embodiments> In the above embodiment, the present disclosure has been described as a hardware configuration, but the present disclosure is not limited to this. Each process of the present disclosure can also be realized by causing a CPU (Central Processing Unit) to execute a program.

[0060] Here, a program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, a transitory computer-readable medium or a communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0061] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0062] 1. Robot arm system 2 robot arm, 2a hand section, 2b arm section, 2c motor, 2d encoder 3 Force sensor 4. Camera 5 Control device, 5a Shape recognition unit, 5b Reinforcement learning unit, 5c Path generation unit, 5d Control unit 6 Database 11 Storage Box 21 Robot Arm System 22 control device, 22a judgment unit 23 Database G Center of gravity of workpiece P Workpiece gripping point Double work

Claims

1. A control device for a robot arm that picks up and moves a workpiece, a reinforcement learning unit that derives the center of gravity of the workpiece when the robot arm picks up the workpiece based on detection information from a force sensor provided on the robot arm, and learns the gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece by a deep reinforcement learning model that receives as inputs the amount of deviation between the center of gravity of the workpiece when the robot arm picks up the workpiece and a gripping point of the workpiece and joint information of the robot arm when the robot arm picks up the workpiece, and outputs the gripping point of the workpiece; A path generation unit that derives joint information of the robot arm so as to pick up the workpiece at the gripping point of the workpiece learned by the deep reinforcement learning model; a control unit that controls the robot arm based on the derived joint information of the robot arm; Equipped with The reinforcement learning unit controls the robot arm based on the derived joint information of the robot arm using the deep reinforcement learning model to re-learn the gripping point of the workpiece by using the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece and the derived joint information of the robot arm.

2. A determination unit that acquires information on the type of work to be picked up by the robot arm, the order of the work to be picked up, and information on the type of work that has been learned, and determines whether the work to be picked up this time has been learned based on the information on the type of work to be picked up by the robot arm, the order of the work to be picked up, and the information on the type of work that has been learned, The control device according to claim 1 , wherein if the work to be picked up this time has already been learned, learning using the deep reinforcement learning model is omitted, and if the work to be picked up this time has not been learned, learning using the deep reinforcement learning model is performed.

3. 3. The control device according to claim 1 or 2, wherein the joint information of the robot arm includes, in addition to angle information of each joint of the robot arm, at least one of angular velocity information, angular acceleration information, and torque information of a motor that drives the joint of the robot arm.

4. A method for controlling a robot arm that picks up and moves a workpiece, comprising the steps of: deriving a center of gravity of the workpiece when the workpiece is picked up by the robot arm based on detection information of a force sensor provided on the robot arm; A step of learning a gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece by a deep reinforcement learning model that uses as inputs the amount of deviation between the center of gravity of the workpiece and a gripping point of the workpiece when the robot arm picks up the workpiece and joint information of the robot arm when the robot arm picks up the workpiece, and outputs the gripping point of the workpiece; Deriving joint information of the robot arm so as to pick up the workpiece at the gripping point of the workpiece learned by the deep reinforcement learning model; controlling the robot arm based on the derived robot arm joint information; A step of re-learning the gripping point of the workpiece using the derived joint information of the robot arm and a deviation amount between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm is controlled based on the derived joint information of the robot arm by the deep reinforcement learning model and the derived joint information of the robot arm; A control method comprising:

5. The method includes a step of acquiring information on the type of workpiece to be picked up by the robot arm, the order of picking up the workpieces, and information on the type of workpiece that has already been learned, and determining whether the workpiece to be picked up this time has already been learned based on the information on the type of workpiece to be picked up by the robot arm, the order of picking up the workpieces, and the information on the type of workpiece that has already been learned; The control method according to claim 4, wherein, if the work to be picked up this time has already been learned, learning by the deep reinforcement learning model is omitted, and if the work to be picked up this time has not been learned, learning by the deep reinforcement learning model is performed.

Images