Packing style influence alleviation method

Abstract

To provide a packing style influence alleviation method which allows for easily gripping an object when a part of the object cannot be seen due to influence of internal material like a buffer material or a packing style.SOLUTION: A packing style influence alleviation method comprises the steps of: recognizing an object from image data including one or more imaged objects; inputting, when a part of the object is hidden because of an internal material, an object image of the object a part of which is hidden, into a self-supervised learning model to generate an object image where an area unseen due to influence of the internal material is reduced; calculating a gripping point of the recognized object; gripping, by a robot, the object at the gripping point; and causing a deep reinforcement learning model to learn the gripping point to enhance a gripping success rate on the basis of gap information between the objects, a force sensor value, and joint information, and causing the deep reinforcement learning model to re-learn, when an image data including one or more objects is imaged next time, the gripping point again to enhance the gripping success rate.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a method for reducing the effect of packaging. [Background technology]

[0002] In the manufacturing process, there is a process in which a robot equipped with a two-fingered hand grasps and removes objects such as parts from a storage container, etc. It is expected that the robot will be able to properly grasp and remove the parts from the storage container.

[0003] A technology has been devised that uses image processing to determine whether an object to be grasped can be grasped (for example, see Patent Document 1). Patent Document 1 discloses a technology that analyzes the distance between the hand and the object to be grasped to determine the grasping success rate, generates a classifier from a feature vector calculated from the distance to the object to be grasped, and uses this classifier to determine whether the object to be grasped can be grasped. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-047505 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the conventional technology, when a part is not visible due to the inner material such as cushioning material or the packaging style, there are problems as follows. When most of a part is obscured by the packaging or inner materials such as cushioning, the gripping point of the part cannot be calculated. The gripping point is calculated incorrectly, causing the part to come into contact with the hand and be damaged. -The parts that can be grasped are limited. - Packaging will need to be changed, increasing costs.

[0006] In view of the above problems, the present invention provides a technique that makes it easier to grasp an object even when part of the object is not visible due to the influence of an inner material such as a cushioning material or the packaging style. [Means for solving the problem]

[0007] In view of the above problems, the present invention provides a method for mitigating the effect of packaging style when a robot equipped with a two-fingered hand grasps an object imaged by an imaging device, the method comprising the steps of: recognizing an object from image data and point cloud data including one or more imaged objects; if there is an object partially hidden by an internal material, inputting an object image of the partially hidden object to a self-supervised learning model; generating an object image in which an area that is invisible due to the influence of the internal material is smaller than that of the object image by the self-supervised learning model; recognizing the object again from image data including the object image generated so that the area that is invisible due to the influence of the internal material is smaller; calculating a grasping point for the recognized object again; grasping the object with the grasping point by the robot; The method includes the steps of: inputting, to the deep reinforcement learning model, information about gaps between objects in image data including an object image generated so as to reduce areas that are invisible due to the influence of external forces, a force sensor value detected when the robot grasps the object, and joint information; and the deep reinforcement learning model learning a grasping point so as to increase the grasping success rate. When image data including one or more objects is captured next time, the deep reinforcement learning model instructs the robot to the grasping point calculated based on the learning result, and inputs, to the deep reinforcement learning model, information about gaps between objects in the next image data, the force sensor value detected when the robot grasps the object at the grasping point, and joint information, and the deep reinforcement learning model re-learns the grasping point so as to increase the grasping success rate. Effect of the Invention

[0008] It is possible to provide a technology that makes it easier to grasp an object even if part of the object is not visible due to the inner materials such as cushioning or the packaging. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram showing an example of a plurality of parts stored in a storage container; [Diagram 2] FIG. 11 is a flowchart illustrating a process performed by the part gripping device. [Diagram 3] 2 is an example of a functional block diagram illustrating functions of the part gripping device divided into blocks. FIG. [Figure 4] 3A to 3C are diagrams for explaining a schematic diagram of a process performed by the part gripping device of the present embodiment. [Diagram 5] 1 is an example of a flowchart illustrating a process in which the part gripping device of the present embodiment recognizes a part and causes the two-fingered hand robot to grip the part. [Figure 6] FIG. 13 is a diagram showing an example of a part having the largest area in image data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, a part gripping device and a packaging style effect mitigation method performed by the part gripping device will be described as an example of an embodiment of the present invention.

[0011] <Example of parts in a storage container> First, referring to FIG. 1, the parts in a storage container will be described. FIG. 1 shows a number of parts stored in a storage container. Note that the parts may be placed as targets for gripping, and are not limited to storage containers. There may be only one part. In FIG. 1, four parts are stored, but for example, the second part from the left 101 is covered by an internal material such as a cushioning material, and most of the part 101 is not visible. The most part means, for example, about 50 to 60% or more. The part gripping device that grips and removes this part captures an image of the storage container with an imaging device, and therefore the following inconvenience occurs when there is a part 101 that is covered by an internal material. The part gripping device cannot correctly recognize the size of the part. The robot's hand interferes with parts that are not visible. In this case, there is a high possibility that parts with decorative surfaces will be damaged. It is necessary to determine whether there are invisible areas, and if so, to generate images of the parts in those invisible areas.

[0012] In addition, situations in which invisible areas of a part occur may be due to internal materials, as well as the way the parts are packed, etc.

[0013] <Terminology> The robot equipped with a two-fingered hand may be any robot that can grasp an object. The robot may also have three or more fingers. In this embodiment, the robot grasps a grasping point, but it may also suck up a suction point, or a mixture of suction points and grasping points may be used.

[0014] Internal materials are materials used to prevent scratches, dents, deformations, etc. that may occur on parts due to shocks during movement between processes or transportation. Internal materials may also be called cushioning materials. Internal materials include cloth, air cushions, various protectors, cushion paper, and polystyrene foam. Packaging appearance refers to the state in which parts are packaged, and is the appearance of the parts when they are packaged and stored in a storage container.

[0015] A part being partially hidden by the inner material or packaging does not include the case where the whole part is completely invisible. If the inner material is transparent, it does not mean that the whole part is completely invisible even if the whole part is covered by the inner material. Also, even if the whole part is completely invisible, it is acceptable if the inner material is in close contact with the part and the shape of the part is clear enough to estimate the gripping point.

[0016] <Outline of the Processing of the Present Embodiment> Therefore, in this embodiment, as described below, part images of invisible areas are generated using a self-supervised learning model, and various information such as gap information between parts (an example of between objects) is input into a deep reinforcement learning model, and an accurate gripping point is found by re-learning.

[0017] 2 is a flow chart for explaining the flow of processing performed by the part gripping device, which is assumed to have an imaging device and a two-fingered hand robot. S1: The part gripping device captures images of one or more parts stored in a storage container and recognizes the parts from the captured image data and point cloud data. The point cloud data is a range image or a 3D point cloud corresponding to the image data, and is detected by a stereo camera or Lidar (Light Detection And Ranging). S2: If there is a part that cannot be recognized due to internal materials or packaging (it does not matter if it is not clear at this point whether it is a part or not), the part grasping device inputs the image data into the self-supervised learning model. S3: The part gripping device uses a self-supervised learning model to generate part images (an example of object images) of parts that are partially invisible due to the internal materials or packaging in the input image data. S4: The part gripping device recognizes the part again using a part image generated to eliminate the effects of internal materials and packaging. S5: The part gripping device analyzes the part recognized in the image data that has been replaced with the generated part image, calculates a gripping point, and picks (removes) the part. S6: The part gripping device inputs information about the gaps between parts (including the gaps with the internal materials) calculated from image data including part images in which the effects of internal materials and packaging have been eliminated, the force sensor values ​​detected during picking, and each joint information into a deep reinforcement learning model. S7: The part gripping device learns the gripping point that maximizes the gripping success rate using a deep reinforcement learning model. The next time an image is captured, the gripping point calculated based on the learning results is assigned to the two-fingered hand robot. S8: The part gripping device again acquires the gripping success rate, the force sensor value, the gap information between the parts, and each joint information when operating at the assigned gripping point. S9: The part gripping device re-learns the acquired information using a deep reinforcement learning model, and based on the re-learned results, instructs the two-fingered hand robot on the gripping point.

[0018] According to the part gripping device of this embodiment, even if most of a part is hidden due to the packaging or the inner material such as cushioning, the gripping point can be calculated as intended. Damage to the part due to interference between the part and the hand can be suppressed. In addition, the number of parts that can be gripped can be increased without changing the packaging.

[0019] <Example of functional configuration> 3 is an example of a functional block diagram explaining the functions of the part gripping device 10 by dividing them into blocks. The part gripping device 10 has, for example, a control unit 11, an imaging device 12, and a two-fingered hand robot 13. The control unit 11 may be a microcomputer, a computer, or a SOC having a CPU, RAM, ROM, HDD (or SSD), an input / output unit, ASIC, FPGA, or the like. The control unit 11 realizes the functions shown in FIG. 3 by the CPU executing a program stored in the ROM or HDD (or SSD). The control unit 11 has a part recognition unit 14, an image generation unit 15, a gripping point calculation unit 16, a self-supervised learning unit 17, and a deep reinforcement learning unit 18.

[0020] The part recognition unit 14 may be an image recognition model that recognizes parts from image data of one or more parts captured by the imaging device 12. The image recognition model is constructed using a learning method such as deep learning. The part recognition unit 14 also recognizes point cloud data such as a range image or a three-dimensional point cloud corresponding to the image data. The range image or the three-dimensional point cloud may be detected by a stereo camera or Lidar.

[0021] The image generation unit 15 uses a self-supervised learning model to generate a part image in which the area that is not visible due to the influence of the internal materials or the packing style is reduced from the part image included in the image data. The part recognition unit 14 recognizes the part again from the generated part image.

[0022] The gripping point calculation unit 16 calculates a gripping point for the recognized part. For example, the gripping point calculation unit 16 estimates the center of gravity from the point cloud data of the recognized part, and determines the gripping point that is closest to the estimated center of gravity and has the largest distance (that can be gripped at a deep position). In addition, when the learning of the deep reinforcement learning model is completed, the gripping point calculation unit 16 can calculate the gripping point using the deep reinforcement learning model constructed by the deep reinforcement learning unit 18 through learning, with the gap information between the parts, the force sensor value, and each joint information as input data. The calculated gripping point is transmitted to the two-fingered hand robot 13.

[0023] The self-supervised learning unit 17 generates a self-supervised learning model (image generation unit 15) that generates a part image with a small invisible area from a part image with a large invisible area due to the influence of internal materials or packaging in the image data. Self-supervised learning refers to a learning method that uses a large amount of unlabeled data set to perform pre-learning to solve a pre-text task (an alternative task for which pseudo-labels are automatically generated).

[0024] The deep reinforcement learning unit 18 generates a deep reinforcement learning model (grasping point calculation unit 16) that outputs a gripping point that maximizes the gripping success rate using gap information between parts calculated from image data including part images from which the effects of internal materials and packaging have been removed, force sensor values, and each joint information. Note that reinforcement learning is a learning method that searches for an optimal action to obtain a "reward (score)" set as a goal, although there is no correct answer in the learning data. In this embodiment, the higher the gripping success rate, the higher the reward setting is considered. Deep reinforcement learning is a learning method that combines reinforcement learning and deep learning, and refers to an approximation of the action value function (Q function) in reinforcement learning by replacing it with deep learning. Note that the learning method is one example, and other learning methods may be adopted.

[0025] The two-fingered hand robot 13 is a robot that is connected to the tip of an industrial robot and grasps, deforms, processes, etc. parts in place of a human hand. In this embodiment, the number of fingers is two, but it may have three or more fingers. The two-fingered hand robot 13 may be mounted on a moving body and be movable, or the two-fingered hand robot 13 may be a humanoid robot. The two-fingered hand robot 13 has a sensor 21 and a joint control unit 22.

[0026] The sensor 21 is a force sensor or pressure sensor built into the surface of the two-fingered hand that comes into contact with a part. A force sensor value may be calculated from a current value for gripping. The joint control unit 22 controls the state of the joints that make up the fingers of the two-fingered hand robot 13. The joint control unit 22 can obtain information regarding the angle, speed, acceleration, and torque of the joint as the state of the joint. Generally, one finger has six joints, but the number of joints is not limited to six.

[0027] <Processing or control flow> FIG. 4 is a diagram for explaining the process performed by the part gripping device 10 of the present embodiment. First, the deep reinforcement learning model 201 will be explained. The deep reinforcement learning model 201 is input with the force sensor value 31 detected by the sensor 21, the joint information 32 acquired by the joint control unit 22, and the gap information 33 between parts (including the gap information between the internal material and the part) recognized by the part recognition unit 14. The deep reinforcement learning model 201 is also input with the gap information 33 between parts (including the gap information between the internal material and the part) in the image data 41 including the part image with the invisible area reduced, generated by the image generation unit 15. The image data 41 may remain the original part image (when the part is not hidden by the internal material, etc.).

[0028] As an example of a deep reinforcement learning model, we will explain using Deep Q-Network (DQN). First, we will explain reinforcement learning.

[0029]

number

[0030] Deep reinforcement learning will be explained. In deep reinforcement learning, deep learning is used to learn the Q-table. That is, in deep reinforcement learning, a state (information on the gap between parts, the force sensor value, and each joint information) is input to the input layer of a neural network, and a reward for each action that can be taken in that state is output from the output layer via a hidden layer. The output layer has a node corresponding to each gripping point, and each node outputs a probability that correlates with the gripping success rate. That is, when an action is a gripping point, the probability of which gripping point to use for the gap information between certain parts, the force sensor value, and each joint information that results in a high gripping success rate (corresponding to a reward) is output with respect to each gripping point.

[0031] We will explain the learning of neural network weights in deep reinforcement learning. In updating the Q table in formula (1), when the second term on the right-hand side approaches zero (when the amount of updates decreases), the update of the Q table also ends. For this reason, in the learning phase of deep reinforcement learning, a loss function (Loss) is defined as shown in formula (2), and learning of the weights between the neural networks that infer the reward is performed. Considering that the reward of the Q table is set so that the grasping success rate is high, the loss function (Loss) is backpropagated, and the weights of the neural network are adjusted so that the grasping success rate is high.

[0032]

number

[0033] Next, the self-supervised learning model 202 will be described. Self-supervised learning is a supervised learning method in which a unique label (supervised data) is mechanically created from the data itself, and learning is performed using this label. In the self-supervised learning of this embodiment, learning is performed to generate a part image in which the unseen area is made smaller from a part image in which the unseen area is large. Specifically, learning is performed in the following flow. · Automatically creates pre-text by dividing a single part image into multiple parts and distorting the color and shape. During this process, two different transformation patterns (positive and negative examples) are applied to each image. Next, the transformed images are transformed using two encoders, one for positive examples and one for counterexamples, to obtain features. Next, the loss is calculated using a loss function that can compare the features obtained by the two encoders, and the encoder weights are trained to minimize the loss. During training, the neural network weights are trained so that the two features obtained from the same original image are closer to each other.

[0034] Self-supervised learning model 202 has a generation unit 37, a real image holding unit 38, a generated image holding unit 39, and an inverse generation unit 40. A part image 42 cut out from image data and having a large area (above a threshold) that is invisible due to the influence of internal materials or packaging style is input to the generation unit 37. The generation unit 37 generates a part image 43 in which the area that is invisible due to the influence of internal materials or packaging style is smaller. Part image 42 in the original image data 41 is replaced with this part image 43.

[0035] The real image storage unit 38, the generated image storage unit 39, and the inverse generation unit 40 are used for learning the self-supervised learning model 202. The real image storage unit 38 stores various part images prepared in advance. The generated image storage unit 39 stores generated part images. The inverse generation unit 40 learns the weights of the neural network so that two feature amounts obtained from two images approach each other, and feeds the weights back to the generation unit 37.

[0036] The image data 41 generated by the self-supervised learning model 202, including the part image 43 with the smaller invisible area, is input to the part recognition process 34. The image data 41 is used to recognize the part, and after recognition, a grip point search process 35 is executed.

[0037] <Detailed process flow> FIG. 5 is a flow chart illustrating the process in which the part gripping device 10 of this embodiment recognizes a part and causes the two-fingered hand robot 13 to grip the part.

[0038] First, the imaging device 12 captures an image of one or more parts stored in a storage container. The part recognition unit 14 recognizes the parts from the image data in which the one or more parts are captured and the point cloud data (S11).

[0039] Next, the part recognition unit 14 identifies the part with the largest area in the image data (S12). An example of the part 203 with the largest area in the image data is shown in FIG.

[0040] Next, the part recognition unit 14 acquires unevenness information of the part 203 with the largest area (S13). When the point cloud data of the part is the distance from the imaging device 12, the unevenness information may be the difference between the maximum and minimum distances, or the variance of the distances.

[0041] Next, the part recognition unit 14 compares the areas of the part with the largest area and the objects thought to be other parts, and the unevenness information of the part with the largest area and the objects thought to be other parts (target parts) (S14). The target parts may be any parts other than the part with the largest area, the part with the smallest area, or the part with the second largest area.

[0042] As an example of a comparison method, "The area of ​​the largest part × 0.9 > the area of ​​the target part" and "The unevenness information of the most placed part × 0.9 > the unevenness information of the target part" This judgment is made by assuming that even if the area or unevenness information of the largest part is estimated slightly low (by multiplying by 0.9), if it is larger than the objects thought to be other parts, it is judged that there are parts that are not visible due to the internal materials or packaging (area is not uniform).

[0043] If the determination in step S14 is Yes, the process proceeds to step S15, and if the determination is No, the process proceeds to step S20.

[0044] In step S15, the image generator 15 inputs the image data to the self-supervised learning model 202 (S15).

[0045] The image generator 15 inputs the part image with the largest area into the self-supervised learning model 202 and generates a part image in which the area that is not visible due to the internal materials or the packaging is reduced (S16). The generated part image replaces the part image in the captured image data.

[0046] The control unit 11 determines whether the number of loop executions matches the "number of parts - 1" (S17). That is, it determines whether part images have been generated for all parts in the image data. If the determination in step S17 is Yes, the process proceeds to step S19, and if the determination is No, the process proceeds to step S14. That is, if the conditions in step S14 are met, part images in which the areas not visible due to the internal materials or packaging are reduced are generated in steps S15 and S16, but if step S14 is no longer met, the process moves to the picking operation. Therefore, part images in which the areas not visible due to the internal materials or packaging are reduced may be generated for only some parts in the image data.

[0047] In step S19, the part recognition unit 14 recognizes the object again using image data including part images that have been generated so as to eliminate the influence of internal materials and packaging styles (S19).

[0048] Next, the gripping point calculation unit 16 analyzes the point cloud data of the recognized part and calculates the optimal gripping point. The control unit 11 also causes the two-fingered hand robot 13 to pick (remove) the part from the gripping point (S20). As a result, all parts are picked up. If learning of the deep reinforcement learning model has already been completed, the gripping point may be calculated by the deep reinforcement learning model.

[0049] Next, the control unit 11 performs learning of the deep reinforcement learning model using the information obtained from the part picking.

[0050] First, the deep reinforcement learning unit 18 inputs the part image with the effects of internal materials and packaging eliminated or the gap information between parts (including the gap with the internal materials) calculated from the original part image, the force sensor value obtained by the pick in step S20, and each joint information into the deep reinforcement learning model 201 (S21).

[0051] The deep reinforcement learning unit 18 learns the grasping point that maximizes the grasping success rate using the deep reinforcement learning model 201 (S22). The control unit 11 instructs the two-fingered hand robot 13 to use the grasping point calculated based on the current learning result when capturing an image of the next part.

[0052] At the next imaging, the control unit 11 acquires from the two-fingered hand robot 13 the grasping success rate, the force sensor value, the gap information between the parts, and the joint information when grasping with the assigned grasping point as a result of learning (S23).

[0053] The control unit 11 re-learns the deep reinforcement learning model 201 from the acquired information, and instructs the two-fingered hand robot 13 based on the re-learning result (S24).

[0054] <Major Effects> According to the part gripping device of this embodiment, even if most of a part is hidden due to the packaging or the inner material such as cushioning, the gripping point can be calculated as intended. Damage to the part due to interference between the part and the hand can be suppressed. In addition, the number of parts that can be gripped can be increased without changing the packaging. [Explanation of symbols]

[0055] 10 Part gripping device 11 Control section 12 Imaging device 13 Two-fingered hand robot

Claims

[Claim 1] A method for mitigating the effect of packaging style when a robot equipped with a two-fingered hand grasps an object captured by an imaging device, comprising: Recognizing objects from image data and point cloud data including one or more captured objects; When there is an object that is partially hidden due to the influence of the inner material or the packaging, inputting an object image of the partially hidden object into a self-supervised learning model; A step of generating an object image in which an area that is not visible due to the influence of an inner material or a packaging style is smaller than that of the object image by a self-supervised learning model; recognizing the object from image data including an object image generated so as to reduce areas that are not visible due to the influence of the inner material or the packaging; again calculating a grasp point for the recognized object; the robot performing a grasp of an object at the grasp point; A step of inputting gap information between objects in image data including an object image generated so as to reduce areas that cannot be seen due to the influence of internal materials or packaging, a force sensor value detected when the robot grasps an object, and joint information into a deep reinforcement learning model; The deep reinforcement learning model learns a gripping point so as to increase a gripping success rate; Next time, when image data including one or more objects is captured, the deep reinforcement learning model instructs the robot to grasp the object based on the learning result. Inputting gap information between objects in the next image data, a force sensor value detected when the robot grasps the object at the grasping point, and joint information into the deep reinforcement learning model; A method for mitigating the effect of packaging shape, in which the deep reinforcement learning model re-learns the gripping points so as to increase the gripping success rate.

Images