Determination device, mounting device, and determination method

Abstract

This determination device is used in a mounting device that executes mounting processing for arranging objects collected by a collection unit on a substrate. The determination device determines abnormality in the collection of an object by means of the collection unit. The determination device comprises: a machine learning unit that executes machine learning using teacher data having associated therein a collection image obtained by capturing, from the collection direction, an image of an object collected by the collection unit, a collection pressure at which the collection unit collected the object, a collection position at which the collection unit collected the object, and information based on an intersection image obtained by capturing an image of the object collected by the collection unit from an intersection direction intersecting the collection direction; and a determination unit that acquires the collection image, the collection pressure, and the collection position at the time of execution of the mounting processing and determines abnormality in collection on the basis of the machine learning.

Description

Determination device, mounting device, and determination method

[0001] The present disclosure relates to a determination device, a mounting device, and a determination method.

[0002] A conventional determination device of this type is used in a mounting device that performs a mounting process in which a component picked up by a suction nozzle is placed on a circuit board, and is proposed to determine an abnormality in the suction posture as an abnormality in component picking by the suction nozzle (see, for example, Patent Document 1). This device determines an abnormality in the suction posture by learning using normal suction images and abnormal suction images, which are obtained by capturing images of the component picked up by the suction nozzle from below, as training data.

[0003] JP 2008-130865 A

[0004] The above-mentioned determination device determines whether the component is in a pickup position abnormality using images captured only from below as training data, and therefore is unable to accurately determine whether the component is in a pickup position abnormality. Therefore, there is a need to improve the determination accuracy. Furthermore, the above-mentioned Patent Document 1 does not disclose determining placement abnormalities, such as the component being picked up by the suction nozzle and then taken away. There is a need to improve the determination accuracy in determining such placement abnormalities as well.

[0005] The main purpose of the determination device, mounting device, and determination method disclosed herein is to improve the accuracy of determining whether a collection unit has abnormally collected or placed an object.

[0006] The determination device, mounting device, and determination method of the present disclosure employ the following means to achieve the above-mentioned main object.

[0007] A first determination device of the present disclosure is used in an implementation device that executes an implementation process in which an object collected by a collection unit is placed on a base, and is a determination device that determines whether the object has been abnormally collected by the collection unit, and is equipped with: a machine learning unit that performs machine learning using training data that associates a collection image of the object collected by the collection unit from a collection direction, the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersection direction that intersects the collection direction; and a determination unit that, during the implementation process, acquires the collection image, the collection pressure, and the collection position, and determines the collection abnormality based on the machine learning.

[0008] The first determination device of the present disclosure performs machine learning using training data that associates a collection image of an object collected by a collection unit from a collection direction, the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from a cross direction that intersects the collection direction. The machine learning uses training data that associates the collection image, the cross image, the collection pressure, and the collection position. This allows the first determination device to improve machine learning performance. Then, during implementation processing, the collection image, the collection pressure, and the collection position are acquired, and a collection abnormality is determined based on the machine learning. As a result, the first determination device can improve the accuracy of determining whether the object is collected by the collection unit.

[0009] 1 is a block diagram showing an outline of the configuration of the mounting system 1. FIG. 1 is a block diagram showing an outline of the configuration of the rotary head 64 and OFC 70. FIG. 1 is a layout diagram showing an example of the arrangement of the rotary head 64 and OFC 70. FIG. 2 is a block diagram showing the electrical connection relationship between the control device 100 and the management device 110. FIG. 3 is an explanatory diagram showing an example of a picked image Gp captured by the OFC 70. FIG. 4 is a block diagram showing an outline of the configuration around the rotary head 64 of the mounting head 160. FIG. 5 is a layout diagram showing an example of the arrangement around the rotary head 64 of the mounting head 160. FIG. 6 is an explanatory diagram showing an example of the intersection image Gc captured by the side camera 76. FIG. 7 is a block diagram showing the electrical connection relationship between the control device 100 and the management device 110. FIG. 8 is a flowchart showing an example of a learning mounting processing routine. FIG. 9 is a flowchart showing an example of a picking-time teacher data creation process. FIG. 10 is an explanatory diagram of the intersection image Gc when a picking abnormality occurs. FIG. 11 is an explanatory diagram of the change over time in the picking position Hpo and the picking pressure Dpr of the suction nozzle 61. FIG. 12 is a flowchart showing an example of a placement-time teacher data creation process. 10 is an explanatory diagram of an intersection image Gc when an arrangement is abnormal. FIG. 11 is an explanatory diagram showing an example of placement training data Dt2. FIG. 12 is a flowchart showing an example of a post-exchange execution processing routine.

[0010] Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram showing an outline of the configuration of a mounting system 1. FIG. 2 is a configuration diagram showing an outline of the configuration of a rotary head 64 and an OFC 70. FIG. 3 is a layout diagram showing an example of the arrangement of the rotary head 64 and the OFC 70. FIG. 4 is a block diagram showing the electrical connection relationship between the control device 100 and the management device 110. FIG. 5 is an explanatory diagram showing an example of a sampled image Gp captured by the OFC 70. Note that in this embodiment, the left-right direction (X-axis), front-back direction (Y-axis), and up-down direction (Z-axis) are as shown in FIGS. 1 to 4.

[0011] The mounting system 1 includes a mounting device 10 that mounts a component P as an object on a substrate S as a base, and a management device 110 that manages the entire system. In this embodiment, the left-right direction in Fig. 1 is the X-axis direction, the front-rear direction is the Y-axis direction, and the up-down direction is the Z-axis direction.

[0012] 1, the mounting apparatus 10 includes a component supply device 20 equipped with reels 22a and trays that store components P, a board transport device 30 that transports a board S, a backup device 40 that backs up the transported board S, a mounting head 60 that picks up components P with suction nozzles 61 and mounts them on the board S, an XY robot 50 that moves the mounting head 60 in the X and Y directions, and a control device 100 that controls the entire mounting machine. In addition to these, the mounting apparatus 10 also includes a parts camera 90 that takes images of the components P picked up by the suction nozzles 61 from below, a mark camera 74 that takes images of board positioning reference marks attached to the board S from above, and a nozzle stocker 94 that stocks multiple types of suction nozzles 61 that can be replaced according to the size of the components P to be mounted.

[0013] 1, the component supply device 20 includes a tape feeder 22 detachably attached to the front side of the housing 12. The tape feeder 22 includes a reel 22a around which a tape is wound, on which components P are stored at predetermined intervals, and feeds the components P to a component supply position by driving a drive motor (not shown) to pull out the tape from the reel 22a.

[0014] 1, the substrate transport device 30 includes a belt conveyor device 32, and transports the substrate S from left to right in Fig. 1 by driving the belt conveyor device 32. A backup device 40 is provided in the center of the substrate transport device 30 in the left-right direction, for backing up the transported substrate S from the back surface side.

[0015] As shown in FIG. 1, the XY robot 50 includes a guide rail 56 provided on the top of the device along the Y-axis direction, a Y-axis slider 58 that can move along the guide rail 56, a guide rail 52 that is provided on the side of the Y-axis slider 58 along the X-axis direction, and an X-axis slider 54 that can move along the guide rail 52 and has a mounting head 60 attached.

[0016] The mounting head 60 is configured to be detachable, and as shown in Figures 2 to 4, is equipped with a rotary head 64 in which a plurality of nozzle holders 62 holding suction nozzles 61 are arranged at predetermined angular intervals (for example, 45 degree intervals) on a circumference coaxial with the rotation axis, an R-axis actuator 66 that rotates the rotary head 64, a Z-axis actuator 67 that moves the nozzle holders 62 up and down, i.e., in the direction in which the component P is picked up, a Q-axis actuator 68 that rotates each nozzle holder 62 individually, and an on-the-fly camera (OFC) 70 that captures images from below of one suction nozzle 61 and the suction posture of the component P picked up by the suction nozzle 61.

[0017] The R-axis actuator 66 intermittently drives a drive motor (not shown) by a predetermined angle (for example, 45 degrees) to intermittently rotate the rotary head 64 by the predetermined angle, thereby causing each suction nozzle 61 arranged on the rotary head 64 to revolve by the predetermined angle in the circumferential direction.

[0018] The Z-axis actuator 67 drives a drive motor (not shown) to rotate, thereby moving the nozzle holder 62 in the Z-axis direction, whereby the suction nozzle 61 moves up and down.

[0019] The Q-axis actuator 68 includes a drive gear that meshes with a gear provided on the cylindrical outer periphery of the nozzle holder 62 and a drive motor connected to the rotation shaft of the drive gear, and rotates the plurality of nozzle holders 62 individually around their axes by rotation of the drive motor, thereby causing the suction nozzles 61 to rotate individually.

[0020] The OFC 70 is provided at the bottom of the mounting head 60 and includes a camera body 71 incorporating multiple image sensors, such as CCDs and CMOSs, and an optical system 72 for focusing an image on the image sensors. The optical system 72 includes multiple mirrors that refract light incident from an entrance on the lower side of the suction nozzle 61 and couple it to the image sensors of the camera body 71. When the OFC 70 captures an image while the suction nozzle 61 is suctioning a component P, it can obtain a captured image Gp captured from the downward direction, i.e., the direction in which the suction nozzle 61 picked up the component P, as shown in FIG. 5 . Because the area of ​​the component P occupies a larger portion of the imaging area in the OFC 70 than in the part camera 90, high-resolution component images can be obtained with fewer image sensors. The OFC 70 has fewer image sensors than the part camera 90, making it lighter and more compact.

[0021] 4, the control device 100 is configured as a microprocessor centered around a CPU 101, and in addition to the CPU 101, includes a ROM 102, a HDD 103, a RAM 104, an input / output interface (I / F) 105, and the like. These are connected via a bus 106. The control device 100 receives, via the I / O interface 105, image signals from the OFC 70, the mark camera 74, and the part camera 90, and a suction pressure Dno from a suction pressure sensor 61a that is built into the suction nozzle 61 and detects the pressure inside the suction nozzle 61 as a suction pressure. The X-axis slider 54 and the Y-axis slider 58 are each equipped with a position sensor, and the control device 100 also receives position information from these position sensors. The control device 100 also outputs drive signals to the component supply device 20, the substrate transport device 30, the backup device 40, the X-axis actuator 55 that moves the X-axis slider 54, the Y-axis actuator 59 that moves the Y-axis slider 58, the R-axis actuator 66, the Z-axis actuator 67, the Q-axis actuator 68, and the solenoid valve 79 that connects and disconnects the vacuum pump (not shown) from the suction nozzle 61, via the input / output interface 105. The control device 100 also calculates the nozzle position Ho of the suction nozzle 61 relative to a predetermined position of the mounting device 10 based on position information from position sensors mounted on the X-axis slider 54 and the Y-axis slider 58 and the rotation angles of the drive motors mounted on the R-axis actuator 66, the Z-axis actuator 67, and the Q-axis actuator 68.

[0022] The management device 110 is, for example, a general-purpose computer. As shown in FIG. 4 , the management device 110 includes a CPU 111, a ROM 112, an HDD 113 for storing board production data, a RAM 114, and an input / output interface (I / F) 115. These components are connected via a bus 116. The management device 110 receives input signals from an input device 117, such as a mouse or keyboard, via the I / F 115. The management device 110 also outputs image signals to a display 118 via the I / F 115. Here, the board production data is data that defines which components P are to be mounted on the board in the mounting device 10, in what order, and how many boards S with the components P mounted thereon are to be produced. This production data is input in advance by an operator and transmitted from the management device 110 to the mounting device 10 when production begins.

[0023] In this mounting device 10, when the later-described picking machine learning and placement machine learning have not been completed, a mounting head 160 that is used for machine learning during production is attached in place of the mounting head 60. Here, the configuration of the mounting head 160 will be described.

[0024] FIG. 6 is a configuration diagram showing an outline of the configuration of the rotary head 64, OFC 70, and side camera 76 of the mounting head 160. FIG. 7 is a layout diagram showing an example of the arrangement of the rotary head 64, OFC 70, and side camera 76 of the mounting head 160. FIG. 8 is an explanatory diagram showing an example of an intersection image Gc captured by the side camera 76. FIG. 9 is a block diagram showing the electrical connection relationship between the control device 100 and the management device 110 when the mounting head 160 is attached. In the mounting head 160, the same configuration as in the mounting head 60 is assigned the same reference numerals, and detailed description thereof will be omitted.

[0025] The mounting head 160 is configured to be detachable, and includes a rotary head 64 in which a plurality of nozzle holders 62 holding suction nozzles 61 are arranged at predetermined angular intervals (for example, 180 degrees) on a circumference coaxial with the rotation axis, an R-axis actuator 66, a Z-axis actuator 67, a Q-axis actuator 68, an OFC 70, and a side camera 76. The R-axis actuator 66, the Z-axis actuator 67, the Q-axis actuator 68, and the OFC 70 have the same configuration as the R-axis actuator 66, the Z-axis actuator 67, the Q-axis actuator 68, and the OFC 70 of the mounting head 60, and therefore detailed description thereof will be omitted.

[0026] The side camera 76 is provided at the bottom of the mounting head 160 and has a built-in camera body that incorporates multiple imaging elements such as CCDs and CMOSs. When the side camera 76 captures an image while the suction nozzle 61 is suctioning a component P, it can obtain an intersection image Gc captured from the left-right direction, i.e., from an intersection direction that intersects with the pickup direction of the component P picked up by the suction nozzle 61, as shown in FIG.

[0027] As shown in FIG. 9, the control device 100 inputs the OFC 70 from the mounting head 160 instead of the mounting head 60 , the mark camera 74 , image signals from the side camera 76 , and the like via an input / output interface 105 .

[0028] Next, the operation of the mounting device 10 configured as described above will be described. The mounting device 10 first performs pick-up machine learning and placement machine learning while mounting components P on a board S with the mounting head 160 attached. Figure 10 is a flowchart showing an example of a learning mounting processing routine executed by the CPU 101 of the control device 100. The learning mounting processing routine is executed when a signal indicating an instruction to start a mounting operation is received from the management device 110 with the mounting head 160 attached.

[0029] In the learning mounting process routine, the CPU 101 of the control device 100 first controls the board transport device 30 to perform a board transport process for transporting the board S (S100). Next, the CPU 101 controls the X-axis actuator 55 and the Y-axis actuator 59 to move the rotary head 64 to the component supply position of the component supply device 20 (S110). The CPU 101 then performs a suction operation to have the suction nozzle 61 pick up the component P (S120). During this suction operation, the CPU controls the R-axis actuator 66 and the Z-axis actuator 67 to lower the suction nozzle 61 while rotating the suction nozzle 61 to a predetermined mounting position, and controls the solenoid valve 79 to apply negative pressure to the suction nozzle 61 to pick up the component P. After performing the suction operation, the CPU 101 determines whether there is a next component P to be picked up (S130). If the CPU 101 determines that there is a next component P to be picked up, it returns to S120 and repeats the pick-up operation, and if it determines that there is no next component P to be picked up, it proceeds to the process of S140.

[0030] Next, the CPU 101 controls the X-axis actuator 55 and the Y-axis actuator 59 so that the rotary head 64 moves directly above the mounting position on the board S, thereby moving the mounting head 160 above the board S (S140).

[0031] Next, the CPU 101 executes a pick-up teacher data creation process to create teacher data used to learn the pick-up orientation of the component P by the suction nozzle 61 (S150). FIG. 11 is a flowchart showing an example of the pick-up teacher data creation process executed by the CPU 101 of the control device 100. In the teacher data creation process, the CPU 101 of the control device 100 first captures an image using the side camera 76 to obtain a cross image Gc (S300), and determines whether a pick-up abnormality has occurred, i.e., an abnormality in the orientation of the component P, based on the cross image Gc (S310). FIG. 12 is an explanatory diagram of the cross image Gc when a pick-up abnormality has occurred. In the figure, the dashed line indicates the component P when no pick-up abnormality has occurred. When a pick-up abnormality has occurred, as shown in the figure, the length Th from the tip of the suction nozzle 61 to the bottom end of the component P in the cross image Gc becomes longer than a reference value Thref, which corresponds to the thickness of the component P in the cross image Gc. In S310, the CPU 101 compares the length Th with a reference value Thref and determines that a collection abnormality has occurred if the length Th is greater than the reference value Thref. Next, the CPU 101 captures an image using the OFC 70 to acquire a collection image Gp (S320). Next, the CPU 101 acquires the suction pressure Dno from the suction pressure sensor 61a as the collection pressure Dpr at which the suction nozzle 61 picked up the component P (S330). Next, the CPU 101 acquires the calculated nozzle position Ho of the suction nozzle 61 as the collection position Hpo at which the suction nozzle 61 picked up the component P (S340). The CPU 101 then creates collection-time teacher data Dt1 as teacher data that associates the judgment result of S310, which is information based on the intersection image Gc, with the collected image Gp, the collection pressure Dpr, and the collection position Hpo, and stores this data in RAM 104 (S350), thereby terminating the collection-time teacher data creation process. 13 is an explanatory diagram showing an example of the collection time training data Dt1. As shown in the figure, the collection time training data Dt1 is created as data that associates the determination result of S310, the collected image Gp, the collection pressure Dpr, and the collection position Hpo.

[0032] When the CPU 101 finishes the collection-time teacher data creation process, it determines whether the number Nt1 of collection-time teacher data Dt1 stored in RAM 104 is equal to or greater than a predetermined number Nt1ref (S160). The predetermined number Nt1ref is a number determined in advance by experiment, analysis, or the like as the number of collection-time teacher data Dt1 stored in RAM 104 at which the accuracy of the collection-time machine learning in S170 described below is equal to or greater than a predetermined value.

[0033] If the number Nt1 is less than the predetermined number Nt1ref in S160, the CPU 101 executes the processing from S190 onward. If the number Nt1 is equal to or greater than the predetermined number Nt1ref in S160, the CPU 101 executes pick-up machine learning, which uses the pick-up training data Dt1 stored in the RAM 104 as input and a neural network to output whether or not the component P has an abnormal posture (S170). Here, a neural network is used for the machine learning, but other techniques such as deep learning and fuzzy logic may also be used. FIG. 14 is an explanatory diagram illustrating an example of the time changes in the pick-up position Hpo and the pick-up pressure Dpr when the component P is placed on the board S. When no placement abnormality occurs, when the component P is picked up, the pick-up position Hpo descends while the pick-up pressure Dpr is negative, and then the pick-up pressure Dpr becomes positive, causing the component P to detach from the suction nozzle 61 and be placed on the board S. Then, the pick-up position Hpo rises while the pick-up pressure Dpr is positive. In this way, since the collection pressure Dpr and collection position Hpo reflect the collection and placement state of the suction nozzle 61, by performing machine learning using the collection-time training data Dt1 that associates the judgment result, which is information based on the intersection image Gc, the collection image Gp, the collection pressure Dpr, and the collection position Hpo, the performance of collection-time machine learning can be improved compared to when the collection pressure Dpr and collection position Hpo are not used as the collection-time training data Dt1.

[0034] When the CPU 101 executes the on-collection machine learning, it notifies the management device 110 of the end of the on-collection machine learning by sending an end signal indicating that the on-collection machine learning has ended to the management device 110 via the input / output interface 105 (S180), and executes the processing from S190 onwards. Upon receiving the end signal, the management device 110 displays on the display 118 that the on-collection machine learning has ended, thereby notifying the operator of the mounting device 10. The operator can confirm that the on-collection machine learning has ended by checking this display.

[0035] Next, the CPU 101 determines whether a pickup abnormality has occurred in S310 (S190). If a pickup abnormality has occurred, the CPU 101 discards the component P (S260), notifies the worker of this information (S270), and proceeds to processing in S280. The worker can check the information reported on the display 118 or the like and perform the necessary work. The information reported to the worker includes information such as the occurrence of a pickup abnormality and the need to discard the component. If a pickup abnormality has not occurred, the CPU 101 executes the mounting operation (S200). In the mounting operation, the CPU 101 controls the R-axis actuator 66 and the Z-axis actuator 67 to lower the suction nozzle 61 while rotating the suction nozzle 61, and waits for the component P picked up by the suction nozzle 61 to come into contact with the board S. The CPU 101 then controls the solenoid valve 79 to apply positive pressure to the suction nozzle 61 to mount the component P on the board S, and controls the Z-axis actuator 67 to raise the suction nozzle 61.

[0036] Upon completing the mounting operation, the CPU 101 executes a placement training data creation process to create training data used to learn how to place the component P using the suction nozzle 61 (S210). FIG. 15 is a flowchart illustrating an example of the placement training data creation process executed by the CPU 101 of the control device 100. In the placement training data creation process, the CPU 101 captures an image using the side camera 76 to acquire a cross image Gc (S400) and determines whether a placement abnormality has occurred, i.e., an abnormality in carrying the component P home, based on the cross image Gc (S410). FIG. 16 is an explanatory diagram of the cross image Gc when a placement abnormality has occurred. When no placement abnormality has occurred, i.e., when the mounting operation is normal, no component P is present in the cross image Gc after the mounting operation, as shown in the figure. However, when a placement abnormality has occurred, a component P is present in the cross image Gc after the mounting operation, as shown in the figure. In S410, the CPU 101 determines that a placement abnormality has occurred when a component P is present in the cross image Gc after the mounting operation. Next, the CPU 101 acquires the sampling pressure Dpr (S420) and the sampling position Hpo (S430). The CPU 101 then creates placement teacher data Dt2 as teacher data that associates the determination result of S410, which is information based on the cross image Gc, with the sampling pressure Dpr and the sampling position Hpo, and saves this data in RAM 104 (S440), thereby terminating the placement teacher data creation process. FIG. 17 is an explanatory diagram showing an example of placement teacher data Dt2. As shown in the figure, the placement teacher data Dt2 is created as data that associates the determination result of S410, which is information based on the cross image Gc, with the sampling pressure Dpr and the sampling position Hpo.

[0037] When the CPU 101 finishes the placement teacher data creation process, it determines whether the number Nt2 of placement teacher data Dt2 stored in RAM 104 is equal to or greater than a predetermined number Nt2ref (S220). The predetermined number Nt2ref is a number determined in advance through experiments, analysis, etc., as the number of placement teacher data Dt2 stored in RAM 104 at which the accuracy of the placement machine learning in S230 described below is equal to or greater than a predetermined value.

[0038] When the number Nt2 is less than the predetermined number Nt2ref in S220, the CPU 101 executes the processing from S250 onward. When the number Nt2 is equal to or greater than the predetermined number Nt2ref in S220, the CPU 101 executes placement machine learning, which is machine learning using a neural network with the placement training data Dt2 stored in the RAM 104 as input, to output whether or not there is a placement abnormality in the component P (S230). Here, a neural network is used for machine learning, but other techniques such as deep learning and fuzzy logic may also be used. As described above, the pickup pressure Dpr and the pickup position Hpo reflect the pickup and placement status of the suction nozzle 61. Therefore, by executing machine learning using the placement training data Dt2 that associates the judgment result, which is information based on the intersection image Gc, with the pickup pressure Dpr and the pickup position Hpo, the CPU 101 can improve the performance of the placement machine learning compared to when the pickup pressure Dpr and the pickup position Hpo are not used as the placement training data Dt2.

[0039] When the CPU 101 executes the placement machine learning, it notifies the management device 110 of the completion of the placement machine learning by sending an end signal indicating that the placement machine learning has ended to the management device 110 via the input / output interface 105 (S240), and executes the processing from S250 onwards. Upon receiving the end signal, the management device 110 displays on the display 118 that the placement machine learning has ended, thereby notifying the operator of the mounting device 10. The operator can confirm that the placement machine learning has ended by checking this display.

[0040] Next, CPU 101 determines whether or not a placement abnormality was determined to have occurred in S410 (S250). If a placement abnormality has occurred, CPU 101 discards component P (S260), notifies the worker of this information (S270), and proceeds to processing of S280. The worker can confirm the notification on display 118 or the like and perform the necessary work. The information notified to the worker includes information such as that a placement abnormality has occurred and that the component will be discarded. CPU 101 notifies management device 110 that component P has been discarded (S270), and executes processing from S280 onward. Upon receiving the notification that component P has been discarded, management device 110 displays on display 118 that component P has been discarded. The worker can confirm the notification on display 118 and perform the necessary work. If a placement abnormality has not occurred, CPU 101 proceeds to S280 and determines whether or not there is a next component P to be mounted (S280). If the CPU 101 determines that there is a component P to be mounted next, the process returns to S140. If the CPU 101 determines that there is no component P to be mounted next, the process subsequently determines whether all components P have been mounted on the board S (S290). If not all components P have been mounted on the board S, the process returns to S110. If all components P have been mounted on the board S, the CPU 101 determines whether a stop instruction has been received from the management device 110 or whether mounting of components P on all boards S has been completed (S295). The stop instruction from the management device 110 is transmitted by a user who recognizes that the collection-time machine learning and placement-time machine learning have been completed and operates the management device 110. If the CPU 101 has not received a stop instruction from the management device 110 and mounting of components P on all boards S has not been completed, the process returns to S100. If a stop instruction has been received from the management device 110 or mounting of components P on all boards S has been completed, the CPU 101 terminates the learning mounting processing routine.

[0041] The user who operates the management device 110 to send a stop instruction to the control device 100 replaces the mounting head 160 with the mounting head 60. Next, the operation after replacing the mounting head 160 with the mounting head 60 will be described. Figure 18 is a flowchart showing an example of a post-replacement mounting processing routine executed by the CPU 101 of the control device 100. This routine is executed when the mounting head 60 is attached in place of the mounting head 160 and a signal indicating an instruction to start post-replacement mounting processing is received from the management device 110.

[0042] In the post-replacement mounting process routine, the CPU 101 of the control device 100 executes steps S100 to S140 described above to move the mounting head 60 above the board S with the suction nozzle 61 picking up the component P. Next, the OFC 70 captures an image Gp (S500), acquires the suction pressure Dno from the suction pressure sensor 61a as the pick-up pressure Dpr at which the suction nozzle 61 picked up the component P (S510), and acquires the calculated nozzle position Ho of the suction nozzle 61 as the pick-up position Hpo at which the suction nozzle 61 picked up the component P (S520). The CPU 101 then determines whether or not there is a pick-up abnormality based on the output of the pick-up machine learning performed in S170, which uses the acquired picked-up image Gp, the pick-up pressure Dpr, and the pick-up position Hpo as input data (S530). In S530, the CPU 101 determines whether or not there is a pick-up abnormality based on the pick-up machine learning performed in S170, thereby improving the accuracy of pick-up abnormality determination. Furthermore, in the mounting device 10, the determination of abnormal picking is performed using the mounting head 60 that is not equipped with the side camera 76, so that the mounting head 60 that has been made smaller can be used to determine abnormal picking.

[0043] Next, CPU 101 determines whether or not a collection abnormality was determined to have occurred in S530 (S540). If a collection abnormality has occurred, CPU 101 executes S260 described above to discard component P and proceeds to processing of S280. If a collection abnormality has not occurred, CPU 101 executes S200 described above to perform the mounting operation.

[0044] Next, the CPU 101 acquires the suction pressure Dno from the suction pressure sensor 61a as the pickup pressure Dpr at which the suction nozzle 61 picked up the component P (S550), and acquires the calculated nozzle position Ho of the suction nozzle 61 as the pickup position Hpo at which the suction nozzle 61 picked up the component P (S560). The CPU 101 then determines whether or not there is a placement abnormality from the output of the placement-time machine learning performed in S230, which uses the acquired pickup pressure Dpr and pickup position Hpo as input data (S570). Because the CPU 101 determines whether or not there is a placement abnormality using the machine learning performed in S230, the accuracy of determining placement abnormalities can be improved. Furthermore, in the mounting device 10, such placement abnormality determination can be performed using a mounting head 60 that does not have a side camera 76, and therefore placement abnormalities can be determined using a miniaturized mounting head 60.

[0045] Next, CPU 101 determines whether or not it was determined in S570 that a placement abnormality has occurred (S580). If a placement abnormality has occurred, CPU 101 executes S260 described above to discard component P, executes S280 and subsequent steps, and ends the post-replacement mounting processing routine. If a placement abnormality has not occurred, CPU 101 executes S280 and subsequent steps described above to end the post-replacement mounting processing routine.

[0046] Here, the correspondence between the components of this embodiment and the components of the present disclosure will be clarified. The suction nozzle 61 of this embodiment corresponds to an example of a collection unit of the present disclosure, the mounting device 10 of this embodiment corresponds to an example of a mounting device, and the control device 100 of this embodiment corresponds to an example of a machine learning unit and a determination unit. Note that in this embodiment, by explaining the operation of the mounting device, an example of a determination device and a determination method of the present disclosure will also be clarified.

[0047] According to the mounting device 10 of this embodiment described above, the control device 100 performs machine learning at the time of collection using training data Dt1 that associates a collection image Gp of the component P picked up by the suction nozzle 61 taken from the collection direction, the collection pressure Dpr at which the suction nozzle 61 picked up the component P, the collection position Hpo at which the suction nozzle 61 picked up the component P, and information based on an intersection image Gc of the component P picked up by the suction nozzle 61 taken from an intersection direction that intersects the collection direction, and when performing the post-replacement mounting process, the control device 100 acquires the collection image Gp, the collection pressure Dpr, and the collection position Hpo and determines a collection abnormality based on the machine learning at the time of collection, thereby improving the accuracy of determining whether the component P is picked up abnormally by the suction nozzle 61.

[0048] Furthermore, the collection abnormality includes an abnormal posture of the component P in the suction nozzle 61, and the control device 100 acquires the collection image Gp, collection pressure Dpr, and collection position Hpo after moving the mounting head 60 onto the substrate S and before the mounting operation, and determines the collection abnormality based on machine learning during collection, so that it can determine the collection abnormality including the abnormal posture of the component P.

[0049] Furthermore, the control device 100 performs machine learning using a mounting head 160 having a side camera 76 that captures the intersection image Gc, and when using a mounting head 60 that does not have a side camera 76, it acquires the collected image Gp, the collection pressure Dpr, and the collection position Hpo and determines a collection abnormality based on the machine learning at the time of collection, so that it can determine a collection abnormality when using a miniaturized mounting head 60 that does not have a side camera 76.

[0050] In addition, when the number Nt1 of collection-time training data Dt1 stored in RAM 104 becomes equal to or greater than a predetermined number Nt1ref and collection-time machine learning is terminated, the control device 100 can notify the user that collection-time machine learning has ended by notifying the management device 110 of the end of collection-time machine learning.

[0051] According to the mounting device 10 of this embodiment, the control device 100 performs placement machine learning using placement training data Dt2 that associates the pickup pressure Dpr at which the suction nozzle 61 picked up the component P, the pickup position Hpo at which the suction nozzle 61 picked up the component P, and information based on an intersection image Gc captured of the component P picked up by the suction nozzle 61 from an intersecting direction that intersects the pickup direction, and when executing the post-replacement mounting process, acquires the pickup pressure Dpr and the pickup position Hpo and determines placement abnormalities based on the placement machine learning, thereby improving the accuracy of determining placement abnormalities of the component P by the suction nozzle 61.

[0052] Furthermore, the control device 100 performs machine learning at the time of placement using a mounting head 160 having a side camera 76 that captures the intersection image Gc, and when using a mounting head 60 that does not have a side camera 76, it acquires the sampling pressure Dpr and the sampling position Hpo and determines a placement abnormality based on the machine learning at the time of placement, so that it can determine a placement abnormality when using a miniaturized mounting head 60 that does not have a side camera 76.

[0053] In addition, when the number Nt2 of placement-time training data Dt2 stored in RAM 104 becomes equal to or greater than a predetermined number Nt2ref and the placement-time machine learning is terminated, the control device 100 can notify the user that the placement-time machine learning has been terminated by notifying the management device 110 of the termination of the placement-time machine learning.

[0054] In the above-described embodiment, the pickup abnormality is determined by acquiring the picked image Gp, the picking pressure Dpr, and the picking position Hpo after the mounting head 60 has been moved above the board S and before the mounting operation, including an abnormality in the orientation of the component P in the suction nozzle 61. However, since the pickup abnormality can be determined by acquiring the picked image Gp, the picking pressure Dpr, and the picking position Hpo at a predetermined time included in the period from when the component P is picked up by the suction nozzle 61 to when the mounting head 60 is placed on the board S, for example, the pickup abnormality may be determined by acquiring the picked image Gp, the picking pressure Dpr, and the picking position Hpo before the mounting head 60 is moved above the board S after the component P has been picked up by the suction nozzle 61.

[0055] In the above-described embodiment, when using a mounting head 60 that does not have a side camera 76, the collected image Gp, the collected pressure Dpr, and the collected position Hpo are acquired, and a collection abnormality is determined based on machine learning during collection. However, the collected image Gp, the collected pressure Dpr, and the collected position Hpo may be acquired using a mounting head 160 that has a side camera 76, and a collection abnormality may be determined based on machine learning during collection.

[0056] In the above-described embodiment, when using a mounting head 60 that does not have a side camera 76, the sampling pressure Dpr and the sampling position Hpo are acquired and a placement abnormality is determined based on placement machine learning. However, the sampling pressure Dpr and the sampling position Hpo may be acquired using a mounting head 160 that has a side camera 76 and a placement abnormality may be determined based on placement machine learning.

[0057] In the above-described embodiment, the control device 100 notifies the management device 110 of the completion of collection-time machine learning and placement-time machine learning. However, if the control device 100 is equipped with a display, a speaker, or the like, the display, the speaker, or the like may be used to notify the user of the completion of collection-time machine learning and placement-time machine learning by displaying a message, sounding an alarm, or making a sound. Furthermore, when the management device 110 is notified of the completion of collection-time machine learning and placement-time machine learning by the control device 100, the management device 110 displays this on the display 118, but the management device 110 may also notify the user of the completion of collection-time machine learning and placement-time machine learning by making an alarm or making a sound using a speaker, or the like.

[0058] In the above embodiment, the control device 100 creates the training data Dt1 at the time of collection as training data that associates the determination result of S310, which is information based on the cross image Gc, with the collected image Gp, the collection pressure Dpr, and the collection position Hpo. However, the information based on the cross image Gc may include the cross image Gc itself, the length Th in the cross image Gc, the judgment value Thref, etc., instead of the determination result of S310 or in addition to the determination of S310.

[0059] In the above embodiment, the control device 100 creates the training data Dt1 at the time of collection as training data that associates the determination result of S410, which is information based on the cross image Gc, with the collected image Gp, the collection pressure Dpr, and the collection position Hpo. However, the information based on the cross image Gc may include the cross image Gc itself instead of or in addition to the determination result of S410.

[0060] In the above-described embodiment, the control device 100 performs collection machine learning and placement machine learning, determines collection abnormalities based on the collection machine learning, and determines placement abnormalities based on the placement machine learning. However, it is also possible to perform at least one of collection machine learning and placement machine learning, and determine at least one of collection abnormalities and placement abnormalities based on the executed machine learning. If only placement machine learning is performed without collection machine learning and placement abnormalities are determined based on the placement machine learning, the mounting heads 60, 160 do not need to be equipped with an OFC 70.

[0061] In the above-described embodiment, the mounting heads 60 and 160 are provided with suction nozzles 61. However, the suction nozzles 61 of the mounting heads 60 and 160 may be any type that can pick up the components P, and may be a mechanical chuck or the like.

[0062] The first and second determination devices of the present disclosure may be configured as follows.

[0063] A second determination device of the present disclosure is used in an implementation device that executes an implementation process of placing an object collected by a collection unit on a base, and determines whether the object has been abnormally placed by the collection unit, and is summarized as comprising: a machine learning unit that performs machine learning using training data that associates the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersection direction that intersects with the collection direction; and a determination unit that, when the implementation process is executed, acquires the collection pressure and the collection position, and determines the abnormal placement based on the machine learning.

[0064] In the second determination device disclosed herein, machine learning is performed using training data that associates the collection pressure at which the collection unit collects the object, the collection position at which the collection unit collects the object, and information based on an intersection image captured from a cross direction intersecting the collection direction of the object collected by the collection unit. The machine learning uses training data that associates the collection pressure and collection position in addition to the cross image. This allows the second determination device to improve machine learning performance. Then, during the implementation process, the collection pressure and collection position are acquired and a placement abnormality is determined based on the machine learning. As a result, the second determination device can improve the accuracy of the determination of an object placement abnormality by the collection unit.

[0065] The mounting device of the present disclosure includes a mounting unit that performs a mounting process of placing an object collected by a collection unit on a base, and a machine learning unit that performs machine learning using teacher data that associates a collection image of the object collected by the collection unit from a collection direction, a collection pressure at which the collection unit collected the object, a collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersection direction that intersects the collection direction ... position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersection direction that intersects the collection direction, and or a determination device used in an implementation device that executes an implementation process that places an object collected by a collection unit on a base and determines an abnormality in placement of the object by the collection unit, the determination device comprising: a machine learning unit that executes machine learning using teacher data that associates the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersecting direction that intersects the collection direction; and a determination unit that acquires the collection pressure and the collection position during execution of the implementation process and determines the abnormality in placement based on the machine learning.

[0066] The mounting device of the present disclosure is equipped with the first and second determination devices of the present disclosure described above, and therefore achieves the effects achieved by the first and second determination devices of the present disclosure, namely, the effect of being able to improve the accuracy of determining whether the collection unit has abnormally collected the object, or the effect of being able to improve the accuracy of determining whether the collection unit has abnormally placed the object.

[0067] A first determination method of the present disclosure is used in an implementation device that executes an implementation process in which an object collected by a collection unit is placed on a base, and is a determination method in which a computer determines whether the object has been abnormally collected by the collection unit, and includes the following steps: (a) performing machine learning using training data that associates a collection image of the object collected by the collection unit from a collection direction, the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersection direction that intersects the collection direction; and (b) acquiring the collection image, the collection pressure, and the collection position during the implementation process, and determining the collection abnormality based on the machine learning.

[0068] In the first determination method disclosed herein, machine learning is performed using training data that associates a collection image of an object collected by a collection unit from a collection direction, the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from a cross direction that intersects the collection direction. The machine learning uses training data that associates the collection image, the cross image, the collection pressure, and the collection position. This allows the first determination method to improve the performance of machine learning. Then, during the implementation process, the collection image, the collection pressure, and the collection position are acquired, and a collection abnormality is determined based on the machine learning. As a result, the first determination method allows for improved accuracy in determining whether the object has been collected by the collection unit.

[0069] A second determination method of the present disclosure is a determination method used in a mounting device that executes a mounting process of placing an object collected by a collection unit on a base, and in which a computer determines an abnormal placement of the object by the collection unit, and includes the following steps: (c) performing machine learning using training data that associates the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from an intersecting direction that intersects with the collection direction; and (d) acquiring the collection pressure and the collection position during the execution of the mounting process, and determining the abnormal placement based on the machine learning.

[0070] In the second determination method disclosed herein, machine learning is performed using training data that associates the collection pressure at which the collection unit collects the object, the collection position at which the collection unit collects the object, and information based on an intersection image captured from a cross direction intersecting the collection direction of the object collected by the collection unit. The machine learning uses training data that associates the collection pressure and collection position in addition to the cross image. This allows the second determination method to improve the performance of machine learning. Then, during the implementation process, the collection pressure and collection position are acquired, and a placement abnormality is determined based on the machine learning. As a result, the second determination method allows for improved accuracy in determining placement abnormalities of objects by the collection unit.

[0071] This specification also discloses the technical idea of ​​changing "the determination device according to claim 1 or 2" in claim 5 as originally filed to "the determination device according to any one of claims 1 to 4," the technical idea of ​​changing "the determination device according to claim 6 or 7" in claim 10 as originally filed to "the determination device according to any one of claims 6 to 9," and the technical idea of ​​changing "the determination device according to claim 1 or 6" in claim 11 as originally filed to "the determination device according to any one of claims 1 to 10."

[0072] The present invention can be used in the manufacturing industry of determination devices and mounting devices.

[0073] 1 Mounting system, 10 Mounting device, 12 Housing, 20 Component supply device, 22 Tape feeder, 22a Reel, 30 Board transport device, 32 Belt conveyor device, 40 Backup device, 50 XY robot, 52 Guide rail, 54 X-axis slider, 55 X-axis actuator, 56 Guide rail, 58 Y-axis slider, 59 Y-axis actuator, 60, 160 Mounting head, 61 Suction nozzle, 61a Suction pressure sensor, 62 Nozzle holder, 64 Rotary head, 66 R-axis actuator, 67 Z-axis actuator, 68 Q-axis actuator, 70 On-the-fly camera (OFC), 71 Camera body, 72 Optical system, 74 Mark camera, 76 Side camera, 79 Solenoid valve, 90 Parts camera, 94 Nozzle stocker, 100 Control device, 101 CPU, 102 ROM, 103 HDD, 104 RAM, 105 input / output interface, 106 bus, 110 management device, 111 CPU, 112 ROM, 113 HDD, 114 RAM, 115 input / output interface, 116 bus, 117 input device, 118 display, Dt1 training data at the time of collection, Dt2 training data at the time of placement, Gc intersection image, Gp collected image, S board.

Claims

1. A judgment device used in a mounting device that executes a mounting process in which an object collected by a collection unit is placed on a base, and that judges abnormal collection of the object by the collection unit, comprising: a machine learning unit that executes machine learning using teacher data that associates a collection image of the object collected by the collection unit from a collection direction, the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from a cross direction that crosses the collection direction; and a judgment unit that acquires the collection image, the collection pressure, and the collection position during the execution of the mounting process, and judges the collection abnormality based on the machine learning.

2. A determination device as described in claim 1, wherein the collection abnormality includes an abnormality in the posture of the object in the collection unit, and the determination unit acquires the collection image, the collection position, and the collection pressure at a predetermined time included in the period from when the object is collected by the collection unit to when it is placed on the base.

3. A judgment device as described in claim 1 or 2, wherein the machine learning unit executes the machine learning using a first mounting head having a cross imaging unit that captures the cross image, and the judgment unit acquires the sampled image, the sampled pressure, and the sampled position when using a second mounting head that does not have the cross imaging unit, and judges the sampled abnormality based on the machine learning.

4. A judgment device as described in claim 3, wherein the machine learning unit executes the machine learning using the first mounting head having a collection imaging unit that captures the collected image, and the judgment unit acquires the collected image, the collection position and the collection pressure when using the second mounting head having the collection imaging unit, and judges the collection abnormality based on the machine learning.

5. A determination device as claimed in claim 1 or 2, comprising an alarm unit that notifies information, wherein the machine learning unit, when the accuracy of the machine learning reaches or exceeds a predetermined value and the machine learning is terminated, causes the alarm unit to notify information that the machine learning is terminated.

6. A judgment device used in a mounting device that executes a mounting process in which an object collected by a collection unit is placed on a base, and that judges abnormal placement of the object by the collection unit, comprising: a machine learning unit that executes machine learning using teacher data that associates the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image captured of the object collected by the collection unit from a cross direction that crosses the collection direction; and a judgment unit that acquires the collection pressure and the collection position when the mounting process is executed, and judges the abnormal placement based on the machine learning.

7. A determination device as described in claim 6, wherein the placement abnormality includes an abnormality in the collection unit bringing back the object placed on the base, and the determination unit acquires the collection position and the collection pressure at a predetermined time included in the period from when the object is placed on the base to when the collection unit starts to collect the next object.

8. A judgment device as described in claim 6 or 7, wherein the machine learning unit executes the machine learning using a first mounting head having a cross imaging unit that captures the cross image, and the judgment unit acquires the sampling position and the sampling pressure when using a second mounting head that does not have the cross imaging unit, and judges the placement anomaly based on the machine learning.

9. A determination device as described in claim 8, wherein the machine learning unit executes the machine learning using the first mounting head having a collection imaging unit that captures a collection image captured from the collection direction, and the determination unit acquires the collection position and the collection pressure when using the second mounting head having the collection imaging unit, and determines the placement anomaly based on the machine learning.

10. A determination device as described in claim 6 or 7, comprising an alarm unit that notifies information, wherein the machine learning unit, when the accuracy of the machine learning reaches a predetermined value or greater and the machine learning is terminated, causes the alarm unit to notify information that the machine learning is terminated.

11. A mounting device comprising: a mounting unit that executes a mounting process for placing an object collected by the collection unit on a base; and a determination device according to claim 1 or 6.

12. A mounting device according to claim 11, wherein the collection unit collects the object using a suction nozzle that adsorbs the object using negative pressure, and the machine learning unit performs the machine learning using the suction pressure of the suction nozzle as the collection pressure.

13. A method for use in an implementation device that executes an implementation process in which an object collected by a collection unit is placed on a base, the method comprising: (a) a step of performing machine learning using teacher data that associates a collection image of the object collected by the collection unit from a collection direction, the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image of the object collected by the collection unit from a cross direction that crosses the collection direction; and (b) a step of acquiring the collection image, the collection pressure, and the collection position during the implementation process, and determining the collection abnormality based on the machine learning.

14. A method for use in a mounting device that performs a mounting process in which an object collected by a collection unit is placed on a base, the method comprising: (c) a step of performing machine learning using training data that associates the collection pressure at which the collection unit collected the object, the collection position at which the collection unit collected the object, and information based on an intersection image captured of the object collected by the collection unit from a cross direction that crosses the collection direction; and (d) a step of acquiring the collection pressure and the collection position during the execution of the mounting process, and determining the placement abnormality based on the machine learning.

Images