Component mounting system and image classification method

Abstract

This component mounting system comprises: at least one camera capable of imaging at least one of a component pickup state and a component mounting state, and a component supply position; an error detection unit which controls the camera so that an image of at least one of the pickup state and the mounting state can be obtained during substrate production, and which is capable of executing at least one of an error detection process based on the pickup state image and an error detection process based on the mounting state image; an inspection unit; and a classification unit. The inspection unit applies a pre-attraction operation image to a trained model, which is obtained by machine learning using a plurality of pre-attraction operation images as input data and the presence or absence of a component in the component supply position as training data, to perform an inspection for the presence or absence of a component in the component supply position. Unless an error is detected, the classification unit during the machine learning classifies a pre-attraction operation image as training data indicating that a component is present.

Description

[Technical Field]

[0001] This specification discloses a component mounting system and an image classification method. [Background technology]

[0002] Conventionally, there is known a component mounter that captures an image of a tape having multiple cavities that can accommodate components and checks whether the cavities are free of components based on the image. For example, Patent Document 1 discloses a component mounter that captures an image for determining whether a component is present, acquires features from the image, inputs the acquired features into a trained model, and determines whether a component is present in the cavity based on the output result of the trained model. This trained model is created by acquiring features from images of cavities in which the presence or absence of components has been preset, and learning using the combinations of the features and the presence or absence of components as training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 205578 Summary of the Invention [Problem to be solved by the invention]

[0004] However, to create a highly accurate trained model, it is necessary to prepare a large amount of training data (combinations of cavity images and the presence or absence of parts). If the training data is to be classified by an operator, the operator must visually classify a large amount of training data, which places a heavy burden on the operator.

[0005] The main purpose of the present disclosure is to make it easier to obtain training data used to create a trained model in a component mounting system that can determine the presence or absence of a component in a cavity. [Means for solving the problem]

[0006] The present disclosure has adopted the following means to achieve the above-mentioned main object.

[0007] The component mounting system of the present disclosure includes: a mounting machine body having a head holding a picking member capable of picking up components supplied from a feeder to a component supply position and a head moving device for moving the head, and capable of mounting the components picked up by the picking member onto a board; one or more cameras capable of capturing images of at least one of a state of component pickup relative to the pickup member and a state of component mounting on a board, and the component supply position; a production control unit that controls the head and the head moving device so that a picking operation of picking up components with the picking member and a mounting operation of mounting the components picked up by the picking member onto a board are performed, and that controls the camera to obtain an image of at least one of the picked-up state of the components with respect to the picking member after the picking operation and the mounted state of the components on the board after the mounting operation, thereby producing the board; an error detection unit capable of executing at least one of an error detection process for detecting a collection error based on the captured image of the collection state during production of the board and an error detection process for detecting a mounting error based on the captured image of the mounting state; an imaging processing unit that images the component supply position with the camera before the picking operation; an inspection unit that inspects the presence or absence of a component at the component supply position by applying the captured image of the component supply position before the picking operation acquired by the imaging processing unit to a trained model obtained by machine learning using a plurality of captured images of the component supply position before the picking operation as input data and the presence or absence of a component at the component supply position as training data; a classification unit that, if no error is detected by the error detection unit during the machine learning, classifies the captured image of the component supply position before the picking operation into an image to be used as the training data of a component present, and, if the error detection unit detects the error, classifies the captured image of the component supply position before the picking operation acquired by the imaging processing unit into an image to be not used as the training data of a component present; The gist of the project is to provide the following:

[0008] In the component mounting system disclosed herein, if no errors are detected by the error detection unit during machine learning, the captured image of the component supply position before the picking operation is classified as an image to be used as training data. Therefore, compared to when an operator visually classifies training data, it is easier to obtain training data with components present. Furthermore, if an error is detected by the error detection unit, the captured image of the component supply position is likely to be unsuitable as training data with components present. Therefore, it is highly significant to classify the captured image of the component supply position before the picking operation as an image not to be used as training data with components present.

[0009] The image classification method of the present disclosure includes: a mounting machine body having a head holding a picking member capable of picking up components to be supplied from a feeder to a component supply position, and a head moving device for moving the head, and capable of mounting the components picked up by the picking member onto a board; one or more cameras capable of capturing images of at least one of a picking state of the components relative to the picking member and a mounting state of the components onto the board, and the component supply position; and controlling the head and the head moving device so that a picking operation of picking up components with the picking member and a mounting operation of mounting the components picked up by the picking member onto the board are performed, and moving the camera so as to obtain a captured image of at least one of a picking state of the components relative to the picking member after the picking operation and a mounting state of the components onto the board after the mounting operation. an error detection unit capable of executing at least one of an error detection process of detecting a picking error based on an image captured in the picking state during production of the board and an error detection process of detecting a mounting error based on an image captured in the mounted state; an imaging processing unit that images the component supply position with the camera before the picking operation; and an inspection unit that inspects the presence or absence of a component at the component supply position by applying the image captured by the imaging processing unit of the component supply position before the picking operation to a trained model obtained by machine learning using a plurality of images captured at the component supply position before the picking operation as input data and the presence or absence of a component at the component supply position as training data, If the error detection unit does not detect an error during the machine learning, the captured image of the component supply position before the picking operation is classified as an image to be used as the training data of a component present, and if the error detection unit detects an error, the captured image of the component supply position before the picking operation acquired by the imaging processing unit is classified as an image to be used as the training data of a component present. The gist of this is as follows.

[0010] In the image classification method disclosed herein, if no errors are detected by the error detection unit during machine learning, the captured image of the component supply position before the picking operation is classified as an image to be used as training data for components present. Therefore, compared to when an operator visually classifies training data, it is easier to obtain training data for components present. Furthermore, if an error is detected by the error detection unit, the captured image of the component supply position is likely to be unsuitable as training data for components present. Therefore, it is highly significant to classify the captured image of the component supply position before the picking operation as an image not to be used as training data for components present. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a configuration diagram showing the configuration of a component mounting system 1. FIG. [Figure 2] FIG. 2 is a perspective view of the component mounting apparatus 10. [Figure 3] FIG. [Figure 4] FIG. 2 is a side view showing the outline of the configuration of the head unit 40. [Figure 5] 2 is a block diagram showing the electrical connection relationship of the component mounting system 1. FIG. [Figure 6] 10 is a flowchart illustrating an example of a production processing routine. [Figure 7] 10 is a flowchart illustrating an example of a side surface inspection subroutine. [Figure 8] 10 is a flowchart illustrating an example of a bottom surface inspection subroutine. [Figure 9] 10 is a flowchart showing an example of a post-mounting component inspection subroutine. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a pre-adsorption operation image Im1. [Figure 11] FIG. 10 is an explanatory diagram showing an example of an image Im2 after the adsorption operation. [Figure 12] FIG. 10 is an explanatory diagram showing an example of a side image Im3. [Figure 13] FIG. 10 is an explanatory diagram showing an example of a bottom image Im4. [Figure 14]FIG. 10 is an explanatory diagram showing an example of a board image Im5. [Figure 15] FIG. 10 is an explanatory diagram illustrating an example of a routine for classifying images containing parts. [Figure 16] FIG. 10 is an explanatory diagram illustrating an example of a routine for classifying images without parts. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, an embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a configuration diagram showing the configuration of a component mounting system 1. Fig. 2 is a perspective view of a component mounting apparatus 10. Fig. 3 is a perspective view showing a component supply position F. Fig. 4 is a side view showing an outline of the configuration of a head unit 40. Fig. 5 is a block diagram showing the electrical connection relationship of the component mounting system 1. In this embodiment, the left-right direction in Figs. 2 to 4 (the direction perpendicular to the paper surface in Figs. 3 and 4) is the X-axis direction, the front-rear direction is the Y-axis direction, and the up-down direction is the Z-axis direction.

[0013] 1, component mounting system 1 includes a solder paste printing device 3, a solder paste inspection device 4, a mounting line 5, a reflow device 6, a board appearance inspection device 7, and a management server 90. Mounting line 5 is made up of a plurality of component mounting devices 10 arranged in a line. Each of these devices is connected to management server 90 via communication network (e.g., LAN) 2 so as to be able to communicate bidirectionally.

[0014] The following describes the operation of each device constituting the component mounting system 1. Each device executes processing according to a production job sent from the management server 90. The production job is information that specifies which component types and in what order to mount them on the boards S in each component mounting device 10, as well as how many boards S the components will be mounted on. The solder paste printing device 3 prints solder paste in a predetermined pattern on the surface of the boards S transported from the upstream side at the positions where each component will be mounted, and transports the boards S to the solder paste inspection device 4 downstream. The solder paste inspection device 4 inspects whether the solder paste has been printed correctly on the transported boards S. The boards S on which the solder paste has been correctly printed are transported via the intermediate conveyor 8a to the component mounting device 10 on the mounting line 5. The multiple component mounting devices 10 arranged on the mounting line 5 mount components on the boards S in order from the upstream side. Once all components have been mounted, the boards S are transported from the component mounting device 10 to the reflow device 6 via the intermediate conveyor 8b. In the reflow device 6, the solder paste on the board S melts and then solidifies, fixing each component onto the board S. The board S is carried out of the reflow device 6 and is carried into the board visual inspection device 7 via an intermediate conveyor 8c. The board visual inspection device 7 judges whether the visual inspection was successful or not based on the visual inspection image obtained by capturing an image of the board S on which all components have been mounted.

[0015] As shown in FIG. 2, the component mounting apparatus 10 includes a mounting apparatus main body 11, a mark camera 70, a side camera 71 (see FIG. 4), a parts camera 72, and a controller 80 (see FIG. 5).

[0016] The mounting device main body 11 picks up components P supplied from the feeder 20 and mounts them on the board S. The mounting device main body 11 includes a board transport device 12, a head moving device 13, and a head unit 40.

[0017] The feeder 20 has a tape reel around which the tape 21 is wound, and a tape feeding mechanism (not shown) unwinds the tape 21 from the tape reel and feeds it to the component supply position F. As shown in FIG. 3, the tape 21 has cavities 21a and sprocket holes 21b formed at predetermined intervals along its length. Components P are accommodated in the cavities 21a. A sprocket of the tape feeding mechanism engages with the sprocket hole 21b. The feeder 20 drives the sprocket by a predetermined number of rotations using a motor to feed the tape 21 engaged with the sprocket by a predetermined amount, thereby sequentially supplying the components P accommodated on the tape 21 to the component supply position. The components P accommodated on the tape 21 are protected by a film covering the surface of the tape 21. The film is peeled off just before the component supply position F, exposing the components P at the component supply position and enabling them to be picked up by the suction nozzle 41.

[0018] The substrate transport device 12 is configured as, for example, a belt conveyor device, and is driven to transport the substrate S from left to right (substrate transport direction) in Fig. 2. A substrate support device is provided at the center of the substrate transport direction (X-axis direction) of the substrate transport device 12, which supports the transported substrate S from the back side with support pins.

[0019] The head moving device 13 is a device that moves the head unit 40 in the horizontal direction. As shown in FIG. 2, the head moving device 13 includes a Y-axis guide rail 14, a Y-axis slider 15, a Y-axis actuator 16 (see FIG. 5), an X-axis guide rail 17, an X-axis slider 18, and an X-axis actuator 19 (see FIG. 5). The Y-axis guide rail 14 is provided on the upper part of the mounting device main body 11 along the Y-axis direction. The Y-axis slider 15 is movable along the Y-axis guide rail 14 by driving the Y-axis actuator 16. The X-axis guide rail 17 is provided on the underside of the Y-axis slider 15 along the X-axis direction. The head unit 40 is attached to the X-axis slider 18, and is movable along the X-axis guide rail 17 by driving the X-axis actuator 19. Therefore, the head moving device 13 can move the head unit 40 in the X and Y directions.

[0020] As shown in FIG. 4, the head unit 40 includes a rotary head 44, an R-axis actuator 46, and a Z-axis actuator 50.

[0021] The rotary head 44 has a plurality of (here, 12) nozzle holders 42, each holding a suction nozzle 41, arranged at a predetermined angular interval (e.g., 30 degrees) on a circumference coaxial with the rotation axis. The nozzle holders 42 are configured as hollow cylindrical members extending in the Z-axis direction. The upper end 42a of the nozzle holder 42 is formed in a cylindrical shape with a diameter larger than that of the shaft portion of the nozzle holder 42. The nozzle holder 42 also has a flange portion 42b, the diameter of which is larger than that of the shaft portion, formed at a predetermined position below the upper end 42a. A spring (coil spring) 45 is disposed between the annular surface below the flange portion 42b and a recess (not shown) formed on the upper surface of the rotary head 44. Therefore, the spring 45 biases the nozzle holder 42 (flange portion 42b) upward, using the recess on the upper surface of the rotary head 44 as a spring seat. The rotary head 44 is equipped with a Q-axis actuator 49 (see FIG. 5) therein, which individually rotates each nozzle holder 42. Although not shown, the Q-axis actuator 49 includes a drive gear meshed with a gear provided on the cylindrical outer periphery of the nozzle holder 42 and a drive motor connected to the rotation shaft of the drive gear. This allows each of the nozzle holders 42 to rotate independently around its axis (in the Q direction), and thus each of the suction nozzles 41 also rotate independently. The suction nozzles 41 are connected to a vacuum pump or air piping via a solenoid valve 60 (see FIG. 5). By driving the solenoid valve 60 so that the suction port of each suction nozzle 41 is connected to a vacuum pump, negative pressure is applied to the suction port, thereby suctioning the component P. By driving the solenoid valve 60 so that the suction port is connected to an air piping, positive pressure is applied to the suction port, thereby releasing the suction of the component P.

[0022] The R-axis actuator 46 includes a rotary shaft 47 connected to the rotary head 44 and a drive motor 48 connected to the rotary shaft 47. The R-axis actuator 46 drives the drive motor 48 intermittently by a predetermined angle (e.g., 30 degrees) to intermittently rotate the rotary head 44 by the predetermined angle. As a result, each nozzle holder 42 arranged on the rotary head 44 pivots by the predetermined angle in the circumferential direction. Here, when the nozzle holder 42 is at a predetermined work position WP (position shown in FIG. 4) among the multiple positions to which it can move, it uses the suction nozzle 41 to pick up a component P supplied from the feeder 20 to the component supply position F, and places the component P picked up by the suction nozzle 41 at a predetermined placement position on the board S.

[0023] The Z-axis actuator 50 is configured as a feed screw mechanism including a screw shaft 54 ​​extending in the Z-axis direction and moving a ball screw nut 52, a Z-axis slider 56 attached to the ball screw nut 52, and a drive motor 58 whose rotation shaft is connected to the screw shaft 54. The Z-axis actuator 50 rotates and drives the drive motor 58 to move the Z-axis slider 56 in the Z-axis direction. The Z-axis slider 56 is formed with a substantially L-shaped lever portion 57 that protrudes toward the rotary head 44. The lever portion 57 is capable of abutting against the upper end portion 42a of the nozzle holder 42 located within a predetermined range that includes the work position WP. Therefore, when the lever portion 57 moves in the Z-axis direction in conjunction with the movement of the Z-axis slider 56 in the Z-axis direction, the nozzle holder 42 (suction nozzle 41) located within the predetermined range can be moved in the Z-axis direction.

[0024] As shown in Fig. 2, the mark camera 70 is provided on the underside of the X-axis slider 18. The mark camera 70 has an imaging range below, and captures an image of an object from above to generate a captured image. Objects imaged by the mark camera 70 include components P held on the tape 21 fed from the feeder 20, marks affixed to the board S, and components P after they have been mounted on the board S.

[0025] The side camera 71 is a camera that captures images from the side of the suction nozzle 41 stopped at the work position WP and the state of suction of the component P to the suction nozzle 41. The side camera 71 is provided at the bottom of the head unit 40, as shown in FIG.

[0026] The part camera 72 has an imaging range above, and captures an image of the state of suction of the component P by the suction nozzle 41 from below the component P to generate a captured image. The part camera 72 is disposed between the feeder 20 and the board transport device 12, as shown in FIG.

[0027] As shown in FIG. 5, the controller 80 is configured as a microprocessor centered around a CPU 81, and includes, in addition to the CPU 81, a ROM 82, a storage (e.g., an HDD or SSD) 83, a RAM 84, and the like. The controller 80 receives image signals from the mark camera 70, the side camera 71, and the parts camera 72. The X-axis slider 18, the Y-axis actuator 16, the R-axis actuator 46, the Q-axis actuator 49, and the Z-axis actuator 50 are each equipped with a position sensor (not shown), and the controller 80 also receives position information from these position sensors. The controller 80 also outputs control signals to the mark camera 70, the side camera 71, and the parts camera 72. The controller 80 also outputs drive signals to the feeder 20, the substrate transport device 12, the Y-axis actuator 16, the X-axis actuator 19, the R-axis actuator 46, the Q-axis actuator 49, the Z-axis actuator 50, the solenoid valve 60, and the like.

[0028] 5, the management server 90 includes a CPU 91, a ROM 92, a storage 93 that stores production jobs for the substrate S, and a RAM 94. The management server 90 receives input signals from an input device 95 such as a mouse or keyboard. The management server 90 also outputs image signals to a display 96.

[0029] Next, the operation of the component mounting apparatus 10 will be described with reference to FIGS. 6 to 14. FIG. 6 is a flowchart showing an example of a production processing routine. FIG. 7 is a flowchart showing an example of a side surface inspection subroutine. FIG. 8 is a flowchart showing an example of a bottom surface inspection subroutine. FIG. 9 is a flowchart showing an example of a post-mounting component inspection subroutine. FIG. 10 is an explanatory diagram showing an example of a pre-suction operation image Im1. FIG. 11 is an explanatory diagram showing an example of a post-suction operation image Im2. FIG. 12 is an explanatory diagram showing an example of a side surface image Im3. FIG. 13 is an explanatory diagram showing an example of a bottom surface image Im4. FIG. 14 is an explanatory diagram showing an example of a board image Im5. The production processing routine is stored in storage 83, and is started when a production job is received from management server 90 and an instruction to start production is issued.

[0030] When this routine starts, the CPU 81 first controls the X-axis actuator 19 and the Y-axis actuator 16 so that the mark camera 70 moves directly above the component supply position F. Then, the CPU 81 controls the mark camera 70 so that an image of the component supply position F before the pickup operation is captured (S100). In this embodiment, this image is referred to as a pre-pickup operation image Im1. An example of the pre-pickup operation image Im1 is shown in FIG.

[0031] Next, the CPU 81 determines whether or not a trained model is stored in the storage 83 (S110). The trained model is used to input a pre-suction operation image Im1 and determine whether the input pre-suction operation image Im1 includes a component P. The trained model is created by machine learning using an image captured by the mark camera 70, data indicating that a component is present in the image (trainee data indicating that a component is present), and an image captured by the mark camera 70, data indicating that a component is not present in the image (trainee data indicating that a component is not present). This trained model is created for each combination of a tape type, which is the type of tape 21, and a component type, which is the type of component P.

[0032] If a negative determination is made in S110, the CPU 81 executes a suction operation to suck up the component P at the component supply position F with the suction nozzle 41 (S120). Specifically, the CPU 81 controls the X-axis actuator 19 and the Y-axis actuator 16 so that the work position WP of the rotary head 44 moves directly above the component supply position F of the feeder 20, controls the Z-axis actuator 50 so that the suction nozzle 41 at the work position WP moves down, and controls the solenoid valve 60 so that negative pressure is applied to the suction nozzle 41 to suck up the component P.

[0033] Next, the CPU 81 controls the X-axis actuator 19 and the Y-axis actuator 16 so that the mark camera 70 moves directly above the component supply position F. Then, the CPU 81 controls the mark camera 70 so that an image of the component supply position F after the suction operation is captured (S130). In this embodiment, this image is referred to as a post-suction operation image Im2. An example of the post-suction operation image Im2 is shown in FIG. 11.

[0034] Next, the CPU 81 executes the side surface inspection subroutine shown in Fig. 7 (S140). When the side surface inspection subroutine starts, the CPU 81 controls the side surface camera 71 so that an image of the suction state of the component P is captured from the side of the suction nozzle 41 at the work position WP (S300). In this embodiment, this image is referred to as a side surface image Im3. An example of the side surface image Im3 is shown in Fig. 12.

[0035] The CPU 81 then determines whether or not a pickup error has occurred based on the side image Im3 (S310). The process of determining whether or not a pickup error has occurred based on the side image Im3 is executed, for example, as follows. That is, if a component P is captured at the tip of the suction nozzle 41 and the vertical length of the captured component P is within the allowable range, the CPU 81 makes a negative determination in S310 and determines that there is no pickup error based on the side image Im3 (S320). Otherwise, the CPU 81 makes a positive determination in S310 and determines that there is a pickup error based on the side image Im3 (S330). For example, if the component P has a rectangular parallelepiped shape and the component P should be picked up so that its longitudinal direction is horizontal, but the longitudinal direction of the component P is oriented at an angle, the vertical length of the captured component P will exceed the allowable range. Therefore, the component P picked up at an angle is determined to have had a pickup error. After S320 or S330, the CPU 81 stores the error presence / absence determination result in the storage 83 (S340), and proceeds to S150 of the production processing routine.

[0036] Next, the CPU 81 executes the bottom surface inspection subroutine shown in Fig. 8 as shown in Fig. 6 (S150). When the bottom surface inspection subroutine starts, the CPU 81 controls the X-axis actuator 19 and the Y-axis actuator 16 so that the rotary head 44 moves from above the feeder 20 to above the part camera 72. Then, the CPU 81 controls the part camera 72 so that an image of the suction state of the component P relative to the suction nozzle 41 is captured from below the suction nozzle 41 (S400). In this embodiment, this image is referred to as a bottom surface image Im4. An example of the bottom surface image Im4 is shown in Fig. 13.

[0037] The CPU 81 then determines whether or not there is a pickup error based on the bottom-side image Im4 (S410). The process of determining whether or not there is a pickup error based on the bottom-side image Im4 is executed, for example, as follows. That is, if a component P is captured at the tip of the suction nozzle 41 and the amount of misalignment of the captured component P is within the allowable range, the CPU 81 makes a negative determination in S410 and determines that there is no pickup error based on the bottom-side image Im4 (S420). Otherwise, the CPU 81 makes a positive determination in S410 and determines that there is a pickup error based on the bottom-side image Im4 (S430). Here, the amount of misalignment is used to correct the position of the component P when placing the component P in a predetermined position on the board S. Therefore, if the amount of misalignment exceeds the allowable range, it is determined that there is a pickup error of the component P relative to the suction nozzle 41. After S420 or S430, the CPU 81 stores the result of the pickup error determination in the storage 83 (S440) and proceeds to S160 of the production processing routine.

[0038] 6, the CPU 81 executes a component mounting operation to mount a component P on the board S (S160). Specifically, the CPU 81 controls the R-axis actuator 46 so that the suction nozzle 41 that is sucking the component P to be mounted comes to the work position WP of the rotary head 44, and controls the X-axis actuator 19 and the Y-axis actuator 16 so that the work position WP moves to a mounting position on the board S. The CPU 81 also controls the Z-axis actuator 50 so that the suction nozzle 41 at the work position WP moves down, and controls the solenoid valve 60 so that a positive pressure is applied to the suction nozzle 41, causing the component P to be released from the suction nozzle 41 and placed in a mounting position on the board S.

[0039] After S160, the CPU 81 executes the post-mounting component inspection routine shown in FIG. 9 (S170). When the post-mounting component inspection routine starts, the CPU 81 controls the mark camera 70 so as to capture an image of the portion of the board S on which the component P is mounted after the mounting operation (S500). In this embodiment, this image is referred to as board image Im5. An example of the board image Im5 is shown in FIG.

[0040] The CPU 81 then determines whether or not there is a mounting error based on the board image Im5 (S510). The process of determining whether or not there is a mounting error based on the board image Im5 is executed, for example, as follows. That is, the CPU 81 recognizes the position of the portion shown in the board image Im5, and if the component P falls within an allowable range from the intended mounting position on the board S, the CPU 81 makes a negative determination in S510 and determines that there is no mounting error based on the board image Im5 (S520). Otherwise, the CPU 81 makes a positive determination in S510 and determines that there is a mounting error based on the board image Im5 (S530). After S520 or S530, the CPU 81 stores the error presence / absence determination result in the storage 83 (S540), and proceeds to S180 of the production processing routine.

[0041] 6, the CPU 81 outputs the before-sucking operation image Im1, the after-sucking operation image Im2, and the error presence / absence determination result (the result of determining whether or not there is a suction error based on the side image Im3, the result of determining whether or not there is a suction error based on the bottom image Im4, and the result of determining whether or not there is a mounting error based on the board image Im5) to the management server 90 (S180). After inputting these, the management server 90 stores the before-sucking operation image Im1, the after-sucking operation image Im2, and the error presence / absence determination result in the storage 93 in association with each other.

[0042] Here, a description will be given of the process to be performed when a positive determination is made in S110. If a positive determination is made in S110, the CPU 81 applies the before-sucking operation image Im1 as input data to the learned model (S200).

[0043] Next, the CPU 81 determines whether or not a component P is present in the before-pickup operation image Im1 based on the output result of the trained model (S210). If the determination in S210 is affirmative, the CPU 81 executes a pickup operation in which the pickup nozzle 41 picks up the component P at the component supply position F (S220), executes a side surface inspection subroutine (S230), executes a bottom surface inspection subroutine (S240), executes a component mounting operation in which the component P is mounted on the board S (S250), and executes a post-mounting component inspection subroutine (S260). Note that the processing in S220 to S260 is the same as the processing in S130 to S170.

[0044] After S180 or S260, the CPU 81 notifies the result of the error presence / absence determination (S190). Specifically, the CPU 81 causes a display device (not shown) of the component mounting apparatus 10 to display the result of the error presence / absence determination.

[0045] On the other hand, if a negative determination is made in S210, the CPU 81 outputs an instruction to replace the feeder 20 to a feeder replacement device (not shown) (S270). After inputting the feeder replacement instruction, the feeder replacement device executes the replacement work of the feeder 20 on the mounting device main body 11.

[0046] After S190 or S270, the CPU 81 ends this routine. Note that the processes of S100 to S270 are executed for the plurality of suction nozzles 41 held by the rotary head 44.

[0047] Next, the operation of the management server 90 will be described. In particular, the operation for classifying images used in creating a trained model (machine learning) will be described. First, a component-present image classification routine executed by the management server 90 will be described with reference to FIG. 15. FIG. 15 is a flowchart showing an example of the component-present image classification routine. This routine is stored in the storage 93 of the management server 90. This routine is executed by the CPU 91 of the management server 90 after the before-suction operation image Im1 is input from the controller 80.

[0048] When this routine starts, the CPU 91 first acquires a feature value from the pre-suction operation image Im1 (S600). Here, the feature value is, for example, the average brightness value of each pixel constituting the pre-suction operation image Im1. Next, the CPU 91 determines whether the feature value acquired in S600 is outside an allowable range (S610). The allowable range is set, for example, based on the average feature value of multiple pre-suction operation images Im1 previously classified as images used for training data containing a component, or the variation in feature values ​​among such multiple pre-suction operation images Im1. If the determination in S610 is negative, the CPU 91 determines whether a suction error based on the side image Im3 is stored in the storage 93 (S620). If the determination in S620 is negative, the CPU 91 determines whether a suction error based on the bottom image is stored in the storage 93 (S630). If a negative determination is made in S630, the CPU 91 determines whether a mounting error based on the board image Im5 is stored in the storage 93 (S640). If a negative determination is made in S640, the CPU 91 classifies the pre-suction operation image Im1 as an image to be used for training data with a component present (S650). On the other hand, if a positive determination is made in S610, S620, S630, or S640, the CPU 91 classifies the pre-suction operation image Im1 as an image not to be used for training data with a component present (S660). After S650 or S660, the CPU 91 terminates this routine.

[0049] Here, if a positive determination is made in S610, the CPU 91 classifies the pre-suction operation image Im1 as an image not to be used in the training data for a component-present (S660) for the following reasons, for example: The pre-suction operation image Im1 used in the training data for a component-present is an image of the component supply position F when a component is present in the cavity 21a of the tape 21, and the pre-suction operation image Im1 not to be used in the training data for a component-present is an image of the component supply position F when no component P is present in the cavity 21a of the tape 21. When a component P is present in the cavity 21a, the pre-suction operation image Im1 captures the component P and the bottom surface of the cavity 21a. On the other hand, when no component is present in the cavity 21a, only the bottom surface of the cavity 21a is captured in the pre-suction operation image Im1. When captured in an image, the brightness values ​​of the component P portion and the bottom surface of the cavity 21a are different. Therefore, the feature amount (average brightness value of each pixel constituting the pre-suction operation image Im1) differs between the pre-suction operation image Im1 when a component is present in the cavity 21a and the pre-suction operation image Im1 when no component P is present in the cavity 21a. Therefore, the CPU 91 classifies the pre-suction operation image Im1 whose feature amount does not fall within the allowable range as an image that will not be used for training data with a component present.

[0050] Furthermore, if a positive determination is made in S620, S630, or S640, the CPU 91 classifies the pre-suction operation image Im1 as an image not to be used for training data with a component present (S660) for the following reasons, for example: These errors occur when there is no component P in the cavity 21a, or when some abnormality has occurred in the cavity 21a or the component P contained in the cavity 21a at the component supply position F. Therefore, if these errors have occurred, the CPU 91 classifies the pre-suction operation image Im1 as an image not to be used for training data with a component present.

[0051] Next, a post-adsorption image classification routine executed by the management server 90 will be described with reference to Fig. 16. Fig. 16 is a flowchart showing an example of the post-adsorption image classification routine. This routine is stored in the storage 93 of the management server 90. This routine is executed by the CPU 91 of the management server 90 after the pre-adsorption operation image Im1 is input from the controller 80.

[0052] When this routine starts, the CPU 91 acquires feature quantities of the post-suction operation image Im2 (S700). Here, the feature quantities are, for example, the average brightness values ​​of the pixels constituting the post-suction operation image Im2. Next, the CPU 91 determines whether the feature quantities acquired in S700 are outside an allowable range (S710). The allowable range is set, for example, based on the average feature quantities of multiple post-suction operation images Im2 previously classified as images to be used for training data without components, or the variation in feature quantities among such multiple post-suction operation images Im2. If a negative determination is made in S710, the CPU 91 classifies the post-suction operation image Im2 as an image to be used for training data without components (S720). On the other hand, if a positive determination is made in S710, the CPU 91 classifies the post-suction operation image Im2 as an image not to be used for training data without components (S730). After S720 or S730, the CPU 91 terminates this routine.

[0053] Here, if a positive determination is made in S710, the CPU 91 classifies the post-suction operation image Im2 as an image not to be used in the training data for no component (S730), for example, for the following reason. That is, the post-suction operation image Im2 used in the training data for no component is an image of the component supply position F when there is no component P in the cavity 21a of the tape 21, and the post-suction operation image Im2 not used in the training data for no component is an image of the component supply position F when there is a component P in the cavity 21a of the tape 21. When there is no component P in the cavity 21a, only the bottom surface of the cavity 21a is captured in the post-suction operation image Im2. On the other hand, when there is a component in the cavity 21a, only the component P and the bottom surface of the cavity 21a are captured in the post-suction operation image Im2. When captured in an image, the brightness values ​​of the part of the component P and the bottom surface of the cavity 21a are different. Therefore, the feature amount (average brightness value of each pixel constituting the post-suction operation image Im2) differs between the post-suction operation image Im2 when there is no component in the cavity 21a and the post-suction operation image Im2 when there is a component P in the cavity 21a. Therefore, the CPU 91 classifies the post-suction operation image Im2 whose feature amount is outside the allowable range as an image that will not be used for training data without a component.

[0054] In this way, in the component mounting system 1, the management server 90 classifies the before-pickup operation image Im1 into an image to be used for training data with components present or an image not to be used for training data with components present. Also, in the component mounting system 1, the management server 90 classifies the after-pickup operation image Im2 into an image to be used for training data without components or an image not to be used for training data without components. In order to create a trained model, a large amount of training data with components present and training data without components must be prepared. Therefore, compared to when an operator visually classifies training data, the component mounting system 1 makes it easier to obtain training data with components present and training data without components.

[0055] Here, the correspondence between the components of this embodiment and the components of the component mounting system of the present disclosure will be described. The component mounting system 1 of this embodiment corresponds to the component mounting system of the present disclosure, the mounting device main body 11 corresponds to the mounting machine main body, the mark camera 70, the side camera 71, and the part camera 72 correspond to the cameras, the controller 80 corresponds to the production control unit, the controller 80 corresponds to the error detection unit, the controller 80 corresponds to the imaging processing unit, the controller 80 corresponds to the inspection unit, and the management server 90 corresponds to the classification unit.

[0056] In the component mounting system 1 described above, if the controller 80 does not detect an error, the pre-pickup operation image Im1 is classified as an image to be used as training data for a component present during machine learning. Therefore, compared to when an operator visually classifies training data, it is easier to obtain training data for a component present. Furthermore, if the controller 80 detects an error, the pre-pickup operation image Im1 is likely to be unsuitable as training data for a component present. Therefore, there is great significance in classifying such a pre-pickup operation image Im1 as an image not to be used as training data for a component present.

[0057] Furthermore, in the component mounting system 1, the management server 90 acquires feature amounts from the pre-pickup operation image Im1, and if the feature amounts are outside the allowable range, classifies the pre-pickup operation image Im1 as an image that will not be used as training data for a component present. If the feature amounts acquired from the pre-pickup operation image are outside the allowable range, there is a high possibility that some kind of abnormality has occurred at the component supply position F. Therefore, it is highly significant to classify the pre-pickup operation image Im1, whose feature amounts are outside the allowable range, as an image that will not be used as training data for a component present.

[0058] Furthermore, in the component mounting system 1, the controller 80 controls the mark camera 70 to capture an image of the component supply position F after the pickup operation. The management server 90 acquires feature values ​​from the post-pickup operation image Im2. If the feature values ​​are within the allowable range, the management server 90 classifies the post-pickup operation image Im2 as an image to be used as training data without components. If the feature values ​​are outside the allowable range, the management server 90 classifies the post-pickup operation image Im2 as an image not to be used as training data without components. Therefore, compared to when an operator visually classifies training data, it is easier to obtain the training data without components necessary for creating a trained model. Furthermore, if the feature values ​​are outside the allowable range, it is highly likely that some kind of abnormality has occurred at the component supply position F. Therefore, it is highly significant to classify the post-pickup operation image Im2, whose feature values ​​are outside the allowable range, as an image not to be used as training data without components.

[0059] Furthermore, in the image classification method of the above-described embodiment, if the controller 80 does not detect an error, the pre-suction operation image Im1 is classified as an image to be used as training data for a part present in machine learning. Therefore, compared to when an operator visually classifies training data, it is easier to obtain training data for a part present. Furthermore, if the controller 80 detects an error, the pre-suction operation image Im1 is likely to be unsuitable as training data for a part present. Therefore, it is highly significant to classify such a pre-suction operation image Im1 as an image not to be used as training data for a part present.

[0060] It goes without saying that the present disclosure is not limited to the above-described embodiments, and can be implemented in various forms as long as they fall within the technical scope of the present disclosure.

[0061] In the above-described embodiment, the component mounting apparatus 10 has, as the cameras of the present disclosure, the mark camera 70, the side camera 71, and the part camera 72. However, the component mounting apparatus 10 may have only the mark camera 70 and the side camera 71, or may have only the mark camera 70 and the part camera 72.

[0062] In the above-described embodiment, the controller 80 executes all of the side surface inspection subroutine, the bottom surface inspection subroutine, and the post-mounting component inspection subroutine in the production processing routine. However, the controller 80 may execute at least one of the side surface inspection subroutine, the bottom surface inspection subroutine, and the post-mounting component inspection subroutine in the production processing routine.

[0063] In the embodiment described above, if the management server 90 detects any one of a pickup error based on the side image Im3, a pickup error based on the bottom image Im4, and a mounting error based on the board image Im5, the management server 90 classifies the pre-pickup operation image Im1 as an image that will not be used for training data with a component present. However, if the management server 90 detects two errors from the pickup error based on the side image Im3, a pickup error based on the bottom image Im4, and a mounting error based on the board image Im5, the management server 90 may classify the pre-pickup operation image Im1 as an image that will not be used for training data with a component present, or if three errors are detected, the management server 90 may classify the pre-pickup operation image Im1 as an image that will not be used for training data with a component present.

[0064] In the above-described embodiment, the side inspection subroutine, bottom surface inspection subroutine, and post-mounting component inspection subroutine are executed by the controller 80, and the component-present image classification routine and component-absent image classification routine are executed by the management server 90. However, the controller 80 may execute at least one of the component-present image classification routine and the component-absent image classification routine, or the management server 90 may execute at least one of the side inspection subroutine, bottom surface inspection subroutine, and post-mounting component inspection subroutine.

[0065] In the embodiment described above, the controller 80 determines whether or not there is a mounting error based on the board image Im5 captured by the mark camera 70. However, the board visual inspection device 7 may determine whether or not there is a mounting error based on an appearance inspection image captured by the device itself.

[0066] In the above-described embodiment, if the worker confirms that a component P is present in the pre-pickup operation image Im1 that has been classified as an image not to be used in training data containing a component, the management server 90 may be configured to allow the worker to input a reclassification instruction via the input device 95. When the reclassification instruction is input, the management server 90 reclassifies the pre-pickup operation image Im1 into an image to be used in training data containing a component.

[0067] In the above-described embodiment, the present disclosure has been described as the component mounting system 1, but it may also be an image classification method. [Industrial Applicability]

[0068] The present disclosure is applicable to industries that involve the operation of mounting components on boards. [Explanation of symbols]

[0069] 1 component mounting system, 3 solder paste printing device, 4 solder paste inspection device, 5 mounting line, 6 reflow device, 7 board appearance inspection device, 8a to 8c intermediate conveyors, 10 component mounting device, 11 mounting device main body, 12 board transport device, 13 head moving device, 14 Y-axis guide rail, 15 Y-axis slider, 16 Y-axis actuator, 17 X-axis guide rail, 18 X-axis slider, 19 X-axis actuator, 20 feeder, 21 tape, 21a cavity, 21b sprocket hole, 40 head unit, 41 suction nozzle, 42 nozzle holder, 42a upper end, 42b flange portion, 44 rotary head, 45 spring, 46 R-axis actuator, 47 rotating shaft, 48 drive motor, 49 Q-axis actuator, 50 Z-axis actuator, 52 ball screw nut, 54 screw shaft, 56 Z-axis slider, 57 Lever part, 58 drive motor, 60 solenoid valve, 61 CPU, 70 mark camera, 71 side camera, 72 parts camera, 80 controller, 81 CPU, 82 ROM, 83 storage, 84 RAM, 90 management server, 91 CPU, 92 ROM, 93 storage, 94 RAM, 95 input device, 96 display, F part supply position, Im1 image before suction operation, Im2 image after suction operation, Im3 side image, Im4 bottom image, Im5 board image, P part, S board, WP work position.

Claims

1. a mounting machine body having a head holding a picking member capable of picking up components supplied from a feeder to a component supply position and a head moving device for moving the head, and capable of mounting the components picked up by the picking member onto a board; one or more cameras capable of capturing images of at least one of a state of component pickup relative to the pickup member and a state of component mounting on a board, and the component supply position; a production control unit that controls the head and the head moving device so that a picking operation of picking up components with the picking member and a mounting operation of mounting the components picked up by the picking member onto a board are performed, and that controls the camera to obtain an image of at least one of the picked-up state of the components with respect to the picking member after the picking operation and the mounted state of the components on the board after the mounting operation, thereby producing the board; an error detection unit capable of executing at least one of an error detection process for detecting a collection error based on the captured image of the collection state during production of the board and an error detection process for detecting a mounting error based on the captured image of the mounting state; an imaging processing unit that images the component supply position with the camera before the picking operation; an inspection unit that inspects the presence or absence of a component at the component supply position by applying the captured image of the component supply position before the picking operation acquired by the imaging processing unit to a trained model obtained by machine learning using a plurality of captured images of the component supply position before the picking operation as input data and the presence or absence of a component at the component supply position as training data; a classification unit that, if no error is detected by the error detection unit during the machine learning, classifies the captured image of the component supply position before the picking operation into an image to be used as the training data of a component present, and, if the error detection unit detects the error, classifies the captured image of the component supply position before the picking operation acquired by the imaging processing unit into an image to be not used as the training data of a component present; A component mounting system comprising:

2. 2. The component mounting system according to claim 1, the classification unit acquires a feature amount from the captured image of the component supply position before the picking operation, and if the feature amount is out of an allowable range, classifies the captured image of the component supply position before the picking operation as an image that does not need to be used as the training data, indicating that a component is present, even if the error is not detected. Component mounting system.

3. 3. The component mounting system according to claim 1, the production control unit controls the camera so that an image of the component supply position after the picking operation is captured; the classification unit acquires a feature amount from the captured image of the component supply position after the picking operation, and if the feature amount is within an allowable range, classifies the captured image of the component supply position after the picking operation as an image to be used as training data without a component, and if the feature amount is outside the allowable range, classifies the captured image of the component supply position after the picking operation as an image not to be used as training data without a component. Component mounting system.

4. a mounting machine body having a head holding a picking member capable of picking up components to be supplied from a feeder to a component supply position, and a head moving device for moving the head, and capable of mounting the components picked up by the picking member onto a board; one or more cameras capable of capturing images of at least one of a picking state of the components relative to the picking member and a mounting state of the components onto the board, and the component supply position; and controlling the head and the head moving device so that a picking operation of picking up components with the picking member and a mounting operation of mounting the components picked up by the picking member onto the board are performed, and moving the camera so as to obtain a captured image of at least one of a picking state of the components relative to the picking member after the picking operation and a mounting state of the components onto the board after the mounting operation. an error detection unit capable of executing at least one of an error detection process of detecting a picking error based on an image captured in the picking state during production of the board and an error detection process of detecting a mounting error based on an image captured in the mounted state; an imaging processing unit that images the component supply position with the camera before the picking operation; and an inspection unit that inspects the presence or absence of a component at the component supply position by applying the image captured by the imaging processing unit of the component supply position before the picking operation to a trained model obtained by machine learning using a plurality of images captured at the component supply position before the picking operation as input data and the presence or absence of a component at the component supply position as training data, If the error detection unit does not detect an error during the machine learning, the captured image of the component supply position before the picking operation is classified as an image to be used as the training data for a component present, and if the error detection unit detects an error, the captured image of the component supply position before the picking operation acquired by the imaging processing unit is classified as an image to be not used as the training data for a component present. Image classification method.

Images