Foreign object detection device and foreign object detection method

Abstract

The foreign object detection device includes an acquisition unit and a determination unit. The acquisition unit captures an image of at least a part of an inspection region of a substrate in accordance with the progress of a substrate work by a substrate work machine that performs a predetermined substrate work on the substrate, and acquires a plurality of pieces of image data obtained by capturing the same inspection region. The determination unit determines the presence or absence of foreign matter adhering to an inspection region on the basis of a difference in feature amounts of the inspection region obtained by image processing each of reference image data, which is one of the plurality of pieces of image data, and inspection image data, which is image data of the inspection target obtained after the reference image data. In addition, when the substrate work is stopped during a period from the acquisition of the reference image data to the acquisition of the inspection image data, the acquisition unit captures an image of the inspection region and acquires first image data, which is image data at a time when the substrate work is stopped.

Description

Technical Field

[0001] This specification discloses a technology related to a foreign matter detection device and a foreign matter detection method. Background Art

[0002] The mounting line described in Patent Document 1 includes a first camera unit, a second camera unit, and an image processing unit. The first camera unit is a camera unit capable of storing at least a portion of a printed circuit board in a field of view, and is configured to photograph the printed circuit board before the mounting operation of the electronic component in any mounting machine. The second camera unit is a camera unit capable of storing the same range as the first camera unit in a field of view, and is configured to photograph the printed circuit board after the mounting operation of the electronic component by any mounting machine or a mounting machine in a process subsequent to the mounting machine.

[0003] The image processing unit compares the image data captured by the second camera unit with the image data captured by the first camera unit to perform image processing. Thus, the mounting line described in Patent Document 1 can detect printing abnormalities of solder when solder is printed on a printed circuit board, or mounting abnormalities of electronic components when electronic components are mounted on a printed circuit board. Prior art literature Patent Literature

[0004] Patent Document 1: Japanese Patent Application Publication No. 2007-335524. Summary of the invention Problems to be solved by the invention

[0005] When foreign matter is attached to the mounting position of a component, the mounting of the component may become defective, and it is necessary to determine whether there is foreign matter attached to the substrate. In this case, it is assumed that a plurality of image data obtained by photographing the same inspection area of ​​at least a portion of the substrate is compared, and the presence or absence of foreign matter is determined based on the difference in the feature amount of the image data.

[0006] However, during the period from obtaining the image data that serves as the reference to obtaining the image data to be compared with the image data, the substrate operation machine may stop the substrate operation. If the substrate operation is stopped, there is a possibility that the state of the substrate changes, and there is a demand to obtain the state of the substrate at the time when the substrate operation is stopped.

[0007] In view of such circumstances, the present specification discloses a foreign matter detection device and a foreign matter detection method that can acquire image data at the time when work on a substrate is stopped. Means for solving problems

[0008] This specification discloses a foreign matter detection device having an acquisition unit and a judgment unit. The acquisition unit photographs at least a portion of an inspection area of ​​the substrate as a substrate operation machine that performs a prescribed substrate operation on the substrate is performed, and acquires a plurality of image data obtained by photographing the same inspection area. The judgment unit judges whether there is a foreign matter attached to the inspection area based on the difference in feature quantities of the inspection area obtained by performing image processing on one image data among the plurality of image data, namely, reference image data, and image data of an inspection object obtained after the reference image data, namely, inspection image data. In addition, when the substrate operation is stopped during the period from the acquisition of the reference image data to the acquisition of the inspection image data, the acquisition unit photographs the inspection area and acquires image data at the moment when the substrate operation is stopped, namely, first image data.

[0009] In addition, the present specification discloses a foreign matter detection method having an acquisition process and a judgment process. In the acquisition process, an inspection area of ​​at least a portion of the substrate is photographed as a substrate operation machine performs a prescribed substrate operation on the substrate, and a plurality of image data obtained by photographing the same inspection area are acquired. In the judgment process, the presence or absence of foreign matter attached to the inspection area is judged based on the difference in feature quantities of the inspection area obtained by performing image processing on one image data of the plurality of image data, namely, the reference image data, and image data of the inspection object acquired after the reference image data, namely, the inspection image data. In addition, in the acquisition process, in the case where the substrate operation is stopped during the period from the acquisition of the reference image data to the acquisition of the inspection image data, the inspection area is photographed to acquire image data at the moment when the substrate operation is stopped, namely, the first image data.

[0010] In addition, in this specification, the technical idea of ​​changing the "foreign matter detection device described in technical solution 1" to "the foreign matter detection device described in any one of technical solutions 1 to technical solutions 5" in technical solution 6 described in the claims originally attached to this application (hereinafter referred to as original claims) is disclosed. In addition, in this specification, the technical idea of ​​changing the "foreign matter detection device described in technical solution 1" to "the foreign matter detection device described in any one of technical solutions 1 to technical solutions 8" in technical solution 9 described in the original claims is disclosed. Moreover, in this specification, the technical idea of ​​changing the "foreign matter detection device described in technical solution 1" to "the foreign matter detection device described in any one of technical solutions 1 to technical solutions 10" in technical solution 11 described in the original claims is disclosed. In addition, in this specification, the technical idea of ​​changing the "foreign matter detection device described in technical solution 1" to "the foreign matter detection device described in any one of technical solutions 1 to technical solutions 10" in technical solution 12 described in the original claims is disclosed. Moreover, in this specification, the technical idea of ​​changing the "foreign matter detection device described in Technical Solution 1" in Technical Solution 13 recorded in the original claim to "the foreign matter detection device described in any one of Technical Solutions 1 to Technical Solution 10 and Technical Solution 12" is disclosed. Effects of the Invention

[0011] According to the above-mentioned foreign matter detection device, it is possible to obtain the first image data, which is the image data at the time when the work on the substrate is stopped. The above-mentioned contents about the foreign matter detection device are also the same in the foreign matter detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a structural diagram showing a structural example of a substrate processing line.

[0013] Figure 2 It is a plan view showing a structural example of a component mounting machine.

[0014] Figure 3 This is a block diagram showing an example of a control block of the foreign object detection device.

[0015] Figure 4 This is a flowchart showing an example of a control procedure of the foreign object detection device.

[0016] Figure 5 : is a schematic diagram showing an example of reference image data.

[0017] Figure 6 It is a schematic diagram showing an example of inspection image data.

[0018] Figure 7 This is a schematic diagram showing an example of the state of the substrate until the work on the substrate is stopped.

[0019] Figure 8 This is a schematic diagram showing an example of the state of the substrate after the work on the substrate is resumed. DETAILED DESCRIPTION 1. Implementation Method 1-1. Configuration Example of Substrate Working Line WL0

[0020] In the substrate operation line WL0, the substrate operation machine WM0 performs a predetermined substrate operation on the substrate 90. The type and number of the substrate operation machines WM0 constituting the substrate operation line WL0 are not limited. Figure 1 As shown, the substrate processing line WL0 of the embodiment includes a plurality of substrate processing machines WM0 including a printer WM1, a print inspection machine WM2, a component mounting machine WM3, a reflow furnace WM4, and a visual inspection machine WM5. The substrate 90 is transported by a substrate transport device in the above order.

[0021] The printer WM1 prints solder to the mounting positions of the plurality of components 91 on the substrate 90. The print inspection machine WM2 inspects the printing state of the solder printed by the printer WM1. Figure 2 As shown, the component mounting machine WM3 mounts a plurality of components 91 on a substrate 90 on which solder is printed by the printer WM1. The component mounting machine WM3 may be one or more. When a plurality of component mounting machines WM3 are provided, the plurality of component mounting machines WM3 can mount the plurality of components 91 in a shared manner.

[0022] The reflow furnace WM4 heats the substrate 90 on which the plurality of components 91 are mounted by the component mounting machine WM3, melts the solder, and performs soldering. The appearance inspection machine WM5 inspects the mounting status of the plurality of components 91 mounted by the component mounting machine WM3, etc. In this way, the substrate operation line WL0 can use a plurality of substrate operation machines WM0 to sequentially transport the substrate 90, perform production processing including inspection processing, and produce substrate products 900. In addition, the substrate operation line WL0 can also be equipped with, for example, a function inspection machine, a buffer device, a substrate supply device, a substrate turning device, a shielding member mounting device, an adhesive coating device, an ultraviolet irradiation device, and other substrate operation machines WM0 as needed.

[0023] The plurality of substrate processing machines WM0 and the line management device LC0 constituting the substrate processing line WL0 are connected in a communicative manner via a communication unit. In addition, the line management device LC0 and the management device HC0 are connected in a communicative manner via a communication unit. The communication unit can connect them in a communicative manner via wire or wirelessly, and various communication methods can be adopted.

[0024] In the embodiment, an internal information communication network (LAN: Local Area Network) is formed by a plurality of substrate work machines WM0, a line management device LC0, and a management device HC0. Therefore, the plurality of substrate work machines WM0 can communicate with each other via a communication unit. In addition, the plurality of substrate work machines WM0 can communicate with the line management device LC0 via a communication unit. Furthermore, the line management device LC0 and the management device HC0 can communicate with each other via a communication unit.

[0025] The line management device LC0 controls the plurality of substrate work machines WM0 constituting the substrate work line WL0 and monitors the operation status of the substrate work line WL0. Various control data for controlling the plurality of substrate work machines WM0 are stored in the line management device LC0. The line management device LC0 sends control data to the plurality of substrate work machines WM0 respectively. In addition, the plurality of substrate work machines WM0 send the operation status and production status to the line management device LC0 respectively.

[0026] The management device HC0 manages at least one line management device LC0. For example, the operation status and production status of the substrate work machine WM0 obtained by the line management device LC0 are sent to the management device HC0 as needed. A storage device is provided in the management device HC0. The storage device can store various acquired data obtained from the substrate work machine WM0. For example, various image data captured by the substrate work machine WM0 are included in the acquired data. Records (log data) of the operating status obtained by the substrate work machine WM0 are included in the acquired data. In addition, the storage device can store various production information related to the production of the substrate product 900.

[0027] The substrate work line WL0 is provided with an input / output device 80. A known input / output device can be used as the input / output device 80. The input / output device 80 includes a display unit that displays various data in a visually recognizable manner. In addition, the display unit is composed of a touch panel and also functions as an input device that accepts various operations of the operator. 1-2. Configuration example of component mounting machine WM3

[0028] The component mounting machine WM3 mounts a plurality of components 91 on the substrate 90. Figure 2 As shown, the component mounting machine WM3 includes a substrate conveying device 11 , a component supplying device 12 , a component transfer device 13 , a component camera 14 , a substrate camera 15 , and a control device 16 .

[0029] The substrate transport device 11 is composed of, for example, a belt conveyor, and transports the substrate 90 along the transport direction (X-axis direction). The substrate 90 is a circuit substrate, and an electronic circuit, an electric circuit, a magnetic circuit, etc. are formed thereon. The substrate transport device 11 carries the substrate 90 into the component mounting machine WM3 and positions the substrate 90 at a predetermined position in the machine. After the mounting process of the plurality of components 91 by the component mounting machine WM3 is completed, the substrate transport device 11 carries the substrate 90 out of the component mounting machine WM3.

[0030] The component supply device 12 supplies a plurality of components 91 to be mounted on the substrate 90. The component supply device 12 has a plurality of feeders 12a arranged along the conveying direction (X-axis direction) of the substrate 90. A tape reel is provided on each of the plurality of feeders 12a. A carrier tape containing a plurality of components 91 is wound on the tape reel. The feeder 12a feeds the carrier tape in intervals and supplies the components 91 in a manner that allows them to be picked up at a supply position located on the front end side of the feeder 12a. In addition, the component supply device 12 can also supply electronic components (for example, lead components, etc.) that are relatively large compared to chip components, etc., in a state arranged on a tray.

[0031] The component transfer device 13 includes a head drive device 13a and a moving table 13b. The head drive device 13a is configured to move the moving table 13b in the X-axis direction and the Y-axis direction (a direction orthogonal to the X-axis direction in a horizontal plane) through a direct-acting mechanism. On the moving table 13b, a mounting head 20 is provided in a detachable (replaceable) manner through a clamping component. The mounting head 20 uses at least one holding component 30 to pick up and hold the component 91 supplied by the component supply device 12, and mounts the component 91 on the substrate 90 positioned by the substrate conveying device 11. The holding component 30 can use, for example, a suction nozzle, a chuck, etc.

[0032] The component camera 14 and the substrate camera 15 can use a known imaging device. The component camera 14 is fixed to the base of the component mounting machine WM3 in such a manner that the optical axis is oriented upward in the vertical direction (Z-axis direction orthogonal to the X-axis direction and the Y-axis direction). The component camera 14 can photograph the component 91 held by the holding member 30 from below. The substrate camera 15 is set on the movable table 13b of the component transfer device 13 in such a manner that the optical axis is oriented downward in the vertical direction (Z-axis direction). The substrate camera 15 can photograph the substrate 90 from above. The component camera 14 and the substrate camera 15 perform imaging based on a control signal sent from the control device 16. The image data of the captured images captured by the component camera 14 and the substrate camera 15 are sent to the control device 16.

[0033] The control device 16 includes a known computing device and a storage device, and constitutes a control circuit. Information output from various sensors provided in the component mounting machine WM3, image data, etc. are input to the control device 16. The control device 16 sends a control signal to each device based on a control program and predetermined mounting conditions, etc.

[0034] For example, the control device 16 causes the substrate camera 15 to photograph the substrate 90 positioned by the substrate transport device 11. The control device 16 performs image processing on the image photographed by the substrate camera 15 to recognize the positioning state of the substrate 90. In addition, the control device 16 causes the holding member 30 to pick up and hold the component 91 supplied by the component supply device 12, and causes the component camera 14 to photograph the component 91 held by the holding member 30. The control device 16 performs image processing on the image photographed by the component camera 14 to recognize the holding posture of the component 91.

[0035] The control device 16 moves the holding member 30 upward toward the predetermined mounting position preset by the control program, etc. In addition, the control device 16 corrects the predetermined mounting position based on the positioning state of the substrate 90, the holding posture of the component 91, etc., and sets the mounting position of the actual mounting component 91. The predetermined mounting position and the mounting position include the rotation angle in addition to the position (X-axis coordinate and Y-axis coordinate).

[0036] The control device 16 corrects the target position (X-axis coordinate and Y-axis coordinate) and the rotation angle of the holding member 30 according to the mounting position. The control device 16 lowers the holding member 30 at the corrected target position and the corrected rotation angle to mount the component 91 on the substrate 90. The control device 16 performs the mounting process of mounting a plurality of components 91 on the substrate 90 by repeatedly performing the above-mentioned pick-and-place cycle. 1-3. Configuration Example of Foreign Matter Detection Device 70

[0037] like Figure 2 As shown, if foreign matter 92 (for example, other components 91, garbage, etc.) is attached to the mounting position of component 91, the mounting of component 91 may become defective (for example, component 91 is not mounted, floated, tilted, etc.), and it is necessary to determine whether there is foreign matter 92 attached to substrate 90. In this case, it is assumed that a plurality of image data PD0 obtained by imaging the same inspection area CA0 of at least a part of substrate 90 are compared, and the presence or absence of foreign matter 92 is determined based on the difference in feature amounts (for example, brightness of pixels, etc.) of image data PD0.

[0038] However, during the period from when the reference image data PD0 is obtained to when the image data PD0 to be compared with the image data PD0 is obtained, the substrate operation performed by the substrate operation machine WM0 may be stopped. If the state of stopping the substrate operation continues, the state of the substrate 90 may change, and there is a demand to obtain the state of the substrate 90 at the time when the substrate operation is stopped. Therefore, the foreign matter detection device 70 capable of obtaining image data at the time when the substrate operation is stopped is provided in the substrate operation line WL0 of the embodiment.

[0039] If the foreign object detection device 70 is understood as a control block, it includes an acquisition unit 71 and a determination unit 72. The foreign object detection device 70 may also include an estimation unit 73. The foreign object detection device 70 may also include a notification unit 74. Figure 3 As shown, the foreign object detection device 70 of the embodiment includes an acquisition unit 71 , a determination unit 72 , an estimation unit 73 , and a notification unit 74 .

[0040] The foreign object detection device 70 can be provided in various control devices. For example, the foreign object detection device 70 can be provided in the control device 16 of the component mounting machine WM3, the line management device LC0, the management device HC0, etc. The foreign object detection device 70 can also be formed on the cloud. Figure 3 As shown, in the foreign matter detection device 70 of the embodiment, the acquisition unit 71, the determination unit 72, the estimation unit 73, and the notification unit 74 are provided in the control device 16 of the component mounting machine WM3.

[0041] In addition, the foreign matter detection device 70 of the embodiment is Figure 4 The control is executed according to the flowchart shown in FIG. 1 . The acquisition unit 71 performs the processing shown in step S11, step S12, step S16, and step S17. The determination unit 72 performs a part of step S13 and the processing shown in step S18. The estimation unit 73 performs a part of the processing of step S13. The notification unit 74 performs the processing shown in step S14, step S15, step S19, and step S20. 1-3-1. Control example when substrate work is proceeding normally

[0042] The acquisition unit 71 images at least a portion of the inspection area CA0 of the substrate 90 as the substrate operation machine WM0 performs a predetermined substrate operation on the substrate 90, and acquires a plurality of image data PD0 obtained by imaging the same inspection area CA0.

[0043] As the inspection area CA0, the acquisition unit 71 can photograph the entire mounting area of ​​the substrate 90, or can photograph a portion of the mounting area of ​​the substrate 90. When photographing a portion of the mounting area of ​​the substrate 90, the acquisition unit 71 can, for example, photograph the mounting area of ​​a component 91 (for example, a BGA (Ball Grid Array) component 91) that has a larger number of electrodes than a chip component and is easily affected by foreign matter 92. In addition, the acquisition unit 71 can also know, based on past mounting performance, mounting areas where poor mounting of the component 91 has occurred due to foreign matter 92, mounting areas where foreign matter 92 is easily attached, and photograph these mounting areas as the inspection area CA0.

[0044] The acquisition unit 71 can also set the area designated by the operator of the component mounting machine WM3 that mounts the component 91 on the substrate 90 as the inspection area CA0. The substrate operation line WL0 of the embodiment includes an input-output device 80. For example, the operator can designate an arbitrary area (the entire mounting area of ​​the substrate 90 or the mounting area of ​​a part of the substrate 90) as the inspection area CA0 using the input-output device 80. In this case, the acquisition unit 71 causes the display unit of the input-output device 80 to schematically display the mounting area of ​​the component 91 on the substrate 90, and the operator can select an arbitrary mounting area.

[0045] The acquisition unit 71 can acquire the image data PD0 using a photographing device that can photograph the inspection area CA0. The photographing device is not limited as long as it can photograph the inspection area CA0. For example, the photographing device includes a substrate camera 15 that can photograph a portion of the mounting area of ​​the substrate 90 from above the substrate 90, a ceiling camera that can photograph the entire mounting area of ​​the substrate 90 from above the substrate 90, and the like. In the embodiment, the acquisition unit 71 sets the area specified by the operator of the component mounting machine WM3 as the inspection area CA0 using the substrate camera 15. In addition, the acquisition unit 71 makes the photographing conditions (such as exposure time, aperture, lighting time, etc.) that can be set by the photographing device the same, photographs the same inspection area CA0 multiple times, and acquires multiple image data PD0.

[0046] The determination unit 72 determines whether there is a foreign object 92 attached to the inspection area CA0 based on the difference in the feature amount of the inspection area CA0 obtained by performing image processing on the reference image data SD0 and the inspection image data CD0. The reference image data SD0 refers to one image data PD0 among the plurality of image data PD0 obtained by the acquisition unit 71. The inspection image data CD0 refers to the image data PD0 of the inspection object obtained by the acquisition unit 71 after the reference image data SD0.

[0047] The determination unit 72 can use various methods as long as it can determine whether there is a foreign object 92 attached to the inspection area CA0 based on the difference in the feature quantity of the inspection area CA0. For example, when the difference in the feature quantity of the inspection area CA0 obtained from the two image data PD0 (in this case, the reference image data SD0 and the inspection image data CD0) exceeds a predetermined threshold, the determination unit 72 determines that there is a foreign object 92 in the inspection area CA0. When the difference in the feature quantity is less than the predetermined threshold, the determination unit 72 determines that there is no foreign object 92 in the inspection area CA0.

[0048] The feature quantity is not limited as long as it can be obtained by performing image processing on the image data PD0. The brightness, chroma, lightness, etc. of each pixel of the two image data PD0 (reference image data SD0 and inspection image data CD0) are included in the feature quantity. In addition, the area of ​​the closed region, the length of the outer periphery of the closed region, etc. obtained by performing image processing (such as binarization processing, etc.) on each of the two image data PD0 (reference image data SD0 and inspection image data CD0) are included in the feature quantity. In the embodiment, the feature quantity is the brightness of each pixel of the two image data PD0 (reference image data SD0 and inspection image data CD0).

[0049] The predetermined threshold is set to be larger than the feature value (e.g., pixel brightness) when the inspection area CA0 is free of foreign matter 92 and smaller than the feature value when the inspection area CA0 is attached with foreign matter 92. The predetermined threshold is obtained in advance, for example, through simulation, actual machine verification, etc.

[0050] Figure 5 and Figure 6 An example of a plurality of (two) pieces of image data PD0 acquired by the acquisition unit 71 is schematically shown. Figure 5 shows reference image data SD0, Figure 6 The inspection image data CD0 is shown. Figure 5 and Figure 6 In FIG. 1 , for the sake of convenience, a plurality of pixels arranged in a grid are also shown. Figure 5 and Figure 6 The illustrated area AR0 represents the same area (a set of the same plurality of pixels) in the inspection area CA0.

[0051] When there is a foreign object 92 in the area AR0, Figure 5 The brightness of the pixels contained in the area AR0 shown is Figure 6 The difference in brightness of the pixels included in the area AR0 shown exceeds the specified threshold. On the contrary, when there is no foreign matter 92 in the area AR0, Figure 5 The brightness of the pixels contained in the area AR0 shown is Figure 6The difference in brightness of pixels included in the illustrated area AR0 is equal to or smaller than a predetermined threshold.

[0052] Therefore, the judgment unit 72 Figure 5 The brightness of the pixels contained in the area AR0 shown is Figure 6 When the difference in brightness of the pixels included in the area AR0 shown in FIG. 1 exceeds a predetermined threshold value, it is determined that a foreign object 92 is present in the inspection area CA0. The determination unit 72 Figure 5 The brightness of the pixels contained in the area AR0 shown is Figure 6 When the difference in brightness of pixels included in the illustrated area AR0 is equal to or smaller than a predetermined threshold value, it is determined that the inspection area CA0 does not contain the foreign matter 92. In addition, brightness comparison is performed for each corresponding pixel.

[0053] The acquisition unit 71 can acquire the reference image data SD0 and the inspection image data CD0 at a predetermined timing. Figure 1 As shown, the substrate 90 is sequentially conveyed to a plurality of (three in this figure) component mounting machines WM3 as the substrate operation machine WM0, and a plurality of components 91 are mounted. For convenience of explanation, the component mounting machine WM3 on the most upstream side among the plurality of (three) component mounting machines WM3 is set as the component mounting machine M1. In addition, the next component mounting machine WM3 on the downstream side of the component mounting machine M1 is set as the component mounting machine M2. And, the next component mounting machine WM3 on the downstream side of the component mounting machine M2 is set as the component mounting machine M3.

[0054] In the above-mentioned substrate working line WL0, for example, the acquisition unit 71 acquires the reference image data SD0 in the component mounting machine WM3 (component mounting machine M1) on the most upstream side among the plurality of (3) component mounting machines WM3. More specifically, the acquisition unit 71 acquires the reference image data SD0 by photographing the inspection area CA0 before the mounting process of the component 91 in the component mounting machine M1 starts.

[0055] In addition, the acquisition unit 71 acquires the inspection image data CD0 in the component mounting machine WM3 (for example, the component mounting machine M3) that mounts the component 91 to the inspection area CA0. More specifically, the acquisition unit 71 photographs the inspection area CA0 before the component mounting machine M3 mounts the component 91 to be mounted in the inspection area CA0 to acquire the inspection image data CD0. In this case, the determination unit 72 can determine whether there is a foreign object 92 attached to the inspection area CA0 during the period when the substrate 90 is transported across multiple (three) component mounting machines WM3. In this specification, the above method is referred to as the first method.

[0056] The acquisition unit 71 can also acquire the reference image data SD0 and the inspection image data CD0 in each of the plurality of (three) component mounting machines WM3. The acquisition unit 71 acquires the reference image data SD0 by photographing the inspection area CA0 before the mounting process of the component 91 is started in each of at least one component mounting machine WM3 (component mounting machine M1 and component mounting machine M2) provided on the upstream side of the component mounting machine WM3 (for example, component mounting machine M3) that mounts the component 91 in the inspection area CA0. The acquisition unit 71 acquires the inspection image data CD0 by photographing the inspection area CA0 after the mounting process of the component 91 is completed in each of the above-mentioned component mounting machines WM3 (component mounting machine M1 and component mounting machine M2).

[0057] In addition, the acquisition unit 71 acquires reference image data SD0 by photographing the inspection area CA0 before the mounting process of the component 91 is started in the component mounting machine WM3 (for example, the component mounting machine M3) that mounts the component 91 in the inspection area CA0. The acquisition unit 71 acquires inspection image data CD0 by photographing the inspection area CA0 before the component 91 to be mounted is mounted in the inspection area CA0 in the above-mentioned component mounting machine WM3 (component mounting machine M3). In these cases, the judgment unit 72 can judge whether there is a foreign object 92 attached to the inspection area CA0 in each of the plurality of (three) component mounting machines WM3. In this specification, the above-mentioned method is referred to as the second method.

[0058] The acquisition unit 71 can also acquire the reference image data SD0 and the inspection image data CD0 between adjacent component mounting machines WM3. The acquisition unit 71 acquires the reference image data SD0 in one component mounting machine WM3 among the plurality of (three) component mounting machines WM3. More specifically, the acquisition unit 71 acquires the reference image data SD0 by photographing the inspection area CA0 after the mounting process of the component 91 in the component mounting machine WM3 is completed and before the substrate 90 is unloaded.

[0059] In addition, the acquisition unit 71 acquires the inspection image data CD0 in the next component mounting machine WM3 on the downstream side of the component mounting machine WM3. More specifically, the acquisition unit 71 acquires the inspection image data CD0 by photographing the inspection area CA0 after the substrate 90 is carried into the component mounting machine WM3 and before the mounting process of the component 91 in the component mounting machine WM3 is started. In these cases, the judgment unit 72 can judge whether there is a foreign object 92 attached to the inspection area CA0 when the substrate 90 is conveyed between adjacent component mounting machines WM3. In this specification, the above method is referred to as the third method. 1-3-2. Example of control when substrate operation is stopped

[0060] For example, the substrate work machine WM0 may stop working on the substrate when the image data PD0 acquired by the imaging device (for example, the component camera 14, the substrate camera 15, etc. in the component mounting machine WM3) is processed and the result is poor. In addition, the substrate work machine WM0 may stop working on the substrate when a situation occurs in which the substrate work cannot be continued (for example, in the component mounting machine WM3, the components 91 to be mounted are insufficient). In addition, the operator may operate the stop button to stop the substrate work of the substrate work machine WM0.

[0061] When the substrate operation is stopped during the period from the acquisition of the reference image data SD0 to the acquisition of the inspection image data CD0, the acquisition unit 71 captures the inspection area CA0 and acquires the first image data PD1 (at Figure 4 1 and step S12). The first image data PD1 refers to the image data PD0 at the time when the substrate operation is stopped. As in the case where the substrate operation is normally performed, the acquisition unit 71 can capture the inspection area CA0 at the timing when the substrate operation is stopped and acquire the first image data PD1. As a result, the foreign matter detection device 70 can acquire the image data PD0 at the time when the substrate operation is stopped, and can grasp the state of the substrate 90 at the time when the substrate operation is stopped.

[0062] In addition, the determination unit 72 can determine the presence or absence of the foreign matter 92 ( Figure 4 The determination unit 72 can determine the presence or absence of the foreign matter 92 in the same manner as when the substrate operation is normally performed.

[0063] Specifically, when the difference between the feature quantities of the inspection area CA0 obtained based on the two image data PD0 (in this case, the reference image data SD0 and the first image data PD1) exceeds a predetermined threshold, the judgment unit 72 judges that there is a foreign object 92 in the inspection area CA0. When the difference between the feature quantities is less than a predetermined threshold, the judgment unit 72 judges that there is no foreign object 92 in the inspection area CA0. In addition, as described above, the feature quantity is not limited. The feature quantity of the embodiment is the brightness of each pixel of the two image data PD0 (reference image data SD0 and first image data PD1).

[0064] Figure 7An example of the state of the substrate 90 before the work on the substrate is stopped is shown. In this figure, taking the first mode as an example, the state of the substrate 90 before the work on the substrate is stopped in the component mounting machine M2 is schematically shown using the image data PD0. Specifically, in the component mounting machine M1 on the most upstream side among the plurality of (three) component mounting machines WM3, the acquisition unit 71 acquires the reference image data SD0.

[0065] Furthermore, it is assumed that the substrate 90 is conveyed to the next component mounting machine M2 on the downstream side of the component mounting machine M1, and the substrate operation is stopped in the component mounting machine M2. In the component mounting machine M2 that has stopped the substrate operation, the acquisition unit 71 captures the inspection area CA0 at the timing of stopping the substrate operation and obtains the first image data PD1. In the reference image data SD0 and the first image data PD1 shown in the figure, the difference in the feature quantity of the inspection area CA0 is small (the difference in the feature quantity is below the specified threshold), and the judgment unit 72 judges that there is no foreign matter 92 in the inspection area CA0. In addition, in the figure, in the next component mounting machine M3 on the downstream side of the component mounting machine M2, the substrate 90 that has not yet been conveyed is represented by a blank in the image data PD0.

[0066] In this way, the determination unit 72 can unconditionally determine the presence or absence of foreign matter 92 when the acquisition unit 71 acquires the first image data PD1. Thus, the determination unit 72 can determine the presence or absence of foreign matter 92 attached to the inspection area CA0 from the time when the acquisition unit 71 acquires the reference image data SD0 to the time when the substrate operation is stopped. In addition, as shown below, the determination unit 72 can also determine the presence or absence of foreign matter 92 when a predetermined condition is satisfied. For example, the feature value of the inspection area CA0 acquired by performing image processing on the image data PD0 may change due to the change over time of the bonding material 93 applied to the inspection area CA0 of the substrate 90 to bond the substrate 90 to the element 91.

[0067] The bonding material 93 is not limited as long as it is a material for bonding the substrate 90 and the element 91. For example, solder, adhesive, etc. are included in the bonding material 93. In the embodiment, the bonding material 93 is solder. The solder changes color from silver to gray as the flux contained therein dries. Therefore, the characteristic value of the inspection area CA0 may change due to the change of the solder over time, and the judgment unit 72 may mistakenly judge the solder as a foreign matter 92.

[0068] Here, the time from when the work on the substrate is stopped to when it is restarted is set as the stop time Q0. In addition, even if the characteristic value of the inspection area CA0 changes due to the time-dependent change of the bonding material 93 applied to the inspection area CA0 to bond the substrate 90 and the component 91, the stop time Q0 is set as the allowable time T0. The shorter the stop time Q0 is than the allowable time T0, the less likely it is to be affected by the time-dependent change of the bonding material 93 when judging the presence or absence of the foreign matter 92 in the component mounting machine M3, and the degree of necessity to judge the presence or absence of the foreign matter 92 in the component mounting machine M2 that has stopped the work on the substrate is reduced.

[0069] On the contrary, if the stop time Q0 is longer than the allowable time T0, when judging the presence or absence of foreign matter 92 in the component mounting machine M3, it is easy to be affected by the change of the bonding material 93 over time, and the judging unit 72 is easy to mistakenly judge the bonding material 93 as foreign matter 92. Therefore, the necessity of judging the presence or absence of foreign matter 92 in the component mounting machine M2 that has stopped the work on the substrate increases. Therefore, the estimating unit 73 estimates the stop time Q0 ( Figure 4 Step S13 shown).

[0070] The estimation unit 73 can adopt various methods as long as it can estimate the stop time Q0. For example, the estimation unit 73 can estimate the stop time Q0 based on the main reason for stopping the work on the substrate (for example, the result of image processing of the image data PD0 is poor, etc.). In this case, the estimation unit 73 stores the main reason for stopping the work on the substrate and the stop time Q0 from the stop of the work on the substrate to the restart of the work on the substrate in advance each time the substrate product 900 is produced. The stop time Q0 can be represented by an average value, a median value, etc., for example.

[0071] When the substrate operation is stopped during the production of the substrate product 900, the estimating unit 73 outputs the stop time Q0 (for example, an average value or a median value) stored in correspondence with the main reason for the stop of the substrate operation to the judging unit 72. In addition, when the stop time Q0 estimated by the estimating unit 73 to be longer than the allowable time T0, the judging unit 72 can judge whether there is a foreign object 92 ( Figure 4 Step S13 shown).

[0072] In addition, for example, when the operator is responding to the substrate operation stop, the actual stop time Q0 may be longer than the stop time Q0 estimated by the estimating unit 73. Therefore, the judging unit 72 can judge the presence or absence of the foreign matter 92 even when the estimating unit 73 estimates the stop time Q0 within the allowable time T0 and then estimates the stop time Q0 longer than the allowable time T0.

[0073] In addition, the allowable time T0 can be acquired in advance, for example, by simulation, verification of an actual machine, etc. In addition, the allowable time T0 may be different depending on the type of the bonding material 93, and the determination unit 72 may use the allowable time T0 corresponding to the type of the bonding material 93. For example, the determination unit 72 may use the allowable time T0 corresponding to the type of solder. In addition, the determination unit 72 may use the allowable time T0 corresponding to the type of adhesive.

[0074] Furthermore, the higher the temperature inside the component mounting machine WM3, the faster the drying of the flux contained in the solder becomes. In addition, the lower the humidity inside the component mounting machine WM3, the faster the drying of the flux contained in the solder becomes. Therefore, the determination unit 72 can also use the allowable time T0 corresponding to at least one of the temperature and humidity inside the component mounting machine WM3 that mounts the component 91 on the substrate 90.

[0075] When the determination unit 72 determines that there is a foreign object 92 in the inspection area CA0, the notification unit 74 notifies the operator of the presence of the foreign object 92 (in Figure 4 The notification unit 74 may be in various forms as long as it can notify the operator of the presence of the foreign matter 92. For example, the substrate operation line WL0 of the embodiment includes an input / output device 80. The notification unit 74 can use the input / output device 80 to notify the operator of the presence of the foreign matter 92 (for example, by display, sound notification, etc.).

[0076] Specifically, the notification unit 74 can notify the existence of the substrate 90 with the foreign matter 92 attached, the position of the foreign matter 92 attached in the substrate 90, the substrate handling machine WM0 where the substrate 90 exists, etc. Thus, the operator can confirm the substrate 90 with the foreign matter 92 attached, and can take measures (for example, carry out the substrate 90, etc.). In addition, the notification unit 74 can also use the portable terminal owned by the operator to similarly notify the operator (for example, display, sound notification, vibration, etc.) of the existence of the foreign matter 92.

[0077] When the work on the substrate is restarted, the acquisition unit 71 captures the inspection area CA0 to acquire the second image data PD2, and updates the reference image data SD0 using the second image data PD2 (in Figure 4 The second image data PD2 refers to the image data PD0 at the time when the substrate operation is restarted. As in the case where the substrate operation is normally performed, the acquisition unit 71 can capture the inspection area CA0 at the time when the substrate operation is restarted and acquire the second image data PD2. Thus, the foreign matter detection device 70 can acquire the image data PD0 at the time when the substrate operation is restarted.

[0078] As described above, if the stop time Q0 from when the work on the substrate is stopped to when it is restarted is longer than the allowed time T0, when judging the presence or absence of the foreign matter 92 in the component mounting machine M3, it is easy to be affected by the change over time of the bonding material 93, and the judging unit 72 is easy to mistakenly judge the bonding material 93 as the foreign matter 92. Therefore, when the stop time Q0 is longer than the allowed time T0, the acquiring unit 71 can update the reference image data SD0 using the second image data PD2.

[0079] In addition, even if the stop time Q0 from when the substrate operation is stopped to when it is restarted is within the allowable time T0, the bonding material 93 changes over time. Therefore, regardless of the stop time Q0, when the substrate operation is restarted, the acquisition unit 71 can capture the inspection area CA0 to acquire the second image data PD2, and update the reference image data SD0 using the second image data PD2.

[0080] The determination unit 72 determines the presence or absence of the foreign matter 92 ( Figure 4 The determination unit 72 can determine the presence or absence of the foreign matter 92 in the same manner as when the substrate operation is normally performed.

[0081] Specifically, when the difference between the feature quantities of the inspection area CA0 obtained from the two image data PD0 (in this case, the updated reference image data SD0 and the inspection image data CD0) exceeds a predetermined threshold, the judgment unit 72 judges that there is a foreign object 92 in the inspection area CA0. When the difference between the feature quantities is less than a predetermined threshold, the judgment unit 72 judges that there is no foreign object 92 in the inspection area CA0. In addition, as described above, the feature quantity is not limited. The feature quantity of the embodiment is the brightness of each pixel of the two image data PD0 (the updated reference image data SD0 and the inspection image data CD0).

[0082] Figure 8 An example of the state of the substrate 90 after the substrate operation is restarted is shown. In this figure, taking the first mode as an example, the state of the substrate 90 after the substrate operation is restarted in the component mounting machine M2 is schematically shown using the image data PD0. Specifically, in the component mounting machine M2 where the substrate operation is restarted, the acquisition unit 71 captures the inspection area CA0 at the timing of restarting the substrate operation to acquire the second image data PD2, and updates the reference image data SD0 using the second image data PD2.

[0083] In this figure, for the sake of convenience, the bonding material 93 is shown in black, schematically showing that the bonding material 93 changes over time during the period when the work on the substrate is stopped. In addition, the substrate 90 is conveyed to the next component mounting machine M3 on the downstream side of the component mounting machine M2. In the component mounting machine M3, the acquisition unit 71 acquires the inspection image data CD0. Assuming that the reference image data SD0 is not updated using the second image data PD2, the judgment unit 72 determines the inspection image data CD0 based on the inspection image data CD0. Figure 7 The reference image data SD0 and Figure 8 The presence or absence of the foreign matter 92 is determined based on the difference in the feature amount of the inspection area CA0 obtained by performing image processing on the inspection image data CD0 shown.

[0084] In this case, Figure 7 The characteristic amount of the inspection area CA0 (the bonding material 93 is white) obtained by performing image processing on the reference image data SD0 shown in FIG. Figure 8 The difference in the characteristic quantity (bonding material 93 is black) of the inspection area CA0 obtained by performing image processing on the inspection image data CD0 shown in FIG. 1 is large, and the difference in the characteristic quantity exceeds the predetermined threshold value. Therefore, the determination unit 72 will mistakenly determine that there is a foreign object 92 in the inspection area CA0. In contrast, when the reference image data SD0 is updated using the second image data PD2, the determination unit 72 makes a false determination based on the reference image data SD0. Figure 8 The presence or absence of the foreign matter 92 is determined based on the difference in the feature amount of the inspection area CA0 obtained by performing image processing on the reference image data SD0 and the inspection image data CD0 shown.

[0085] In this case, Figure 8 The characteristic value of the inspection area CA0 (the bonding material 93 is black) obtained by performing image processing on the reference image data SD0 shown in FIG. Figure 8 The difference in the characteristic quantity (bonding material 93 is black) of the inspection area CA0 obtained by image processing of the inspection image data CD0 shown is small, and the difference in the characteristic quantity is below the specified threshold. Therefore, the judgment unit 72 judges that there is no foreign matter 92 in the inspection area CA0. That is, it is possible to suppress the erroneous judgment of the foreign matter 92 caused by the change of the bonding material 93 over time. In addition, Figure 8 The state of the substrate 90 after the work on the substrate is restarted is shown, and the image data PD0 of the component mounting machine M1 on the upstream side of the component mounting machine M2 is displayed as blank.

[0086] When the determination unit 72 determines that there is a foreign object 92 in the inspection area CA0, the notification unit 74 notifies the operator of the presence of the foreign object 92 (in Figure 4In the case of YES in step S19 and step S20 shown in FIG. 1 , the notification unit 74 can notify the operator of the presence of the foreign matter 92 in the same manner as when notifying the operator of the presence of the foreign matter 92 immediately after the work on the substrate is stopped. Then, the control of the foreign matter detection device 70 is temporarily terminated.

[0087] In addition, Figure 4 If the answer is no in step S11 and step S19, the control of the foreign matter detection device 70 is temporarily terminated. If the answer is no in step S14, the control of the foreign matter detection device 70 enters the control shown in step S16. If the answer is no in step S16, the control of the foreign matter detection device 70 returns to the control shown in step S16 and waits until the substrate operation is restarted.

[0088] In addition, the matters described in this specification can be appropriately combined and can be selected. For example, in this specification, the first method is used as an example for explanation, but the matters described in this specification can also be applied to the second method or the third method. In addition, the matters described in this specification can also be applied to a method formed by combining the second method and the third method. It should be noted that in the third method, the substrate operation is stopped when the substrate 90 is transported between adjacent component mounting machines WM3, so a shooting device capable of obtaining the first image data PD1 and the second image data PD2 is provided between adjacent component mounting machines WM3. In addition, in the first method, the second method and the third method, when the substrate operation is not stopped and is carried out normally, a method can be selected in which the time required from the acquisition of the reference image data SD0 by the acquisition unit 71 to the acquisition of the inspection image data CD0 is shorter than the above-mentioned allowed time T0. 2. Foreign matter detection method

[0089] The contents already described about the foreign matter detection device 70 are also the same for the foreign matter detection method. Specifically, the foreign matter detection method includes an acquisition process and a judgment process. The acquisition process is equivalent to the control performed by the acquisition unit 71. The judgment process is equivalent to the control performed by the judgment unit 72. In addition, the foreign matter detection method can include an estimation process. The estimation process is equivalent to the control performed by the estimation unit 73. The foreign matter detection method can also include a notification process. The notification process is equivalent to the control performed by the notification unit 74. 3. An example of the effect of the embodiment

[0090] According to the foreign matter detection device 70, the image data PD0 at the time when the work on the substrate is stopped, that is, the first image data PD1 can be acquired. The above contents about the foreign matter detection device 70 are also the same as the foreign matter detection method. Description of Reference Numerals

[0091] 70: foreign matter detection device, 71: acquisition unit, 72: judgment unit, 73: estimation unit, 74: notification unit, 90: substrate, 91: component, 92: foreign matter, 93: bonding material, CA0: inspection area, PD0: image data, SD0: reference image data, CD0: inspection image data, PD1: first image data, PD2: second image data, Q0: stop time, T0: allowed time, WM0: substrate operation machine, WM3: component mounting machine.

Claims

1. A foreign body detection device, comprising: an acquisition unit that captures an inspection area of ​​at least a portion of the substrate as a substrate operation machine performs a predetermined substrate operation on the substrate, and acquires a plurality of image data obtained by capturing the same inspection area; and The determination unit determines whether there is a foreign object attached to the inspection area based on a difference in feature amounts of the inspection area obtained by performing image processing on one image data among the plurality of image data, namely, a reference image data, and image data of the inspection object obtained after the reference image data, namely, the inspection image data, respectively, to determine whether there is a foreign object attached to the inspection area. When the substrate operation is stopped during the period from when the reference image data is acquired to when the inspection image data is acquired, the acquisition unit captures the inspection region and acquires first image data, which is image data at the time when the substrate operation is stopped.

2. The foreign matter detection device according to claim 1, wherein: The determination unit determines the presence or absence of the foreign matter based on a difference in the feature amount of the inspection region obtained by performing image processing on the reference image data and the first image data.

3. The foreign matter detection device according to claim 2, wherein: The foreign matter detection device includes an estimating unit that estimates a stop time from when the operation on the substrate is stopped to when it is restarted. When it is estimated that the stop time is longer than the allowed time, the judgment unit judges the presence or absence of the foreign matter, and the allowed time is a time during which the bonding material that bonds the substrate and the element to each other and is applied to the inspection area will not be misjudged as the foreign matter even if the characteristic value of the inspection area changes due to the temporal change of the bonding material.

4. The foreign matter detection device according to claim 2, wherein: The foreign matter detection device includes an estimating unit that estimates a stop time from when the operation on the substrate is stopped to when it is restarted. When it is estimated that the stop time is longer than the allowed time after it is estimated that the stop time is within the allowed time, the judgment unit judges whether the foreign matter exists, and the allowed time is a time during which the bonding material that bonds the substrate and the element to each other and is applied to the inspection area will not be misjudged as the foreign matter even if the characteristic value of the inspection area changes due to the temporal change of the bonding material.

5. The foreign matter detection device according to any one of claims 1 to 4, wherein: The foreign object detection device includes a notification unit configured to notify an operator of the presence of the foreign object when the determination unit determines that the foreign object is present in the inspection region.

6. The foreign matter detection device according to claim 1, wherein: When the work on the substrate is resumed, the acquisition unit captures the inspection region to acquire second image data, which is image data at the time when the work on the substrate is resumed, and updates the reference image data using the second image data.

7. The foreign matter detection device according to claim 6, wherein: In a case where the stop time from when the operation on the substrate is stopped to when it is restarted is longer than the allowed time, the acquisition unit updates the reference image data using the second image data, and the allowed time is a time during which the bonding material that bonds the substrate to the component and is applied to the inspection area will not be misjudged as the foreign matter even if the characteristic value of the inspection area changes due to the temporal change of the bonding material.

8. The foreign matter detection device according to claim 6 or 7, wherein: The determination unit determines the presence or absence of the foreign matter based on a difference in the feature amount of the inspection region obtained by performing image processing on each of the updated reference image data and the inspection image data.

9. The foreign matter detection device according to claim 1, wherein: When the difference between the characteristic quantities of the inspection area obtained based on the two image data exceeds a specified threshold, the judgment unit judges that the inspection area contains the foreign matter; when the difference between the characteristic quantities is below the specified threshold, the judgment unit judges that the inspection area contains no foreign matter.

10. The foreign matter detection device according to claim 9, wherein: The feature amount is the brightness of each pixel of two image data.

11. The foreign matter detection device according to claim 1, wherein: The substrate is sequentially conveyed to a plurality of component mounting machines as the substrate operation machine, and a plurality of components are mounted thereon. The acquisition unit acquires the reference image data in a component mounting machine on the most upstream side among the plurality of component mounting machines, and acquires the inspection image data in a component mounting machine that mounts the component in the inspection area. The determination unit determines whether the foreign matter is attached to the inspection area while the substrate is being transported across the plurality of component mounting machines.

12. The foreign matter detection device according to claim 1, wherein: The substrate is sequentially conveyed to a plurality of component mounting machines as the substrate operation machine, and a plurality of components are mounted thereon. The acquisition unit acquires the reference image data and the inspection image data in each of the plurality of component mounting machines. The determination unit determines whether or not the foreign matter is attached to the inspection area in each of the plurality of component mounting machines.

13. The foreign matter detection device according to claim 1, wherein: The substrate is sequentially conveyed to a plurality of component mounting machines as the substrate operation machine, and a plurality of components are mounted thereon. The acquisition unit acquires the reference image data in one of the plurality of component mounters, and acquires the inspection image data in a next component mounter on the downstream side of the one component mounter. The determination unit determines whether or not the foreign matter is attached to the inspection area when the substrate is conveyed between adjacent component mounting machines.

14. A foreign body detection method comprising: an acquisition step of photographing at least a portion of an inspection region of the substrate as a substrate operation machine performs a predetermined substrate operation on the substrate, and acquiring a plurality of image data obtained by photographing the same inspection region; and a judging step of judging whether there is a foreign object attached to the inspection area based on a difference in feature quantities of the inspection area obtained by performing image processing on one image data among the plurality of image data, namely the reference image data, and image data of the inspection object obtained after the reference image data, namely the inspection image data, respectively, to determine whether there is a foreign object attached to the inspection area; The acquiring step acquires first image data, which is image data at the time when the substrate operation is stopped, by photographing the inspection region when the reference image data is acquired and when the inspection image data is acquired.

Images