Solder inspection device

Abstract

Provided is a solder inspection device with which it is possible to reduce labor and burden in obtaining an identification means, and to allow for common use of the identification means for different land sizes. Inspection image data based on image data acquired by means of a camera (32d) is input to an AI model (101) generated by training a prescribed neural network using only image data relating to non-defective cream solder as training data, and reconstructed image data is acquired. The inspection image data and the reconstructed image data are compared to determine the quality of cream solder. The training data and the inspection image data comprise each of four extracted images which, corresponding to four sides of a land, are provided in an image frame. An image frame related to the training data and an image frame related to the inspection image data have the same size.

Description

Solder inspection device

[0001] The present invention relates to a solder inspection device for inspecting the cream solder provided on a substrate.

[0002] Generally, in a substrate manufacturing line for mounting electronic components on a printed circuit board, first, cream solder is printed on the lands of the printed circuit board (solder printing process). Next, the electronic components are temporarily fixed on the printed circuit board based on the viscosity of the cream solder (mounting process). Then, the printed circuit board is introduced into a reflow furnace, and soldering is performed by heating and melting the cream solder (reflow process). In such a substrate manufacturing line, an inspection device for inspecting the printed circuit board may be provided.

[0003] Recently, as an inspection device for inspecting a printed circuit board, a device using an AI model has been proposed. As an inspection device using an AI model, for example, by comparing inspection image data (original image data) of an inspection area on a printed circuit board with reconstructed image data generated by inputting the inspection image data into an AI model (identification means), a device for inspecting the presence or absence of foreign matter on the printed circuit board is known (see, for example, Patent Document 1, etc.).

[0004] Japanese Unexamined Patent Application Publication No. 2022-88818

[0005] By the way, if there is an abnormality in the shape (two-dimensional shape or three-dimensional shape) of the cream solder, or if foreign matter adheres to the land before printing the cream solder, there is a risk of defective mounting of electronic components. Therefore, it is conceivable to use the above AI model to inspect whether there is an abnormality in the cream solder that may lead to defective mounting of electronic components before mounting the electronic components.

[0006] However, in order to obtain an AI model, it is necessary to perform learning using image data related to cream solder as learning data. However, since the shapes of individual cream solders are different from each other, if image data related to the entire cream solder is used as learning data, a large number of image data must be prepared to obtain the desired accuracy, and efficient learning is difficult.

[0007] Furthermore, in terms of improving inspection accuracy, it is preferable to match the size (width and height) of the training data with the size of the inspection image data. However, if the sizes of the training data and inspection image data are matched by scaling the image data, inspection accuracy may decrease. To address this, it is conceivable to prepare various training data of different sizes corresponding to the sizes of various lands, and to separately prepare multiple AI models that have been trained only on training data of the same size. Then, during inspection, inspection image data of a size corresponding to the size of the land is acquired, and an appropriate AI model that matches the size of the land (i.e., the inspection image data) is selected from among the multiple AI models and used. However, preparing different AI models for each size of land (inspection image data) requires complicated work and considerable effort and time. In addition, obtaining multiple AI models requires preparing a huge amount of training data.

[0008] The present invention has been made in view of the above circumstances, and its purpose is to provide a solder inspection device that can reduce the effort and burden required to obtain an identification means as an AI model, and that can use the identification means in common even when the land sizes are different.

[0009] Below, we will describe, in separate sections, each means suitable for achieving the above objectives. Furthermore, we will add notes on the effects and benefits specific to each means as needed.

[0010] Means 1. A solder inspection device for inspecting abnormalities in solder paste printed on a printed circuit board, comprising: an image data acquisition means capable of acquiring image data of a predetermined area to be inspected on the printed circuit board, including solder paste printed on rectangular lands; an identification means generated by training a neural network having an encoding unit for extracting feature quantities from input image data and a decoding unit for reconstructing image data from the feature quantities, using only image data relating to good solder paste as training data; an inspection image data acquisition means for acquiring inspection image data including an image of the solder paste to be inspected based on the image data acquired by the image data acquisition means; a reconstructed image data acquisition means capable of acquiring reconstructed image data as reconstructed image data by inputting the inspection image data to the identification means; and a comparison means capable of comparing the inspection image data and the reconstructed image data, wherein the device is configured to determine whether or not there are abnormalities in the solder paste based on the comparison result by the comparison means, and the training data consists of four extracted images, each containing the outer edge of the solder paste and corresponding to the four sides of the land, extracted from a solder image which is an image relating to solder paste corresponding to one land, and each of these images is placed in an image frame larger than the size of the extracted image. The solder inspection apparatus is characterized in that the inspection image data acquisition means acquires inspection image data in which each of the four extracted images extracted from the solder image in the image data acquired by the image data acquisition means is placed in an image frame of the same size as the image frame of the learning data.

[0011] Furthermore, the training data may be generated from image data (actual image data) obtained by imaging a printed circuit board with good solder paste printed on it (i.e., an image of actual solder paste), or it may be a virtually generated image of good solder paste. Examples of the actual image data include image data accumulated from previous inspections, and image data of good printed circuit boards that have been visually sorted by an operator after the solder paste has been printed.

[0012] Furthermore, the above-mentioned "neural network" includes, for example, a convolutional neural network having multiple convolutional layers. The above-mentioned "learning" includes, for example, deep learning. The above-mentioned "discrimination means (generative model)" includes, for example, an autoencoder or a convolutional autoencoder.

[0013] In addition, the "identification means" is generated by training it only with image data related to good solder paste. Therefore, when inspection image data related to defective solder paste is input to the identification means, the reconstructed image data generated will be almost identical to the inspection image data after the defective parts have been corrected (for example, foreign matter has been removed, or the shape and size have been corrected). In other words, when there are defective parts in the solder paste, the reconstructed image data related to the solder paste will be a hypothetical image data of the solder paste assuming that there are no defective parts.

[0014] According to the above-described method 1, the inspection image data is provided by placing the extracted image within an image frame. Therefore, the size (width and height) of the inspection image data will match the size of the image frame and will remain constant without fluctuating significantly depending on the land size. This eliminates the need to prepare numerous different identification means for each land size, reducing the effort and time required to obtain the identification means. Furthermore, the identification means can be used in common even when the land sizes are different.

[0015] Furthermore, according to the above means 1, the image frame of the training data and the image frame of the inspection image data are the same size, and the sizes of the training data and the inspection image data are identical. Therefore, when the inspection image data is input to the identification means, the appropriate reconstructed image data corresponding to the inspection image data can be output more reliably, and consequently, the presence or absence of abnormalities in the solder paste can be determined more accurately. As a result, good inspection accuracy can be obtained more reliably.

[0016] Furthermore, the training data consists of extracted images corresponding to each of the four sides of a land, placed within an image frame. Therefore, compared to using image data of the entire solder paste as training data, a recognition method with the desired accuracy can be obtained with less training data. Moreover, since four training data can be obtained from a single solder image, the required number of training data can be easily obtained. These factors combined effectively reduce the effort and time required to obtain the recognition method.

[0017] In addition, since the training data and inspection image data consist of four extracted images corresponding to the four sides of the land, each placed in an image frame, the size of both data can be made relatively small compared to an image of the entire solder paste placed in an image frame. Therefore, the processing burden related to training and inspection can be reduced, and training and inspection can be expedited.

[0018] Means 2. The solder inspection apparatus according to Means 1, characterized in that the extracted images in the learning data and the inspection image data are set so that the outer edge of the solder paste is in a predetermined orientation.

[0019] According to the above-described means 2, the extracted images in the training data and inspection image data are set so that the outer edge of the solder paste (i.e., the side opposite the center of the solder paste) is oriented in a predetermined direction. For example, the extracted images in both image data are set so that the outer edge of the solder paste faces downwards. Therefore, the learning efficiency is improved, and the effort and time required to obtain the identification means can be further reduced. In addition, the accuracy of the reconstructed image data output from the identification means is improved, making it possible to further enhance the inspection accuracy.

[0020] Means 3. The solder inspection apparatus according to Means 1, characterized in that the extracted images in the learning data and the inspection image data are set so that the long side or short side extends along a predetermined direction.

[0021] According to the above-described means 3, the extracted images in the training data and the test image data are set so that the long side or short side extends along a predetermined direction. For example, the extracted images in both image data are set so that the long side extends in the left-right direction. Therefore, the learning efficiency can be further improved, and the effort and time required to obtain the identification means can be further reduced. In addition, the accuracy of the reconstructed image data can be further improved, leading to an even greater improvement in inspection accuracy.

[0022] Means 4. The solder inspection apparatus according to Means 1, wherein the extracted image is configured to extract a region within an extraction frame that has a certain positional relationship with the outer edge of a reference solder paste or land from the solder image, the extraction frame is rectangular in shape with two short sides, a central long side positioned on the center side of the solder paste, and an outer long side corresponding to the side opposite the central long side, the extraction frame for extracting the extracted image constituting the learning data and the extraction frame for extracting the extracted image constituting the inspection image data each have equal lengths W in the short side direction, and when extracting the extracted image, the central long side and the outer long side are positioned so as to sandwich the outer edge of the solder paste, and the outer long side is positioned at a distance W0 away from the reference in the short side direction.

[0023] According to the above means 4, the extraction frame for the training data and the extraction frame for the inspection image data have equal lengths W in the short-side direction, and when extracting the image, it is positioned at a fixed position with respect to a reference (outer edge of the solder paste or outer edge of the land). Therefore, large variations in the position and range of the solder paste are less likely to occur in the extracted images that make up the training data and inspection image data. This makes it possible to further improve the learning efficiency and further enhance the inspection accuracy.

[0024] Furthermore, the technical aspects related to each of the above means may be combined as appropriate. For example, the technical aspects related to means 2 may be combined with the technical aspects related to means 3 or 4.

[0025] This is a partially enlarged plan view of a part of a printed circuit board. This is a partially enlarged cross-sectional view of a part of a printed circuit board. This is a block diagram showing the configuration of a printed circuit board manufacturing line. This is a schematic diagram illustrating a solder inspection device. This is a block diagram showing the functional configuration of a solder inspection device. This is a schematic diagram for explaining the structure of a neural network. This is a flowchart showing the learning process of a neural network. This is a flowchart showing the inspection process. This is a schematic diagram showing the source image data for training. This is a schematic diagram showing the solder image in the source image data for training. This is a schematic diagram for explaining the size and position of the extraction frame. This is a schematic diagram for explaining the extraction of the extracted image Cag. This is a schematic diagram for explaining the extraction of the extracted image Cbg. This is a schematic diagram for explaining the extraction of the extracted image Ccg. This is a schematic diagram for explaining the extraction of the extracted image Cdg. This is a schematic diagram showing the image frame and training data Ga. This is a schematic diagram showing the image frame and training data Gb. This is a schematic diagram showing the image frame and training data Gc. This is a schematic diagram showing the image frame and training data Gd. This is a schematic diagram showing an example of source image data for inspection. This is a partially enlarged cross-sectional view of a printed circuit board with foreign matter attached. This is a schematic diagram showing a solder image in the original image data for inspection. This is a schematic diagram for explaining the extraction of extracted image Cak, etc. This is a schematic diagram for explaining the extraction of extracted image Cbk, etc. This is a schematic diagram for explaining the extraction of extracted image Cck, etc. This is a schematic diagram for explaining the extraction of extracted image Cdk, etc. This is a schematic diagram showing an image frame and inspection image data Ka. This is a schematic diagram showing an image frame and inspection image data Kb. This is a schematic diagram showing an image frame and inspection image data Kc. This is a schematic diagram showing an image frame and inspection image data Kd. This is a schematic diagram showing reconstructed image data output from the AI ​​model when inspection image data Ka is input. This is a schematic diagram showing reconstructed image data output from the AI ​​model when inspection image data Kd is input. In another embodiment, this is a schematic diagram for explaining the size and position of the extraction frame, etc. In another embodiment, this is a schematic diagram showing an example of original image data for inspection that includes solder paste with abnormal shape or size, or solder paste covering foreign matter.This is a schematic diagram illustrating the extraction of an extracted image Cbk relating to solder paste with abnormal shape or size in another embodiment. This is a schematic diagram illustrating the extraction of an extracted image Cck relating to solder paste with abnormal shape or size in another embodiment. This is a schematic diagram illustrating the extraction of an extracted image Cak relating to solder paste covering foreign matter in another embodiment. This is a schematic diagram showing an image frame and inspection image data Kb in another embodiment. This is a schematic diagram showing an image frame and inspection image data Kc in another embodiment. This is a schematic diagram showing an image frame and inspection image data Ka in another embodiment. This is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Kb is input in another embodiment. This is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Kc is input in another embodiment. This is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Ka is input in another embodiment.

[0026] The following describes one embodiment with reference to the drawings. First, the configuration of the printed circuit board will be described. Figure 1 is a partially enlarged plan view of a part of the printed circuit board, and Figure 2 is a partially enlarged cross-sectional view of a part of the printed circuit board.

[0027] As shown in Figures 1 and 2, the printed circuit board 1 has a flat base substrate 2 made of glass epoxy resin or the like, on which wiring patterns (not shown) made of copper foil and a plurality of rectangular lands 3 in plan view are formed. The portion of the base substrate 2 surface excluding the lands 3 is coated with a resist film 4.

[0028] Furthermore, solder paste 5, made by mixing solder granules with flux, is printed on the land 3. In Figure 1, etc., for convenience, a scattered dot pattern is added to the area representing the solder paste 5. Also, in Figure 1, etc., only a small portion of the multiple land 3 and solder paste 5 on the printed circuit board 1 are shown.

[0029] Next, the manufacturing line (manufacturing process) for producing the printed circuit board 1 will be explained with reference to Figure 3. As shown in Figure 3, the manufacturing line 10 is equipped with, in order from the upstream side (upper side of Figure 3), a solder printing machine 12, a solder inspection device 13, a component mounting machine 14, a reflow device 15, and a post-reflow inspection device 16.

[0030] The solder printing machine 12 performs a solder printing process to print solder paste 5 onto each land 3 of the printed circuit board 1. In the solder printing process, for example, the solder paste 5 is printed by screen printing. In screen printing, first, the lower surface of the screen mask is brought into contact with the printed circuit board 1, and solder paste 5 is supplied to the upper surface of the screen mask. The screen mask has a plurality of openings corresponding to each land 3 of the printed circuit board 1. Next, the openings are filled with solder paste 5 by moving a predetermined squeegee while it is in contact with the upper surface of the screen mask. After that, the printed circuit board 1 is separated from the lower surface of the screen mask, and solder paste 5 is printed onto each land 3 of the printed circuit board 1.

[0031] The solder inspection device 13 inspects whether foreign matter is attached to the land 3 and solder paste 5, and the shape of the solder paste 5 printed on the land 3. Details of the solder inspection device 13 will be described later.

[0032] The component mounting machine 14 mounts electronic components 25 (see Figures 1 and 2) on a land 3 on which solder paste 5 is printed. The electronic component 25 has multiple electrodes (not shown), and each of these electrodes is temporarily fixed to a predetermined amount of solder paste 5.

[0033] The reflow apparatus 15 heats and melts the solder paste 5 to perform a reflow process in which the land 3 and the electrodes of the electronic component 25 are soldered together.

[0034] The post-reflow inspection device 16 checks whether the solder joints were properly formed during the reflow process by, for example, using brightness image data to confirm whether there is any misalignment in the electronic components 25.

[0035] In addition, although not shown in the diagram, the manufacturing line 10 is equipped with conveyors for transporting printed circuit boards 1 between the aforementioned devices, such as between the solder printing machine 12 and the solder inspection device 13. Branching devices are also provided between the solder inspection device 13 and the component mounting machine 14, and downstream of the post-reflow inspection device 16. Printed circuit boards 1 that are judged as good by the solder inspection device 13 or the post-reflow inspection device 16 are guided downstream, while printed circuit boards 1 that are judged as defective are discharged to the defective product storage section by the branching devices.

[0036] Next, the configuration of the solder inspection device 13 will be explained in detail with reference to Figures 4 and 5. Figure 4 is a schematic diagram showing the configuration of the solder inspection device 13. Figure 5 is a block diagram showing the functional configuration of the solder inspection device 13.

[0037] The solder inspection device 13 includes a transport mechanism 31 that transports and positions the printed circuit board 1, an inspection unit 32 for obtaining image data of the printed circuit board 1, and a control device 33 (see Figure 5) that performs various controls, image processing, and calculation processing in the solder inspection device 13, including drive control of the transport mechanism 31 and the inspection unit 32.

[0038] The transport mechanism 31 comprises a pair of transport rails 31a arranged along the loading and unloading direction of the printed circuit board 1, and an endless conveyor belt 31b rotatably mounted on each transport rail 31a. Although not shown in the figures, the transport mechanism 31 is also provided with a drive means such as a motor for driving the conveyor belt 31b, and a chuck mechanism for positioning the printed circuit board 1 at a predetermined position. The transport mechanism 31 is driven and controlled by a control device 33 (a transport mechanism control unit 79, which will be described later).

[0039] Under the above configuration, the printed circuit board 1, when transported to the solder inspection device 13, has both side edges in the width direction perpendicular to the transport rail 31a inserted into the transport rail 31a and is placed on the conveyor belt 31b. Subsequently, the conveyor belt 31b starts moving, and the printed circuit board 1 is transported to a predetermined inspection position. When the printed circuit board 1 reaches the inspection position, the conveyor belt 31b stops and the chuck mechanism activates. This chuck mechanism pushes up the conveyor belt 31b, and both side edges of the printed circuit board 1 are clamped between the conveyor belt 31b and the upper edges of the transport rail 31a. This positions and fixes the printed circuit board 1 in the inspection position. When the inspection is completed, the fixing by the chuck mechanism is released, and the conveyor belt 31b starts moving again. As a result, the printed circuit board 1 is transported out of the solder inspection device 13. Of course, the configuration of the transport mechanism 31 is not limited to the above form, and other configurations may be adopted.

[0040] The inspection unit 32 is positioned above the transport rail 31a (the transport path for the printed circuit board 1). The inspection unit 32 includes a first lighting device 32a, a second lighting device 32b, a third lighting device 32c, and a camera 32d. In this embodiment, the camera 32d constitutes the "image data acquisition means".

[0041] Furthermore, the inspection unit 32 is also equipped with an X-axis movement mechanism 32e (see Figure 5) that enables movement in the X-axis direction (left-right direction in Figure 4), and a Y-axis movement mechanism 32f (see Figure 5) that enables movement in the Y-axis direction (front-back direction in Figure 4). These movement mechanisms 32e and 32f are driven and controlled by a control device 33 (movement mechanism control unit 76, which will be described later).

[0042] The first illumination device 32a and the second illumination device 32b, in order to perform three-dimensional measurement of the printed circuit board 1, each illuminate a predetermined area to be inspected on the printed circuit board 1 with predetermined light for three-dimensional measurement (patterned light having a striped light intensity distribution) from diagonally above.

[0043] Specifically, the first lighting device 32a includes a first light source 32a1 that emits predetermined light, and a first liquid crystal shutter 32a2 that forms a first grating for converting the light from the first light source 32a1 into first pattern light having a striped light intensity distribution. The first lighting device 32a is driven and controlled by a control device 33 (a lighting control unit 72 described later).

[0044] The second lighting device 32b includes a second light source 32b1 that emits predetermined light, and a second liquid crystal shutter 32b2 that forms a second grating for converting the light from the second light source 32b1 into second pattern light having a striped light intensity distribution. The second lighting device 32b is driven and controlled by a control device 33 (a lighting control unit 72 described later).

[0045] Under the above configuration, the light emitted from each of the light sources 32a1 and 32b1 is guided to a condenser lens (not shown), made into parallel light there, and then guided to a projection lens (not shown) through the liquid crystal shutters 32a2 and 32b2, and projected onto the printed circuit board 1 as pattern light. Further, in the present embodiment, the switching control of the liquid crystal shutters 32a2 and 32b2 is performed so that the phases of the respective pattern lights are shifted by 1 / 4 pitch each.

[0046] By using the liquid crystal shutters 32a2 and 32b2 as gratings, it is possible to irradiate pattern light close to an ideal sine wave. Thereby, the measurement resolution of the three-dimensional measurement is improved. Further, the phase shift control of the pattern light can be performed electrically, and the device can be made compact.

[0047] When performing two-dimensional measurement of the printed circuit board 1, the third lighting device 32c irradiates a predetermined inspection region on the printed circuit board 1 with predetermined light (for example, uniform light) for two-dimensional measurement. The third lighting device 32c includes a ring light capable of irradiating blue light, a ring light capable of irradiating green light, and a ring light capable of irradiating red light. Since the third lighting device 32c has the same configuration as the known technology, detailed description thereof will be omitted.

[0048] The camera 32d images a predetermined inspection area of the printed circuit board 1 from directly above. The camera 32d has an imaging element such as a CCD (Charge Coupled Device) type image sensor or a CMOS (Complementary Metal Oxide Semiconductor) type image sensor, and an optical system (such as a lens unit and an aperture) that forms an image of the printed circuit board 1 on the imaging element, and is arranged such that its optical axis is along the vertical direction (Z-axis direction). Of course, the imaging element is not limited to these, and other imaging elements may be adopted.

[0049] The camera 32d is driven and controlled by a control device 33 (a camera control unit 73 described later). More specifically, the control device 33 executes imaging processing by the camera 32d while synchronizing with the irradiation processing by each of the lighting devices 32a, 32b, 32c. As a result, among the light irradiated from any of the lighting devices 32a, 32b, 32c, the light reflected by the printed circuit board 1 is imaged by the camera 32d. As a result, image data of the inspection area of the printed circuit board 1 including the cream solder 5 printed on the land 3 is acquired. The "inspection area" of the printed circuit board 1 is one area among a plurality of areas preset on the printed circuit board 1 with the size of the imaging field (imaging range) of the camera 32d as one unit.

[0050] In addition, the camera 32d in the present embodiment is configured as a color camera. As a result, the light of each color irradiated simultaneously from the color ring lights of the third lighting device 32c and reflected by the printed circuit board 1 can be imaged at once.

[0051] The image data imaged and generated by the camera 32d is converted into a digital signal inside the camera 32d and then transferred to the control device 33 (an image acquisition unit 74 described later) in the form of a digital signal. Then, the control device 33 stores the transferred image data and performs various image processing and arithmetic processing based on the image data.

[0052] The control device 33 consists of a computer including a CPU (Central Processing Unit) that executes predetermined calculation processes, a ROM (Read Only Memory) that stores various programs and fixed value data, a RAM (Random Access Memory) that temporarily stores various data when various calculation processes are executed, and peripheral circuits for these components.

[0053] The control device 33 functions as various functional units, such as the main control unit 71, lighting control unit 72, camera control unit 73, image acquisition unit 74, data processing unit 75, movement mechanism control unit 76, learning unit 77, inspection unit 78, and transport mechanism control unit 79, as the CPU operates according to various programs.

[0054] However, the various functional units described above are realized through the cooperation of various hardware components such as the CPU, ROM, and RAM, and there is no need to clearly distinguish between functions realized in hardware and functions realized in software. Some or all of these functions may be realized by hardware circuits such as ICs.

[0055] Furthermore, the control device 33 includes an input unit 55 consisting of a keyboard, mouse, touch panel, etc., a display unit 56 equipped with a display screen consisting of a liquid crystal display, etc., a storage unit 57 capable of storing various data, programs, calculation results, inspection results, etc., and a communication unit 58 capable of sending and receiving various data with the outside.

[0056] Here, the various functional units that constitute the control device 33 will be described in detail.

[0057] The main control unit 71 is a functional unit that controls the entire solder inspection device 13 and is configured to send and receive various signals with other functional units such as the lighting control unit 72 and the camera control unit 73.

[0058] The lighting control unit 72 is a functional unit that drives and controls the lighting devices 32a, 32b, and 32c, and performs switching control of the illuminated light based on command signals from the main control unit 71.

[0059] The camera control unit 73 is a functional unit that drives and controls the camera 32d, and controls the imaging timing and other parameters based on command signals from the main control unit 71.

[0060] The image acquisition unit 74 is a functional unit for capturing image data acquired by the camera 32d.

[0061] The data processing unit 75 is a functional unit that performs predetermined image processing on image data acquired by the image acquisition unit 74, and performs two-dimensional measurement processing, three-dimensional measurement processing, and the like using the image data.

[0062] The movement mechanism control unit 76 is a functional unit that drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, and controls the position of the inspection unit 32 based on command signals from the main control unit 71. By driving and controlling the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, the movement mechanism control unit 76 can move the inspection unit 32 to a position above any area to be inspected on the printed circuit board 1, which is positioned and fixed at the inspection position. Then, as the inspection unit 32 is sequentially moved to multiple areas to be inspected set on the printed circuit board 1, the inspection of the area to be inspected is performed, thereby performing an inspection of the entire printed circuit board 1.

[0063] The learning unit 77 is a functional unit that uses training data to train the deep neural network 90 (hereinafter simply referred to as "neural network 90"; see Figure 6) and constructs an AI (Artificial Intelligence) model 101 as a "discrimination means".

[0064] As will be described later, the AI ​​model 101 in this embodiment is a generative model constructed by deep learning a neural network 90 using only image data related to good solder paste 5 as training data, and has the structure of a so-called autoencoder.

[0065] Here, the structure of the neural network 90 will be explained with reference to Figure 6. Figure 6 is a schematic diagram conceptually showing the structure of the neural network 90. ​​As shown in Figure 6, the neural network 90 has the structure of a convolutional auto-encoder (CAE), comprising an encoder unit 91 as an "encoding unit" that extracts feature quantities (latent variables) TA from the input image data GA, and a decoder unit 92 as a "decoding unit" that reconstructs image data GB from the feature quantities TA.

[0066] The structure of the convolutional autoencoder is well known, so a detailed explanation will be omitted. The encoder unit 91 has multiple convolutional layers 93, and in each convolutional layer 93, the result of a convolution operation using multiple filters (kernels) 94 on the input data is output as input data for the next layer. Similarly, the decoder unit 92 has multiple deconvolutional layers 95, and in each deconvolutional layer 95, the result of a deconvolution operation using multiple filters (kernels) 96 on the input data is output as input data for the next layer. Then, in the learning process described later, the weights (parameters) of each filter 94, 96 are updated.

[0067] The inspection unit 78 is a functional unit that inspects for abnormalities related to the solder paste 5. In this embodiment, the inspection unit 78 inspects whether the solder paste 5 is properly printed in terms of the presence, size, and shape of foreign matter.

[0068] The transport mechanism control unit 79 is a functional unit that drives and controls the transport mechanism 31, and controls the position of the printed circuit board 1 based on command signals from the main control unit 71.

[0069] The memory unit 57 is composed of an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and has a predetermined memory area for storing, for example, the AI ​​model 101 (the neural network 90 and the learning information acquired through its learning).

[0070] The communication unit 58 is equipped with a wireless communication interface conforming to communication standards such as wired LAN (Local Area Network) or wireless LAN, and is configured to send and receive various data to and from the outside. For example, the results of inspections performed by the inspection unit 78 are output to the outside via the communication unit 58, or the results of inspections performed by the post-reflow inspection device 16 are input via the communication unit 58.

[0071] Next, the learning process of the neural network 90 performed by the solder inspection device 13 will be explained with reference to the flowchart in Figure 7.

[0072] When the learning process starts based on the execution of a predetermined learning program, the main control unit 71 first performs pre-processing in step S101 for training the neural network 90.

[0073] In this preprocessing step, inspection information for a large number of printed circuit boards 1 stored in the post-reflow inspection device 16 is first obtained via the communication unit 58. Subsequently, based on this inspection information, learning source image data Ig, which is image data related to good solder paste 5 that passed the post-reflow inspection, is obtained from the storage unit 57 (see, for example, Figure 9). In Figure 9, only a portion of the learning source image data Ig is shown, and the actual learning source image data Ig contains many more solder paste 5 and lands 3.

[0074] The training source image data Ig pertains to the printed circuit board 1 after the solder paste 5 has been printed and before the electronic components 25 have been mounted, and is used to obtain the training data Ga, Gb, Gc, and Gd (hereinafter sometimes abbreviated as "training data Ga to Gd") used for training the neural network 90. ​​The training source image data Ig includes three-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while pattern light is irradiated from the first illumination device 32a or the second illumination device 32b, and two-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while uniform light is irradiated from the third illumination device 32c.

[0075] Furthermore, the training source image data Ig may be image data obtained by the camera 32d without any special processing (for example, monochrome luminance image data or RGB luminance image data), or it may be image data obtained by applying predetermined processing to the image data obtained by the camera 32d (for example, HLS image data obtained by converting RGB image data, or height image data obtained by converting image data).

[0076] Next, training data Ga to Gd are created from the acquired training source image data Ig (see Figures 16 to 19). In order to obtain the training data Ga to Gd, first, the solder image Ihg (see Figure 10) in the acquired training source image data Ig is identified. This solder image Ihg corresponds to the solder paste 5 of land 3 and the surrounding area of ​​land 1 in the training source image data Ig.

[0077] In identifying the solder image Ihg, first, the area occupied by the solder paste 5 is identified. If the training source image data Ig is two-dimensional data, the area occupied by the solder paste 5 is identified using, for example, brightness, hue, and saturation. If the training source image data Ig is three-dimensional data, the area occupied by the solder paste 5 is identified using, for example, height information. Then, the portion of the training source image data Ig corresponding to the identified area occupied by the solder paste 5 and its surrounding area is identified as the solder image Ihg.

[0078] Next, four extracted images, Cag, Cbg, Ccg, and Cdg (hereinafter sometimes abbreviated as "extracted images Cag to Cdg") are extracted from the identified solder image Ihg (see Figures 12 to 15). In this embodiment, as extracted images Cag to Cdg, regions located within the extraction frame Wc that have a certain positional relationship with the outer edge of the reference solder paste 5 are extracted from the solder image Ihg.

[0079] As shown in Figure 11, the extraction frame Wc is rectangular in shape, having two parallel short sides X1 and X2, and parallel central long side Y1 and outer long side Y2 connecting the ends of these short sides X1 and X2. The central long side Y1 is the long side positioned on the central side of the solder paste 5, and the outer long side Y2 is the long side corresponding to the side opposite the central long side Y1. In obtaining the extracted images Cag to Cdg, the extraction frame Wc is positioned such that the central long side Y1 and the outer long side Y2 are positioned so as to sandwich the outer edge of the solder paste 5, the long sides Y1 and Y2 are parallel to the outer edge of the solder paste 5, and the outer long side Y2 is positioned at a distance W0 away from the outer edge of the solder paste 5 in the direction of the short sides X1 and X2. Alternatively, the Gerber data (design data) of the printed circuit board 1 can be used to set the outer edge (side) of the land 3 to be parallel to the long sides Y1 and Y2, thereby ensuring that the outer edge of the solder paste 5 is parallel to the long sides Y1 and Y2.

[0080] Furthermore, the extraction frame Wc has a constant length W along the short sides X1 and X2 (for example, a constant value greater than 0.1 mm), while the length L along the long sides Y1 and Y2 is adjusted according to the width Ls of the solder paste 5. The width Ls is the maximum width of the solder paste 5. Alternatively, the maximum width of the portion of the solder paste 5 located between the long sides Y1 and Y2 may be used as the width Ls. The length L is set to a value that is greater than or equal to the width Ls and less than the value obtained by adding twice the tolerance value L0 to the width Ls. The tolerance value L0 is set to a value of 0.25 mm, for example. In order to prevent solder paste 5 other than the target solder paste 5 (non-target solder paste 5) from entering the extraction frame Wc, the tolerance value L0 may be set to a smaller value (for example, 0.125 mm).

[0081] Furthermore, in obtaining the extracted images Cag to Cdg, the extraction frame Wc is positioned such that its center along the long sides Y1 and Y2 coincides with the center along the long sides Y1 and Y2 of the solder paste 5 located between the long sides Y1 and Y2. The distance W1 from the outer edge of the reference solder paste 5 to the central long side Y1 along the short sides X1 and X2 is set, for example, to 0.1 mm or more and 0.2 mm or less, in order to make the size of the extracted images Cag to Cdg relatively small while ensuring that the solder paste 5 is appropriately included within the extracted images Cag to Cdg. Note that the above-mentioned values ​​such as distances W, W0, W1 and tolerance L0 are values ​​in relation to the actually printed solder paste 5, and are only relative in relation to the solder paste 5 in the image. Therefore, for example, the position and size of the extraction frame Wc in the image are set based on the ratio of the actual length to the length (number of pixels) in the image.

[0082] Then, four extraction frames Wc are set corresponding to the solder image Ihg, and by extracting the images within these extraction frames Wc, four extracted images Cag to Cdg are obtained that include the outer edge of the solder paste 5 and correspond to the four sides of the printed land 3 of the solder paste 5 (see the left and center figures of Figures 12 to 15).

[0083] Next, rotation processing is performed on the extracted images Cag to Cdg that require rotation (in this embodiment, extracted images Cbg, Ccg, and Cdg), so that the outer edge of the solder paste 5 in the extracted images Cag to Cdg is oriented in a predetermined direction, and the long or short side of the extracted images Cag to Cdg extends along a predetermined direction (see the right-hand figures of Figures 13 to 15). In this embodiment, the outer edge of the solder paste 5 in the extracted images Cag to Cdg is oriented downwards, and the long side of the extracted images Cag to Cdg extends in the left-right direction.

[0084] Then, by pasting the extracted images Cag to Cdg onto the image frame W1, we obtain the training data Ga to Gd (see Figures 16 to 19) which consist of the extracted images Cag to Cdg placed on the image frame W1.

[0085] The image frame W1 is a rectangular image with a height (width in the vertical direction of the paper, such as in Figure 16) of m (pixels) and a width (width in the horizontal direction of the paper, such as in Figure 16) of n (pixels). Its size (width and height) is set to be larger than the sizes of each extracted image Cag to Cdg, based on design data, etc. Note that m and n are natural numbers, and in this embodiment, m < n is satisfied. Furthermore, by adjusting the pasting position, each learning data Ga to Gd is made to coincide with the center or centroid of the extracted images Cag to Cdg and the center of the image frame W1.

[0086] Then, by repeatedly performing the above processes of identifying the solder image Ihg, extracting the extracted images Cag to Cdg, rotating the extracted images Cag to Cdg, and pasting the extracted images Cag to Cdg onto the image frame W1, multiple training data Ga to Gd are obtained from one training source image data Ig. At this time, four training data Ga to Gd are obtained for each of the planar rectangular lands 3 contained in the training source image data Ig. Furthermore, by using multiple training source image data Ig, the final number of training data Ga to Gd is obtained. In this embodiment, the training data Ga to Gd include those obtained based on two-dimensional data and those obtained based on three-dimensional data.

[0087] In step S101, once the necessary number of training data Ga to Gd have been acquired, in the following step S102, the learning unit 77 prepares an untrained neural network 90 based on a command from the main control unit 71. For example, it reads a neural network 90 that has been previously stored in the memory unit 57 or the like. Alternatively, it constructs a neural network 90 based on network configuration information (for example, the number of layers in the neural network and the number of nodes in each layer) stored in the memory unit 57 or the like.

[0088] In this embodiment, two neural networks 90 are constructed separately: one that performs learning using training data Ga to Gd acquired based on two-dimensional data, and another that performs learning using training data Ga to Gd acquired based on three-dimensional data. Therefore, in this embodiment, two neural networks 90 are constructed.

[0089] In step S103, reconstructed image data is acquired. Specifically, based on a command from the main control unit 71, the learning unit 77 provides the learning data Ga to Gd acquired in step S102 as input data to the input layer of the neural network 90, thereby acquiring the reconstructed image data output from the output layer of the neural network 90. ​​More specifically, the learning unit 77 provides the input layer of the neural network 90 with the learning data Ga to Gd acquired in step S102 that corresponds to the neural network 90 as input data, thereby acquiring the reconstructed image data output from the output layer of the neural network 90. ​​For example, the learning unit 77 provides the learning data Ga to Gd as input data to the input layer of a neural network 90 that performs learning using learning data Ga to Gd obtained from two-dimensional data, and acquires the reconstructed image data output from the neural network 90. ​​In other words, the learning unit 77 inputs appropriate learning data Ga to Gd to each of the two neural networks 90 and acquires the output reconstructed image data.

[0090] In the following step S104, the learning unit 77 compares the input learning data Ga to Gd with the reconstructed image data output by the neural network 90 and determines whether the error is sufficiently small (whether it is below a predetermined threshold).

[0091] If the error is sufficiently small, in step S106, the learning unit 77 determines whether the learning termination conditions are met. For example, if a certain number of consecutive affirmative determinations are made in step S104 without going through the process of step S105 described later, or if learning using all of the prepared learning data Ga to Gd is repeated a predetermined number of times, it is determined that the termination conditions are met. If the termination conditions are met, the neural network 90 and its learning information (updated parameters, etc., described later) are stored in the storage unit 57 as the AI ​​model 101, and the learning process is terminated.

[0092] In this embodiment, the AI ​​model 101 ultimately stores an AI model trained using training data Ga to Gd acquired from two-dimensional data, and an AI model trained using training data Ga to Gd acquired from three-dimensional data.

[0093] On the other hand, if the termination condition is not met in step S106, the process returns to step S102 and the neural network 90 is trained again.

[0094] Furthermore, if the error is not sufficiently small in step S104, the network update process (training of the neural network 90) is performed in step S105, and then the process returns to step S103 and the above series of processes is repeated.

[0095] Specifically, in the network update process of step S105, known learning algorithms such as backpropagation are used to update the weights (parameters) of each filter 94, 96 in the neural network 90 to more appropriate values ​​so that the loss function representing the difference between the training data Ga to Gd and the reconstructed image data is minimized. For example, BCE (Binary Cross-entropy) can be used as the loss function.

[0096] By repeatedly performing steps S103 to S105, the neural network 90 minimizes the error between the training data Ga to Gd and the reconstructed image data, resulting in the output of more accurate reconstructed image data.

[0097] The resulting AI model 101 will generate reconstructed image data that closely matches the image data of a good solder paste 5 when that image data is input. Furthermore, when the AI ​​model 101 receives image data of a defective solder paste 5 in terms of shape, size, or presence of foreign matter, it will generate reconstructed image data that closely matches the image data after correcting the shape and size of the solder paste 5 or removing noise (parts corresponding to foreign matter). In other words, when the solder paste 5 is defective, the reconstructed image data of the solder paste 5 will be a virtual image data of the solder paste 5 assuming that there are no defects.

[0098] Next, the inspection process performed by the solder inspection device 13 will be explained with reference to the flowchart in Figure 8. This inspection process is performed for each area to be inspected on the printed circuit board 1.

[0099] Once the printed circuit board 1 is brought into the solder inspection device 13 and positioned at a predetermined inspection location, the inspection process is started based on the execution of a predetermined inspection program.

[0100] When the inspection process is started, the first step, S301, is the image data acquisition step. In the image data acquisition step, the raw inspection image data Ik (see, for example, Figure 20) related to the printed circuit board 1 to be inspected is acquired. The raw inspection image data Ik is the image data used to obtain the inspection image data Ka, Kb, Kc, and Kd (hereinafter sometimes abbreviated as "inspection image data Ka to Kd") which will be described later.

[0101] In this embodiment, as an example of a printed circuit board 1 to be inspected, a foreign matter F1 is present between the land 3 and the solder paste 5, and the foreign matter F1 protrudes from the outer edge of the solder paste 5 (see Figure 21). This foreign matter F1 adhered to the land 3 before the solder paste 5 was printed. If foreign matter F is present under the solder paste 5, the solder paste 5 may move unintended during the reflow process, potentially resulting in mounting defects of the electronic components 25.

[0102] The raw image data Ik for inspection includes three-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while pattern light is irradiated from the first illumination device 32a or the second illumination device 32b, and two-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while uniform light is irradiated from the third illumination device 32c. The image data acquisition process consists of acquiring three-dimensional data and acquiring two-dimensional data.

[0103] First, let's explain the process for acquiring three-dimensional data. In this process, the phase of the first pattern light emitted from the first illumination device 32a is changed, and imaging is performed four times under the first pattern light with different phases. Then, the phase of the second pattern light emitted from the second illumination device 32b is changed, and imaging is performed four times under the second pattern light with different phases, thereby acquiring a total of eight different sets of three-dimensional data. This will be explained in detail below.

[0104] As described above, once the printed circuit board 1, which has been brought into the solder inspection device 13, is positioned and fixed at a predetermined inspection location, the movement mechanism control unit 76, based on a command from the main control unit 71, first drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f to move the inspection unit 32, and adjusts the imaging field of view (imaging range) of the camera 32d to match the predetermined area to be inspected on the printed circuit board 1.

[0105] In addition, the lighting control unit 72 switches the liquid crystal shutters 32a2 and 32b2 of both lighting devices 32a and 32b, and sets the positions of the first and second grids formed on both liquid crystal shutters 32a2 and 32b2 to predetermined reference positions.

[0106] Once the switching settings for the first and second grids are complete, the lighting control unit 72 causes the first light source 32a1 of the first lighting device 32a to emit light and irradiate with the first pattern light, while the camera control unit 73 drives and controls the camera 32d to perform the first imaging process under the first pattern light. The image data generated by the imaging process is taken into the image acquisition unit 74 as it occurs (the same applies hereafter). As a result, three-dimensional data of the area under inspection, including multiple lands 3 and multiple solder pastes 5, is acquired.

[0107] Subsequently, the lighting control unit 72, simultaneously with the completion of the first imaging process under the first pattern light, turns off the first light source 32a1 of the first lighting device 32a and performs a switching process for the first liquid crystal shutter 32a2. Specifically, it switches the position of the first grid formed on the first liquid crystal shutter 32a2 from the reference position to a second position where the phase of the first pattern light is shifted by a quarter pitch (90°).

[0108] Once the first grid switching setting is complete, the lighting control unit 72 causes the light source 32a1 of the first lighting device 32a to emit light and irradiate with the first pattern light, while the camera control unit 73 drives and controls the camera 32d to perform a second imaging process under the first pattern light. Thereafter, the same process is repeated to acquire four types of three-dimensional data under the first pattern light with a phase difference of 90°.

[0109] Next, the lighting control unit 72 causes the second light source 32b1 of the second lighting device 32b to emit light and irradiate with the second pattern light, while the camera control unit 73 drives and controls the camera 32d to perform the first imaging process under the second pattern light.

[0110] Subsequently, the lighting control unit 72, upon completion of the first imaging process under the second pattern light, turns off the second light source 32b1 of the second lighting device 32b and simultaneously executes a switching process for the second liquid crystal shutter 32b2. Specifically, it switches the position of the second grid formed on the second liquid crystal shutter 32b2 from the reference position to a second position where the phase of the second pattern light is shifted by a quarter pitch (90°).

[0111] Once the switching settings for the second grid are complete, the lighting control unit 72 causes the light source 32b1 of the second lighting device 32b to emit light, illuminating with the second pattern light, and the camera control unit 73 drives and controls the camera 32d to perform a second imaging process under the second pattern light. Thereafter, the same process is repeated to acquire four types of three-dimensional data under the second pattern light with a phase difference of 90°.

[0112] Next, the process for acquiring two-dimensional data will be described. In this process, based on a command from the main control unit 71, the illumination control unit 72 causes the third illumination device 32c to emit light, irradiating a predetermined area to be inspected with uniform light, while the camera control unit 73 drives and controls the camera 32d to perform imaging under the uniform light. As a result, a predetermined area to be inspected on the printed circuit board 1 is imaged, and two-dimensional data relating to the area to be inspected is acquired.

[0113] The acquired raw image data Ik for inspection (three-dimensional data and two-dimensional data) is stored in the storage unit 57.

[0114] Furthermore, the raw image data Ik for inspection may be image data obtained by the camera 32d without any special processing (for example, monochrome luminance image data or RGB luminance image data), or it may be image data obtained by applying predetermined processing to the image data obtained by the camera 32d (for example, HLS image data obtained by converting RGB image data, or height image data obtained by converting image data).

[0115] Next, in step S302, the inspection image data acquisition process is performed. In the inspection image data acquisition process, inspection image data Ka to Kd (see Figures 27 to 30) are acquired based on the original inspection image data Ik obtained in the image data acquisition process. The method for acquiring the inspection image data Ka to Kd is the same as the method for acquiring the training data Ga to Gd described above.

[0116] Specifically, first, the solder image Ihk (see Figure 22) in the acquired inspection source image data Ik is identified. This solder image Ihk corresponds to the solder paste 5 and its surrounding area corresponding to land 3 of 1 in the inspection source image data Ik. The identification of the solder image Ihk is performed in the same way as the identification of the solder image Ihg in the training source image data Ig.

[0117] Next, four extracted images Cak, Cbk, Cck, and Cdk (hereinafter sometimes abbreviated as "extracted images Cak to Cdk") are extracted from the identified solder image Ihk (see the left and center figures of Figures 23 to 26). In this embodiment, the four extracted images Cak to Cdk are extracted from the solder image Ihk in the same way as the extraction method for extracted images Cag to Cdg in the learning process. That is, as extracted images Cak to Cdk, regions located within the extraction frame Wc that have a certain positional relationship with the outer edge of the reference solder paste 5 are extracted from the solder image Ihk. The extraction frame Wc is the same as that used in the learning process, and for example, the length W along the short side directions X1 and X2 is set to a constant value (for example, a constant value greater than 0.1 mm). Furthermore, in obtaining the extracted images Cak to Cdk, the extraction frame Wc is positioned such that, as during the learning process, the central long side Y1 and the outer long side Y2 are positioned so as to sandwich the outer edge of the solder paste 5, the long sides Y1 and Y2 are parallel to the outer edge of the solder paste 5, and the outer long side Y2 is positioned at a distance W0 away from the outer edge of the solder paste 5 in the direction of the short sides X1 and X2.

[0118] Next, rotation processing is performed on the extracted images Cak to Cdk that require rotation (in this embodiment, extracted images Cbk, Cck, and Cdk) (see the right-hand figures in Figures 24 to 26). As a result, the extracted images Cak to Cdk are set so that the outer edge of the solder paste 5 is oriented in a predetermined direction, and the long or short side extends along a predetermined direction. In this embodiment, the outer edge of the solder paste 5 in the extracted images Cak to Cdk is facing downwards, and the long side of the extracted images Cak to Cdk is extended in the left-right direction.

[0119] Next, by pasting the extracted images Cak to Cdk onto the image frame W1, inspection image data Ka to Kd is obtained, which consists of the extracted images Cak to Cdk placed on the image frame W1 (see Figures 27 to 30). Since the image frame W1 for the inspection image data Ka to Kd is the same size as the image frame W1 for the training data Ga to Gd, the size (width and height) of the inspection image data Ka to Kd is the same as the size of the training data Ga to Gd.

[0120] Then, by repeatedly performing the above processes such as identifying the solder image Ihk, extracting the extracted images Cak to Cdk, rotating the extracted images Cak to Cdk, and pasting the extracted images Cak to Cdk onto the image frame W1, multiple inspection image data Ka to Kd are obtained from one inspection source image data Ik. At this time, four inspection image data Ka to Kd are obtained for each rectangular land 3 in plan view included in the inspection source image data Ik. In this embodiment, the inspection image data Ka to Kd include those obtained based on two-dimensional data and those obtained based on three-dimensional data. In this embodiment, the inspection unit 78 that obtains the inspection image data Ka to Kd constitutes the "inspection image data acquisition means".

[0121] In the following step S303, the reconstructed image data acquisition process is performed. Specifically, based on a command from the main control unit 71, the inspection unit 78 inputs the inspection image data Ka to Kd acquired in step S302 into the input layer of the AI ​​model 101 corresponding to the type of inspection image data Ka to Kd. Therefore, inspection image data Ka to Kd acquired based on two-dimensional data is input into the AI ​​model 101 corresponding to the two-dimensional data, and inspection image data Ka to Kd acquired based on three-dimensional data is input into the AI ​​model 101 corresponding to the three-dimensional data. The image data that is reconstructed by the AI ​​model 101 and output from the output layer is acquired as the reconstructed image data. The acquired reconstructed image data is stored in association with the inspection image data Ka to Kd from which the reconstructed image data was derived.

[0122] Here, if the AI ​​model 101 receives, for example, inspection image data Ka (see Figure 27) relating to solder paste 5 with foreign matter F1 attached, it will output, as reconstructed image data S, image data relating to good solder paste 5 from which the foreign matter F1 has been removed (see, for example, Figure 31).

[0123] On the other hand, when the AI ​​model 101 receives, for example, inspection image data Kd relating to a good solder paste 5, it outputs reconstructed image data S relating to a good solder paste 5 that is substantially the same as the inspection image data Kd (see Figure 32). The size (width and height) of the reconstructed image data S is the same as the size of the original inspection image data Ka to Kd. In this embodiment, the inspection unit 78 that acquires the reconstructed image data S constitutes the "reconstructed image data acquisition means".

[0124] In step S304, a pass / fail judgment process is performed based on the acquired reconstructed image data S. In the pass / fail judgment process, based on a command from the main control unit 71, the inspection unit 78 compares the entirety of the inspection image data Ka to Kd acquired in step S302 with the entirety of the reconstructed image data S acquired in step S303 using the inspection image data Ka to Kd, and calculates the difference between the two image data Ka to Kd and S. For example, by comparing dots (pixels) at the same coordinates in both image data Ka to Kd and S, the area (number of dots) of clusters of dots where the difference in brightness is greater than or equal to a predetermined value is calculated. Note that the difference between the inspection image data Ka in which the foreign object F1 exists and the reconstructed image data S composed of the inspection image data Ka will be relatively large. In this embodiment, the inspection unit 78 that compares the inspection image data Ka to Kd and the reconstructed image data S constitutes the "comparison means".

[0125] Next, the inspection unit 78 determines whether the calculated difference is smaller than a predetermined threshold. If the calculated difference is smaller than the predetermined threshold, the inspection unit 78 determines it to be a "good product," while if the difference is larger than the predetermined threshold, it determines it to be a "defective product."

[0126] Furthermore, the inspection unit 78 performs the above determination on all inspection image data Ka to Kd related to the area to be inspected on the printed circuit board 1. If all inspection image data Ka to Kd are determined to be "good," the inspection unit 78 determines that the area to be inspected is "good" and stores this result in the storage unit 57. On the other hand, if, as a result of performing the above determination on all inspection image data Ka to Kd related to the area to be inspected, at least one inspection image data Ka to Kd is determined to be "defective," the inspection unit 78 determines that the area to be inspected is "defective" and stores this result in the storage unit 57.

[0127] Then, the solder inspection device 13, after performing the above inspection process on all areas to be inspected on the printed circuit board 1, determines that the printed circuit board 1 is free of defects in the solder paste 5 (pass judgment) if all areas to be inspected are judged to be "good" as a result, and stores this result in the storage unit 57.

[0128] On the other hand, if the solder inspection device 13 finds even one area under inspection that is determined to be a "defective product," it determines that the printed circuit board 1 has an abnormality in the solder paste 5 (failure judgment), stores this result in the storage unit 57, and notifies the outside of this fact via the display unit 56, communication unit 58, etc.

[0129] As detailed above, according to this embodiment, the inspection image data Ka to Kd is obtained by placing the extracted images Cak to Cdk in the image frame W1. Therefore, the size (width and height) of the inspection image data Ka to Kd will match the size of the image frame W1 and will remain constant without fluctuating depending on the size of the land 3. This eliminates the need to prepare many different identification means (AI models) for each size of the land 3, and reduces the effort and time required to obtain the AI ​​model 101. Furthermore, the AI ​​model 101 can be used in common even when the size of the land 3 is different.

[0130] Furthermore, the image frames W1 for the training data Ga to Gd and the image frames W1 for the inspection image data Ka to Kd are the same size, and the sizes of the training data Ga to Gd and the inspection image data Ka to Kd are identical. Therefore, when the inspection image data Ka to Kd is input to the AI ​​model 101, the appropriate reconstructed image data S corresponding to the inspection image data Ka to Kd can be output more reliably, and consequently, the presence or absence of abnormalities in the solder paste 5 can be determined more accurately. As a result, good inspection accuracy can be obtained more reliably.

[0131] Furthermore, the training data Ga to Gd are obtained by placing each of the extracted images Ca to Cdg, corresponding to the four sides of the land 3, in the image frame W1. Therefore, compared to the case where image data relating to the entire solder paste 5 is used as training data, an AI model 101 with the desired accuracy can be obtained with less training data Ga to Gd. Moreover, since four training data Ga to Gd can be obtained from one solder image Ihg, the required number of training data Ga to Gd can be easily obtained. These factors together effectively reduce the effort and time required to obtain the AI ​​model 101.

[0132] In addition, since the training data Ga to Gd and the inspection image data Ka to Kd are each composed of four extracted images Cag to Cdg and Cak to Cdk, corresponding to the four sides of the land 3, placed in the image frame W1, the size of both data Ga to Gd and Ka to Kd can be made relatively small compared to a model in the image frame W1 containing an image of the entire solder paste 5. Therefore, the processing burden related to training and inspection can be reduced, and training and inspection can be expedited.

[0133] Furthermore, the extracted images Cag to Cdg and Cak to Cdk from the training data Ga to Gd and inspection image data Ka to Kd are set so that the outer edge side of the solder paste 5 (i.e., the side opposite the center of the solder paste 5) is in a predetermined orientation. Therefore, the learning efficiency is improved, and the effort and time required to obtain the AI ​​model 101 can be further reduced. In addition, the accuracy of the reconstructed image data S output from the AI ​​model 101 is improved, making it possible to further enhance inspection accuracy.

[0134] Furthermore, the extracted images Cag-Cdg and Cak-Cdk from the training data Ga-Gd and inspection image data Ka-Kd are set so that their long or short sides extend along a predetermined orientation. Therefore, the learning efficiency can be further improved, and the effort and time required to obtain the AI ​​model 101 can be further reduced. In addition, the accuracy of the reconstructed image data S can be further improved, leading to an even greater improvement in inspection accuracy.

[0135] In addition, the extraction frames Wc for the learning data Ga to Gd and the extraction frames Wc for the inspection image data Ka to Kd are set to have equal lengths W in the short side direction, and when extracting the extracted images Cag to Cdg and Cak to Cdk, they are positioned at a constant position relative to the reference (outer edge of the solder paste 5). Therefore, large variations in the position and range of the solder paste 5 are less likely to occur in the extracted images Cag to Cdg and Cak to Cdk. This further improves learning efficiency and inspection accuracy.

[0136] Furthermore, the embodiment is not limited to the description above, and may be implemented as follows, for example. Of course, other applications and modifications not exemplified below are also possible.

[0137] (a) In the above embodiment, the extraction frame Wc is positioned to have a fixed positional relationship with the outer edge of the reference solder paste 5. In contrast, the extraction frame Wc may be positioned to have a fixed positional relationship with the outer edge of the reference land 3 (see Figure 33; the solder paste 5 is not shown in Figure 33). In this case, the numerical range of distance W1 may be wider than the numerical range of distance W1 in the above embodiment, taking into account dimensional errors (for example, 0.1 mm ≤ W1 ≤ 0.3 mm). Also, considering that the outer edge of the land 3 may be hidden by the solder paste 5, the position of the outer edge of the reference land 3 may be determined using the Gerber data (design data) of the printed circuit board 1. Furthermore, instead of the distance Ls indicating the width of the solder paste 5, the distance Lp indicating the width of the land 3 may be used.

[0138] (b) In the above embodiment, an example is given in which the solder inspection device 13 is used to inspect the solder paste 5 in a state where foreign matter F1 is protruding from the outer edge. However, the solder inspection device 13 can also be used to inspect the solder paste 5 in other states. For example, the solder inspection device 13 may be used to inspect the solder paste 5 that is partially bulging and covering foreign matter F2 on the land 3, as shown in Figure 34, or the solder paste 5 that has abnormalities in size or shape.

[0139] In this case, similar to the above embodiment, inspection image data Ka, Kb, Kc (see Figures 38-40) are obtained by performing processes such as identifying the solder image Ihk, extracting the extracted images Cak, Cbk, and Cck using the extraction frame Wc (see the left and center figures in Figures 35-37), rotating the extracted images Cbk and Cck (see the right figures in Figures 35 and 36), and pasting the extracted images Cak, Cbk, and Cck onto the image frame W1. For the sake of explanation, only a portion of the extracted images and inspection image data obtained for each of the four land 3 are shown here as representative examples.

[0140] Then, by comparing the entirety of the inspection image data Ka, Kb, and Kc with the entirety of the reconstructed image data S obtained using the inspection image data Ka, Kb, and Kc, the presence or absence of abnormalities in the solder paste 5 is determined. If inspection image data Kb and Kc relating to solder paste 5 with abnormalities in shape or size are input to the AI ​​model 101, the reconstructed image data S will be image data relating to good solder paste 5 with normal shape and size (see Figures 41 and 42). Also, if inspection image data Ka relating to solder paste 5 covering foreign matter F2 is input to the AI ​​model 101, the reconstructed image data S will be image data relating to good solder paste 5 with normal height and shape of the part related to foreign matter F2 (see Figure 43).

[0141] (c) In the above embodiment, when training the neural network 90, training data Ga to Gd is obtained using training source image data Ig related to the printed circuit board 1 that has passed the post-reflow inspection. Alternatively, training data Ga to Gd may be obtained using training source image data related to good quality solder paste 5 that has been visually selected by an operator after printing the solder paste 5.

[0142] Furthermore, the learning unit 77 may acquire learning data Ga to Gd using image data of a virtually generated good solder paste 5.

[0143] (d) In the above embodiment, separate AI models 101 are provided for two-dimensional data and for three-dimensional data. However, a common AI model may be provided for each of the two-dimensional and three-dimensional data.

[0144] (e) The configuration of the AI ​​model 101 (neural network 90) and its learning method are not limited to the above embodiment. For example, when performing the learning process of the neural network 90 or the reconstruction image data acquisition process, the configuration may include normalization or other processing on various data as needed. Also, the structure of the neural network 90 is not limited to that shown in Figure 6, and for example, a pooling layer may be provided after the convolutional layer 93. Of course, the number of layers in the neural network 90, the number of nodes in each layer, and the connection structure of each node may also be different configurations.

[0145] Furthermore, in the above embodiment, the AI ​​model 101 (neural network 90) is a generative model having the structure of a convolutional autoencoder (CAE), but it is not limited to this, and may be a generative model having the structure of a different type of autoencoder, such as a variational autoencoder (VAE).

[0146] Furthermore, in the above embodiment, the neural network 90 is trained using the backpropagation method, but the system is not limited to this, and various other learning algorithms may be used for training.

[0147] In addition, the neural network 90 may be composed of a dedicated AI processing circuit, such as a so-called AI chip. In that case, only learning information such as parameters may be stored in the storage unit 57, and the dedicated AI processing circuit may read this and set it in the neural network 90 to construct the AI ​​model 101.

[0148] In addition, in the above embodiment, the control device 33 is equipped with a learning unit 77 and is configured to perform training of the neural network 90 within the control device 33, but it is not limited to this. For example, the learning unit 77 may be omitted, and the training of the neural network 90 may be performed outside the control device 33, and the AI ​​model 101 (trained neural network 90) that has been trained externally may be stored in the storage unit 57.

[0149] (f) In the above embodiment, two-dimensional data and three-dimensional data are acquired as the raw image data Ik for inspection, but it is also possible to configure the system to acquire only one of the two-dimensional data or the three-dimensional data. Furthermore, depending on the data to be acquired, only the AI ​​model 101 corresponding to one of the two-dimensional data or the three-dimensional data may be provided.

[0150] (g) In the above embodiment, the image frame W1 is rectangular in shape satisfying m < n, but the image frame W1 may be square in shape.

[0151] (h) The size and position of the extraction frame Wc may be changed as appropriate. For example, by making the size of the extraction frame Wc larger, the solder paste 5 located towards the center may be included in the extracted image.

[0152] (i) The system may be configured to delete non-target solder paste 5 from the image data if a solder paste 5 other than the target solder paste 5 (non-target solder paste 5) is placed in the extraction frame Wc.

[0153] 1...Printed circuit board, 3...Land, 5...Solder paste, 13...Solder inspection device, 32d...Camera (image data acquisition means), 78...Inspection unit (inspection image data acquisition means, reconstructed image data acquisition means, comparison means), 101...AI model (identification means), Cag, Cbg, Ccg, Cdg, Cak, Cbk, Cck, Cdk...Extracted images, Ga, Gb, Gc, Gd...Training data, Ka, Kb, Kc, Kd...Inspection image data, W1...Image frame, Wc...Extraction frame, X1, X2...Short side, Y1...Center side long side, Y2...Outer side long side.

Claims

1. A solder inspection apparatus for inspecting abnormalities in solder paste printed on a printed circuit board, comprising: an image data acquisition means capable of acquiring image data of a predetermined area to be inspected on the printed circuit board, including solder paste printed on rectangular lands; an identification means generated by training a neural network having an encoding unit for extracting feature quantities from input image data and a decoding unit for reconstructing image data from the feature quantities, using only image data relating to good solder paste as training data; an inspection image data acquisition means for acquiring inspection image data including an image of the solder paste to be inspected based on the image data acquired by the image data acquisition means; a reconstructed image data acquisition means capable of acquiring reconstructed image data as reconstructed image data by inputting the inspection image data to the identification means; and a comparison means capable of comparing the inspection image data and the reconstructed image data, wherein the apparatus is configured to determine whether or not there are abnormalities in the solder paste based on the comparison result by the comparison means, and the training data consists of four extracted images, each containing the outer edge of the solder paste and corresponding to the four sides of the land, extracted from a solder image which is an image relating to solder paste corresponding to one land, and each of these images is placed in an image frame larger than the size of the extracted image. The solder inspection apparatus is characterized in that the inspection image data acquisition means acquires inspection image data in which each of the four extracted images extracted from the solder image in the image data acquired by the image data acquisition means is placed in an image frame of the same size as the image frame of the learning data.

2. The solder inspection apparatus according to claim 1, characterized in that the extracted images in the learning data and the inspection image data are set so that the outer edge of the solder paste is in a predetermined orientation.

3. The solder inspection apparatus according to claim 1, characterized in that the extracted image in the learning data and the inspection image data is set so that the long side or short side extends along a predetermined direction.

4. The solder inspection apparatus according to claim 1, wherein the extracted image is configured to extract a region within an extraction frame that has a certain positional relationship with the outer edge of a reference solder paste or land from the solder image, the extraction frame is rectangular in shape with two short sides, a central long side positioned on the center side of the solder paste, and an outer long side corresponding to the side opposite the central long side, the extraction frame for extracting the extracted image constituting the learning data and the extraction frame for extracting the extracted image constituting the inspection image data each have equal lengths W in the short side direction, and when extracting the extracted image, the central long side and the outer long side are positioned so as to sandwich the outer edge of the solder paste, and the outer long side is positioned at a distance W0 away from the reference in the short side direction.

Images