Solder inspection device

Abstract

This invention provides a solder inspection device that can reduce the effort and burden required to obtain identification means as an AI model, and that can use the identification means in common even when the land sizes differ. [Solution] An AI model 101 is generated by training a predetermined neural network using only image data related to good solder paste as training data. Inspection image data based on image data acquired by camera 32d is input to the AI ​​model 101 to obtain reconstructed image data, and the quality of the solder paste is determined by comparing the inspection image data and the reconstructed image data. The training data and inspection image data are provided as divided solder images, which are obtained by dividing the solder region image into 2 or 4 parts, and the image frame for the training data and the image frame for the inspection image data are the same size.

Description

Technical Field

[0001] The present invention relates to a solder inspection device for inspecting cream solder provided on a substrate.

Background Art

[0002] Generally, in a substrate manufacturing line for mounting electronic components on a printed circuit board, first, cream solder is printed on the lands of the printed circuit board (solder printing process). Next, the electronic components are temporarily fixed on the printed circuit board based on the viscosity of the cream solder (mounting process). Then, such a printed circuit board is led into a reflow furnace, and soldering is performed by heating and melting the cream solder (reflow process). In such a substrate manufacturing line, an inspection device for inspecting the printed circuit board may be provided.

[0003] Recently, as an inspection device for inspecting a printed circuit board, one using an AI model has been proposed. As an inspection device using an AI model, for example, by comparing inspection image data (original image data) of an inspection area on a printed circuit board with reconstructed image data generated by inputting the inspection image data into an AI model (identification means), an inspection is performed on whether the shape of the upper part of the cream solder on the printed circuit board is good (see, for example, Patent Document 1, etc.).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in order to obtain an AI model, it is necessary to train it using image data related to solder paste as training data. However, since the shape of each individual solder paste is different, if image data related to all solder pastes is used as training data, a large amount of image data must be prepared to obtain the desired accuracy, making efficient training difficult.

[0006] Furthermore, in order to improve inspection accuracy, it is preferable to match the size (width and height) of the training data with the size of the inspection image data. However, if the sizes of the training data and inspection image data are matched by scaling the image data, there is a risk that the inspection accuracy will decrease. To address this, it is conceivable to prepare various training data of different sizes corresponding to the sizes of various lands, and to separately prepare multiple AI models that have been trained only on training data of the same size. Then, during inspection, inspection image data of a size corresponding to the size of the land is acquired, and an appropriate AI model that matches the size of the land (i.e., the inspection image data) is selected from among the multiple AI models and used. However, preparing different AI models for each size of land (inspection image data) requires complicated work and considerable effort and time. In addition, obtaining multiple AI models requires preparing a huge amount of training data.

[0007] The present invention has been made in view of the above circumstances, and its purpose is to provide a solder inspection device that can reduce the effort and burden required to obtain an identification means as an AI model, and that can use the identification means in common even when the land sizes are different. [Means for solving the problem]

[0008] Below, we will describe, in separate sections, each means suitable for achieving the above objectives. Furthermore, we will add notes on the effects and benefits specific to each means as needed.

[0009] Method 1. A solder inspection device for inspecting solder paste printed on a printed circuit board, Image data acquisition means capable of acquiring image data of a predetermined area to be inspected on the printed circuit board including printed solder paste, An identification means generated by training a neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, using only image data related to good quality solder paste as training data, An inspection image data acquisition means that acquires inspection image data, including an image of the solder paste to be inspected, based on the image data acquired by the aforementioned image data acquisition means, A reconstructed image data acquisition means capable of inputting the aforementioned inspection image data into the identification means and acquiring the reconstructed image data as reconstructed image data, The system includes a comparison means capable of comparing the aforementioned inspection image data and the aforementioned reconstructed image data, Based on the comparison results obtained by the aforementioned comparison means, the system is configured to determine the quality of the solder paste. The aforementioned training data is obtained by dividing a solder region image representing solder paste corresponding to a land into four sections using virtual cross lines that allow for four sections of the same shape, and each of these four divided solder images is placed in an image frame larger than the size of the divided solder image. The solder inspection apparatus is characterized in that the inspection image data acquisition means acquires inspection image data in which each of the four divided solder images, obtained by dividing the solder region image in the image data acquired by the image data acquisition means into four parts by the virtual cross lines, is placed in an image frame the same size as the image frame of the learning data.

[0010] Furthermore, the training data may be generated from image data (actual image data) obtained by imaging a printed circuit board with good solder paste printed on it (i.e., an image relating to actual solder paste), or it may be a virtually generated image relating to good solder paste. Examples of the actual image data include image data accumulated from previous inspections, and image data of good printed circuit boards selected by an operator visually after the solder paste has been printed (the same applies to means 5 described later).

[0011] Furthermore, the above-mentioned "neural network" includes, for example, a convolutional neural network having multiple convolutional layers. The above-mentioned "learning" includes, for example, deep learning. The above-mentioned "discrimination means (generative model)" includes, for example, an autoencoder or a convolutional autoencoder (the same applies to means 5 described later).

[0012] In addition, the "identification means" is generated by training it only with image data related to good solder paste. Therefore, when inspection image data related to defective solder paste is input to the identification means, the reconstructed image data generated will be almost identical to the inspection image data in which the defective parts have been corrected (for example, foreign matter has been removed, or the shape and size have been corrected). In other words, when there are defective parts in the solder paste, the reconstructed image data related to the solder paste will be a virtual image data of the solder paste assuming that there are no defective parts (the same applies to means 5 described later).

[0013] According to the above method 1, the inspection image data is provided by placing a segmented solder image within an image frame. Therefore, the size (width and height) of the inspection image data will match the size of the image frame and will remain constant without varying slightly depending on the size of the land. This eliminates the need to prepare numerous different identification means for each land size, reducing the effort and time required to obtain the identification means. Furthermore, the identification means can be used in common even when the land sizes are different.

[0014] Furthermore, the image frames for the training data and the inspection image data are the same size, and the sizes of both the training data and the inspection image data are identical. Therefore, when the inspection image data is input to the identification means, the appropriate reconstructed image data corresponding to the inspection image data can be output more reliably, and consequently, the quality of the solder paste can be determined more accurately. This makes it possible to more reliably obtain good inspection accuracy.

[0015] Furthermore, the training data consists of four divided solder region images, each of which is placed within an image frame. Therefore, compared to using image data of the entire solder paste as training data, a recognition method with the desired accuracy can be obtained with less training data. Moreover, since four training data can be obtained from a single solder region image, the required number of training data can be easily obtained. These factors combined effectively reduce the effort and time required to obtain the recognition method.

[0016] In addition, since the training data and inspection image data are created by placing segmented solder images within image frames, the size of both data can be made relatively smaller compared to data created by placing an image of the entire solder paste within an image frame. Therefore, the processing burden related to training and inspection can be reduced, and training and inspection can be expedited.

[0017] Means 2. The divided solder images in the learning data and the inspection image data are set such that two sides corresponding to the virtual cross line and the angle formed by the two sides have a predetermined orientation, according to the solder inspection apparatus described in Means 1.

[0018] According to the above Means 2, the divided solder images in the learning data and the inspection image data are set such that two sides corresponding to the virtual cross line and the angle formed by the two sides have a predetermined orientation. For example, in both image data, one of the two sides corresponding to the virtual cross line faces downward, the other faces left, and the angle formed by the two sides faces the lower left. Therefore, the learning efficiency is improved, and the labor and effort in obtaining the discrimination means can be further reduced. Also, the accuracy of the reconstructed image data output from the discrimination means is improved, and the inspection accuracy can be further enhanced.

[0019] Furthermore, according to the above Means 2, in all the learning data and the inspection image data, the two sides corresponding to the virtual cross line and the corner formed by the two sides are common, so these do not become characteristic parts. Therefore, the accuracy of the reconstructed image data reconstructed by the discrimination means can be further enhanced, and the comparison by the comparison means can be performed more accurately. As a result, better inspection accuracy can be obtained.

[0020] Means 3. The virtual cross line is composed of a first center line connecting the centers of two parallel sides in the circumscribed rectangle that circumscribes the solder region image and has the minimum area, and a second center line connecting the centers of the other two parallel sides in the circumscribed rectangle, according to the solder inspection apparatus described in Means 1.

[0021] According to the above Means 3, even if the solder region image is rectangular or circular, the solder region image can be appropriately divided into four parts.

[0022] Means 4. The outer edge of the land corresponding to the solder region image has a rectangular shape in plan view. The virtual cross line is composed of a first center line connecting the centers of two parallel sides in an outer circumscribed rectangle that circumscribes the solder region image and extends in a direction consistent with the extending direction of the land of 1, and a second center line connecting the centers of the other two parallel sides in the outer circumscribed rectangle. The solder inspection apparatus according to means 1 is characterized by this.

[0023] Incidentally, the "outer circumscribed rectangle extending in a direction consistent with the extending direction of the land of 1" can be paraphrased as an outer circumscribed rectangle having two parallel sides parallel to two opposing sides constituting the outer edge of the land of 1 (the same applies in means 8 described later).

[0024] According to the above means 4, the process of determining the outer circumscribed rectangle becomes relatively easy. As a result, the processing burden related to learning and inspection can be further reduced, and the learning and inspection can be made faster.

[0025] Means 5. A solder inspection apparatus for inspecting the cream solder printed on a printed circuit board, Image data acquisition means capable of acquiring image data of a predetermined inspection region on the printed circuit board including the printed cream solder, An identification means generated by learning only the image data related to the good cream solder as learning data for a neural network having an encoding unit that extracts feature amounts from the input image data and a decoding unit that reconstructs the image data from the feature amounts, Inspection image data acquisition means for acquiring inspection image data including an image of the cream solder to be inspected based on the image data acquired by the image data acquisition means, Reconstructed image data acquisition means capable of acquiring the image data reconstructed by inputting the inspection image data into the identification means as reconstructed image data, Comparing means capable of comparing the inspection image data and the reconstructed image data, Based on the comparison result by the comparing means, it is configured to be able to determine the quality of the cream solder, The aforementioned training data is obtained by dividing a solder region image representing solder paste corresponding to a land into two parts by a virtual line that allows for the division into two parts of the same shape, and each of these two divided solder images is placed in an image frame larger than the size of the divided solder image. The solder inspection apparatus is characterized in that the inspection image data acquisition means acquires inspection image data in which each of the two divided solder images, obtained by dividing the solder region image in the image data acquired by the image data acquisition means into two parts by the virtual line, is placed in an image frame the same size as the image frame of the learning data.

[0026] According to method 5 described above, the size (width and height) of the inspection image data will match the size of the image frame and will remain constant without fluctuating significantly depending on the land size. This eliminates the need to prepare numerous different identification methods for each land size, reducing the effort and time required to obtain the identification method. Furthermore, the identification method can be used in common even when the land sizes are different.

[0027] Furthermore, since the sizes of the training data and the inspection image data are the same, when the inspection image data is input to the identification means, it is possible to more reliably output appropriate reconstructed image data corresponding to the inspection image data. As a result, the quality of the solder paste can be determined more accurately, and good inspection accuracy can be more reliably obtained.

[0028] Furthermore, the training data consists of two divided solder region images, each of which is placed within an image frame. Therefore, compared to using image data of the entire solder paste as training data, a recognition method with the desired accuracy can be obtained with less training data. Moreover, since two training data can be obtained from one solder region image, the required number of training data can be easily obtained. These factors together effectively reduce the effort and time required to obtain the recognition method.

[0029] In addition, since the training data and inspection image data are created by placing segmented solder images within image frames, the size of both data can be made relatively smaller compared to data created by placing an image of the entire solder paste within an image frame. Therefore, the processing burden related to training and inspection can be reduced, and training and inspection can be expedited.

[0030] Means 6. The solder inspection apparatus according to means 5, characterized in that the divided solder images in the learning data and the inspection image data are set so that the sides corresponding to the virtual lines are in a predetermined orientation.

[0031] According to the above-described means 6, the segmented solder images in the training data and inspection image data are set so that the edges corresponding to the virtual lines are oriented in a predetermined direction. For example, the segmented solder images in both image data are set so that the edges corresponding to the virtual lines face downwards. Therefore, the learning efficiency is improved, and the effort and time required to obtain the identification means can be further reduced. In addition, the accuracy of the reconstructed image data output from the identification means is improved, making it possible to further enhance the inspection accuracy.

[0032] Furthermore, according to the above means 6, since the edges corresponding to the virtual lines are common in all training data and inspection image data, these edges cease to be feature parts. Therefore, the accuracy of the reconstructed image data reconstructed by the identification means can be further improved, and the comparison by the comparison means can be performed more accurately. As a result, even better inspection accuracy can be obtained.

[0033] Means 7. The solder inspection apparatus according to Means 5, characterized in that the virtual line consists of a center line connecting the centers of two parallel sides in a circumscribing rectangle that is circumscribing the solder region image and has the smallest area.

[0034] According to the above means 7, even if the solder region image is rectangular or circular, the solder region image can be appropriately divided into two parts.

[0035] Means 8. The outer edge of the land corresponding to the solder region image is rectangular in plan view. The solder inspection apparatus according to means 5, characterized in that the virtual line consists of a center line connecting the centers of two parallel sides in a circumscribing rectangle that circumscribing the solder region image and extending in a direction that coincides with the direction in which the land 1 extends.

[0036] According to the above-described method 8, the process of defining the circumscribed rectangle becomes relatively easy, and the processing burden related to learning and testing can be further reduced. As a result, learning and testing can be accelerated more effectively.

[0037] Furthermore, the technical aspects related to each of the above means may be combined as appropriate. For example, the technical aspects related to means 2 may be combined with the technical aspects related to means 3 or 4. [Brief explanation of the drawing]

[0038] [Figure 1] This is a magnified plan view of a portion of a printed circuit board. [Figure 2] This is a magnified cross-sectional view of a portion of a printed circuit board. [Figure 3] This is a block diagram showing the configuration of a printed circuit board manufacturing line. [Figure 4] This is a schematic diagram illustrating the configuration of a solder inspection device. [Figure 5] This is a block diagram showing the functional configuration of a solder inspection device. [Figure 6] This is a schematic diagram illustrating the structure of a neural network. [Figure 7] This is a flowchart showing the learning process of a neural network. [Figure 8] This is a flowchart showing the inspection process. [Figure 9] This is a schematic diagram showing the source image data used for training. [Figure 10] This is a schematic diagram showing the solder region image Hg1 in the first embodiment. [Figure 11]This is a schematic diagram showing the solder region image Hg2 in the first embodiment. [Figure 12] This is a schematic diagram illustrating the acquisition process of segmented solder images Bg1, Bg2, Bg3, and Bg4, as well as image processing of segmented solder images Bg1, Bg2, and Bg3, in the first embodiment. [Figure 13] This is a schematic diagram illustrating the acquisition process of segmented solder images Bg5, Bg6, Bg7, and Bg8, as well as image processing of segmented solder images Bg5, Bg6, and Bg7, in the first embodiment. [Figure 14] This is a schematic diagram showing the image frame and training data G1 in the first embodiment. [Figure 15] This is a schematic diagram showing the image frame and training data G2 in the first embodiment. [Figure 16] This is a schematic diagram showing the image frame and training data G3 in the first embodiment. [Figure 17] This is a schematic diagram showing the image frame and training data G4 in the first embodiment. [Figure 18] This is a schematic diagram showing the image frame and training data G5 in the first embodiment. [Figure 19] This is a schematic diagram showing the image frame and training data G6 in the first embodiment. [Figure 20] This is a schematic diagram showing the image frame and training data G7 in the first embodiment. [Figure 21] This is a schematic diagram showing the image frame and training data G8 in the first embodiment. [Figure 22] This is a schematic diagram showing the source image data used for inspection. [Figure 23] This is a schematic diagram showing the solder region image Hk1 in the first embodiment. [Figure 24] This is a schematic diagram showing the solder region image Hk2 in the first embodiment. [Figure 25] This is a schematic diagram illustrating the acquisition process of segmented solder images Bk1, Bk2, Bk3, and Bk4, and the image processing applied to the segmented solder images Bk1, Bk2, and Bk3 in the first embodiment. [Figure 26]This is a schematic diagram illustrating the acquisition process of segmented solder images Bk5, Bk6, Bk7, and Bk8, as well as image processing of the segmented solder images Bk5, Bk6, and Bk7, in the first embodiment. [Figure 27] This is a schematic diagram showing the image frame and inspection image data K1 in the first embodiment. [Figure 28] This is a schematic diagram showing the image frame and inspection image data K2 in the first embodiment. [Figure 29] This is a schematic diagram showing the image frame and inspection image data K3 in the first embodiment. [Figure 30] This is a schematic diagram showing the image frame and inspection image data K4 in the first embodiment. [Figure 31] This is a schematic diagram showing the image frame and inspection image data K5 in the first embodiment. [Figure 32] This is a schematic diagram showing the image frame and inspection image data K6 in the first embodiment. [Figure 33] This is a schematic diagram showing the image frame and inspection image data K7 in the first embodiment. [Figure 34] This is a schematic diagram showing the image frame and inspection image data K8 in the first embodiment. [Figure 35] This is a schematic diagram showing the reconstructed image data output from the AI ​​model when the inspection image data K1 is input in the first embodiment. [Figure 36] This is a schematic diagram showing the reconstructed image data output from the AI ​​model when the inspection image data K5 is input in the first embodiment. [Figure 37] This is a schematic diagram illustrating the acquisition process of segmented solder images Bg1 and Bg2, and image processing of segmented solder image Bg1, in the second embodiment. [Figure 38] This is a schematic diagram illustrating the acquisition process of segmented solder images Bg3 and Bg4, and image processing of segmented solder image Bg3, in the second embodiment. [Figure 39] This is a schematic diagram showing the image frame and training data G1 in the second embodiment. [Figure 40] This is a schematic diagram showing the image frame and training data G2 in the second embodiment. [Figure 41] This is a schematic diagram showing the image frame and training data G3 in the second embodiment. [Figure 42] This is a schematic diagram showing the image frame and training data G4 in the second embodiment. [Figure 43] This is a schematic diagram illustrating the acquisition process of segmented solder images Bk1 and Bk2, and image processing of segmented solder image Bk1, etc., in the second embodiment. [Figure 44] This is a schematic diagram illustrating the acquisition process of segmented solder images Bk5 and Bk6, and image processing of the segmented solder image Bk5, in the second embodiment. [Figure 45] This is a schematic diagram showing the image frame and inspection image data K1 in the second embodiment. [Figure 46] This is a schematic diagram showing the image frame and inspection image data K2 in the second embodiment. [Figure 47] This is a schematic diagram showing the image frame and inspection image data K3 in the second embodiment. [Figure 48] This is a schematic diagram showing the image frame and inspection image data K4 in the second embodiment. [Figure 49] In another embodiment, this is a schematic diagram showing the circumscribed rectangle, virtual cross lines, etc., when the solder region image is divided into four sections. [Figure 50] In another embodiment, this is a schematic diagram showing the circumscribed rectangle, virtual lines, etc., when the solder region image is divided into two parts. [Figure 51] In another embodiment, this is a schematic diagram showing a bounding rectangle or the like that has a shape corresponding to the outer edge of a land. [Modes for carrying out the invention]

[0039] The embodiments will be described below with reference to the drawings. [First Embodiment] First, let's explain the configuration of the printed circuit board. Figure 1 is a magnified plan view of a portion of the printed circuit board, and Figure 2 is a magnified cross-sectional view of a portion of the printed circuit board.

[0040] As shown in Figures 1 and 2, the printed circuit board 1 has a flat base substrate 2 made of glass epoxy resin or the like, on which wiring patterns (not shown) made of copper foil and multiple rectangular or circular lands 3 in plan view are formed. The portion of the base substrate 2 surface excluding the lands 3 is coated with a resist film 4.

[0041] Furthermore, a solder paste 5, made by mixing solder granules with flux, is printed on the land 3. In this embodiment, there are at least two types of solder paste 5: one printed on a rectangular land 3 that is rectangular in shape when viewed from above, and another printed on a circular land 3 that is circular in shape when viewed from above. For convenience, in Figure 1 and other images, a dotted pattern is added to the areas representing the solder paste 5. Also, in Figure 1 and other images, only a small portion of the multiple land 3s and solder paste 5 present on the printed circuit board 1 are shown.

[0042] Next, the manufacturing line (manufacturing process) for producing the printed circuit board 1 will be explained with reference to Figure 3. As shown in Figure 3, the manufacturing line 10 is equipped with, in order from the upstream side (upper side of Figure 3), a solder printing machine 12, a solder inspection device 13, a component mounting machine 14, a reflow device 15, and a post-reflow inspection device 16.

[0043] The solder printing machine 12 performs a solder printing process to print solder paste 5 onto each land 3 of the printed circuit board 1. In the solder printing process, for example, the solder paste 5 is printed by screen printing. In screen printing, first, the lower surface of the screen mask is brought into contact with the printed circuit board 1, and solder paste 5 is supplied to the upper surface of the screen mask. The screen mask has a plurality of openings corresponding to each land 3 of the printed circuit board 1. Next, the openings are filled with solder paste 5 by moving a predetermined squeegee while it is in contact with the upper surface of the screen mask. After that, the printed circuit board 1 is separated from the lower surface of the screen mask, and solder paste 5 is printed onto each land 3 of the printed circuit board 1.

[0044] The solder inspection device 13 inspects the solder paste 5, determining its quality based on factors such as shape and the presence or absence of foreign matter. Details of the solder inspection device 13 will be described later.

[0045] The component mounting machine 14 mounts electronic components 25 (see Figures 1 and 2) on land 3 on which solder paste 5 is printed. The electronic component 25 has multiple electrodes (not shown), and each electrode is temporarily fixed to a predetermined amount of solder paste 5.

[0046] The reflow apparatus 15 heats and melts the solder paste 5 to perform a reflow process in which the land 3 and the electrodes of the electronic component 25 are soldered together.

[0047] The post-reflow inspection device 16 checks whether the solder joints were properly formed during the reflow process by, for example, using brightness image data to confirm whether there is any misalignment in the electronic components 25.

[0048] In addition, although not shown in the diagram, the manufacturing line 10 is equipped with conveyors for transporting printed circuit boards 1 between the aforementioned devices, such as between the solder printing machine 12 and the solder inspection device 13. Branching devices are also provided between the solder inspection device 13 and the component mounting machine 14, and downstream of the post-reflow inspection device 16. Printed circuit boards 1 that are judged as good by the solder inspection device 13 or the post-reflow inspection device 16 are guided downstream, while printed circuit boards 1 that are judged as defective are discharged to the defective product storage section by the branching devices.

[0049] Next, the configuration of the solder inspection device 13 will be explained in detail with reference to Figures 4 and 5. Figure 4 is a schematic diagram showing the configuration of the solder inspection device 13. Figure 5 is a block diagram showing the functional configuration of the solder inspection device 13.

[0050] The solder inspection device 13 includes a transport mechanism 31 that transports and positions the printed circuit board 1, an inspection unit 32 for obtaining image data of the printed circuit board 1, and a control device 33 (see Figure 5) that performs various controls, image processing, and calculation processing in the solder inspection device 13, including drive control of the transport mechanism 31 and the inspection unit 32.

[0051] The transport mechanism 31 comprises a pair of transport rails 31a arranged along the loading and unloading direction of the printed circuit board 1, and an endless conveyor belt 31b rotatably mounted on each transport rail 31a. Although not shown in the figures, the transport mechanism 31 is also provided with a drive means such as a motor for driving the conveyor belt 31b, and a chuck mechanism for positioning the printed circuit board 1 at a predetermined position. The transport mechanism 31 is driven and controlled by a control device 33 (a transport mechanism control unit 79, described later).

[0052] Under the above configuration, the printed circuit board 1, when brought into the solder inspection device 13, has both side edges in the width direction perpendicular to the loading / unloading direction inserted into the transport rails 31a and is placed on the conveyor belt 31b. Subsequently, the conveyor belt 31b starts moving, and the printed circuit board 1 is transported to a predetermined inspection position. When the printed circuit board 1 reaches the inspection position, the conveyor belt 31b stops and the chuck mechanism activates. This chuck mechanism pushes up the conveyor belt 31b, and both side edges of the printed circuit board 1 are clamped between the conveyor belt 31b and the upper edges of the transport rails 31a. This positions and fixes the printed circuit board 1 in the inspection position. When the inspection is completed, the fixing by the chuck mechanism is released, and the conveyor belt 31b starts moving again. As a result, the printed circuit board 1 is discharged from the solder inspection device 13. Of course, the configuration of the transport mechanism 31 is not limited to the above form, and other configurations may be adopted.

[0053] The inspection unit 32 is positioned above the transport rail 31a (the transport path for the printed circuit board 1). The inspection unit 32 includes a first illumination device 32a, a second illumination device 32b, a third illumination device 32c, and a camera 32d. In this embodiment, the camera 32d constitutes the "image data acquisition means".

[0054] Furthermore, the inspection unit 32 is also equipped with an X-axis movement mechanism 32e (see Figure 5) that enables movement in the X-axis direction (left-right direction in Figure 4), and a Y-axis movement mechanism 32f (see Figure 5) that enables movement in the Y-axis direction (front-back direction in Figure 4). These movement mechanisms 32e and 32f are driven and controlled by a control device 33 (movement mechanism control unit 76, which will be described later).

[0055] The first illumination device 32a and the second illumination device 32b, in order to perform three-dimensional measurement of the printed circuit board 1, each illuminate a predetermined area to be inspected on the printed circuit board 1 with predetermined light for three-dimensional measurement (patterned light having a striped light intensity distribution) from diagonally above.

[0056] Specifically, the first lighting device 32a includes a first light source 32a1 that emits predetermined light, and a first liquid crystal shutter 32a2 that forms a first grid that converts the light from the first light source 32a1 into first pattern light having a striped light intensity distribution, and is driven and controlled by a control device 33 (lighting control unit 72, which will be described later).

[0057] The second lighting device 32b includes a second light source 32b1 that emits predetermined light, and a second liquid crystal shutter 32b2 that forms a second grating that converts the light from the second light source 32b1 into a second pattern light having a striped light intensity distribution, and is driven and controlled by a control device 33 (lighting control unit 72, described later).

[0058] Under the above configuration, the light emitted from each light source 32a1 and 32b1 is guided to a focusing lens (not shown), where it is converted into parallel light, and then guided to a projection lens (not shown) via liquid crystal shutters 32a2 and 32b2, where it is projected onto the printed circuit board 1 as pattern light. In this embodiment, the switching control of the liquid crystal shutters 32a2 and 32b2 is performed so that the phase of each pattern light is shifted by a quarter pitch.

[0059] Furthermore, by using liquid crystal shutters 32a2 and 32b2 as the grating, it is possible to irradiate pattern light that is close to an ideal sine wave. This improves the measurement resolution of three-dimensional measurements. In addition, the phase shift of the pattern light can be controlled electrically, allowing for a more compact device.

[0060] The third illumination device 32c irradiates a predetermined area of ​​the printed circuit board 1 under inspection with a predetermined light (e.g., uniform light) for two-dimensional measurement of the printed circuit board 1. The third illumination device 32c is equipped with a ring light capable of emitting blue light, a ring light capable of emitting green light, and a ring light capable of emitting red light. Since the third illumination device 32c has a configuration similar to that of known technology, a detailed explanation thereof will be omitted.

[0061] Camera 32d captures a predetermined inspection area of ​​the printed circuit board 1 from directly above. Camera 32d has an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor, and an optical system (lens unit, aperture, etc.) that forms an image of the printed circuit board 1 onto the image sensor, and its optical axis is arranged to align with the vertical direction (Z-axis direction). Of course, the image sensor is not limited to these, and other image sensors may be used.

[0062] Camera 32d is driven and controlled by control device 33 (camera control unit 73, described later). More specifically, control device 33 performs imaging processing by camera 32d in synchronization with the illumination processing by each illumination device 32a, 32b, and 32c. As a result, the light reflected from the printed circuit board 1 from any of the illumination devices 32a, 32b, and 32c is imaged by camera 32d. Consequently, image data of the area of ​​the printed circuit board 1 to be inspected, including the solder paste 5 printed on the land 3, is acquired. The "area to be inspected" of the printed circuit board 1 is one of several areas pre-set on the printed circuit board 1, with the size of the camera 32d's imaging field of view (imaging range) as one unit.

[0063] Furthermore, the camera 32d in this embodiment is a color camera. This allows for simultaneous imaging of each color of light illuminating the printed circuit board 1 from each color ring light of the third illumination device 32c.

[0064] Image data captured and generated by camera 32d is converted into a digital signal within camera 32d and then transferred in digital form to control device 33 (image acquisition unit 74, described later). The control device 33 then stores the transferred image data and performs various image processing and calculation processing based on the image data.

[0065] The control device 33 consists of a computer including a CPU (Central Processing Unit) that executes predetermined arithmetic processing, a ROM (Read Only Memory) that stores various programs and fixed value data, a RAM (Random Access Memory) that temporarily stores various data when executing various arithmetic processing, and peripheral circuits for these.

[0066] The control device 33 functions as various functional units, such as the main control unit 71, lighting control unit 72, camera control unit 73, image acquisition unit 74, data processing unit 75, movement mechanism control unit 76, learning unit 77, inspection unit 78, and transport mechanism control unit 79, as the CPU operates according to various programs.

[0067] However, the various functional units described above are realized through the cooperation of various hardware components such as the CPU, ROM, and RAM, and there is no need to clearly distinguish between functions realized in hardware and functions realized in software. Some or all of these functions may be realized by hardware circuits such as ICs.

[0068] Furthermore, the control device 33 includes an input unit 55 consisting of a keyboard, mouse, touch panel, etc., a display unit 56 equipped with a display screen consisting of a liquid crystal display, etc., a storage unit 57 capable of storing various data, programs, calculation results, inspection results, etc., and a communication unit 58 capable of sending and receiving various data with the outside.

[0069] Here, the various functional units that constitute the control device 33 will be described in detail.

[0070] The main control unit 71 is a functional unit that controls the entire solder inspection device 13 and is configured to send and receive various signals with other functional units such as the lighting control unit 72 and the camera control unit 73.

[0071] The lighting control unit 72 is a functional unit that drives and controls the lighting devices 32a, 32b, and 32c, and performs switching control of the illuminated light based on command signals from the main control unit 71.

[0072] The camera control unit 73 is a functional unit that drives and controls the camera 32d, and controls the imaging timing and other parameters based on command signals from the main control unit 71.

[0073] The image acquisition unit 74 is a functional unit for capturing image data acquired by the camera 32d.

[0074] The data processing unit 75 is a functional unit that performs predetermined image processing on image data acquired by the image acquisition unit 74, and performs two-dimensional measurement processing, three-dimensional measurement processing, and the like using the image data.

[0075] The movement mechanism control unit 76 is a functional unit that drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, and controls the position of the inspection unit 32 based on command signals from the main control unit 71. By driving and controlling the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, the movement mechanism control unit 76 can move the inspection unit 32 to a position above any area to be inspected on the printed circuit board 1, which is positioned and fixed at the inspection position. Then, as the inspection unit 32 is sequentially moved to multiple areas to be inspected set on the printed circuit board 1, the inspection of the area to be inspected is performed, thereby performing an inspection of the entire printed circuit board 1.

[0076] The learning unit 77 is a functional unit that uses training data to train the deep neural network 90 (hereinafter simply referred to as "neural network 90"; see Figure 6) and constructs an AI (Artificial Intelligence) model 101 as a "discrimination tool".

[0077] As will be described later, the AI ​​model 101 in this embodiment is a generative model constructed by deep learning a neural network 90 using only image data related to good solder paste 5 as training data, and has the structure of a so-called autoencoder.

[0078] Here, the structure of the neural network 90 will be explained with reference to Figure 6. Figure 6 is a schematic diagram conceptually showing the structure of the neural network 90. ​​As shown in Figure 6, the neural network 90 has the structure of a convolutional auto-encoder (CAE), comprising an encoder unit 91 as an "encoding unit" that extracts feature quantities (latent variables) TA from the input image data GA, and a decoder unit 92 as a "decoding unit" that reconstructs image data GB from the feature quantities TA.

[0079] The structure of the convolutional autoencoder is well known, so a detailed explanation will be omitted. The encoder unit 91 has multiple convolutional layers 93, and in each convolutional layer 93, the result of a convolution operation using multiple filters (kernels) 94 on the input data is output as input data for the next layer. Similarly, the decoder unit 92 has multiple deconvolutional layers 95, and in each deconvolutional layer 95, the result of a deconvolution operation using multiple filters (kernels) 96 on the input data is output as input data for the next layer. Then, in the learning process described later, the weights (parameters) of each filter 94, 96 are updated.

[0080] The inspection unit 78 is a functional unit that inspects the quality of the solder paste 5. In this embodiment, the inspection unit 78 inspects whether the solder paste 5 is properly printed in terms of the presence, size, and shape of foreign matter.

[0081] The transport mechanism control unit 79 is a functional unit that drives and controls the transport mechanism 31, and controls the position of the printed circuit board 1 based on command signals from the main control unit 71.

[0082] The memory unit 57 is composed of an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and has a predetermined memory area for storing, for example, the AI ​​model 101 (neural network 90 and the learning information acquired through its learning).

[0083] The communication unit 58 is equipped with a wireless communication interface conforming to communication standards such as wired LAN (Local Area Network) or wireless LAN, and is configured to send and receive various data to and from the outside. For example, the results of inspections performed by the inspection unit 78 are output to the outside via the communication unit 58, and the results of inspections performed by the post-reflow inspection device 16 are input via the communication unit 58.

[0084] Next, the learning process of the neural network 90 performed by the solder inspection device 13 will be explained with reference to the flowchart in Figure 7.

[0085] When the learning process starts based on the execution of a predetermined learning program, the main control unit 71 first performs pre-processing in step S101 for training the neural network 90.

[0086] In this preprocessing step, inspection information for numerous printed circuit boards 1 stored in the post-reflow inspection device 16 is first obtained via the communication unit 58. Subsequently, based on this inspection information, learning source image data Ig, which is image data related to good solder paste 5 that passed the post-reflow inspection, is obtained from the storage unit 57 (see, for example, Figure 9). Figure 9 shows only a portion of the learning source image data Ig; the actual learning source image data Ig contains many more solder paste 5 and lands 3.

[0087] The training source image data Ig pertains to the printed circuit board 1 after the solder paste 5 has been printed and before the electronic components 25 have been mounted, and is used to obtain the training data G1, G2, G3, G4, G5, G6, G7, G8 (hereinafter referred to as "training data G1-G8") used for training the neural network 90. ​​The training source image data Ig includes three-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while pattern light is irradiated from the first illumination device 32a or the second illumination device 32b, and two-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while uniform light is irradiated from the third illumination device 32c.

[0088] Furthermore, the training source image data Ig may be image data obtained by camera 32d without any special processing (for example, monochrome luminance image data or RGB luminance image data), or it may be image data obtained by applying predetermined processing to the image data obtained by camera 32d (for example, HLS image data obtained by converting RGB image data, or height image data obtained by converting image data) (the same applies to the inspection source image data Ik described later).

[0089] Next, the area occupied by the solder paste 5 in the acquired training image data Ig is identified. If the training image data Ig is two-dimensional data, the area occupied by the solder paste 5 is identified using, for example, brightness, hue, and saturation. If the training image data Ig is three-dimensional data, the area occupied by the solder paste 5 is identified using, for example, height information.

[0090] Next, images of the connected components (clumps) in the region occupied by the identified solder paste 5 are extracted as solder region images Hg1 and Hg2 (see, for example, Figures 10 and 11). Solder region image Hg1 relates to the rectangular solder paste 5 printed on the rectangular land 3. Solder region image Hg2 relates to the circular solder paste 5 printed on the circular land 3. Figures 10 and 11 show some examples of solder region images Hg1 and Hg2.

[0091] The solder region images Hg1 and Hg2 correspond to the land 3 of 1. In this embodiment, the connected component (clump portion) of the area occupied by the solder paste 5 that is located on the land 3 of 1 according to the design is extracted as the solder region images Hg1 and Hg2 of 1. Alternatively, the connected component (clump portion) of 1 may be extracted as the solder region images Hg1 and Hg2 of 1 without considering the position of the land 3. Furthermore, in order to eliminate the influence of minute shape differences at the outer edge (base) of the solder paste 5, the solder region images Hg1 and Hg2 may be subjected to a shrinkage process to obtain solder region images Hg1 and Hg2 with the outer edge removed. In addition, the solder region images Hg1 and Hg2 with the outer edge (base) removed may be obtained by cutting off the portion below a predetermined height.

[0092] Next, divided solder images Bg1, Bg2, Bg3, Bg4, Bg5, Bg6, Bg7, Bg8 (see Figures 12 and 13; hereinafter referred to as "divided solder images Bg1-Bg8") are obtained from the solder region images Hg1 and Hg2. Divided solder images Bg1, Bg2, Bg3, Bg4 (hereinafter referred to as "divided solder images Bg1-Bg4") are images obtained by dividing the solder region image Hg1 into four parts using a virtual cross line Xga (see Figure 12) that can divide it into four parts of the same shape. Divided solder images Bg5, Bg6, Bg7, Bg8 (hereinafter referred to as "divided solder images Bg5-Bg8") are images obtained by dividing the solder region image Hg2 into four parts using a virtual cross line Xgb (see Figure 13) that can divide it into four parts of the same shape. Therefore, the divided solder images Bg1 to Bg4 relate to rectangular solder paste 5 printed on rectangular land 3, and the divided solder images Bg5 to Bg8 relate to circular solder paste 5 printed on circular land 3.

[0093] The virtual cross line Xga consists of a first center line Cga1 connecting the centers of two parallel sides in the circumscribed rectangle Sga, and a second center line Cga2 perpendicular to the first center line Cga1 and connecting the centers of two other parallel sides in the circumscribed rectangle Sga. The virtual cross line Xgb consists of a first center line Cgb1 connecting the centers of two parallel sides in the circumscribed rectangle Sgb, and a second center line Cgb2 perpendicular to the first center line Cgb1 and connecting the centers of two other parallel sides in the circumscribed rectangle Sgb. As described above, the virtual cross lines Xga and Xgb allow the solder area images Hg1 and Hg2 to be divided into four identical shapes. However, "divisible into four identical shapes" more precisely means that when the solder paste 5 is printed in the ideal shape (i.e., the design shape), the solder area image related to the solder paste 5 can be divided into four identical shapes (the same applies to the virtual cross lines Xka and Xkb described later). Furthermore, the circumscribed rectangle Sga is the rectangle with the smallest area among the rectangles that circumscribe the solder region image Hg1. The circumscribed rectangle Sgb is the rectangle with the smallest area among the rectangles that circumscribe the solder region image Hg2.

[0094] Next, image processing (rotation and inversion) is performed on the images Bg1 to Bg8 that require image processing (in this embodiment, Bg1, Bg2, Bg3, Bg5, Bg6, Bg7) from the divided solder images Bg1 to Bg8, so that the two sides corresponding to the virtual cross lines Xga and Xgb in the divided solder images Bg1 to Bg8 and the corner formed by these two sides are set to a predetermined orientation (see the lower figures in Figures 12 and 13). In this embodiment, the divided solder images Bg1 to Bg8 are set so that the sides corresponding to the first center lines Cga1 and Cgb1 point downwards, the sides corresponding to the second center lines Cga2 and Cgb2 point lefts, and the corner formed by these two sides points downwards to the left.

[0095] Then, by pasting the divided solder images Bg1 to Bg4 onto the image frame W1, we obtain training data G1, G2, G3, and G4 (see Figures 14 to 17; hereinafter referred to as "training data G1 to G4") which consist of the divided solder images Bg1 to Bg4 placed on the image frame W1. In addition, by pasting the divided solder images Bg5 to Bg8 onto the image frame W1, we obtain training data G5, G6, G7, and G8 (see Figures 18 to 21; hereinafter referred to as "training data G5 to G8") which consist of the divided solder images Bg5 to Bg8 placed on the image frame W1.

[0096] Image frame W1 is a rectangular image with a height (width in the vertical direction of the paper, such as in Figure 14) of m (pixels) and a width (width in the horizontal direction of the paper, such as in Figure 14) of n (pixels). Its size (width and height) is set to be larger than the size of each of the divided solder images Bg1 to Bg8, based on the design data, etc. Note that m and n are natural numbers, and in this embodiment, m=n is satisfied. Furthermore, by adjusting the pasting position, each learning data G1 to G8 is set to coincide with the center or centroid of the divided solder images Bg1 to Bg8 and the center of image frame W1.

[0097] Then, by repeatedly performing the above processes, such as extracting solder region images Hg1 and Hg2, acquiring segmented solder images Bg1 to Bg8, performing image processing on segmented solder images Bg1 to Bg8 as needed, and pasting the segmented solder images Bg1 to Bg8 onto the image frame W1, multiple training data G1 to G8 are obtained from one training source image data Ig. At this time, four training data G1 to G4 are obtained for each rectangular land 3 (cream solder 5) contained in the training source image data Ig. Also, four training data G5 to G8 are obtained for each circular land 3 (cream solder 5) contained in the training source image data Ig. Furthermore, by using multiple training source image data Ig, the required number of training data G1 to G8 are finally obtained. In this embodiment, the training data G1 to G8 include data obtained based on two-dimensional data and data obtained based on three-dimensional data.

[0098] In step S101, once the necessary number of training data G1 to G8 have been acquired, in the following step S102, the learning unit 77 prepares an untrained neural network 90 based on a command from the main control unit 71. For example, it reads a neural network 90 that has been previously stored in the memory unit 57 or the like. Alternatively, it constructs a neural network 90 based on network configuration information (for example, the number of layers in the neural network and the number of nodes in each layer) stored in the memory unit 57 or the like.

[0099] In this embodiment, two neural networks 90 are constructed separately: one that learns using training data G1-G8 acquired based on two-dimensional data, and another that learns using training data G1-G8 acquired based on three-dimensional data. Therefore, in this embodiment, two neural networks 90 are constructed.

[0100] In step S103, reconstructed image data is acquired. Specifically, based on a command from the main control unit 71, the learning unit 77 provides the learning data G1 to G8 acquired in step S102 as input data to the input layer of the neural network 90, thereby acquiring the reconstructed image data output from the output layer of the neural network 90. ​​More specifically, the learning unit 77 provides the input layer of the neural network 90 with the learning data G1 to G8 acquired in step S102 that corresponds to the neural network 90 as input data, thereby acquiring the reconstructed image data output from the output layer of the neural network 90. ​​For example, the learning unit 77 provides the learning data G1 to G8 as input data to the input layer of a neural network 90 that performs learning using learning data G1 to G8 obtained from two-dimensional data, and acquires the reconstructed image data output from the neural network 90. ​​In other words, the learning unit 77 inputs the appropriate learning data G1 to G8 to each of the two neural networks 90 and acquires the output reconstructed image data.

[0101] In the following step S104, the learning unit 77 compares the input learning data G1 to G8 with the reconstructed image data output by the neural network 90 and determines whether the error is sufficiently small (whether it is below a predetermined threshold).

[0102] If the error is sufficiently small, in step S106, the learning unit 77 determines whether the learning termination conditions are met. For example, if a certain number of consecutive affirmative judgments are made in step S104 without going through the process of step S105 described later, or if learning using all of the prepared learning data G1 to G8 is repeated a predetermined number of times, it is determined that the termination conditions are met. If the termination conditions are met, the neural network 90 and its learning information (updated parameters, etc., described later) are stored in the storage unit 57 as the AI ​​model 101, and this learning process is terminated.

[0103] In this embodiment, the AI ​​model 101 ultimately stores an AI model trained on training data G1 to G8 acquired from two-dimensional data, and an AI model trained on training data G1 to G8 acquired from three-dimensional data.

[0104] On the other hand, if the termination condition is not met in step S106, the process returns to step S102 and the neural network 90 is trained again.

[0105] Furthermore, if the error is not sufficiently small in step S104, the network update process (training of the neural network 90) is performed in step S105, and then the process returns to step S103 and the above series of processes is repeated.

[0106] Specifically, in the network update process of step S105, known learning algorithms such as backpropagation are used to update the weights (parameters) of each filter 94 and 96 in the neural network 90 to more appropriate values ​​so that the loss function representing the difference between the training data G1 to G8 and the reconstructed image data is minimized. For example, BCE (Binary Cross-entropy) can be used as the loss function.

[0107] By repeatedly performing steps S103 to S105, the neural network 90 minimizes the error between the training data G1 to G8 and the reconstructed image data, resulting in the output of more accurate reconstructed image data.

[0108] The resulting AI model 101 will generate reconstructed image data that closely matches the image data of a good solder paste 5 when that image data is input. Furthermore, when the AI ​​model 101 receives image data of a defective solder paste 5 in terms of shape, size, or presence of foreign matter, it will generate reconstructed image data that closely matches the image data after correcting the shape and size of the solder paste 5 or removing noise (parts corresponding to foreign matter). In other words, when the solder paste 5 is defective, the reconstructed image data of the solder paste 5 will be a virtual image data of the solder paste 5 assuming that there are no defects.

[0109] Next, the inspection process performed by the solder inspection device 13 will be explained with reference to the flowchart in Figure 8. This inspection process is performed for each area to be inspected on the printed circuit board 1.

[0110] Once the printed circuit board 1 is brought into the solder inspection device 13 and positioned at a predetermined inspection location, the inspection process is started based on the execution of a predetermined inspection program.

[0111] When the inspection process is started, the first step, S301, is the image data acquisition process. In the image data acquisition process, the raw inspection image data Ik (see, for example, Figure 22) related to the printed circuit board 1 to be inspected is acquired. The raw inspection image data Ik is image data used to obtain the inspection image data K1, K2, K3, K4, K5, K6, K7, K8 (hereinafter referred to as "inspection image data K1 to K8") described later. In this embodiment, as an example of the printed circuit board 1 to be inspected, one is given that has solder paste 5 with foreign matter F1 attached and solder paste 5 with a shape abnormality (chip).

[0112] The inspection source image data Ik includes three-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while pattern light is irradiated from the first illumination device 32a or the second illumination device 32b, and two-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while uniform light is irradiated from the third illumination device 32c. The image data acquisition process consists of acquiring three-dimensional data and acquiring two-dimensional data.

[0113] First, let's explain the process for acquiring three-dimensional data. In this process, the phase of the first pattern light emitted from the first illumination device 32a is changed, and imaging is performed four times under the first pattern light with different phases. Then, the phase of the second pattern light emitted from the second illumination device 32b is changed, and imaging is performed four times under the second pattern light with different phases, thereby acquiring a total of eight different sets of three-dimensional data. This will be explained in detail below.

[0114] As described above, once the printed circuit board 1, which has been brought into the solder inspection device 13, is positioned and fixed at a predetermined inspection location, the movement mechanism control unit 76, based on a command from the main control unit 71, first drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f to move the inspection unit 32, and adjusts the imaging field of view (imaging range) of the camera 32d to match the predetermined area to be inspected on the printed circuit board 1.

[0115] In addition, the lighting control unit 72 switches the liquid crystal shutters 32a2 and 32b2 of both lighting devices 32a and 32b, and sets the positions of the first and second grids formed on both liquid crystal shutters 32a2 and 32b2 to predetermined reference positions.

[0116] Once the switching settings for the first and second grids are complete, the lighting control unit 72 causes the first light source 32a1 of the first lighting device 32a to emit light and irradiate with the first pattern light, while the camera control unit 73 drives and controls the camera 32d to perform the first imaging process under the first pattern light. The image data generated by the imaging process is taken into the image acquisition unit 74 as needed (the same applies hereafter). As a result, three-dimensional data of the area under inspection, including multiple lands 3 and multiple solder pastes 5, is acquired.

[0117] Subsequently, the lighting control unit 72, simultaneously with the completion of the first imaging process under the first pattern light, turns off the first light source 32a1 of the first lighting device 32a and performs a switching process for the first liquid crystal shutter 32a2. Specifically, it switches the position of the first grid formed on the first liquid crystal shutter 32a2 from the reference position to a second position where the phase of the first pattern light is shifted by a quarter pitch (90°).

[0118] Once the switching settings for the first grid are complete, the lighting control unit 72 causes the light source 32a1 of the first lighting device 32a to emit light, illuminating with the first pattern light, and the camera control unit 73 drives and controls the camera 32d to perform a second imaging process under the first pattern light. Thereafter, the same process is repeated to acquire four different sets of three-dimensional data under the first pattern light with a phase difference of 90°.

[0119] Next, the lighting control unit 72 causes the second light source 32b1 of the second lighting device 32b to emit light and irradiate with the second pattern light, while the camera control unit 73 drives and controls the camera 32d to perform the first imaging process under the second pattern light.

[0120] Subsequently, the lighting control unit 72, simultaneously with the completion of the first imaging process under the second pattern light, turns off the second light source 32b1 of the second illumination device 32b and performs a switching process for the second liquid crystal shutter 32b2. Specifically, it switches the position of the second grid formed on the second liquid crystal shutter 32b2 from the reference position to a second position where the phase of the second pattern light is shifted by a quarter pitch (90°).

[0121] Once the switching settings for the second grid are complete, the lighting control unit 72 causes the light source 32b1 of the second lighting device 32b to emit light, illuminating with the second pattern light, and the camera control unit 73 drives and controls the camera 32d to perform a second imaging process under the second pattern light. Thereafter, the same process is repeated to acquire four types of three-dimensional data under the second pattern light with a phase difference of 90°.

[0122] Next, the process for acquiring two-dimensional data will be described. In this process, based on a command from the main control unit 71, the illumination control unit 72 causes the third illumination device 32c to emit light, irradiating a predetermined area to be inspected with uniform light, while the camera control unit 73 drives and controls the camera 32d to perform imaging under the uniform light. As a result, a predetermined area to be inspected on the printed circuit board 1 is imaged, and two-dimensional data relating to the area to be inspected is acquired.

[0123] The acquired raw image data Ik (three-dimensional data and two-dimensional data) for inspection is stored in the storage unit 57.

[0124] Next, in step S302, the inspection image data acquisition process is executed. In the inspection image data acquisition process, inspection image data K1 to K8 (see Figures 27 to 34) are acquired based on the original inspection image data Ik obtained in the image data acquisition process. The method for acquiring inspection image data K1 to K8 is the same as the method for acquiring training data G1 to G8 described above.

[0125] Specifically, first, solder region images Hk1 and Hk2 (see Figures 23 and 24) are extracted from the acquired inspection source image data Ik. The extraction of solder region images Hk1 and Hk2 is performed in the same way as the extraction of solder region images Hg1 and Hg2 from the training source image data Ig. Note that solder region image Hk1 relates to rectangular cream solder 5 printed on rectangular land 3. Solder region image Hk2 relates to circular cream solder 5 printed on circular land 3. Figures 23 and others show some examples of solder region images Hk1 and Hk2.

[0126] Next, using the same method as for obtaining the divided solder images Bg1 to Bg8 in the learning process, divided solder images Bk1, Bk2, Bk3, Bk4, Bk5, Bk6, Bk7, and Bk8 (see Figures 25 and 26; hereinafter referred to as "divided solder images Bk1 to Bk8") are obtained from the solder region images Hk1 and Hk2. That is, the solder region image Hk1 is divided into four parts by a virtual cross line Xka that can be divided into four parts of the same shape, thereby obtaining four divided solder images Bk1, Bk2, Bk3, and Bk4 (see Figure 25; hereinafter referred to as "divided solder images Bk1 to Bk4") from the solder region image Hk1. Furthermore, by dividing the solder region image Hk2 into four sections using a virtual cross line Xkb that can divide it into four sections of the same shape, four divided solder images Bk5, Bk6, Bk7, and Bk8 (see Figure 26; hereinafter referred to as "divided solder images Bk5-Bk8") are obtained from the solder region image Hk2.

[0127] The virtual cross line Xka consists of a first center line Cka1 connecting the centers of two parallel sides in the circumscribed rectangle Ska, and a second center line Cka2 perpendicular to the first center line Cka1 and connecting the centers of two other parallel sides in the circumscribed rectangle Ska. Similarly, the virtual cross line Xkb consists of a first center line Ckb1 connecting the centers of two parallel sides in the circumscribed rectangle Skb, and a second center line Ckb2 perpendicular to the first center line Ckb1 and connecting the centers of two other parallel sides in the circumscribed rectangle Skb. The circumscribed rectangle Ska is the rectangle with the smallest area among the rectangles circumscribed around the solder region image Hk1. The circumscribed rectangle Skb is the rectangle with the smallest area among the rectangles circumscribed around the solder region image Hk2.

[0128] Next, image processing (rotation and inversion) is performed on the images from the divided solder images Bk1 to Bk8 that require it (in this embodiment, divided solder images Bk1, Bk2, Bk3, Bk5, Bk6, Bk7), so that the two sides corresponding to the virtual cross lines Xka and Xkb in the divided solder images Bk1 to Bk8 and the corner formed by these two sides are set to a predetermined orientation (see the lower figures in Figures 25 and 26). In this embodiment, the divided solder images Bk1 to Bk8 are set in the same way as the divided solder images Bg1 to Bg8 in the training data G1 to G8, so that the sides corresponding to the first center lines Cka1 and Ckb1 point downwards, the sides corresponding to the second center lines Cka2 and Ckb2 point lefts, and the corner formed by these two sides points left downwards.

[0129] Next, by attaching the divided solder images Bk1 to Bk8 to the image frame W1, inspection image data K1 to K8 are obtained, which consists of the divided solder images Bk1 to Bk8 on the image frame W1 (see Figures 27 to 34). Since the image frame W1 of the inspection image data K1 to K8 is the same size as the image frame W1 of the training data G1 to G8, the size (width and height) of the inspection image data K1 to K8 is the same as the size of the training data G1 to G1.

[0130] Then, by repeatedly performing the above processes, such as extracting solder region images Hk1 and Hk2, acquiring divided solder images Bk1 to Bk8, performing image processing on the divided solder images Bk1 to Bk8 as needed, and pasting the divided solder images Bk1 to Bk8 onto the image frame W1, multiple inspection image data K1 to K8 are obtained from one inspection source image data Ik. At this time, four inspection image data K1, K2, K3, and K4 (hereinafter referred to as "inspection image data K1 to K4") are obtained for each rectangular land 3 (cream solder 5) contained in the inspection source image data Ik. In addition, four inspection image data K5, K6, K7, and K8 (hereinafter referred to as "inspection image data K5 to K8") are obtained for each circular land 3 (cream solder 5) contained in the learning source image data Ig. In this embodiment, inspection image data K1 to K8 includes both those acquired based on two-dimensional data and those acquired based on three-dimensional data. In this embodiment, the inspection unit 78 that acquires inspection image data K1 to K8 constitutes the "inspection image data acquisition means".

[0131] In the following step S303, the reconstructed image data acquisition process is performed. Specifically, based on a command from the main control unit 71, the inspection unit 78 inputs the inspection image data K1 to K8 acquired in step S302 into the input layer of the AI ​​model 101 corresponding to the type of inspection image data K1 to K8. Therefore, inspection image data K1 to K8 acquired based on two-dimensional data is input into the AI ​​model 101 corresponding to two-dimensional data, and inspection image data K1 to K8 acquired based on three-dimensional data is input into the AI ​​model 101 corresponding to three-dimensional data. The image data reconstructed by the AI ​​model 101 and output from the output layer is then acquired as the reconstructed image data. The acquired reconstructed image data is stored in association with the inspection image data K1 to K8 from which the reconstructed image data was derived.

[0132] Here, if the AI ​​model 101 receives, for example, inspection image data K1 (see Figure 27) relating to solder paste 5 with a shape abnormality, it will output, as reconstructed image data S, image data relating to good solder paste 5 with the shape abnormality corrected (see, for example, Figure 35). Also, if the AI ​​model 101 receives, inspection image data K5 (see Figure 31) relating to solder paste 5 with foreign matter F1 attached, it will output, as reconstructed image data S, image data relating to good solder paste 5 with the foreign matter F1 removed (see, for example, Figure 36).

[0133] On the other hand, if the AI ​​model 101 receives, for example, inspection image data K2 and K6 (see Figures 28 and 32) relating to good solder paste 5, it outputs reconstructed image data S relating to good solder paste 5 that is almost identical to the inspection image data K2 and K6. The size (width and height) of the reconstructed image data S is the same as the size of the original inspection image data K1 to K8. In this embodiment, the inspection unit 78 that acquires the reconstructed image data S constitutes the "reconstructed image data acquisition means".

[0134] In step S304, a pass / fail judgment process is performed based on the acquired reconstructed image data S. In the pass / fail judgment process, based on a command from the main control unit 71, the inspection unit 78 compares the entirety of the inspection image data K1 to K8 acquired in step S302 with the entirety of the reconstructed image data S acquired in step S303 using the inspection image data K1 to K8, and calculates the difference between the two image data K1 to K8, S. For example, by comparing dots (pixels) at the same coordinates in both image data K1 to K8, S, the area (number of dots) of clusters of dots where the difference in brightness is greater than or equal to a predetermined value is calculated. Note that the difference between the inspection image data K1, K5 in which abnormalities exist and the reconstructed image data S composed of the inspection image data K1, K5 will be relatively large. In this embodiment, the inspection unit 78 that compares the inspection image data K1 to K8 and the reconstructed image data S constitutes the "comparison means".

[0135] Next, the inspection unit 78 determines whether the calculated difference is smaller than a predetermined threshold. If the calculated difference is smaller than the predetermined threshold, the inspection unit 78 determines it to be a "good product," while if the difference is larger than the predetermined threshold, it determines it to be a "defective product."

[0136] Furthermore, the inspection unit 78 performs the above determination on all inspection image data K1 to K8 related to the area to be inspected on the printed circuit board 1. If all inspection image data K1 to K8 are determined to be "good," the inspection unit 78 determines that the area to be inspected is "good" and stores this result in the storage unit 57. On the other hand, if, as a result of performing the above determination on all inspection image data K1 to K8 related to the area to be inspected, at least one inspection image data K1 to K8 is determined to be "defective," the inspection unit 78 determines that the area to be inspected is "defective" and stores this result in the storage unit 57.

[0137] Then, if the solder inspection device 13 determines that all areas to be inspected on the printed circuit board 1 are "good" as a result of the inspection process, it determines that the printed circuit board 1 is free of defects in the solder paste 5 (pass judgment) and stores this result in the storage unit 57.

[0138] On the other hand, if the solder inspection device 13 finds even one area under inspection that is determined to be a "defective product," it determines that the printed circuit board 1 has an abnormality in the solder paste 5 (failure judgment), stores this result in the storage unit 57, and notifies the outside of this fact via the display unit 56, communication unit 58, etc.

[0139] As detailed above, according to this embodiment, the inspection image data K1 to K8 are formed by providing divided solder images Bk1 to Bk8 in the image frame W1. Therefore, the size (width and height) of the inspection image data K1 to K8 will match the size of the image frame W1 and will remain constant without varying slightly depending on the size of the land 3. This eliminates the need to prepare many different AI models 101 for each size of the land 3, thereby reducing the effort and time required to obtain the AI ​​model 101. Furthermore, the AI ​​model 101 can be used in common even when the size of the land 3 is different.

[0140] Furthermore, the image frame W1 of the training data G1-G8 and the image frame W1 of the inspection image data K1-K8 are the same size, and the sizes of the training data G1-G8 and the inspection image data K1-K8 are identical. Therefore, when the inspection image data K1-K8 is input to the AI ​​model 101, the appropriate reconstructed image data S corresponding to the inspection image data K1-K8 can be output more reliably, and consequently, the quality of the solder paste 5 can be determined more accurately. This makes it possible to more reliably obtain good inspection accuracy.

[0141] In addition, the training data G1 to G8 are obtained by dividing the solder region images Hg1 and Hg2 into four parts, and then placing each of these divided solder images Bg1 to Bg8 within the image frame W1. Therefore, compared to the case where image data relating to the entire solder paste 5 is used as the training data G1 to G8, an AI model 101 with the desired accuracy can be obtained with less training data. Furthermore, since four training data sets G1 to G4 and G5 to G8 can be obtained from the solder region images Hg1 and Hg2 respectively, the required number of training data sets G1 to G8 can be easily obtained. These factors together effectively reduce the effort and time required to obtain the AI ​​model 101.

[0142] In addition, since the training data G1-G8 and inspection image data K1-K8 are created by placing divided solder images Bg1-Bg8 and Bk1-Bk8 in image frame W1, the size of both data G1-G8 and K1-K8 can be made relatively small compared to a system where the entire image of the solder paste 5 is placed in image frame W1. Therefore, the processing burden related to training and inspection can be reduced, and training and inspection can be expedited.

[0143] Furthermore, the segmented solder images Bg1-Bg8 and Bk1-Bk8 in the training data G1-G8 and inspection image data K1-K8 are set so that two sides corresponding to the virtual cross lines Xga, Xgb, Xka, and Xkb, and the corner formed by those two sides, are in a predetermined orientation. Therefore, the learning efficiency is improved, and the effort and time required to obtain the AI ​​model 101 can be further reduced. In addition, the accuracy of the reconstructed image data S output from the AI ​​model 101 is improved, making it possible to further enhance inspection accuracy.

[0144] Furthermore, in all training data G1-G8 and inspection image data K1-K8, the two sides corresponding to the virtual cross lines Xga, Xgb, Xka, and Xkb, and the corner formed by these two sides, are common, and therefore these no longer constitute feature parts. Consequently, the accuracy of the reconstructed image data S reconstructed by the AI ​​model 101 can be further improved, and the comparison (pass / fail judgment process) by the inspection unit 78 can be performed more accurately. As a result, even better inspection accuracy can be obtained.

[0145] In addition, the rectangular images Sga, Sgb, Ska, and Ska are defined as rectangles that are circumscribing the solder region images Hg1, Hg2, Hk1, and Hk2 while minimizing their area. This allows the solder region images Hg1, Hg2, Hk1, and Hk2 to be appropriately divided into four parts, regardless of whether they are rectangular or circular. [Second Embodiment] Next, the second embodiment will be described, focusing on the differences from the first embodiment. In the first embodiment, the solder region images Hg1 and Hg2 are each divided into four parts in order to obtain the training data G1 to G8. In contrast, in this second embodiment, the solder region images Hg1 and Hg2 are each divided into two parts in order to obtain the training data G1 to G4 (see Figures 39 to 42).

[0146] To explain in more detail how to acquire the training data G1 to G4, first, as in the first embodiment described above, solder region images Hg1 and Hg2 are extracted from the training source image data Ig. Next, divided solder images Bg1 to Bg4 (see Figures 37 and 38) are acquired from the solder region images Hg1 and Hg2. Divided solder images Bg1 and Bg2 are images obtained by dividing the single solder region image Hg1 into two by a virtual line Lga (see Figure 37) that can divide it into two identical shapes. Divided solder images Bg3 and Bg4 are images obtained by dividing the single solder region image Hg2 into two by a virtual line Lgb (see Figure 38) that can divide it into two identical shapes.

[0147] The virtual line Lga consists of a center line Cga connecting the centers of two parallel sides in the circumscribed rectangle Sga. The virtual line Lgb consists of a center line Cgb connecting the centers of two parallel sides in the circumscribed rectangle Sgb. As described above, the virtual lines Lga and Lgb are capable of dividing the solder area images Hg1 and Hg2 into two identical shapes. However, "capable of being divided into two identical shapes" means, more precisely, that when the solder paste 5 is printed in its ideal shape (i.e., the design shape), the solder area image related to the solder paste 5 can be divided into two identical shapes (the same applies to the virtual lines Lka and Lkb described later).

[0148] Next, image processing is performed on the images Bg2 and Bg4 among the divided solder images Bg1 to Bg4 that require image processing (rotation or inversion), so that the edges corresponding to the virtual lines Lga and Lgb in the divided solder images Bg1 to Bg4 are set to a predetermined orientation (see the lower figures in Figures 37 and 38). For example, in the divided solder images Bg1 to Bg4, the edges corresponding to the center lines Cga and Cgb are set to point downwards.

[0149] Then, by attaching the divided solder images Bg1 to Bg4 to the image frame W1, training data G1 to G4 (see Figures 39 to 42) is obtained, which consists of the divided solder images Bg1 to Bg4 placed on the image frame W1. The training of the AI ​​model 101 using the training data G1 to G4 is carried out in the same manner as in the first embodiment described above.

[0150] Furthermore, in order to acquire training data G1 to G4 based on the two divided solder region images Hg1 and Hg2 as described above, in this second embodiment, inspection image data K1 to K4 (see Figures 45 to 48) are acquired based on the two divided solder region images Hk1 and Hk2.

[0151] To explain in more detail how to acquire inspection image data K1 to K4, first, as in the first embodiment described above, solder region images Hk1 and Hk2 are extracted from the original inspection image data Ik. Next, divided solder images Bk1 to Bk4 (see Figures 43 and 44) ​​are obtained from the solder region images Hk1 and Hk2. Divided solder images Bk1 and Bk2 are images obtained by dividing one solder region image Hk1 into two by a virtual line Lka (see Figure 43) that can divide it into two of the same shape. Divided solder images Bk3 and Bk4 are images obtained by dividing one solder region image Hk2 into two by a virtual line Lkb (see Figure 44) that can divide it into two of the same shape.

[0152] The virtual line Lka consists of the center line Cka, which connects the centers of two parallel sides in the circumscribed rectangle Ska. The virtual line Lkb consists of the center line Ckb, which connects the centers of two parallel sides in the circumscribed rectangle Skb.

[0153] Next, image processing is performed on the images from the divided solder images Bk1 to Bk4 that require image processing (rotation or inversion) (in this second embodiment, divided solder images Bk2 and Bk4), so that the edges corresponding to the virtual lines Lka and Lkb in the divided solder images Bk1 to Bk4 are set to a predetermined orientation (see the lower figures in Figures 43 and 44). For example, in the divided solder images Bk1 to Bk4, the edges corresponding to the center lines Cka and Ckb are set to point downwards.

[0154] Then, by attaching the divided solder images Bk1 to Bk4 to the image frame W1, inspection image data K1 to K4 (see Figures 45 to 48) are obtained, which consist of the divided solder images Bk1 to Bk4 placed on the image frame W1. The inspection of the solder paste 5 using the inspection image data K1 to K4 is performed in the same manner as in the first embodiment described above.

[0155] As described above, according to this second embodiment, the size (width and height) of the inspection image data K1 to K4 will match the size of the image frame W1 and will remain constant without fluctuating significantly depending on the size of the land 3. This eliminates the need to prepare numerous different AI models 101 for each land 3 size, thereby reducing the effort and time required to obtain the AI ​​model 101. Furthermore, the AI ​​model 101 can be used in common even when the land 3 sizes are different.

[0156] Furthermore, since the sizes of the training data G1-G4 and the inspection image data K1-K4 are the same, when the inspection image data K1-K4 is input to the AI ​​model 101, it is possible to more reliably output appropriate reconstructed image data S corresponding to the inspection image data K1-K4. As a result, the quality of the solder paste 5 can be determined more accurately, and good inspection accuracy can be more reliably obtained.

[0157] Furthermore, the training data G1 to G4 are obtained by dividing the solder region images Hg1 and Hg2 into two parts, and then placing each of these divided solder images Bg1 to Bg4 within the image frame W1. Therefore, compared to the case where image data relating to the entire solder paste 5 is used as training data, an AI model 101 with the desired accuracy can be obtained with less training data G1 to G4. Moreover, since two training data can be obtained from each of the solder region images Hg1 and Hg2, the required number of training data G1 to G4 can be easily obtained. These factors together effectively reduce the effort and time required to obtain the AI ​​model 101.

[0158] In addition, since the training data G1-G4 and inspection image data K1-K4 are created by placing segmented solder images Bg1-Bg4 and Bk1-Bk4 in an image frame W1, the size of both data G1-G4 and K1-K4 can be made relatively small compared to a system where the entire image of the solder paste 5 is placed in an image frame. Therefore, the processing burden related to training and inspection can be reduced, and training and inspection can be expedited.

[0159] Furthermore, the segmented solder images Bg1-Bg4 and Bk1-Bk4 in the training data G1-G4 and inspection image data K1-K4 are set so that the edges corresponding to the virtual lines Lga, Lgb, Lka, and Lkb are in a predetermined orientation. Therefore, learning efficiency is improved, and the effort and time required to obtain the AI ​​model 101 can be further reduced. In addition, the accuracy of the reconstructed image data output from the AI ​​model 101 is improved, making it possible to further enhance inspection accuracy.

[0160] Furthermore, since the edges corresponding to the virtual lines Lga, Lgb, Lka, and Lkb are common to all training data G1-K4 and inspection image data K1-K4, these edges cease to be feature areas. Consequently, the accuracy of the reconstructed image data reconstructed by the AI ​​model 101 can be further improved, and the comparison (pass / fail judgment process) by the inspection unit 78 can be performed more accurately. As a result, even better inspection accuracy can be obtained.

[0161] (a) In the first embodiment described above, the virtual cross line Xka is defined as relating to a circumscribed rectangle Ska that is circumscribed with the solder region image Hk1 and has the smallest area. In contrast, as shown in Figure 49, the virtual cross line Xkc may be defined as relating to a circumscribed rectangle Skc that is circumscribed with the solder region image Hk1 and extends in a direction that coincides with the direction in which one rectangular land 3 corresponding to the solder region image Hk1 extends. In this case, the virtual cross line Xkc consists of a first center line Ckc1 connecting the centers of two parallel sides in the circumscribed rectangle Skc, and a second center line Ckc2 connecting the centers of two other parallel sides in the circumscribed rectangle Skc. In other words, "a circumscribed rectangle Skc that extends in a direction that coincides with the direction in which one land 3 extends" can be defined as a circumscribed rectangle Skc having two sides parallel to the two opposing sides that constitute the outer edge of one land 3. The "direction in which the land extends" can be determined, for example, from design data or manufacturing data (such as Gerber data).

[0162] Furthermore, in the second embodiment described above, the virtual line Lka is defined as relating to the circumscribed rectangle Ska that is circumscribed with the solder region image Hk1 and has the smallest area. In contrast, as shown in Figure 50, the virtual line Lkc may be defined as relating to the circumscribed rectangle Skc that is circumscribed with the solder region image Hk1 and extends in a direction that coincides with the direction in which the 1 rectangular land 3 corresponding to the solder region image Hk1 extends. In this case, the virtual line Lkc consists of a center line Ckc that connects the centers of two parallel sides in the circumscribed rectangle Skc.

[0163] By adopting the circumscribed rectangle Skc described above, the process of defining the circumscribed rectangle Skc becomes relatively easy. This further reduces the processing burden related to learning and testing, and enables more effective acceleration of learning and testing.

[0164] Furthermore, as shown in Figure 51, the circumscribing rectangle Skd may be one whose shape coincides with the outer edge of the rectangular land 3 corresponding to the solder area image Hk1. In this case, the virtual cross line Xkd consists of a first center line Ckd1 connecting the centers of two parallel sides in the circumscribing rectangle Skd, and a second center line Ckd2 connecting the centers of two other parallel sides in the circumscribing rectangle Skd. The "outer edge of the land 3" can be identified, for example, from design data or manufacturing data (e.g., Gerber data). Alternatively, a virtual line for dividing the solder area image into two may be set based on the circumscribing rectangle Skd described above.

[0165] Furthermore, the same circumscribing rectangles Skc and Skd described above may be used as the circumscribing rectangle for obtaining training data.

[0166] (b) In the above embodiment, the AI ​​model 101 is trained by combining the training data G1 to G4 related to rectangular solder paste 5 printed on rectangular lands 3 and the training data G5 to G8 related to circular solder paste 5 printed on circular lands 3. That is, the AI ​​model 101 corresponds to the rectangular solder paste 5 and the circular solder paste 5, respectively. Alternatively, an AI model corresponding to the rectangular solder paste 5, which is trained on the training data G1 to G4, and an AI model corresponding to the circular solder paste 5, which is trained on the training data G5 to G8, may be provided separately. In this case, the learning efficiency and inspection accuracy can be further improved.

[0167] (c) In the above embodiment, separate AI models 101 are provided for two-dimensional data and for three-dimensional data. However, a common AI model may be provided for each of the two-dimensional and three-dimensional data.

[0168] (d) The configuration of the AI ​​model 101 (neural network 90) and its learning method are not limited to the above embodiment. For example, when performing the learning process of the neural network 90 or the reconstruction image data acquisition process, the configuration may include normalization or other processing on various data as needed. Also, the structure of the neural network 90 is not limited to that shown in Figure 6, and for example, a pooling layer may be provided after the convolutional layer 93. Of course, the number of layers in the neural network 90, the number of nodes in each layer, and the connection structure of each node may also be different configurations.

[0169] Furthermore, in the above embodiment, the AI ​​model 101 (neural network 90) is a generative model having the structure of a convolutional autoencoder (CAE), but it is not limited to this, and may be a generative model having the structure of a different type of autoencoder, such as a variational autoencoder (VAE).

[0170] Furthermore, in the above embodiment, the neural network 90 is trained using the backpropagation method, but the system is not limited to this, and various other learning algorithms may be used for training.

[0171] In addition, the neural network 90 may be composed of a dedicated AI processing circuit, such as a so-called AI chip. In that case, only learning information such as parameters may be stored in the memory unit 57, and the dedicated AI processing circuit may read this information and set it in the neural network 90 to construct the AI ​​model 101.

[0172] In addition, in the above embodiment, the control device 33 is equipped with a learning unit 77, and the neural network 90 is trained within the control device 33, but the configuration is not limited to this. For example, the learning unit 77 may be omitted, and the neural network 90 may be trained outside the control device 33, and the AI ​​model 101 (trained neural network 90) trained externally may be stored in the storage unit 57.

[0173] (e) In the above embodiment, two-dimensional data and three-dimensional data are acquired as the raw image data Ik for inspection, but it is also possible to configure the system to acquire only one of the two-dimensional data or the three-dimensional data. Furthermore, depending on the data to be acquired, only one of the two-dimensional data or the three-dimensional data may be provided as the AI ​​model 101.

[0174] (f) In the above embodiment, the image frame W1 is rectangular (square) in shape, satisfying m=n, but it may also be rectangular in shape, satisfying m≠n. [Explanation of Symbols]

[0175] 1…Printed circuit board, 3…Land, 5…Solder paste, 13…Solder inspection device, 32d…Camera (image data acquisition means), 78…Inspection unit (inspection image data acquisition means, reconstructed image data acquisition means, comparison means), 101…AI model (identification means), Bg1~Bg8,Bk1~Bk8…Divided solder images, Cga,Cgb,Cka,Ckb,Ckc…Center line, Cga1,Cgb1,Cka1,Ckb1,Ckc1,Ckd1…Number One centerline, Cga2, Cgb2, Cka2, Ckb2, Ckc2, Ckd2... Second centerline, G1~G8... Training data, Hg1, Hg2, Hk1, Hk2... Solder region image, K1~K8... Inspection image data, Lga, Lgb, Lka, Lkb, Lkc... Virtual lines, Sga, Sgb, Ska, Skb, Skc, Skd... Bounded rectangle, W1... Image frame, Xga, Xgb, Xka, Xkb, Xkc, Xkd... Virtual cross lines.

Claims

1. A solder inspection device for inspecting solder paste printed on a printed circuit board, Image data acquisition means capable of acquiring image data of a predetermined area to be inspected on the printed circuit board including printed solder paste, An identification means generated by training a neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, using only image data related to good quality solder paste as training data, An inspection image data acquisition means that acquires inspection image data, including an image of the solder paste to be inspected, based on the image data acquired by the aforementioned image data acquisition means, A reconstructed image data acquisition means capable of inputting the aforementioned inspection image data into the identification means and acquiring the reconstructed image data as reconstructed image data, The system includes a comparison means capable of comparing the aforementioned inspection image data and the aforementioned reconstructed image data, Based on the comparison results obtained by the aforementioned comparison means, the system is configured to determine the quality of the solder paste. The aforementioned training data is obtained by dividing a solder region image representing solder paste corresponding to a land into four sections using virtual cross lines that allow for four divisions of the same shape, and each of these four divided solder images is placed in an image frame larger than the size of the divided solder image. The solder inspection apparatus is characterized in that the inspection image data acquisition means acquires inspection image data in which each of the four divided solder images, obtained by dividing the solder region image in the image data acquired by the image data acquisition means into four parts by the virtual cross lines, is placed in an image frame the same size as the image frame of the learning data.

2. The solder inspection apparatus according to claim 1, characterized in that the divided solder images in the learning data and the inspection image data are set so that the two sides corresponding to the virtual cross line and the corner formed by the two sides are in a predetermined orientation.

3. The solder inspection apparatus according to claim 1, characterized in that the virtual cross line consists of a first center line connecting the centers of two parallel sides in a circumscribing rectangle that circumscribes the solder region image and has the smallest area, and a second center line connecting the centers of two other parallel sides in the circumscribing rectangle.

4. The outer edge of the land corresponding to the solder region image is rectangular in plan view. The solder inspection apparatus according to claim 1, characterized in that the virtual cross line comprises a first center line connecting the centers of two parallel sides in a circumscribing rectangle that circumscribing the solder region image and extending in a direction coinciding with the direction in which the land 1 extends, and a second center line connecting the centers of two other parallel sides in the circumscribing rectangle.

5. A solder inspection device for inspecting solder paste printed on a printed circuit board, Image data acquisition means capable of acquiring image data of a predetermined area to be inspected on the printed circuit board including printed solder paste, An identification means generated by training a neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, using only image data related to good quality solder paste as training data, An inspection image data acquisition means that acquires inspection image data, including an image of the solder paste to be inspected, based on the image data acquired by the aforementioned image data acquisition means, A reconstructed image data acquisition means capable of inputting the aforementioned inspection image data into the identification means and acquiring the reconstructed image data as reconstructed image data, The system includes a comparison means capable of comparing the aforementioned inspection image data and the aforementioned reconstructed image data, Based on the comparison results obtained by the aforementioned comparison means, the system is configured to determine the quality of the solder paste. The aforementioned training data is obtained by dividing a solder region image representing solder paste corresponding to a land into two parts by a virtual line that can divide it into two parts of the same shape, and each of these two divided solder images is placed in an image frame larger than the size of the divided solder image. The solder inspection apparatus is characterized in that the inspection image data acquisition means acquires inspection image data in which each of the two divided solder images, obtained by dividing the solder region image in the image data acquired by the image data acquisition means into two parts by the virtual line, is placed in an image frame the same size as the image frame of the learning data.

6. The solder inspection apparatus according to claim 5, characterized in that the divided solder images in the learning data and the inspection image data are set so that the sides corresponding to the virtual lines are in a predetermined orientation.

7. The solder inspection apparatus according to claim 5, characterized in that the virtual line consists of a center line connecting the centers of two parallel sides in a bounding rectangle that is tangent to the solder region image and has the smallest area.

8. The outer edge of the land corresponding to the solder region image is rectangular in plan view. The solder inspection apparatus according to claim 5, characterized in that the virtual line consists of a center line connecting the centers of two parallel sides in a circumscribing rectangle that circumscribes the solder region image and extends in a direction that coincides with the direction in which the land of 1 extends.

Images