Solder inspection device and solder inspection method

Abstract

To reduce labor and a burden for obtaining identification means as an AI model, and utilize the identification means commonly even if the size of a land differs.SOLUTION: A solder inspection device acquires reconfiguration image data by inputting inspection image data based on image data acquired by a camera 32d to AI models 101 to 103 generated by allowing a predetermined neural network to learn only image data pertaining to conforming cream solder as learning data, and compares the inspection image data and the reconfiguration image data, and hence determines whether cream solder is appropriate. The learning data are obtained by providing one solder area image showing cream solder corresponding to one land in an image frame with a size larger than the size of the first solder area image. Inspection image data are obtained by providing one solder area image extracted from image data acquired by the camera 32d in an image frame having the same size as that of the image frame of the learning data.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a solder inspection apparatus and a solder inspection method for inspecting solder provided on a substrate.

Background Art

[0002] Generally, in a substrate manufacturing line for mounting electronic components on a printed circuit board, first, cream solder is printed on the lands of the printed circuit board (solder printing process). Next, the electronic components are temporarily fixed on the printed circuit board based on the viscosity of the cream solder (mounting process). Then, such a printed circuit board is led into a reflow oven, and soldering is performed by heating and melting the cream solder (reflow process). In such a substrate manufacturing line, an inspection apparatus for inspecting the printed circuit board may be provided.

[0003] Recently, as an inspection apparatus for inspecting a printed circuit board, one using an AI model has been proposed. As an inspection apparatus using an AI model, for example, an apparatus for inspecting the presence or absence of foreign matter on a printed circuit board by comparing inspection image data (original image data) of an inspection area on the printed circuit board with reconstructed image data generated by inputting the inspection image data into an AI model (identifying means) is known (see, for example, Patent Document 1, etc.).

[0004] Also, in the inspection of a printed circuit board, for example, the inspection of cream solder may be performed by determining the shape (two-dimensional shape or three-dimensional shape) of the cream solder, the presence or absence of foreign matter adhering to the cream solder, and the like. Therefore, it is conceivable to perform the inspection of cream solder using the above-described AI model.

[0005] By the way, the configuration of the printed circuit board varies depending on the type. Here, in terms of enabling the AI model to support inspections of more types of printed circuit boards, it is effective to acquire inspection image data related to solder paste for each land and perform inspections using this inspection image data. This is because the configurations (shape, size, etc.) of the lands and the solder paste printed on the lands often do not vary extremely among various printed circuit boards.

[0006] When acquiring inspection image data related to solder paste for each land, it is preferable that the size (width and height) of the inspection image data to be acquired does not protrude from the land. For example, as shown in FIGS. 31 and 32, it is preferable to match the size of the inspection image data with the size of land 3 (for example, the size of the rectangle indicated by the thick line in FIGS. 31 and 32). In FIGS. 31 and the like, a scatter pattern is attached to the solder paste 5. By using such inspection image data, the influence of the solder paste, circuit patterns, resists, etc. existing around the land can be reduced.

[0007] Also, in terms of improving inspection accuracy, it is preferable to match the size (width and height) of the training data with the size of the inspection image data. Here, although it is possible to adjust the sizes of the training data and the inspection image data by enlarging or reducing the image data, for example, for the standard land size in the general component size "0603" with a width of 0.25 to 0.35 mm × a height of 0.30 to 0.40 mm, the diameter of the particles constituting the cream solder (for example, an average particle size of about 30 μm) is relatively large. Therefore, adjusting the sizes of both image data by enlargement or reduction may lead to a decrease in inspection accuracy. Thus, in terms of improving inspection accuracy, while matching the sizes of both image data to the land size, and preparing a plurality of AI models obtained by training different training data for each land size (that is, the sizes of both image data), at the time of inspection, according to the size of the inspection image data (that is, the land size), an appropriate AI model is selected from among the plurality of AI models, and it is conceivable to determine the quality of the cream solder using the selected AI model.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0009] However, even for lands on which the same type of component is mounted, their sizes are not constant and may be somewhat different. For example, in two printed circuit boards each having a land on which the same component is mounted, the size of the land may be somewhat different between the two printed circuit boards. Therefore, in order to prepare different AI models for each land size, complicated work has to be carried out, and there is a risk that a great deal of labor and effort will be required.

[0010] The present invention has been made in view of the above circumstances, and an object thereof is to reduce the labor and burden in obtaining discrimination means as an AI model, and to provide a solder inspection apparatus or the like that can commonly use the discrimination means even when the size of the land is different.

Means for Solving the Problems

[0011] Hereinafter, each means suitable for solving the above object will be described separately. In addition, the effects specific to the corresponding means will be appended as necessary.

[0012] Means 1. A solder inspection apparatus for inspecting cream solder printed on a printed circuit board, Image data acquisition means capable of acquiring image data of a predetermined inspection area on the printed circuit board including the printed portion of the cream solder, Discrimination means generated by learning only the image data related to good cream solder as learning data for a neural network having an encoding unit that extracts feature amounts from the input image data and a decoding unit that reconstructs the image data from the feature amounts, Inspection image data acquisition means for acquiring inspection image data including an image of the cream solder to be inspected based on the image data acquired by the image data acquisition means, Reconstructed image data acquisition means capable of acquiring, as reconstructed image data, the image data reconstructed by inputting the inspection image data to the discrimination means, Comparison means capable of comparing the inspection image data and the reconstructed image data, It is configured to be able to determine the quality of the cream solder based on the comparison result by the comparison means, The learning data is formed by providing an image of a single solder area indicating cream solder corresponding to a single land in an image frame having a size larger than the size of the image of the single solder area. The inspection image data acquisition means acquires the inspection image data of the same size as the learning data, wherein the solder region image of 1 extracted from the image data acquired by the image data acquisition means is provided in an image frame of the same size as the image frame of the learning data. A solder inspection apparatus characterized by the above.

[0013] In addition, the solder region image of 1 may be an image (image of means 2 and 3 described later) showing the entire lump portion of the cream solder at least partially located on 1 land, or an image showing only the portion existing on 1 land among the lump portions of the cream solder. The former image becomes an image of the entire cream solder including the protruding portion when the cream solder protrudes from 1 land. On the other hand, the latter image becomes an image of a part of the cream solder excluding the protruding portion when the cream solder protrudes from 1 land. Further, the solder region image of 1 only needs to correspond to 1 land, and includes not only an image composed of 1 lump portion but also an image composed of a plurality of lump portions.

[0014] In addition, the solder region image of 1 constituting the learning data may be extracted from image data (actual image data) obtained by imaging a printed circuit board on which good-quality cream solder is printed (that is, an image of actual cream solder), or may be an image of good-quality cream solder generated virtually. Examples of the actual image data include image data accumulated in previous inspections and image data of good-quality printed circuit boards visually selected by an operator after printing the cream solder.

[0015] Furthermore, the above "neural network" includes, for example, a convolutional neural network having a plurality of convolutional layers. The above "learning" includes, for example, deep learning (deep learning). The above "discrimination means (generation model)" includes, for example, an autoencoder (autoencoder) and a convolutional autoencoder (convolutional autoencoder).

[0016] In addition, the "identification means" is generated by learning only the image data related to the good cream solder (learning data obtained by providing an image frame with a solder area image of 1 related to the good cream solder). Therefore, when the inspection image data related to the defective cream solder is input into the identification means, the reconstructed image data generated is almost identical to the inspection image data in which the defective part has been corrected (for example, foreign matter has been removed, or the shape and size have been made correct). That is, when there is a defective part in the cream solder, virtual image data related to the cream solder assuming no defective part is generated as the reconstructed image data related to the cream solder.

[0017] According to the above means 1, the inspection image data is obtained by providing an image frame with a solder area image of 1 extracted from the image data acquired by the image data acquisition means. Therefore, the size (width and height) of the inspection image data does not vary finely depending on the size of the land and is constant. This eliminates the need to prepare a large number of different identification means for each land size, reducing the labor and effort required to obtain the identification means. Also, the identification means can be commonly used even when the land sizes are different.

[0018] Furthermore, according to the above means 1, the image frame of the learning data and the image frame of the inspection image data are the same size, and the sizes of the learning data and the inspection image data are the same. Therefore, when the inspection image data is input into the identification means, the appropriate reconstructed image data corresponding to the inspection image data can be more reliably output, and thus the quality determination of the cream solder can be performed more accurately. As a result, good inspection accuracy can be obtained more reliably.

[0019] In addition, according to the above means 1, the inspection image data is compared with the reconstructed image data that is input to the identification means and reconstructed, and based on the comparison result, the quality of the cream solder is determined. Therefore, both pieces of image data to be compared are related to the same cream solder. Accordingly, unlike the method of determining quality by comparison with a separately prepared standard, there is no need to set relatively loose inspection conditions to prevent false detection, and stricter inspection conditions can be set. Furthermore, in both pieces of image data to be compared, the imaging conditions of the printed circuit board to be inspected (for example, the placement position, placement angle, deflection, etc. of the printed circuit board) and the imaging conditions on the inspection device side (for example, the lighting state, the angle of view of the camera, etc.) can be made to match. These factors combined enable the quality determination of the cream solder to be performed more accurately.

[0020] Means 2. A solder region image extraction means for extracting the solder region image of 1 that constitutes the inspection image data from the image data acquired by the image data acquisition means. The solder region image extraction means is capable of specifying the region occupied by the cream solder in the image data acquired by the image data acquisition means, and extracting the image of the connected component in the specified region as the solder region image of 1 that constitutes the inspection image data. The solder inspection apparatus according to means 1, characterized by the above.

[0021] According to the above-mentioned means 2, the area occupied by the cream solder in the image data acquired by the image data acquisition means is specified by the solder area image extraction means, and the image of the connected component in the specified area is extracted as one solder area image constituting the inspection image data. Therefore, one solder area image includes not only the image of the portion located on the land in the cream solder but also the image of the portion protruding from the land in the cream solder. That is, as shown in FIGS. 33 and 34, even if the cream solder 5 protrudes from the land 3, as shown in FIGS. 35 and 36, the one solder area image Ih includes this protruding portion. Thereby, it becomes possible to appropriately perform the pass / fail determination regarding the cream solder partially protruding from the land, and the inspection accuracy can be further enhanced.

[0022] Means 3. The solder area image extraction means is characterized in that, among the connected components, an image of the whole of which at least a part overlaps with one land on the design data or the manufacturing data can be extracted as the one solder area image constituting the inspection image data. The solder inspection apparatus according to means 2.

[0023] According to the above-mentioned means 3, when the cream solder printed on one land is separated into a plurality, the one solder area image includes the images of the whole of the cream solder separated into the plurality. Therefore, for example, as shown in FIG. 37, when the cream solder 5 printed on one land 3 is separated into two, as shown in FIG. 38, the one solder area image Ih is an image including the whole of the cream solder 5 separated into two. Thereby, even if the cream solder printed on one land is in a state of being separated into a plurality, the pass / fail determination regarding the cream solder can be performed more appropriately.

[0024] Means 4. Second discrimination means generated by learning only the image data related to the good cream solder as second learning data for a neural network having an encoding unit that extracts a feature amount from the input image data and a decoding unit that reconstructs the image data from the feature amount is provided. The second learning data is obtained by providing the solder region image of 1 in a second image frame having a size larger than that of the solder region image of 1 and smaller than that of the image frame of the learning data. When the size of the solder region image of 1 extracted from the image data acquired by the image data acquisition means is smaller than the size of the second image frame, the inspection image data acquisition means acquires the inspection image data of the same size as the second learning data, which is obtained by providing the solder region image of 1 in the second image frame. The reconstructed image data acquisition means acquires the reconstructed image data reconstructed by inputting the inspection image data into the second identification means, and the comparison means is configured to compare the inspection image data and the reconstructed image data. The solder inspection apparatus according to means 1, characterized in that.

[0025] According to the above means 4, when the size of the solder region image of 1 is relatively small, the inspection image data acquisition means acquires the inspection image data of a relatively small size, in which the solder region image of 1 is provided in a second image frame of a relatively small size. Then, the reconstructed image data acquisition means inputs this relatively small inspection image data into the second identification means to output the reconstructed image data, and the comparison means compares the relatively small inspection image data and the reconstructed image data, respectively. Therefore, compared with the case where the image frame of the inspection image data is always set to a constant size, the processing for acquiring the reconstructed image data and the comparison processing by the comparison means can be speeded up, and thus the inspection speed can be further improved.

[0026] Means 5. A circular correspondence identification means is provided, which is generated by training only the image data related to the good cream solder printed on the circular land as circular correspondence learning data for a neural network having an encoding unit that extracts feature amounts from the input image data and a decoding unit that reconstructs the image data from the feature amounts. The circular-corresponding learning data is formed by providing the solder region image of the good cream solder printed on the circular land in a circular-corresponding image frame having a size larger than the size of the solder region image of 1. When the solder region image of 1 extracted from the image data acquired by the image data acquisition means corresponds to the circular land, the inspection image data acquisition means acquires the inspection image data of the same size as the circular-corresponding learning data, which is formed by providing the solder region image of 1 in the circular-corresponding image frame. The reconstructed image data acquisition means acquires the reconstructed image data reconstructed by inputting the inspection image data to the circular-corresponding identification means. The comparison means is configured to compare the inspection image data and the reconstructed image data. The solder inspection apparatus according to means 1 is characterized in that.

[0027] Regarding the number of learning data used for the learning of the identification means, usually, the number of data related to the cream solder printed on the rectangular land is overwhelmingly larger than that related to the cream solder printed on the circular land. Therefore, when the inspection image data K1 related to the circular cream solder 5 printed on the circular land (see, for example, FIG. 39) is input to the identification means, there is a possibility that an image (see, for example, FIG. 41) showing the rectangular cream solder 5 is output as the reconstructed image data S1, similar to when the inspection image data K2 related to the cream solder 5 with rounded corners in the rectangular shape is input. When such reconstructed image data is output, there is a possibility that the good cream solder is erroneously determined as a defective product.

[0028] In this regard, according to the above means 5, there is provided a circular-corresponding identification means dedicated to inspecting the cream solder printed on the circular land, which is generated by learning only the image data related to the good cream solder printed on the circular land. Therefore, it is possible to accurately perform the pass / fail determination regarding the circular cream solder printed on the circular land. As a result, it is possible to further improve the inspection accuracy.

[0029] Means 6. The learning data and the inspection image data are set such that the center or centroid of the solder region image of item 1 coincides with the center of the image frame, and the long side or short side of the solder region image of item 1 extends along a predetermined direction. The solder inspection apparatus according to Means 1, characterized in that.

[0030] Incidentally, the technical matters related to the above Means 6 may be applied to the above Means 4 and 5. That is, the second learning data and the learning data for circular correspondence may be set such that the center or centroid of the solder region image of item 1 coincides with the center of the second image frame or the image frame for circular correspondence, and the long side or short side of the solder region image of item 1 extends along a predetermined direction. Of course, the inspection image data related to the above Means 4 and 5 may be set in the same manner.

[0031] According to the above Means 6, the orientation and position of the cream solder in the learning data and the inspection image data can be generally aligned. Therefore, even if the learning data used for generating the discrimination means is relatively small, it is possible to accurately determine the quality of the cream solder. That is, it is possible to obtain good inspection accuracy while more effectively reducing the labor and effort required to obtain the discrimination means.

[0032] Means 7. The comparison means is configured to be able to compare the inspection image data and the reconstructed image data by using only the solder region image of item 1 in the inspection image data as a comparison target. The solder inspection apparatus according to Means 1, characterized in that.

[0033] According to the above-described means 7, the comparison means compares the inspection image data and the reconstructed image data with only the solder region image of 1 in the inspection image data as the comparison target. That is, when performing the comparison of the two image data, the comparison means does not use the portions other than the solder region image of 1 in the inspection image data as the comparison target. Therefore, compared with the case of comparing the entire two image data, the processing load related to the comparison of the two image data can be reduced, and the inspection can be speeded up and made more efficient. Further, it is possible to more reliably prevent the portions other than the solder region image of 1 in the inspection image data, that is, the portions unrelated to the cream solder, from affecting the pass / fail determination, and thus it is possible to further improve the inspection accuracy.

[0034] Means 8. A solder inspection method for inspecting cream solder printed on a printed circuit board, an image data acquisition step capable of acquiring image data of a predetermined inspection region on the printed circuit board including the printed portion of the cream solder; an inspection image data acquisition step of acquiring inspection image data including an image of the cream solder to be inspected based on the image data acquired in the image data acquisition step; a reconstruction image data acquisition step capable of using discrimination means generated by learning only the image data related to good cream solder as learning data for a neural network having an encoding unit that extracts feature amounts from the input image data and a decoding unit that reconstructs the image data from the feature amounts, inputting the inspection image data acquired in the inspection image data acquisition step to the discrimination means, and acquiring the reconstructed image data as the reconstructed image data; including a comparison step of comparing the inspection image data and the reconstructed image data; determining the pass / fail of the cream solder based on the comparison result in the comparison step; the learning data is formed by providing a solder region image of 1 indicating the cream solder corresponding to a land of 1 in an image frame having a size larger than the size of the solder region image of 1; In the inspection image data acquisition step, inspection image data of the same size as the learning data is acquired, wherein the solder region image of 1 extracted from the image data acquired in the image data acquisition step is provided in an image frame of the same size as the image frame of the learning data. A solder inspection method characterized by this.

[0035] According to the above means 8, the same operational effects as those of the above means 1 are achieved.

[0036] In addition, the technical matters related to the above respective means may be appropriately combined. Therefore, for example, the technical matters related to the above means 4 etc. may be combined with respect to the technical matters related to the above means 2. Also, for example, at least one of the technical matters related to the above means 2 to 7 may be applied to the above means 8.

Brief Description of the Drawings

[0037]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Best Mode for Carrying Out the Invention

[0038] Hereinafter, an embodiment will be described with reference to the drawings. First, the configuration of the printed circuit board will be described. FIG. 1 is a partially enlarged plan view of a part of the printed circuit board.

[0039] As shown in FIG. 1, the printed circuit board 1 is formed by forming a wiring pattern (not shown) made of copper foil and a plurality of lands 3 on the surface of a flat base board 2 made of glass epoxy resin or the like. A resist film 4 is coated on the portion of the surface of the base board 2 excluding the lands 3.

[0040] Also, on the lands 3, a cream solder 5 formed by kneading solder grains with flux is printed. In FIG. 1 and the like, for the sake of convenience, a scatter pattern is attached to the portion showing the cream solder 5. In the present embodiment, as the cream solder 5, there are relatively large rectangular solders 5a and 5b printed on relatively large lands 3, relatively small rectangular solders 5c and 5d printed on relatively small lands 3, and a circular solder 5e printed on a circular land 3.

[0041] Next, the manufacturing line (manufacturing process) for manufacturing the printed circuit board 1 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the configuration of the manufacturing line 10 of the printed circuit board 1. As shown in FIG. 2, in the manufacturing line 10, a solder printer 12, a solder inspection device 13, a component mounter 14, a reflow device 15, and a post-reflow inspection device 16 are installed in order from the upstream side (upper side in FIG. 2).

[0042] The solder printer 12 performs a solder printing process for printing the cream solder 5 on each land 3 of the printed circuit board 1. In the solder printing process, for example, the cream solder 5 is printed by screen printing. In screen printing, first, with the lower surface of the screen mask in contact with the printed circuit board 1, the cream solder 5 is supplied onto the upper surface of the screen mask. The screen mask is formed with a plurality of openings corresponding to each land 3 of the printed circuit board 1. Next, by moving while bringing a predetermined squeegee into contact with the upper surface of the screen mask, the cream solder 5 is filled into the openings. Thereafter, by separating the printed circuit board 1 from the lower surface of the screen mask, the cream solder 5 is printed on each land 3 of the printed circuit board 1.

[0043] The solder inspection device 13 inspects the shape of the cream solder 5 printed on the land 3 and the presence or absence of foreign matter adhering to the cream solder 5. Details of the solder inspection device 13 will be described later.

[0044] The component mounter 14 mounts the electronic component 25 (see FIG. 1) on the land 3 on which the cream solder 5 is printed. The electronic component 25 is provided with a plurality of electrodes (not shown), and the electrodes are temporarily fixed to the respective predetermined cream solder 5.

[0045] The reflow device 15 performs a reflow process of heating and melting the cream solder 5 to solder-join (solder) the land 3 and the electrodes of the electronic component 25.

[0046] The post-reflow inspection device 16 inspects whether the solder joint is properly performed in the reflow process, for example, by checking the presence or absence of misalignment in the electronic component 25 using luminance image data or the like.

[0047] In addition, although not shown in the figures, the manufacturing line 10 is provided with a conveyor or the like for transferring the printed circuit board 1 between the above-described devices such as between the solder printer 12 and the solder inspection device 13. Further, a branching device is provided between the solder inspection device 13 and the component mounter 14 and on the downstream side of the post-reflow inspection device 16. The printed circuit boards 1 that are judged to be good products by the solder inspection device 13 or the post-reflow inspection device 16 are guided directly to the downstream side, while the printed circuit boards 1 that are judged to be defective products are discharged to the defective product storage section by the branching device.

[0048] Next, the configuration of the solder inspection device 13 will be described in detail with reference to FIGS. 3 and 4. FIG. 3 is a schematic configuration diagram schematically showing the solder inspection device 13. FIG. 4 is a block diagram showing the functional configuration of the solder inspection device 13.

[0049] The solder inspection device 13 includes a transport mechanism 31 that transports and positions the printed circuit board 1, an inspection unit 32 that obtains image data of the printed circuit board 1, and a control device 33 (see FIG. 4) that executes various controls, image processing, and arithmetic processing in the solder inspection device 13, including driving control of the transport mechanism 31 and the inspection unit 32.

[0050] The transport mechanism 31 includes a pair of transport rails 31a arranged along the loading / unloading direction of the printed circuit board 1, and an endless conveyor belt 31b rotatably disposed with respect to each transport rail 31a. Although not shown in the figures, the transport mechanism 31 is provided with drive means such as a motor for driving the conveyor belt 31b and a chuck mechanism for positioning the printed circuit board 1 at a predetermined position. The transport mechanism 31 is driven and controlled by a control device 33 (a transport mechanism control unit 79 described later).

[0051] Under the above configuration, the printed circuit board 1 carried into the solder inspection apparatus 13 has both side edges in the width direction orthogonal to the carry-in / carry-out direction inserted into the conveyance rails 31a, respectively, and is placed on the conveyor belt 31b. Subsequently, the conveyor belt 31b starts operating, and the printed circuit board 1 is conveyed to a predetermined inspection position. When the printed circuit board 1 reaches the inspection position, the conveyor belt 31b stops and the chuck mechanism operates. By the operation of this chuck mechanism, the conveyor belt 31b is pushed up, and both side edges of the printed circuit board 1 are sandwiched between the upper side portions of the conveyor belt 31b and the conveyance rails 31a. As a result, the printed circuit board 1 is positioned and fixed at the inspection position. When the inspection is completed, the fixing by the chuck mechanism is released, and the conveyor belt 31b starts operating. Thereby, the printed circuit board 1 is carried out from the solder inspection apparatus 13. Of course, the configuration of the conveyance mechanism 31 is not limited to the above form, and other configurations may be adopted.

[0052] The inspection unit 32 is disposed above the conveyance rail 31a (the conveyance path of the printed circuit board 1). The inspection unit 32 includes a first illumination device 32a, a second illumination device 32b, a third illumination device 32c, and a camera 32d. In the present embodiment, the camera 32d constitutes the "image data acquisition means".

[0053] Further, the inspection unit 32 also includes an X-axis movement mechanism 32e (see FIG. 4) capable of moving in the X-axis direction (the left-right direction in FIG. 3) and a Y-axis movement mechanism 32f (see FIG. 4) capable of moving in the Y-axis direction (the front-rear direction in FIG. 3). These movement mechanisms 32e, 32f are driven and controlled by a control device 33 (a movement mechanism control unit 76 described later).

[0054] When performing three-dimensional measurement of the printed circuit board 1, the first illumination device 32a and the second illumination device 32b irradiate a predetermined inspection area on the printed circuit board 1 with predetermined light for three-dimensional measurement (pattern light having a striped light intensity distribution) from obliquely above, respectively.

[0055] Specifically, the first lighting device 32a includes a first light source 32a1 that emits predetermined light, and a first liquid crystal shutter 32a2 that forms a first grating for converting the light from the first light source 32a1 into first pattern light having a striped light intensity distribution. The first lighting device 32a is driven and controlled by a control device 33 (a lighting control unit 72 described later).

[0056] The second lighting device 32b includes a second light source 32b1 that emits predetermined light, and a second liquid crystal shutter 32b2 that forms a second grating for converting the light from the second light source 32b1 into second pattern light having a striped light intensity distribution. The second lighting device 32b is driven and controlled by a control device 33 (a lighting control unit 72 described later).

[0057] Under the above configuration, the light emitted from each of the light sources 32a1 and 32b1 is respectively guided to a condenser lens (not shown), made into parallel light there, and then guided to a projection lens (not shown) through the liquid crystal shutters 32a2 and 32b2, and projected onto the printed circuit board 1 as pattern light. Further, in the present embodiment, the switching control of the liquid crystal shutters 32a2 and 32b2 is performed so that the phases of the respective pattern lights are shifted by a quarter pitch each.

[0058] In addition, by using the liquid crystal shutters 32a2 and 32b2 as gratings, it is possible to irradiate pattern light close to an ideal sine wave. As a result, the measurement resolution of the three-dimensional measurement is improved. Further, the phase shift control of the pattern light can be electrically performed, and the device can be made compact.

[0059] When performing two-dimensional measurement of the printed circuit board 1, the third lighting device 32c irradiates a predetermined inspection area on the printed circuit board 1 with predetermined light (for example, uniform light) for two-dimensional measurement. The third lighting device 32c includes a ring light capable of irradiating blue light, a ring light capable of irradiating green light, and a ring light capable of irradiating red light. Since the third lighting device 32c has the same configuration as the known technology, a detailed description thereof will be omitted.

[0060] The camera 32d images a predetermined inspection area of the printed circuit board 1 from directly above. The camera 32d has an imaging element such as a CCD (Charge Coupled Device) type image sensor or a CMOS (Complementary Metal Oxide Semiconductor) type image sensor, and an optical system (such as a lens unit and an aperture) that forms an image of the printed circuit board 1 on the imaging element, and is arranged such that its optical axis is along the vertical direction (Z-axis direction). Of course, the imaging element is not limited to these, and other imaging elements may be adopted.

[0061] The camera 32d is driven and controlled by a control device 33 (a camera control unit 73 described later). More specifically, the control device 33 executes an imaging process by the camera 32d while synchronizing with the irradiation processes by the respective lighting devices 32a, 32a, 32c. As a result, among the light irradiated from any of the lighting devices 32a, 32b, 32c, the light reflected by the printed circuit board 1 is imaged by the camera 32d. Consequently, image data of the inspection area of the printed circuit board 1 including the printed portion of the cream solder 5 is acquired. Note that the "inspection area" of the printed circuit board 1 is one area among a plurality of areas preset on the printed circuit board 1 with the size of the imaging field (imaging range) of the camera 32d as one unit.

[0062] Also, the camera 32d in the present embodiment is configured as a color camera. Thereby, the light of each color irradiated simultaneously from the respective color ring lights of the third lighting device 32c and reflected by the printed circuit board 1 can be imaged at once.

[0063] The image data imaged and generated by the camera 32d is converted into a digital signal inside the camera 32d and then transferred to the control device 33 (an image acquisition unit 74 described later) in the form of a digital signal. Then, the control device 33 stores the transferred image data and performs various image processes, arithmetic processes, etc. based on the image data.

[0064] The control device 33 consists of a computer including a CPU (Central Processing Unit) that executes predetermined arithmetic processing, a ROM (Read Only Memory) that stores various programs and fixed-value data, etc., a RAM (Random Access Memory) that temporarily stores various data when executing various arithmetic processes, and peripheral circuits thereof.

[0065] Then, when the CPU of the control device 33 operates according to various programs, it functions as various functional units such as a main control unit 71, a lighting control unit 72, a camera control unit 73, an image acquisition unit 74, a data processing unit 75, a movement mechanism control unit 76, a learning unit 77, an inspection unit 78, and a conveyance mechanism control unit 79.

[0066] However, the above various functional units are realized by the cooperation of various hardware such as the above CPU, ROM, and RAM, and there is no need to clearly distinguish between functions realized in a hardware or software manner. Some or all of these functions may be realized by a hardware circuit such as an IC.

[0067] Furthermore, the control device 33 is provided with an input unit 55 composed of a keyboard, a mouse, a touch panel, etc., a display unit 56 provided with a display screen composed of a liquid crystal display, etc., a storage unit 57 capable of storing various data, programs, calculation results, inspection results, etc., and a communication unit 58 capable of transmitting and receiving various data to and from the outside.

[0068] Here, the above various functional units constituting the control device 33 will be described in detail.

[0069] The main control unit 71 is a functional unit that controls the entire solder inspection device 13, and is configured to be able to transmit and receive various signals to and from other functional units such as the lighting control unit 72 and the camera control unit 73.

[0070] The lighting control unit 72 is a functional unit that drives and controls the lighting devices 32a, 32b, and 32c, and performs switching control of irradiation light, etc., based on a command signal from the main control unit 71.

[0071] The camera control unit 73 is a functional unit that drives and controls the camera 32d, and controls imaging timing and the like based on a command signal from the main control unit 71.

[0072] The image acquisition unit 74 is a functional unit for capturing the image data captured and acquired by the camera 32d.

[0073] The data processing unit 75 is a functional unit that performs predetermined image processing on the image data captured by the image acquisition unit 74, or performs two-dimensional measurement processing, three-dimensional measurement processing, etc. using the image data.

[0074] The movement mechanism control unit 76 is a functional unit that drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, and controls the position of the inspection unit 32 based on a command signal from the main control unit 71. By driving and controlling the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, the movement mechanism control unit 76 can move the inspection unit 32 to a position above an arbitrary inspection area of the printed circuit board 1 fixed at the inspection position. Then, while the inspection unit 32 is sequentially moved to a plurality of inspection areas set on the printed circuit board 1, the inspection related to the inspection area is executed, so that the inspection of the entire printed circuit board 1 is executed.

[0075] The learning unit 77 is a functional unit that performs learning of the deep neural network 90 (hereinafter simply referred to as "neural network 90"; see FIG. 5) using learning data, and constructs the first AI (Artificial Intelligence) model 101 as "identification means", the second AI model 102 as "second identification means", and the third AI model 103 as "circular correspondence identification means".

[0076] Furthermore, each of the AI models 101, 102, and 103 in the present embodiment (hereinafter, may be simply abbreviated as "AI models 101 to 103") is a generative model constructed by deep learning (deep neural network) of a neural network 90 using only image data related to good cream solder 5 as learning data, and has a structure of a so-called autoencoder (self-encoder).

[0077] Here, the structure of the neural network 90 will be described with reference to FIG. 5. FIG. 5 is a schematic diagram conceptually showing the structure of the neural network 90. As shown in FIG. 5, the neural network 90 includes an encoder unit 91 as an "encoding unit" that extracts a feature amount (latent variable) TA from the input image data GA, and a decoder unit 92 as a "decoding unit" that reconstructs the image data GB from the feature amount TA, and has a structure of a convolutional autoencoder (CAE).

[0078] Since the structure of the convolutional autoencoder is well-known, a detailed description will be omitted. The encoder unit 91 has a plurality of convolutional layers 93. In each convolutional layer 93, the result of performing a convolution operation on the input data using a plurality of filters (kernels) 94 is output as the input data of the next layer. Similarly, the decoder unit 92 has a plurality of deconvolution layers 95. In each deconvolution layer 95, the result of performing a deconvolution operation on the input data using a plurality of filters (kernels) 96 is output as the input data of the next layer. Then, in the learning process described later, the weights (parameters) of each of the filters 94 and 96 will be updated.

[0079] The inspection unit 78 is a functional unit that inspects the cream solder 5. In the present embodiment, the inspection unit 78 inspects whether the cream solder 5 is properly printed in terms of size, shape, presence or absence of foreign matter, and the like.

[0080] The transport mechanism control unit 79 is a functional unit that drives and controls the transport mechanism 31, and controls the position of the printed circuit board 1 based on a command signal from the main control unit 71.

[0081] The storage unit 57 is composed of an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and has a predetermined storage area for storing, for example, each AI model 101 to 103 (the neural network 90 and the learning information acquired by its learning).

[0082] The communication unit 58 is provided with a wireless communication interface conforming to a communication standard such as a wired LAN (Local Area Network) or a wireless LAN, and is configured to be able to transmit and receive various data to and from the outside. For example, the results of inspections performed by the inspection unit 78 are output to the outside via the communication unit 58, or the results of inspections performed by the post-reflow inspection device 16 are input via the communication unit 58.

[0083] Next, the learning process of the neural network 90 performed by the solder inspection device 13 will be described with reference to the flowchart of FIG. 6.

[0084] When the learning process is started based on the execution of a predetermined learning program, the main control unit 71 first performs preprocessing for learning the neural network 90 in step S101.

[0085] In this preprocessing, first, a large amount of inspection information of the printed circuit boards 1 stored in the post-reflow inspection device 16 is acquired via the communication unit 58. Subsequently, based on the inspection information, learning original image data Ig, which is image data related to the good cream solder 5 that passed the post-reflow inspection, is acquired from the storage unit 57 (see, for example, FIG. 8).

[0086] This original image data Ig for learning pertains to the printed circuit board 1 after the printing of the cream solder 5 and before the mounting of the electronic components 25, and is used to obtain the following learning data Ga, Gb, Gc, Gd, Ge (hereinafter, may be abbreviated as "learning data Ga~Ge") used for the learning of the neural network 90. Further, this original image data Ig for learning includes three-dimensional data which is image data obtained by imaging the printed circuit board 1 with the camera 32d in a state where pattern light is irradiated from the first lighting device 32a or the second lighting device 32b, and two-dimensional data which is image data obtained by imaging the printed circuit board 1 with the camera 32d in a state where uniform light is irradiated from the third lighting device 32c.

[0087] Note that the original image data Ig for learning may be image data that has not been subjected to any special processing as obtained by the camera 32d (for example, monochromatic luminance image data, RGB luminance image data, etc.), or may be image data obtained by subjecting the image data obtained by the camera 32d to a predetermined process (for example, HLS image data obtained by converting RGB image data, height image data obtained by converting image data, etc.).

[0088] Next, the first learning data Ga, Gb, the second learning data Gc, Gd, and the third learning data Ge are created from the acquired original image data Ig for learning (refer to FIGS. 11 to 15 for the learning data Ga~Ge). In this embodiment, the first learning data Ga, Gb corresponds to the "learning data", and the third learning data Ge corresponds to the "learning data for circular correspondence". The first learning data Ga, Gb is used for the generation of the first AI model 101, the second learning data Gc, Gd is used for the generation of the second AI model 102, and the third learning data Ge is used for the generation of the third AI model 103, respectively.

[0089] In obtaining each of the learning data Ga to Ge, first, the region occupied by the cream solder 5 in the acquired original learning image data Ig is specified (see, for example, FIG. 9). When the original learning image data Ig is two-dimensional data, for example, the region occupied by the cream solder 5 is specified using luminance, hue, saturation, etc. When the original learning image data Ig is three-dimensional data, for example, the region occupied by the cream solder 5 is specified using height information, etc.

[0090] Next, an image of the connected component (block portion) in the region occupied by the specified cream solder 5 is extracted as the solder region image Ih of 1 (see, for example, FIG. 10). The solder region image Ih of 1 corresponds to the land 3 of 1. In this embodiment, among the regions occupied by the cream solder 5, the connected component (block portion) located on the designed land 3 of 1 is extracted as the solder region image Ih of 1. Note that, without considering the position of the land 3, simply extracting the connected component (block portion) of 1 as the solder region image Ih of 1 may also be acceptable.

[0091] Next, an image frame for pasting the extracted solder region image Ih of 1 is selected from among the first image frame W1, the second image frame W2, and the third image frame W3 (see FIGS. 11 to 15 for each of the image frames W1, W2, and W3).

[0092] In this embodiment, the first image frame W1 is a rectangular image with a height (the width in the vertical direction of the paper surface in FIG. 11, etc.) of n (pixels) and a width (the width in the horizontal direction of the paper surface in FIG. 11, etc.) of n (pixels), and its size (width and height) is set to be larger than the size of the extracted solder region image Ih of 1 based on design data, etc.

[0093] The second image frame W2 is a rectangular image with a height of m (pixels) and a width of m (pixels), and its size (width and height) is set to be smaller than the size of the first image frame W1.

[0094] The third image frame W3 is a rectangular image with a height of l (pixels) and a width of l (pixels), and its size is set to be smaller than the size of the second image frame W2. In this embodiment, the first image frame W1 corresponds to the "image frame", and the third image frame W3 corresponds to the "circular-corresponding image frame". Note that n, m, and l are natural numbers, respectively, and satisfy n>m>l.

[0095] Also, each of the image frames W1, W2, W3 is assumed to have all pixels with the same value (when nothing is pasted). For example, the luminance, height information, etc. of each pixel constituting each of the image frames W1, W2, W3 are set to "0". Note that this same value is preferably significantly different from the value of the constituent pixels of the solder region image Ih of 1. Therefore, as this same value, it is preferable to use a value other than the commonly used value (for example, a negative number, etc.).

[0096] The selection of the image frames W1, W2, W3 is performed based on the size and shape of the solder region image Ih of 1. When the size (width and height) of the solder region image Ih of 1 is larger than the size of the second image frame W2, the first image frame W1 is selected. On the other hand, when the size of the solder region image Ih of 1 is smaller than the size of the second image frame, the second image frame W2 is selected.

[0097] Also, when the solder region image Ih of 1 pertains to the cream solder 5 printed on the circular land 3, the third image frame W3 is selected. Note that whether the solder region image Ih of 1 pertains to the cream solder 5 printed on the circular land 3 can be determined based on, for example, the shape of the land 3 in the design data or manufacturing data.

[0098] Next, by pasting the solder region image Ih of 1 onto the selected image frames W1, W2, W3, each learning data Ga~Ge provided with the solder region image Ih of 1 in the image frames W1, W2, W3 is obtained.

[0099] In this embodiment, by attaching a solder region image Ih of 1 corresponding to the solder 5a to the first image frame W1, first training data Ga (see, for example, FIG. 11) is obtained, and by attaching a solder region image Ih of 1 corresponding to the solder 5b to the first image frame W1, first training data Gb (see, for example, FIG. 12) is obtained.

[0100] Also, by attaching a solder region image Ih of 1 corresponding to the solder 5c to the second image frame W2, second training data Gc (see, for example, FIG. 13) is obtained, and by attaching a solder region image Ih of 1 corresponding to the solder 5d to the second image frame W2, second training data Gd (see, for example, FIG. 14) is obtained.

[0101] Furthermore, by attaching a solder region image Ih of 1 corresponding to the solder 5e to the third image frame W3, third training data Gd (see, for example, FIG. 15) is obtained.

[0102] In addition, by performing adjustment of the attachment position, rotation processing of the solder region image Ih of 1, etc., each of the training data Ga to Ge is set such that the center or centroid of the solder region image Ih of 1 coincides with the center of the image frames W1, W2, W3, and the long side or short side of the solder region image Ih of 1 extends along a predetermined direction. This long side or short side may be the long side or short side of a rectangle circumscribing the solder region image Ih of 1. Also, among the virtual lines passing through the center or centroid of the solder region image Ih of 1, the one with the longest length of the portion overlapping the solder region image Ih of 1 may be defined as the long side, and the virtual line orthogonal to the long side may be defined as the short side.

[0103] Then, by repeatedly performing the above processes such as extracting the solder region image Ih of 1 and pasting the solder region image Ih of 1 onto the selected image frames W1, W2, and W3, each learning data Ga to Ge is obtained from the original learning image data Ig of 1. Further, by using a plurality of original learning image data Ig, finally, the required number of first learning data Ga and Gb, second learning data Gc and Gd, and third learning data Ge are obtained. In addition, in the present embodiment, each of the learning data Ga to Ge includes data obtained based on two-dimensional data and data obtained based on three-dimensional data.

[0104] When the required number of learning data Ga to Ge for learning is obtained in step S101, in the subsequent step S102, based on a command from the main control unit 71, the learning unit 77 prepares an unlearned neural network 90. For example, the neural network 90 stored in the storage unit 57 or the like in advance is read out. Alternatively, based on the network configuration information (for example, the number of layers of the neural network and the number of nodes in each layer) stored in the storage unit 57 or the like, the neural network 90 is constructed.

[0105] In the present embodiment, as the neural network 90, one for performing learning using the first learning data Ga and Gb, one for performing learning using the second learning data Gc and Gd, and one for performing learning using the third learning data Ge are separately constructed. Also, as the neural network 90, one for performing learning using the learning data Ga to Ge obtained based on two-dimensional data and one for performing learning using the learning data Ga to Ge obtained based on three-dimensional data are separately constructed. Therefore, in the present embodiment, a total of six neural networks 90 are constructed.

[0106] In step S103, reconstructed image data is acquired. That is, based on a command from the main control unit 71, the learning unit 77 supplies the learning data Ga to Ge acquired in step S102 as input data to the input layer of the neural network 90, and thereby acquires the reconstructed image data output from the output layer of the neural network 90. More specifically, the learning unit 77 supplies, to the input layer of the neural network 90, the learning data Ga to Ge corresponding to the neural network 90 among the learning data Ga to Ge acquired in step S102 as input data, and thereby acquires the reconstructed image data output from the output layer of the neural network 90. For example, the learning unit 77 supplies the first learning data Ga and Gb obtained from two-dimensional data as input data to the input layer of the neural network 90 that performs learning using the first learning data Ga and Gb, and acquires the reconstructed image data output from the neural network 90. That is, the learning unit 77 inputs appropriate learning data Ga to Ge to each of the six types of neural networks 90 and acquires the output reconstructed image data.

[0107] In the subsequent step S104, the learning unit 77 compares the input learning data Ga to Ge with the reconstructed image data output by the neural network 90 and determines whether the error therebetween is sufficiently small (whether it is equal to or less than a predetermined threshold).

[0108] Here, when the error is sufficiently small, in step S106, the learning unit 77 determines whether the learning end condition is satisfied. For example, when an affirmative determination is made in step S104 without going through the process of step S105 described later for a predetermined number of consecutive times, or when learning using all of the prepared learning data Ga to Ge is repeated a predetermined number of times, it is determined that the end condition is satisfied. When the end condition is satisfied, the neural network 90 and its learning information (parameters after update, etc., described later) are stored in the storage unit 57 as the AI models 101 to 103, and this learning process is terminated.

[0109] In this embodiment, finally, as the first AI model 101, an AI model obtained by learning the first learning data Ga and Gb acquired from two-dimensional data and an AI model obtained by learning the first learning data Ga and Gb acquired from three-dimensional data are stored.

[0110] Also, as the second AI model 102, an AI model obtained by learning the second learning data Gc and Gd acquired from two-dimensional data and an AI model obtained by learning the second learning data Gc and Gd acquired from three-dimensional data are stored.

[0111] Furthermore, as the third AI model 103, an AI model obtained by learning the third learning data Ge acquired from two-dimensional data and an AI model obtained by learning the third learning data Ge acquired from three-dimensional data are stored. Therefore, a total of six types of AI models are stored.

[0112] On the other hand, if the end condition is not satisfied in step S106, the process returns to step S102 to perform the learning of the neural network 90 again.

[0113] Also, in step S104, if the error is not small enough, after performing the network update process (learning of the neural network 90) in step S105, the process returns to step S103 again to repeat the above series of processes.

[0114] Specifically, in the network update process of step S105, using a known learning algorithm such as the backpropagation method, the weights (parameters) of the respective filters 94 and 96 in the neural network 90 are updated to more appropriate values so that the loss function representing the difference between the learning data Ga to Ge and the reconstructed image data becomes as small as possible. Note that, for example, BCE (Binary Cross-entropy) can be used as the loss function.

[0115] By repeatedly performing the processes of steps S103 to S105, in the neural network 90, the error between the learning data Ga to Ge and the reconstructed image data becomes as small as possible, and more accurate reconstructed image data is output.

[0116] And each of the finally obtained AI models 101 to 103 generates reconstructed image data that substantially matches the input image data when the image data related to the good cream solder 5 is input. Also, when the image data related to the defective cream solder 5 is input in terms of shape, size, and presence or absence of foreign matter, each of the AI models 101 to 103 generates reconstructed image data that substantially matches the image data obtained by correcting the shape and size of the cream solder 5 or removing the noise part (the part corresponding to the foreign matter). That is, when the cream solder 5 is defective, virtual image data related to the cream solder 5 assuming no defective part is generated as the reconstructed image data related to the cream solder 5.

[0117] Next, the inspection process performed by the solder inspection device 13 will be described with reference to the flowchart of FIG. 7. However, the inspection process shown in FIG. 7 is a process executed for each inspection area on the printed circuit board 1.

[0118] When the printed circuit board 1 is carried into the solder inspection device 13 and positioned at a predetermined inspection position, the inspection process is started based on the execution of a predetermined inspection program.

[0119] When the inspection process starts, first, in step S301, an image data acquisition process is performed. In the image data acquisition process, inspection original image data Ik (for example, refer to FIG. 16) related to the printed circuit board 1 to be inspected is acquired. The inspection original image data Ik is image data for obtaining inspection image data Ka, Kb, Kc, Kd, Ke (hereinafter, may be abbreviated as "inspection image data Ka to Ke") described later. In addition, in the present embodiment, as the printed circuit board 1 to be inspected, a plurality of cream solders 5 are printed on one land 3 in a separated state, or the cream solder 5 protrudes from the land 3, or foreign matter X adheres to the cream solder 5, or there are abnormalities in the size or shape of the cream solder 5, etc. are cited as an example.

[0120] The inspection original image data Ik includes three-dimensional data which is image data obtained by imaging the printed circuit board 1 with the camera 32d in a state where pattern light is irradiated from the first illumination device 32a or the second illumination device 32b, and two-dimensional data which is image data obtained by imaging the printed circuit board 1 with the camera 32d in a state where uniform light is irradiated from the third illumination device 32c. In the image data acquisition process, a three-dimensional data acquisition process and a two-dimensional data acquisition process are performed.

[0121] First, the three-dimensional data acquisition process will be described. In this process, while changing the phase of the first pattern light irradiated from the first illumination device 32a, four imaging processes are performed under the first pattern light with different phases, and then while changing the phase of the second pattern light irradiated from the second illumination device 32b, four imaging processes are performed under the second pattern light with different phases, and a total of eight kinds of three-dimensional data are acquired. This will be described in detail below.

[0122] As described above, when the printed circuit board 1 carried into the solder inspection device 13 is positioned and fixed at a predetermined inspection position, based on a command from the main control unit 71, the movement mechanism control unit 76 first drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f to move the inspection unit 32, and adjusts the imaging field (imaging range) of the camera 32d to a predetermined inspection area of the printed circuit board 1.

[0123] At the same time, the lighting control unit 72 switches and controls the liquid crystal shutters 32a2 and 32b2 of both lighting devices 32a and 32b, and sets the positions of the first grid and the second grid formed on the both liquid crystal shutters 32a2 and 32b2 to a predetermined reference position.

[0124] When the switching setting of the first grid and the second grid is completed, the lighting control unit 72 causes the first light source 32a1 of the first lighting device 32a to emit light and irradiates the first pattern light, and the camera control unit 73 drives and controls the camera 32d to execute the first imaging process under the first pattern light. The image data generated by the imaging process is taken into the image acquisition unit 74 at any time (the same applies hereinafter). Thereby, three-dimensional data of an inspection area including a plurality of lands 3 (cream solder 5) is acquired.

[0125] Thereafter, at the same time as the end of the first imaging process under the first pattern light, the lighting control unit 72 turns off the first light source 32a1 of the first lighting device 32a and executes the switching process of the first liquid crystal shutter 32a2. Specifically, the position of the first grid formed on the first liquid crystal shutter 32a2 is switched and set from the reference position to a second position where the phase of the first pattern light is shifted by a quarter pitch (90°).

[0126] When the switching setting of the first grid is completed, the lighting control unit 72 causes the light source 32a1 of the first lighting device 32a to emit light and irradiates the first pattern light, and the camera control unit 73 drives and controls the camera 32d to execute the second imaging process under the first pattern light. Thereafter, by repeating the same process, four kinds of three-dimensional data under the first pattern lights having different phases by 90° are acquired.

[0127] Subsequently, the lighting control unit 72 causes the second light source 32b1 of the second lighting device 32b to emit light and irradiates the second pattern light, and the camera control unit 73 drives and controls the camera 32d to execute the first imaging process under the second pattern light.

[0128] Thereafter, upon completion of the first imaging process under the second pattern light, the illumination control unit 72 turns off the second light source 32b1 of the second illumination device 32b and executes a switching process of the second liquid crystal shutter 32b2. Specifically, the position of the second grating formed on the second liquid crystal shutter 32b2 is switched and set from the reference position to a second position where the phase of the second pattern light is shifted by a quarter pitch (90°).

[0129] When the switching setting of the second grating is completed, the illumination control unit 72 turns on the light source 32b1 of the second illumination device 32b to irradiate the second pattern light, and the camera control unit 73 drives and controls the camera 32d to execute the second imaging process under the second pattern light. Thereafter, by repeating the same process, four-dimensional three-dimensional data under the second pattern light with different phases by 90° each is acquired.

[0130] Next, the process of acquiring two-dimensional data will be described. In this process, based on a command from the main control unit 71, the illumination control unit 72 turns on the third illumination device 32c to irradiate uniform light to a predetermined inspection area, and the camera control unit 73 drives and controls the camera 32d to execute an imaging process under the uniform light. As a result, a predetermined inspection area on the printed circuit board 1 is imaged, and two-dimensional data related to the inspection area is acquired.

[0131] The acquired original inspection image data Ik (three-dimensional data and two-dimensional data) is stored in the storage unit 57.

[0132] Note that the original inspection image data Ik may be image data that has not been subjected to any special processing as obtained by the camera 32d (for example, monochromatic luminance image data or RGB luminance image data), or may be image data obtained by subjecting the image data obtained by the camera 32d to a predetermined process (for example, HLS image data obtained by converting RGB image data or height image data obtained by converting the image data).

[0133] Next, in step S302, an inspection image data acquisition process is executed. In the inspection image data acquisition process, based on the inspection original image data Ik obtained in the image data acquisition process, first inspection image data Ka, Kb, second inspection image data Kc, Kd, and third inspection image data Ke, which will be described later, are acquired respectively.

[0134] In obtaining these inspection image data Ka to Ke, first, the area occupied by the cream solder 5 in the acquired inspection original image data Ik is specified (see, for example, FIG. 17). The area occupied by the cream solder 5 can be specified using luminance, hue, chroma, height information, etc.

[0135] Next, a solder area image Ih of 1 in the specified area occupied by the cream solder 5 is extracted (see, for example, FIGS. 18 and 19). The solder area image Ih of 1 corresponds to the land 3 of 1. In this embodiment, among the connected components (block portions) in the specified area, the entire image of the one whose at least a part overlaps with the land 3 of 1 in the design data or manufacturing data is extracted as the solder area image Ih of 1. Therefore, when there are two or more connected components on the land 3 of 1 in the design data or manufacturing data, the solder area image Ih of 1 is composed of two or more connected components (see, for example, FIG. 18). Also, when a part of the connected component protrudes from the land 3 in the design data or manufacturing data, the solder area image Ih of 1 is composed of the entire connected component including the part protruding from the land 3 (see, for example, FIG. 19). Note that, without using design data or the like, a single connected component may be simply extracted as the solder area image Ih of 1. In this embodiment, an inspection unit 78 that extracts the solder area image Ih of 1 from the inspection original image data Ik constitutes "solder area image extraction means".

[0136] Then, based on the size and shape of the extracted solder area image Ih of 1, an appropriate image frame is selected from the first image frame W1, the second image frame W2, and the third image frame W3, and inspection image data Ka to Ke, which are formed by providing the solder area image Ih of 1 to the selected image frames W1, W2, W3, are acquired (see FIGS. 20 to 26 for the inspection image data Ka to Ke).

[0137] That is, when the size of the extracted solder region image Ih of 1 is larger than the size of the second image frame W2, the solder region image Ih of 1 is pasted onto the first image frame W1 to obtain the first inspection image data Ka, Kb (see, for example, FIGS. 20 to 23). Therefore, the size (width and height) of the first inspection image data Ka, Kb is the same as the size of the first learning data Ga, Gb.

[0138] Also, when the size of the extracted solder region image Ih of 1 is smaller than the size of the second image frame W2, the solder region image Ih of 1 is pasted onto the second image frame W2 to obtain the second inspection image data Kc, Kd (see, for example, FIGS. 24 and 25). Therefore, the size of the second inspection image data Kc, Kd is the same as the size of the second learning data Gc, Gd. In FIGS. 24 and 25, for reference, the normal-shaped and -sized cream solder 5 is virtually shown by a two-dot chain line.

[0139] Furthermore, when the extracted solder region image Ih of 1 is related to the cream solder 5 printed on the circular land 3, the solder region image Ih of 1 is pasted onto the third image frame W3 to obtain the third inspection image data Ke (see, for example, FIG. 26). Therefore, the size of the third inspection image data Ke is the same as the size of the third learning data Ge. Whether the solder region image Ih of 1 is related to the cream solder 5 printed on the circular land 3 can be determined based on, for example, the shape of the land 3 in the design data or manufacturing data.

[0140] Also, when pasting the solder region image Ih of 1 onto the image frames W1, W2, W3, adjustment of the pasting position, rotation processing of the image, etc. are performed. Thereby, each inspection image data Ka to Ke, similar to each learning data Ga to Ge, has the center or centroid of the solder region image Ih of 1 coinciding with the center of the image frames W1, W2, W3, and the long side or short side of the solder region image Ih of 1 extending along a predetermined direction.

[0141] Then, by repeatedly performing the above processes such as extracting the solder region image Ih of 1 and pasting the solder region image Ih of 1 onto the selected image frames W1, W2, and W3, each inspection image data Ka to Ke is obtained from the original inspection image data Ik of 1. In addition, in the present embodiment, each inspection image data Ka to Ke includes data obtained based on two-dimensional data and data obtained based on three-dimensional data. In the present embodiment, the inspection unit 78 that obtains the inspection image data Ka to Ke constitutes the "inspection image data acquisition means".

[0142] In the subsequent step S303, a reconstructed image data acquisition process is executed. Specifically, based on a command from the main control unit 71, the inspection unit 78 inputs the inspection image data Ka to Ke acquired in step S302 into the input layers of the AI models 101, 102, and 103 corresponding to the types of the inspection image data Ka to Ke. Therefore, the first inspection image data Ka and Kb are input into the first AI model 101, the second inspection image data Kc and Kd are input into the second AI model 102, and the third inspection image data Ke is input into the third AI model 103, respectively. Also, the inspection image data Ka to Ke obtained based on two-dimensional data are input into the AI models 101 to 103 corresponding to two-dimensional data, and the inspection image data Ka to Ke obtained based on three-dimensional data are input into the AI models 101 to 103 corresponding to three-dimensional data. Then, the image data reconstructed by the AI models 101 to 103 and output from the output layer is obtained as the reconstructed image data. The obtained reconstructed image data is stored in association with the inspection image data Ka to Ke that is the source of the reconstructed image data.

[0143] Here, when the first inspection image data Kb (see FIG. 23) related to the cream solder 5 with the foreign object X attached or the inspection image data Ka, Kc, Kd (see FIGS. 20, 21, 24, 25) related to the cream solder 5 with inappropriate shape or size is input into each of the AI models 101 to 103, by learning as described above, as the reconstructed image data S, image data related to the good cream solder 5 in which the foreign object X is removed or the shape and size are corrected is output (for example, see FIGS. 27 to 30).

[0144] On the one hand, when inspection image data Ka to Ke related to the good cream solder 5 is input to each of the AI models 101 to 103, as the reconstructed image data S, image data related to the good cream solder 5 that is substantially the same as the inspection image data Ka to Ke is output. Incidentally, the size (width and height) of the reconstructed image data S is the same as the size of the original inspection image data Ka to Ke. In the present embodiment, the inspection unit 78 that acquires the reconstructed image data S constitutes the "reconstructed image data acquisition means".

[0145] In step S304, a pass / fail determination process based on the acquired reconstructed image data S is performed. In the pass / fail determination process, based on a command from the main control unit 71, the inspection unit 78 compares the entirety of the inspection image data Ka to Ke acquired in step S302 with the entirety of the reconstructed image data S acquired in step S303 using the inspection image data Ka to Ke, and calculates the difference between the two image data Ka to Ke, S. For example, the dots (pixels) at the same coordinates in the two image data Ka to Ke, S are compared respectively, and the area (number of dots) of the block of dots where the luminance difference is equal to or greater than a predetermined value is calculated. In the present embodiment, the inspection unit 78 that compares the inspection image data Ka to Ke and the reconstructed image data S constitutes the "comparison means". Also, the process of comparing the inspection image data Ka to Ke and the reconstructed image data S corresponds to the "comparison process".

[0146] Subsequently, the inspection unit 78 determines whether the calculated difference is greater than a predetermined threshold. Then, when the calculated difference is greater than the predetermined threshold, the inspection unit 78 determines "good product", while when the difference is smaller than the predetermined threshold, it determines "defective product".

[0147] Furthermore, the inspection unit 78 performs the above determination on all the inspection image data Ka to Ke related to the inspection area of the printed circuit board 1. When it is determined that all the inspection image data Ka to Ke are "good products", it is determined that the inspection area is a "good product", and this result is stored in the storage unit 57. On the other hand, as a result of performing the above determination on all the inspection image data Ka to Ke related to the inspection area, if it is determined that at least one of the inspection image data Ka to Ke is a "defective product", it is determined that the inspection area is a "defective product", and this result is stored in the storage unit 57.

[0148] Then, as a result of performing the above inspection process on all the inspection areas on the printed circuit board 1, if it is determined that all the inspected areas are "good products", the solder inspection apparatus 13 determines that the printed circuit board 1 has no abnormality in the solder paste 5 (qualified determination), and stores this result in the storage unit 57.

[0149] On the other hand, if there is even one inspected area determined to be a "defective product", the solder inspection apparatus 13 determines that the printed circuit board 1 has an abnormality in the solder paste 5 (unqualified determination), stores this result in the storage unit 57, and notifies the outside of this fact via the display unit 56, the communication unit 58, etc.

[0150] As described in detail above, according to the present embodiment, the inspection image data Ka to Ke are formed by providing one solder area image Ih in the image frames W1, W2, and W3. Therefore, the size (width and height) of each inspection image data Ka to Ke does not vary finely depending on the size of the land 3 and is constant. As a result, it is not necessary to prepare a large number of different AI models (identifying means) for each size of the land 3, and the labor and effort required to obtain the AI models 101 to 103 can be reduced. In addition, even when the sizes of the lands 3 are different, the AI models 101 to 103 can be used in common.

[0151] Furthermore, the image frames W1, W2, and W3 of the learning data Ga to Ge and the image frames W1, W2, and W3 of the inspection image data Ka to Ke are of the same size, and the sizes of the learning data Ga to Ge and the inspection image data Ka to Ke are the same. Therefore, when the inspection image data Ka to Ke is input into the AI models 101 to 103, it is possible to more reliably output the appropriate reconstructed image data S corresponding to the inspection image data Ka to Ke, and thus more accurately determine the quality of the cream solder 5. As a result, good inspection accuracy can be obtained more reliably.

[0152] In addition, the inspection image data Ka to Ke and the reconstructed image data S to be compared are each related to the same cream solder 5. Therefore, unlike the method of determining quality by comparison with a separately prepared standard, it is not necessary to set relatively loose inspection conditions to prevent false detection, and relatively strict inspection conditions can be set. Furthermore, in both image data Ka to Ke and S to be compared, the imaging conditions of the printed circuit board 1 to be inspected (for example, the placement position, placement angle, deflection, etc. of the printed circuit board 1) and the imaging conditions on the solder inspection device 13 side (for example, the illumination state, the angle of view of the camera, etc.) can be made to match. These factors combined enable more accurate determination of the quality of the cream solder 5.

[0153] Also, in the present embodiment, the solder region image Ih that constitutes the inspection image data Ka to Ke includes not only the image of the portion located on the land 3 in the cream solder 5 but also the image of the portion protruding from the land 3 in the cream solder 5. This makes it possible to appropriately determine the quality of the cream solder 5 with a part protruding from the land 3, and the inspection accuracy can be further improved.

[0154] Furthermore, when the cream solder 5 printed on the land 3 of 1 is separated into a plurality, the solder region image Ih of 1 that constitutes the inspection image data Ka to Ke includes an image of the entire cream solder 5 separated into a plurality. Therefore, even if the cream solder 5 printed on the land 3 of 1 is in a separated state, it is possible to more appropriately perform the pass / fail determination regarding the cream solder 5.

[0155] Also, in the present embodiment, when the size of the solder region image Ih of 1 is relatively small, inspection image data Kc and Kd of a relatively small size are obtained by providing the solder region image Ih of 1 in a second image frame W2 of a relatively small size. Then, by inputting this inspection image data Kc and Kd of a relatively small size into the second AI model 102, reconstructed image data S is output, and the inspection image data Kc and Kd of a relatively small size and the reconstructed image data S are compared respectively. Therefore, compared with the case where the image frame of the inspection image data is always a constant size, it is possible to speed up the process for obtaining the reconstructed image data S and the comparison process by the inspection unit 78, and thus the inspection speed can be further improved.

[0156] Furthermore, in the present embodiment, a third AI model 103 is provided for exclusively inspecting the cream solder 5 printed on the circular land 3. Therefore, it is possible to accurately perform the pass / fail determination regarding the circular cream solder 5 printed on the circular land 3. As a result, it is possible to further improve the inspection accuracy.

[0157] In addition, since the orientations and positions of the cream solder 5 in the learning data Ga to Ge and the inspection image data Ka to Ke are generally aligned, even if the learning data Ga to Ge used for generating the AI models 101 to 103 is relatively small, it is possible to accurately perform the pass / fail determination of the cream solder 5. That is, it is possible to obtain good inspection accuracy while more effectively reducing the labor and effort for obtaining the AI models 101 to 103.

[0158] It is not limited to the description content of the above embodiment, and for example, it may be implemented as follows. Of course, other application examples and modification examples not illustrated below are also naturally possible.

[0159] (a) In the above embodiment, in the pass / fail determination step of step S304, the entire inspection image data Ka to Ke and the entire reconstructed image data S are configured to be compared.

[0160] On the other hand, only the solder region image Ih of 1 in the inspection image data Ka to Ke may be set as the comparison target, and the inspection image data Ka to Ke and the reconstructed image data S may be compared. That is, the solder region image Ih of 1 in the inspection image data Ka to Ke and the region overlapping the solder region image Ih of 1 in the reconstructed image data S may be configured to be compared.

[0161] When configured in this way, since the portions other than the solder region image Ih of 1 in the inspection image data Ka to Ke are not used as comparison targets, the processing load related to the comparison of the two image data Ka to Ke, S can be reduced compared to the case of comparing the entire two image data Ka to Ke, S. Therefore, the inspection can be speeded up and the efficiency can be improved. In addition, it is possible to more reliably prevent the portions other than the solder region image Ih of 1 in the inspection image data Ka to Ke, that is, the portions unrelated to the cream solder 5, from affecting the pass / fail determination, and thus it is possible to further improve the inspection accuracy.

[0162] In addition, only the region related to the cream solder 5 in the reconstructed image data S may be set as the comparison target, and the inspection image data Ka to Ke and the reconstructed image data S may be compared. That is, the region related to the cream solder 5 in the reconstructed image data S and the region overlapping the region related to the cream solder 5 in the inspection image data Ka to Ke may be configured to be compared. Of course, the above two comparison methods may be used in combination.

[0163] (b) In the above embodiment, when training the neural network 90, training data Ga to Ge are obtained using the original training image data Ig related to the printed circuit board 1 that passed the post-reflow inspection. On the other hand, for example, training data Ga to Ge may be obtained using the original training image data related to the good-quality solder paste 5 visually selected by an operator after printing the solder paste 5.

[0164] Further, the training unit 77 may obtain the training data Ga to Ge using the image data of the virtual good-quality solder paste 5 generated.

[0165] (c) In the above embodiment, as the AI models 101 to 103, those corresponding to two-dimensional data and those corresponding to three-dimensional data are provided separately, but a common AI model corresponding to each of the two-dimensional data and the three-dimensional data may be provided.

[0166] Furthermore, in the above embodiment, the first AI model 101 and the second AI model 102 are provided, but the configuration may be such that the second AI model 102 is omitted. In this case, the sizes of the second inspection image data Kc and Kd may be the same as the sizes of the first inspection image data Ka and Kd, and the first AI model 101 may perform inspections based on the inspection image data Ka, Kb, Kc, and Kd. Also, the configuration may be such that the third AI model 103 is omitted.

[0167] (d) The configurations and learning methods of the AI models 101 to 103 (neural network 90) are not limited to the above embodiment. For example, when performing the learning process of the neural network 90, the process of obtaining the reconstructed image data, etc., a configuration may be adopted in which various data are subjected to processing such as normalization as necessary. Also, the structure of the neural network 90 is not limited to that shown in FIG. 5, and for example, a configuration may be adopted in which a pooling layer is provided after the convolutional layer 93. Of course, the number of layers of the neural network 90, the number of nodes in each layer, the connection structure of each node, etc. may be different configurations.

[0168] Furthermore, in the above embodiment, the AI models 101 to 103 (neural network 90) are generative models having the structure of a convolutional autoencoder (CAE). However, the present invention is not limited to this. For example, a generative model having the structure of a different type of autoencoder such as a variational autoencoder (VAE) may be used.

[0169] Also, in the above embodiment, the neural network 90 is configured to be learned by the error backpropagation method. However, the present invention is not limited to this, and other various learning algorithms may be used for learning.

[0170] In addition, the neural network 90 may be configured by an AI processing dedicated circuit such as a so-called AI chip. In that case, only learning information such as parameters is stored in the storage unit 57, and the AI processing dedicated circuit reads this and sets it in the neural network 90, so that the AI models 101 to 103 may be configured.

[0171] In addition, in the above embodiment, the control device 33 includes a learning unit 77 and is configured to perform learning of the neural network 90 within the control device 33. However, the present invention is not limited to this. For example, the learning unit 77 may be omitted, and the learning of the neural network 90 may be performed outside the control device 33, and the AI models 101 to 103 (the learned neural network 90) learned outside may be stored in the storage unit 57.

[0172] (e) In the above embodiment, the inspection original image data Ik is obtained as two-dimensional data and three-dimensional data. However, a configuration may be adopted in which only one of the two-dimensional data and the three-dimensional data is obtained. Further, in accordance with the data to be obtained, as the AI models 101 to 103, only those corresponding to one of the two-dimensional data and the three-dimensional data may be provided.

[0173] (f) In the above embodiment, the inspection image data Ka to Ke are obtained by attaching one solder region image Ih to the image frames W1, W2, and W3. On the other hand, the inspection image data Ka to Ke may be obtained as follows. That is, first, when extracting the image of the connected component (block portion) in the region occupied by the identified cream solder 5 as the one solder region image Ih, by extracting the one solder region image Ih and its surrounding portion, an extracted image of the same size as the image frames W1, W2, and W3 is obtained. Then, the inspection image data Ka to Ke may be obtained by replacing the values of each pixel in the surrounding portion of the extracted image with the same value (for example, setting the luminance and height to "0"). Of course, the learning data Ga to Ge may be obtained using a similar method.

Explanation of Signs

[0174] 1... Printed circuit board, 3... Land, 5... Cream solder, 13... Solder inspection device, 32d... Camera (image data acquisition means), 78... Inspection unit (inspection image data acquisition means, reconstructed image data acquisition means, comparison means, solder region image extraction means), 90... Neural network, 91... Encoder unit (encoding unit), 92... Decoder unit (decoding unit), 101... First AI model (identification means), 102... Second AI model (second identification means), 103... Third AI model (circular correspondence identification means), Ih... One solder region image, W1... First image frame (image frame), W2... Second image frame, W3... Third image frame (circular correspondence image frame).

Claims

1. A solder inspection apparatus for inspecting cream solder printed on a printed circuit board, comprising: image data acquisition means capable of acquiring image data of a predetermined inspection area on the printed circuit board including the printed portion of the cream solder; identification means generated by learning only the image data related to good cream solder as learning data for a neural network having an encoding unit that extracts feature amounts from the input image data and a decoding unit that reconstructs the image data from the feature amounts; inspection image data acquisition means for acquiring inspection image data including an image of the cream solder to be inspected based on the image data acquired by the image data acquisition means; reconstructed image data acquisition means capable of acquiring, as reconstructed image data, the image data reconstructed by inputting the inspection image data to the identification means; comparison means capable of comparing the inspection image data and the reconstructed image data; configured to be able to determine the quality of the cream solder based on the comparison result by the comparison means; the learning data is formed by providing an image of a solder area corresponding to one land of the cream solder in an image frame having a size larger than the size of the image of the one solder area; The inspection image data acquisition means is characterized in that the image of the one solder area extracted from the image data acquired by the image data acquisition means is provided in an image frame having the same size as the image frame of the learning data, and the inspection image data having the same size as the learning data is acquired. A solder inspection apparatus according to claim 1.

2. comprising solder area image extraction means for extracting the image of the one solder area constituting the inspection image data from the image data acquired by the image data acquisition means; The solder area image extraction means is characterized in that it can identify the area occupied by the cream solder in the image data acquired by the image data acquisition means and extract, as the image of the one solder area constituting the inspection image data, the image of the connected component in the identified area. The solder inspection apparatus according to claim 1.

3. The solder area image extraction means is characterized in that it can extract, as the image of the one solder area constituting the inspection image data, the entire image of the connected component that at least partially overlaps one land in the design data or manufacturing data. The solder inspection apparatus according to claim 2.

4. For a neural network having an encoding unit that extracts feature amounts from input image data and a decoding unit that reconstructs image data from the feature amounts, it is provided with second discrimination means generated by learning only image data related to good cream solder as second learning data. The second learning data is obtained by providing the solder region image of 1 in a second image frame having a size larger than the size of the solder region image of 1 and smaller than the size of the image frame of the learning data. When the size of the solder region image of 1 extracted from the image data acquired by the image data acquisition means is smaller than the size of the second image frame, the inspection image data acquisition means acquires inspection image data of the same size as the second learning data, in which the solder region image of 1 is provided in the second image frame. The reconstructed image data acquisition means acquires the reconstructed image data reconstructed by inputting the inspection image data to the second discrimination means, and the comparison means is configured to compare the inspection image data and the reconstructed image data. The solder inspection apparatus according to claim 1, characterized in that.

5. For a neural network having an encoding unit that extracts feature amounts from input image data and a decoding unit that reconstructs image data from the feature amounts, it is provided with circular correspondence discrimination means generated by learning only image data related to good cream solder printed on a circular land as circular correspondence learning data. The circular correspondence learning data is obtained by providing the solder region image of 1 related to good cream solder printed on a circular land in a circular correspondence image frame having a size larger than the size of the solder region image of 1. When the solder region image of 1 extracted from the image data acquired by the image data acquisition means corresponds to a circular land, the inspection image data acquisition means acquires inspection image data of the same size as the circular correspondence learning data, in which the solder region image of 1 is provided in the circular correspondence image frame. The reconstructed image data acquisition means acquires the reconstructed image data reconstructed by inputting the inspection image data to the circular correspondence discrimination means, and the comparison means is configured to compare the inspection image data and the reconstructed image data. The solder inspection apparatus according to claim 1, characterized in that.

6. The solder inspection apparatus according to claim 1, wherein the learning data and the inspection image data are set such that the center or the centroid of the solder region image of 1 coincides with the center of the image frame, and the long side or the short side of the solder region image of 1 extends along a predetermined direction.

7. The solder inspection apparatus according to claim 1, wherein the comparison means is configured to be able to compare the inspection image data and the reconstructed image data by using only the solder region image of 1 in the inspection image data as a comparison target.

8. A solder inspection method for inspecting cream solder printed on a printed circuit board, comprising: an image data acquisition step of acquiring image data of a predetermined inspection region on the printed circuit board including a printed portion of cream solder; an inspection image data acquisition step of acquiring inspection image data including an image of cream solder to be inspected based on the image data acquired in the image data acquisition step; a reconstruction image data acquisition step of using discrimination means generated by learning only image data related to good cream solder as learning data for a neural network having an encoding unit that extracts feature amounts from the input image data and a decoding unit that reconstructs image data from the feature amounts, and inputting the inspection image data acquired in the inspection image data acquisition step into the discrimination means to acquire the reconstructed image data as reconstructed image data; a comparison step of comparing the inspection image data and the reconstructed image data; determining the quality of the cream solder based on the comparison result in the comparison step; the learning data is formed by providing a solder region image of 1 showing cream solder corresponding to 1 land in an image frame having a size larger than the size of the solder region image of 1; In the inspection image data acquisition step, the inspection image data having the same size as the learning data is acquired by providing the solder region image of 1 extracted from the image data acquired in the image data acquisition step in an image frame having the same size as the image frame of the learning data.

Images