Solder Inspection Equipment

Abstract

To provide a solder inspection device which can reduce the labor and burden required to obtain an identification means and can use the identification means in common even when the sizes of lands are different. [Solution] A predetermined neural network is trained using only image data related to good cream solder as training data to generate an AI model 101, which then receives test image data based on image data acquired by a camera 32d to obtain reconstructed image data, and the quality of the cream solder is determined by comparing the test image data and the reconstructed image data. The training data and the test image data are each configured with an image frame containing four extracted images corresponding to the four sides of the land, and the image frame related to the training data and the image frame related to the test image data are the same size.

Description

[Technical Field]

[0001] The present invention relates to a solder inspection device for inspecting cream solder applied to a circuit board. [Background technology]

[0002] Generally, in a board manufacturing line where electronic components are mounted on a printed circuit board, cream solder is first printed on the lands of the printed circuit board (solder printing process). Next, the electronic components are temporarily attached to the printed circuit board using the viscosity of the cream solder (mounting process). After that, the printed circuit board is introduced into a reflow furnace, where the cream solder is heated and melted to perform soldering (reflow process). Such board manufacturing lines may be equipped with an inspection device that inspects the printed circuit board.

[0003] Recently, inspection devices that use AI models have been proposed for inspecting printed circuit boards. Known examples of inspection devices that use AI models include one that inspects the presence or absence of foreign matter on a printed circuit board by comparing inspection image data (original image data) of an area to be inspected on the printed circuit board with reconstructed image data generated by inputting the inspection image data into an AI model (identification means) (see, for example, Patent Document 1, etc.). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-88818 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if there is an abnormality in the shape (two-dimensional or three-dimensional) of the cream solder or if foreign matter adheres to the land before the cream solder is printed, there is a risk of defective mounting of the electronic component. Therefore, it is conceivable to use the AI ​​model described above to inspect the cream solder for any abnormalities that could lead to defective mounting of the electronic component before mounting the electronic component.

[0006] However, to obtain an AI model, it is necessary to use image data related to the cream solder as training data. However, since the shape of each individual cream solder is different, if image data related to the entire cream solder is used as training data, a large amount of image data must be prepared to achieve the desired accuracy, making efficient training difficult.

[0007] Furthermore, in terms of improving inspection accuracy, it is preferable to match the size (width and height) of the training data with the size of the test image data. However, matching the sizes of the training data and test image data by enlarging or reducing the image data may result in a decrease in inspection accuracy. To address this issue, it is possible to prepare various training data of different sizes corresponding to the size of the lands, and to prepare multiple AI models trained only on training data of the same size. Then, during inspection, test image data of a size corresponding to the size of the lands is acquired, and an appropriate AI model that matches the size of the lands (i.e., the test image data) is selected from the multiple AI models and used. However, preparing different AI models for each size of land (test image data) requires complex work and requires a great deal of effort. Furthermore, obtaining multiple AI models requires the preparation of a huge amount of training data.

[0008] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a solder inspection device that can reduce the effort and burden involved in obtaining an identification means as an AI model, and that can use the identification means in common even when the land sizes are different. [Means for solving the problem]

[0009] The following describes each of the means suitable for achieving the above object, with specific effects of the corresponding means added as necessary.

[0010] Means 1. A solder inspection device for inspecting abnormalities related to cream solder printed on a printed circuit board, an image data acquisition means for acquiring image data of a predetermined inspection area on the printed circuit board including cream solder printed on rectangular lands; a recognition means for generating the image data by training only image data relating to non-defective cream solders in a neural network having an encoding unit for extracting feature values ​​from input image data and a decoding unit for reconstructing image data from the feature values; an inspection image data acquisition means for acquiring inspection image data including an image of the cream solder to be inspected based on the image data acquired by the image data acquisition means; a reconstructed image data acquisition means for inputting the inspection image data to the identification means and acquiring reconstructed image data as reconstructed image data; a comparison means for comparing the inspection image data and the reconstructed image data; The apparatus is configured to be able to determine whether or not there is an abnormality in the cream solder based on the comparison result by the comparison means, The learning data is composed of four extracted images extracted from a solder image that is an image related to cream solder corresponding to one land, the extracted images including the outer edge of the cream solder and corresponding to the four sides of the land, each of which is arranged in an image frame that is larger than the size of the extracted image, A solder inspection device characterized in that the inspection image data acquisition means acquires the inspection image data in which each of the four extracted images extracted from the solder image in the image data acquired by the image data acquisition means is placed in an image frame of the same size as the image frame of the learning data.

[0011] The learning data may be generated from image data (actual image data) obtained by capturing an image of a printed circuit board on which good quality cream solder has been printed (i.e., an image relating to the actual cream solder), or may be a virtually generated image relating to good quality cream solder. Examples of the actual image data include image data accumulated in previous inspections and image data of good quality printed circuit boards visually selected by an operator after the cream solder has been printed.

[0012] Furthermore, the "neural network" includes, for example, a convolutional neural network having multiple convolutional layers. The "learning" includes, for example, deep learning. The "identification means (generative model)" includes, for example, an autoencoder and a convolutional autoencoder.

[0013] In addition, the "identification means" is generated by learning only image data related to good cream solder. Therefore, when inspection image data related to defective cream solder is input to the identification means, the reconstructed image data generated will be nearly identical to the inspection image data in which the defective parts have been corrected (for example, foreign matter has been removed, or the shape and size have been corrected). In other words, when there is a defect in the cream solder, virtual image data related to the cream solder as if there were no defective parts is generated as the reconstructed image data related to the cream solder.

[0014] According to the above-mentioned means 1, the test image data is formed by placing an extracted image in an image frame. Therefore, the size (width and height) of the test image data matches the size of the image frame and is constant without varying slightly depending on the land size. This eliminates the need to prepare multiple different identification means for each land size, reducing the labor and time required to obtain the identification means. Furthermore, the identification means can be used in common even when the land sizes are different.

[0015] Furthermore, according to the above-mentioned means 1, the image frame of the learning data and the image frame of the test image data are the same size, and the sizes of the learning data and the test image data are the same. Therefore, when the test image data is input to the identification means, appropriate reconstructed image data corresponding to the test image data can be more reliably output, and thus the presence or absence of an abnormality in the cream solder can be more accurately determined. This makes it possible to more reliably obtain good test accuracy.

[0016] Furthermore, the training data is configured by providing an image frame for each extracted image corresponding to the four sides of the land. Therefore, compared to using image data of the entire cream solder as training data, a discrimination method with the desired accuracy can be obtained with less training data. Furthermore, since four training data can be obtained from one solder image, the required number of training data can be easily obtained. These combined features can very effectively reduce the effort and time required to obtain a discrimination method.

[0017] In addition, since the learning data and the test image data are each composed of four extracted images corresponding to the four sides of the land, each set in an image frame, the size of both data can be made relatively small compared to when an image of the entire cream solder is set in an image frame. This reduces the processing load for learning and testing, and allows for faster learning and testing.

[0018] Means 2: A solder inspection device according to Means 1, characterized in that the extracted images in the learning data and the inspection image data are set so that the outer edge side of the cream solder is oriented in a predetermined direction.

[0019] According to the above-mentioned means 2, the extracted images in the learning data and the inspection image data are set so that the outer edge side of the cream solder (i.e., the side opposite the center of the cream solder) faces a predetermined direction. For example, the extracted images in both image data are set so that the outer edge side of the cream solder faces downward. Therefore, the learning efficiency is improved, and the labor and effort required to obtain the identification means can be further reduced. Furthermore, the accuracy of the reconstructed image data output from the identification means is improved, making it possible to further increase the inspection accuracy.

[0020] Means 3: The solder inspection device according to Means 1, characterized in that the extracted images in the learning data and the inspection image data are set so that the long or short sides extend in a predetermined direction.

[0021] According to the above-mentioned means 3, the extracted images in the learning data and the test image data are set so that their long or short sides extend in a predetermined direction. For example, the extracted images in both sets of image data are set so that their long sides extend in the left-right direction. Therefore, the learning efficiency can be further improved, and the labor and effort required to obtain the identification means can be further reduced. In addition, the accuracy of the reconstructed image data can be further improved, and the inspection accuracy can be further improved.

[0022] Means 4. The extracted image is configured to extract an area within an extraction frame that has a certain positional relationship with the outer edge of the solder paste or the outer edge of the land, which is a reference, from the solder image; the extraction frame is rectangular in shape having two short sides, a central long side disposed on the center side of the cream solder, and an outer long side corresponding to the side opposite the central long side; The solder inspection device described in means 1 is characterized in that the extraction frame for extracting the extracted images that constitute the learning data and the extraction frame for extracting the extracted images that constitute the inspection image data each have the same length W in the short side direction, and when extracting the extracted images, the central long side and the outer long side are positioned so as to sandwich the outer edge of the cream solder, and the outer long side is positioned at a distance W0 in the short side direction from the reference.

[0023] According to the above-mentioned means 4, the extraction frame for the learning data and the extraction frame for the test image data have the same length W in the short side direction, and when extracting the extracted image, they are positioned at a fixed position relative to the reference (the outer edge of the cream solder or the outer edge of the land). Therefore, it is less likely that the position and range of the cream solder will vary significantly in the extracted images that make up the learning data and test image data. This can further improve learning efficiency and further increase inspection accuracy.

[0024] The technical matters relating to the above means may be combined as appropriate. For example, the technical matters relating to the above means 2 may be combined with the technical matters relating to the above means 3 or 4. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 2 is a partially enlarged plan view of a portion of the printed circuit board. [Figure 2] FIG. 2 is a partially enlarged cross-sectional view of a portion of the printed circuit board. [Figure 3] FIG. 1 is a block diagram showing the configuration of a manufacturing line for printed circuit boards. [Figure 4] FIG. 1 is a schematic diagram illustrating a solder inspection device. [Figure 5] FIG. 2 is a block diagram showing the functional configuration of the solder inspection device. [Figure 6] FIG. 1 is a schematic diagram for explaining the structure of a neural network. [Figure 7]10 is a flowchart showing the flow of a neural network learning process. [Figure 8] 10 is a flowchart showing the flow of an inspection process. [Figure 9] FIG. 2 is a schematic diagram showing original image data for learning; [Figure 10] FIG. 10 is a schematic diagram showing a solder image in the learning original image data. [Figure 11] 10A and 10B are schematic diagrams for explaining the size and position of an extraction frame. [Figure 12] FIG. 10 is a schematic diagram for explaining extraction of an extracted image Cag, etc.; [Figure 13] 10A and 10B are schematic diagrams for explaining extraction of an extracted image Cbg and the like. [Figure 14] 10A and 10B are schematic diagrams for explaining extraction of an extraction image Ccg and the like. [Figure 15] FIG. 10 is a schematic diagram for explaining extraction of an extracted image Cdg, etc.; [Figure 16] FIG. 1 is a schematic diagram showing an image frame and learning data Ga. [Figure 17] FIG. 10 is a schematic diagram showing an image frame and learning data Gb. [Figure 18] FIG. 10 is a schematic diagram showing an image frame and learning data Gc. [Figure 19] FIG. 1 is a schematic diagram showing an image frame and learning data Gd. [Figure 20] FIG. 10 is a schematic diagram showing an example of original inspection image data. [Figure 21] FIG. 10 is an enlarged partial cross-sectional view of a printed circuit board with foreign matter attached thereto. [Figure 22] FIG. 10 is a schematic diagram showing a solder image in the original image data for inspection. [Figure 23] 10A and 10B are schematic diagrams for explaining extraction of an extracted image Cak, etc. FIG. [Figure 24] 10A and 10B are schematic diagrams for explaining extraction of an extracted image Cbk, etc. FIG. [Figure 25] 10A and 10B are schematic diagrams for explaining extraction of an extracted image Cck, etc. FIG. [Figure 26]10A and 10B are schematic diagrams for explaining extraction of an extracted image Cdk, etc. FIG. [Figure 27] 10 is a schematic diagram showing an image frame and test image data Ka. FIG. [Figure 28] 10 is a schematic diagram showing an image frame and test image data Kb. FIG. [Figure 29] 10 is a schematic diagram showing an image frame and test image data Kc. FIG. [Figure 30] 10 is a schematic diagram showing an image frame and test image data Kd. FIG. [Figure 31] 10 is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Ka is input. FIG. [Figure 32] 10 is a schematic diagram showing reconstructed image data output from an AI model when test image data Kd is input. FIG. [Figure 33] FIG. 10 is a schematic diagram for explaining the size, position, etc. of the extraction frame in another embodiment. [Figure 34] 10 is a schematic diagram showing an example of original inspection image data including cream solder with an abnormal shape or size, or cream solder covering a foreign object, in another embodiment. [Figure 35] 10A and 10B are schematic diagrams for explaining extraction of an extracted image Cbk relating to cream solder having an abnormal shape or size in another embodiment. [Figure 36] 10A and 10B are schematic diagrams for explaining extraction of an extracted image Cck relating to cream solder having an abnormal shape or size in another embodiment. [Figure 37] 10 is a schematic diagram for explaining extraction of an extracted image Cak relating to cream solder covering a foreign substance in another embodiment. FIG. [Figure 38] 10A and 10B are schematic diagrams showing an image frame and test image data Kb in another embodiment. [Figure 39] 10A and 10B are schematic diagrams showing an image frame and test image data Kc in another embodiment. [Figure 40] 10A and 10B are schematic diagrams showing an image frame and test image data Ka in another embodiment. [Figure 41] FIG. 10 is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Kb is input in another embodiment. [Figure 42] FIG. 10 is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Kc is input in another embodiment. [Figure 43] FIG. 10 is a schematic diagram showing reconstructed image data output from an AI model when inspection image data Ka is input in another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] An embodiment will be described below with reference to the drawings. First, the configuration of the printed circuit board will be described. Fig. 1 is a partially enlarged plan view of a portion of the printed circuit board, and Fig. 2 is a partially enlarged cross-sectional view of a portion of the printed circuit board.

[0027] 1 and 2, the printed circuit board 1 has a wiring pattern (not shown) made of copper foil and a plurality of lands 3 that are rectangular in plan view formed on the surface of a flat base substrate 2 made of glass epoxy resin or the like. A resist film 4 is coated on the surface of the base substrate 2 except for the lands 3.

[0028] Furthermore, cream solder 5, which is made by kneading solder particles with flux, is printed on the lands 3. For convenience, in Figure 1 and other figures, a scattered dot pattern is used to indicate the cream solder 5. Furthermore, in Figure 1 and other figures, only a small portion of the multiple lands 3 and cream solder 5 that the printed circuit board 1 has is shown.

[0029] Next, a manufacturing line (manufacturing process) for manufacturing the printed circuit board 1 will be described with reference to Fig. 3. As shown in Fig. 3, a manufacturing line 10 is provided with, in order from the upstream side (upper side in Fig. 3), a solder printer 12, a solder inspection device 13, a component mounter 14, a reflow device 15, and a post-reflow inspection device 16.

[0030] The solder printer 12 performs a solder printing process for printing cream solder 5 on each land 3 of the printed circuit board 1. In the solder printing process, the cream solder 5 is printed by, for example, screen printing. In screen printing, first, the bottom surface of a screen mask is brought into contact with the printed circuit board 1, and cream solder 5 is supplied to the top surface of the screen mask. The screen mask has a plurality of openings formed therein, each corresponding to each land 3 of the printed circuit board 1. Next, a predetermined squeegee is brought into contact with the top surface of the screen mask and moved, thereby filling the openings with the cream solder 5. Thereafter, the printed circuit board 1 is separated from the bottom surface of the screen mask, and the cream solder 5 is printed on each land 3 of the printed circuit board 1.

[0031] The solder inspection device 13 inspects whether or not foreign matter is attached to the lands 3 and the cream solder 5, and inspects the shape of the cream solder 5 printed on the lands 3. Details of the solder inspection device 13 will be described later.

[0032] The component mounter 14 mounts an electronic component 25 (see FIGS. 1 and 2) on the land 3 on which the cream solder 5 is printed. The electronic component 25 has a plurality of electrodes (not shown), and each of the electrodes is temporarily fixed to a predetermined cream solder 5.

[0033] The reflow device 15 performs a reflow process in which the cream solder 5 is heated and melted to solder the lands 3 and the electrodes of the electronic component 25 together.

[0034] The post-reflow inspection device 16 checks whether the solder joints have been properly made in the reflow process by checking for the presence or absence of misalignment in the electronic components 25 using, for example, brightness image data.

[0035] In addition, although not shown, the manufacturing line 10 is equipped with conveyors or the like for transporting the printed circuit board 1 between the above-mentioned devices, such as between the solder printer 12 and the solder inspection device 13. Branching devices are also provided between the solder inspection device 13 and the component mounter 14 and downstream of the post-reflow inspection device 16. The printed circuit boards 1 that have been determined to be non-defective by the solder inspection device 13 or the post-reflow inspection device 16 are guided directly downstream, while the printed circuit boards 1 that have been determined to be defective are discharged by the branching device to a defective product storage area.

[0036] Next, the configuration of the solder inspection device 13 will be described in detail with reference to Figures 4 and 5. Figure 4 is a schematic diagram showing the configuration of the solder inspection device 13. Figure 5 is a block diagram showing the functional configuration of the solder inspection device 13.

[0037] The solder inspection device 13 includes a transport mechanism 31 that transports and positions the printed circuit board 1, an inspection unit 32 that obtains image data of the printed circuit board 1, and a control device 33 (see Figure 5) that controls the drive of the transport mechanism 31 and the inspection unit 32, as well as performs various controls, image processing, and calculation processing in the solder inspection device 13.

[0038] The transport mechanism 31 includes a pair of transport rails 31a arranged along the direction in which the printed circuit board 1 is carried in and out, and an endless conveyor belt 31b rotatably disposed relative to each of the transport rails 31a. Although not shown, the transport mechanism 31 also includes a driving means such as a motor for driving the conveyor belt 31b, and a chuck mechanism for positioning the printed circuit board 1 at a predetermined position. The transport mechanism 31 is driven and controlled by a control device 33 (a transport mechanism control unit 79, which will be described later).

[0039] With the above configuration, the printed circuit board 1 is carried into the solder inspection device 13. Both side edges in the width direction perpendicular to the carrying-in / out direction are inserted into the conveyor rails 31a, and the printed circuit board 1 is placed on the conveyor belt 31b. The conveyor belt 31b then starts operating, transporting the printed circuit board 1 to a predetermined inspection position. When the printed circuit board 1 reaches the inspection position, the conveyor belt 31b stops and the chucking mechanism is activated. The operation of this chucking mechanism pushes up the conveyor belt 31b, and both side edges of the printed circuit board 1 are clamped between the conveyor belt 31b and the upper edge of the conveyor rail 31a. This positions and fixes the printed circuit board 1 at the inspection position. When the inspection is completed, the chucking mechanism releases the fixation, and the conveyor belt 31b starts operating. The printed circuit board 1 is then carried out of the solder inspection device 13. Of course, the configuration of the transport mechanism 31 is not limited to the above, and other configurations may be adopted.

[0040] The inspection unit 32 is disposed above the transport rail 31a (the transport path of the printed circuit board 1). The inspection unit 32 includes a first lighting device 32a, a second lighting device 32b, a third lighting device 32c, and a camera 32d. In this embodiment, the camera 32d constitutes the "image data acquisition means."

[0041] The inspection unit 32 also includes an X-axis movement mechanism 32e (see FIG. 5) that allows movement in the X-axis direction (left-right direction in FIG. 4), and a Y-axis movement mechanism 32f (see FIG. 5) that allows movement in the Y-axis direction (front-back direction in FIG. 4). These movement mechanisms 32e and 32f are driven and controlled by the control device 33 (a movement mechanism control unit 76, which will be described later).

[0042] When performing three-dimensional measurement of the printed circuit board 1, the first lighting device 32a and the second lighting device 32b each irradiate a predetermined inspection area on the printed circuit board 1 from diagonally above with predetermined light for three-dimensional measurement (pattern light having a striped light intensity distribution).

[0043] Specifically, the first lighting device 32a includes a first light source 32a1 that emits a predetermined light, and a first liquid crystal shutter 32a2 that forms a first grating that converts the light from the first light source 32a1 into a first pattern light having a striped light intensity distribution, and is driven and controlled by the control device 33 (the lighting control unit 72 described later).

[0044] The second lighting device 32b includes a second light source 32b1 that emits a predetermined light, and a second liquid crystal shutter 32b2 that forms a second grating that converts the light from the second light source 32b1 into a second pattern light having a striped light intensity distribution, and is driven and controlled by the control device 33 (the lighting control unit 72 described later).

[0045] With the above configuration, the light emitted from each of the light sources 32a1 and 32b1 is guided to a condenser lens (not shown), where it is converted into parallel light, and then guided to a projection lens (not shown) via the liquid crystal shutters 32a2 and 32b2, and projected as patterned light onto the printed circuit board 1. In this embodiment, the liquid crystal shutters 32a2 and 32b2 are switched and controlled so that the phase of each patterned light is shifted by a quarter pitch.

[0046] Furthermore, by using liquid crystal shutters 32a2 and 32b2 as the grating, it is possible to irradiate pattern light that is close to an ideal sine wave. This improves the measurement resolution of three-dimensional measurement. In addition, the phase shift of the pattern light can be electrically controlled, allowing for a more compact device.

[0047] When performing two-dimensional measurement of the printed circuit board 1, the third illumination device 32c irradiates a predetermined inspection area on the printed circuit board 1 with predetermined light for two-dimensional measurement (for example, uniform light). The third illumination device 32c is equipped with a ring light capable of irradiating blue light, a ring light capable of irradiating green light, and a ring light capable of irradiating red light. Note that the third illumination device 32c has a configuration similar to that of known technology, and therefore detailed description thereof will be omitted.

[0048] The camera 32d captures an image of a predetermined inspection area of ​​the printed circuit board 1 from directly above. The camera 32d has an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and an optical system (lens unit, diaphragm, etc.) that forms an image of the printed circuit board 1 on the imaging element, and is arranged so that its optical axis is aligned in the vertical direction (Z-axis direction). Of course, the imaging element is not limited to these, and other imaging elements may be used.

[0049] The camera 32d is driven and controlled by the control device 33 (camera control unit 73, which will be described later). More specifically, the control device 33 executes an image capturing process by the camera 32d in synchronization with the irradiation process by each of the lighting devices 32a, 32b, and 32c. As a result, light irradiated from one of the lighting devices 32a, 32b, and 32c and reflected by the printed circuit board 1 is captured by the camera 32d. As a result, image data of the inspection area of ​​the printed circuit board 1, including the cream solder 5 printed on the lands 3, is acquired. Note that the "inspection area" of the printed circuit board 1 is one of multiple areas preset on the printed circuit board 1, with the size of the imaging field of view (imaging range) of the camera 32d being one unit.

[0050] Furthermore, the camera 32d in this embodiment is configured as a color camera, which makes it possible to simultaneously capture images of the light of each color that is simultaneously irradiated from the ring lights of each color of the third illumination device 32c and reflected by the printed circuit board 1.

[0051] The image data captured and generated by the camera 32d is converted into a digital signal inside the camera 32d and then transferred in the form of a digital signal to the control device 33 (an image acquisition unit 74 described below). The control device 33 then stores the transferred image data and performs various image processing, arithmetic processing, and the like based on the image data.

[0052] The control device 33 consists of a computer including a CPU (Central Processing Unit) that executes predetermined arithmetic processing, a ROM (Read Only Memory) that stores various programs and fixed value data, a RAM (Random Access Memory) that temporarily stores various data when executing various arithmetic processing, and peripheral circuits for these.

[0053] The control device 33 functions as various functional units such as a main control unit 71, a lighting control unit 72, a camera control unit 73, an image acquisition unit 74, a data processing unit 75, a movement mechanism control unit 76, a learning unit 77, an inspection unit 78, and a transport mechanism control unit 79, as the CPU operates in accordance with various programs.

[0054] However, the various functional units are realized by the cooperation of various hardware such as the CPU, ROM, RAM, etc., and there is no need to clearly distinguish between functions realized by hardware and functions realized by software, and some or all of these functions may be realized by hardware circuits such as ICs.

[0055] Furthermore, the control device 33 is provided with an input unit 55 consisting of a keyboard, mouse, touch panel, etc., a display unit 56 with a display screen consisting of an LCD display, etc., a memory unit 57 capable of storing various data, programs, calculation results, test results, etc., and a communication unit 58 capable of sending and receiving various data to and from the outside.

[0056] Here, the above-mentioned various functional units that constitute the control device 33 will be described in detail.

[0057] The main control unit 71 is a functional unit that controls the entire solder inspection device 13, and is configured to be able to send and receive various signals to and from other functional units such as the illumination control unit 72 and camera control unit 73.

[0058] The illumination control unit 72 is a functional unit that controls the driving of the illumination devices 32a, 32b, and 32c, and performs switching control of the illumination light based on a command signal from the main control unit 71.

[0059] The camera control unit 73 is a functional unit that controls the driving of the camera 32d, and controls the timing of image capture and the like based on a command signal from the main control unit 71.

[0060] The image acquisition unit 74 is a functional unit for taking in image data captured and acquired by the camera 32d.

[0061] The data processing unit 75 is a functional unit that performs predetermined image processing on the image data captured by the image acquisition unit 74, and performs two-dimensional measurement processing, three-dimensional measurement processing, and the like using the image data.

[0062] The movement mechanism control unit 76 is a functional unit that drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, and controls the position of the inspection unit 32 based on a command signal from the main control unit 71. By driving and controlling the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f, the movement mechanism control unit 76 can move the inspection unit 32 to a position above any inspection area of ​​the printed circuit board 1 that is positioned and fixed at the inspection position. Then, the inspection unit 32 is moved sequentially to a plurality of inspection areas set on the printed circuit board 1, and inspections of the inspection areas are performed, thereby inspecting the entire area of ​​the printed circuit board 1.

[0063] The learning unit 77 is a functional unit that uses learning data to learn a deep neural network 90 (hereinafter simply referred to as "neural network 90"; see FIG. 6) and constructs an AI (Artificial Intelligence) model 101 as an "identification means."

[0064] As will be described later, the AI ​​model 101 in this embodiment is a generative model constructed by deep learning a neural network 90 using only image data relating to good cream solder 5 as training data, and has the structure of a so-called autoencoder.

[0065] Here, the structure of the neural network 90 will be described with reference to Fig. 6. Fig. 6 is a schematic diagram conceptually showing the structure of the neural network 90. ​​As shown in Fig. 6, the neural network 90 has a convolutional auto-encoder (CAE) structure that includes an encoder unit 91 as an "encoding unit" that extracts a feature (latent variable) TA from input image data GA, and a decoder unit 92 as a "decoding unit" that reconstructs image data GB from the feature TA.

[0066] The structure of a convolutional autoencoder is well known, and therefore a detailed description will be omitted. However, the encoder unit 91 has a plurality of convolution layers 93, and in each convolution layer 93, a convolution operation is performed on input data using a plurality of filters (kernels) 94, and the result is output as input data for the next layer. Similarly, the decoder unit 92 has a plurality of deconvolution layers 95, and in each deconvolution layer 95, a deconvolution operation is performed on input data using a plurality of filters (kernels) 96, and the result is output as input data for the next layer. Then, in the learning process described below, the weights (parameters) of each filter 94, 96 are updated.

[0067] The inspection unit 78 is a functional unit that inspects for abnormalities related to the cream solder 5. In this embodiment, the inspection unit 78 inspects whether the cream solder 5 has been printed properly in terms of the presence, size, and shape of foreign matter.

[0068] The transfer mechanism control unit 79 is a functional unit that controls the drive of the transfer mechanism 31 , and controls the position of the printed circuit board 1 based on a command signal from the main control unit 71 .

[0069] The memory unit 57 is composed of a hard disk drive (HDD) or a solid state drive (SSD), and has a predetermined memory area for storing, for example, an AI model 101 (neural network 90 and learning information acquired by its learning).

[0070] The communication unit 58 is equipped with a wireless communication interface conforming to communication standards such as a wired LAN (Local Area Network) or a wireless LAN, and is configured to be able to transmit and receive various data to and from the outside. For example, the results of the inspection performed by the inspection unit 78 are output to the outside via the communication unit 58, and the results of the inspection performed by the post-reflow inspection device 16 are input via the communication unit 58.

[0071] Next, the learning process of the neural network 90 performed by the solder inspection device 13 will be described with reference to the flowchart of FIG.

[0072] When the learning process is started based on the execution of a predetermined learning program, the main control unit 71 first performs pre-processing for learning the neural network 90 in step S101.

[0073] In this pre-processing, first, inspection information of a large number of printed circuit boards 1 stored in the post-reflow inspection device 16 is obtained via the communication unit 58. Next, based on the inspection information, learning original image data Ig, which is image data relating to non-defective cream solder 5 that has passed the post-reflow inspection, is obtained from the storage unit 57 (see, for example, FIG. 9). FIG. 9 shows only a portion of the learning original image data Ig, and the actual learning original image data Ig contains many more cream solders 5 and lands 3.

[0074] The original learning image data Ig pertains to the printed circuit board 1 after the printing of the cream solder 5 and before the mounting of the electronic components 25, and is used to obtain the later-described learning data Ga, Gb, Gc, and Gd (hereinafter, sometimes simply referred to as "learning data Ga-Gd") used for training the neural network 90. ​​The original learning image data Ig includes three-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while irradiating it with patterned light from the first illumination device 32a or the second illumination device 32b, and two-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while irradiating it with uniform light from the third illumination device 32c.

[0075] The original learning image data Ig may be image data obtained by the camera 32d without any special processing (for example, monochrome luminance image data or RGB luminance image data), or may be image data obtained by performing predetermined processing on the image data obtained by the camera 32d (for example, HLS image data obtained by converting RGB image data, or height image data obtained by converting image data).

[0076] Next, learning data Ga to Gd are created from the acquired learning original image data Ig (see FIGS. 16 to 19). To obtain the learning data Ga to Gd, first, a solder image Ihg (see FIG. 10) is identified in the acquired learning original image data Ig. This solder image Ihg corresponds to one cream solder 5 corresponding to one land 3 and its surrounding area in the learning original image data Ig.

[0077] To identify the solder image Ihg, first, the area occupied by the cream solder 5 is identified. If the learning original image data Ig is two-dimensional data, the area occupied by the cream solder 5 is identified using, for example, brightness, hue, saturation, etc. If the learning original image data Ig is three-dimensional data, the area occupied by the cream solder 5 is identified using, for example, height information, etc. Then, the portion of the learning original image data Ig that corresponds to the area occupied by the identified cream solder 5 and its surrounding area is identified as the solder image Ihg.

[0078] Next, four extracted images Cag, Cbg, Ccg, and Cdg (hereinafter, sometimes simply referred to as "extracted images Cag to Cdg") are extracted from the identified solder image Ihg (see FIGS. 12 to 15). In this embodiment, the extracted images Cag to Cdg are areas of the solder image Ihg that are located within an extraction frame Wc that has a certain positional relationship with the outer edge of the cream solder 5, which serves as a reference.

[0079] As shown in Fig. 11, the extraction frame Wc has a rectangular shape with two parallel short sides X1, X2 and parallel central long sides Y1 and Y2 connecting the ends of these short sides X1, X2. The central long side Y1 is the long side located toward the center of the cream solder 5, and the outer long side Y2 is the long side opposite the central long side Y1. When obtaining the extracted images Cag to Cdg, the extraction frame Wc is positioned so that the central long side Y1 and the outer long side Y2 sandwich the outer edge of the cream solder 5, the outer edge of the cream solder 5 is parallel to the long sides Y1, Y2, and the outer long side Y2 is located a distance W0 away from the outer edge of the cream solder 5 in the direction of the short sides X1 and X2. Furthermore, by using the Gerber data (design data) of the printed circuit board 1, the outer edge (side) of the land 3 can be set to be parallel to the long sides Y1 and Y2, so that the outer edge of the cream solder 5 can be parallel to the long sides Y1 and Y2.

[0080] The extraction frame Wc has a constant length W along the short side directions X1 and X2 (e.g., a constant value greater than 0.1 mm), while its length L along the long side directions Y1 and Y2 is adjusted according to the width Ls of the cream solder 5. The width Ls is the maximum width of the cream solder 5. Note that the width Ls may also be the maximum width of the portion of the cream solder 5 located between the long sides Y1 and Y2. The length L is set to, for example, a value equal to or greater than the width Ls but smaller than the width Ls plus twice the tolerance L0. The tolerance L0 is set to, for example, 0.25 mm. Note that the tolerance L0 may be set to a smaller value (e.g., 0.125 mm) to prevent cream solder 5 other than the target cream solder 5 (non-target cream solder 5) from entering the extraction frame Wc.

[0081] Furthermore, when obtaining the extracted images Cag-Cdg, the extraction frame Wc is positioned so that its center along the long sides Y1-Y2 coincides with the center along the long sides Y1-Y2 of the cream solder 5 located between the long sides Y1-Y2. The distance W1 from the outer edge of the reference cream solder 5 to the center long side Y1 along the short sides X1-X2 is set to, for example, 0.1 mm or more and 0.2 mm or less so that the size of the extracted images Cag-Cdg is relatively small while the cream solder 5 is appropriately included within the extracted images Cag-Cdg. The above-mentioned distances W, W0, W1, tolerance L0, and other numerical values ​​are relative to the actual printed cream solder 5 and are merely relative to the cream solder 5 in the image. Therefore, for example, the position and size of the extraction frame Wc within the image are set based on the ratio between the actual length and the length (number of pixels) within the image.

[0082] Then, a total of four extraction frames Wc are set corresponding to one solder image Ihg, and by extracting the images within these extraction frames Wc, four extracted images Cag to Cdg are obtained that include the outer edge of the cream solder 5 and correspond to the four sides of the land 3 on which the cream solder 5 is printed (see the left and center figures in Figures 12 to 15).

[0083] Next, rotation processing is performed on the images that require rotation among the extracted images Cag to Cdg (in this embodiment, the extracted images Cbg, Ccg, and Cdg). As a result, the outer edge side of the cream solder 5 in the extracted images Cag to Cdg is set to a predetermined direction, and the long side or short side of the extracted images Cag to Cdg extends along a predetermined direction (see the right diagrams in FIGS. 13 to 15). In this embodiment, the outer edge side of the cream solder 5 in the extracted images Cag to Cdg is set to face downward, and the long side of the extracted images Cag to Cdg extends in the left-right direction.

[0084] Then, by pasting the extracted images Cag to Cdg onto the image frame W1 respectively, each learning data Ga to Gd (see FIGS. 16 to 19) formed by providing the extracted images Cag to Cdg in the image frame W1 is obtained.

[0085] The image frame W1 is a rectangular image with a height (the width in the vertical direction of the paper surface in FIGS. 16 etc.) of m (pixels) and a width (the width in the horizontal direction of the paper surface in FIGS. 16 etc.) of n (pixels). Its size (width and height) is set to be larger than the respective sizes of the extracted images Cag to Cdg based on design data etc. Here, m and n are natural numbers respectively, and in this embodiment, it is satisfied that m < n. Also, by adjusting the pasting position, each learning data Ga to Gd is made to coincide with the center or centroid of the extracted images Cag to Cdg and the center of the image frame W1.

[0086] Then, by repeatedly performing the above processes of identifying the solder image Ihg, extracting the extracted images Cag-Cdg, rotating the extracted images Cag-Cdg, and pasting the extracted images Cag-Cdg into the image frame W1, multiple pieces of learning data Ga-Gd are obtained from one learning original image data Ig. At this time, four pieces of learning data Ga-Gd are obtained for each land 3 that is rectangular in plan view and included in the learning original image data Ig. Furthermore, by using multiple learning original image data Ig, the required number of learning data Ga-Gd is finally obtained. In this embodiment, the learning data Ga-Gd includes data obtained based on two-dimensional data and data obtained based on three-dimensional data.

[0087] In step S101, once the number of pieces of learning data Ga to Gd required for learning has been acquired, in the following step S102, the learning unit 77 prepares an unlearned neural network 90 based on a command from the main control unit 71. For example, the learning unit 77 reads out the neural network 90 stored in advance in the storage unit 57 or the like. Alternatively, the neural network 90 is constructed based on network configuration information (for example, the number of layers of the neural network, the number of nodes in each layer, etc.) stored in the storage unit 57 or the like.

[0088] In this embodiment, two neural networks 90 are constructed: one for learning using learning data Ga to Gd acquired based on two-dimensional data, and the other for learning using learning data Ga to Gd acquired based on three-dimensional data. Therefore, in this embodiment, two neural networks 90 are constructed.

[0089] In step S103, reconstructed image data is acquired. That is, based on a command from the main control unit 71, the learning unit 77 provides the training data Ga-Gd acquired in step S102 as input data to the input layer of the neural network 90, thereby acquiring reconstructed image data output from the output layer of the neural network 90. ​​More specifically, the learning unit 77 provides the training data Ga-Gd acquired in step S102 that corresponds to the neural network 90 as input data to the input layer of the neural network 90, thereby acquiring reconstructed image data output from the output layer of the neural network 90. ​​For example, the learning unit 77 provides the training data Ga-Gd obtained from two-dimensional data as input data to the input layer of the neural network 90 that performs training using the training data Ga-Gd, and acquires the reconstructed image data output from the neural network 90. ​​That is, the learning unit 77 inputs appropriate training data Ga-Gd to each of the two neural networks 90 and acquires the output reconstructed image data.

[0090] In the following step S104, the learning unit 77 compares the input learning data Ga to Gd with the reconstructed image data output by the neural network 90, and determines whether the error is sufficiently small (whether it is equal to or smaller than a predetermined threshold).

[0091] If the error is sufficiently small, then in step S106, the learning unit 77 determines whether or not a learning termination condition is met. For example, if a positive determination is made in step S104 a predetermined number of times in succession without going through the processing of step S105 (described later), or if learning using all of the prepared learning data Ga to Gd is repeated a predetermined number of times, then it is determined that the termination condition is met. If the termination condition is met, then the neural network 90 and its learning information (such as updated parameters (described later)) are stored in the storage unit 57 as the AI ​​model 101, and the learning process is terminated.

[0092] In this embodiment, the AI ​​model 101 ultimately stores an AI model obtained by learning the learning data Ga to Gd acquired from two-dimensional data and an AI model obtained by learning the learning data Ga to Gd acquired from three-dimensional data.

[0093] On the other hand, if the termination condition is not met in step S106, the process returns to step S102, and the neural network 90 is trained again.

[0094] If the error is not sufficiently small in step S104, the network update process (learning of the neural network 90) is performed in step S105, and then the process returns to step S103 to repeat the above series of processes.

[0095] Specifically, in the network update process of step S105, a known learning algorithm such as backpropagation is used to update the weights (parameters) of the filters 94, 96 in the neural network 90 to more appropriate ones so that the loss function representing the difference between the training data Ga to Gd and the reconstructed image data is minimized. Note that, for example, BCE (Binary Cross-entropy) can be used as the loss function.

[0096] By repeating the processes of steps S103 to S105 many times, the neural network 90 minimizes the error between the learning data Ga to Gd and the reconstructed image data, and outputs more accurate reconstructed image data.

[0097] When image data relating to a good cream solder 5 is input, the finally obtained AI model 101 generates reconstructed image data that substantially matches the image data. Furthermore, when image data relating to a defective cream solder 5 in terms of shape, size, and the presence or absence of foreign matter is input, the AI ​​model 101 generates reconstructed image data that substantially matches the image data after correcting the shape and size of the cream solder 5 and removing noise portions (portions corresponding to foreign matter). In other words, when the cream solder 5 is defective, virtual image data relating to the cream solder 5, assuming that there are no defective portions, is generated as the reconstructed image data relating to the cream solder 5.

[0098] Next, the inspection process performed by the solder inspection device 13 will be described with reference to the flowchart of Fig. 8. This inspection process is executed for each area to be inspected on the printed circuit board 1.

[0099] When the printed circuit board 1 is carried into the solder inspection device 13 and positioned at a predetermined inspection position, inspection processing is started based on the execution of a predetermined inspection program.

[0100] When the inspection process starts, first, in step S301, an image data acquisition process is performed. In the image data acquisition process, original inspection image data Ik (see, for example, FIG. 20) related to the printed circuit board 1 to be inspected is acquired. The original inspection image data Ik is image data for obtaining inspection image data Ka, Kb, Kc, and Kd (hereinafter sometimes abbreviated as "inspection image data Ka to Kd") described later.

[0101] In this embodiment, an example of a printed circuit board 1 to be inspected is one in which a foreign matter F1 exists between the land 3 and the cream solder 5, and the foreign matter F1 protrudes from the outer edge of the cream solder 5 (see FIG. 21). This foreign matter F1 adheres to the land 3 before the cream solder 5 is printed. If a foreign matter F exists under the cream solder 5, the cream solder 5 may move unintentionally during the reflow process, which may result in defective mounting of the electronic component 25.

[0102] The original inspection image data Ik includes three-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while irradiating it with patterned light from the first illumination device 32a or the second illumination device 32b, and two-dimensional data, which is image data obtained by imaging the printed circuit board 1 with the camera 32d while irradiating it with uniform light from the third illumination device 32c. In the image data acquisition process, a three-dimensional data acquisition process and a two-dimensional data acquisition process are performed.

[0103] First, the three-dimensional data acquisition process will be described. In this process, the phase of the first pattern light irradiated from the first illumination device 32a is changed, and four imaging processes are performed under the first pattern light with different phases. Then, the phase of the second pattern light irradiated from the second illumination device 32b is changed, and four imaging processes are performed under the second pattern light with different phases, and a total of eight sets of three-dimensional data are acquired. This will be described in detail below.

[0104] As described above, when the printed circuit board 1 carried into the solder inspection device 13 is positioned and fixed at a predetermined inspection position, based on instructions from the main control unit 71, the movement mechanism control unit 76 first drives and controls the X-axis movement mechanism 32e and the Y-axis movement mechanism 32f to move the inspection unit 32, and aligns the imaging field of view (imaging range) of the camera 32d with the predetermined inspection area of ​​the printed circuit board 1.

[0105] Additionally, the illumination control unit 72 controls the switching of the liquid crystal shutters 32a2, 32b2 of the illumination devices 32a, 32b, and sets the positions of the first grating and second grating formed on the liquid crystal shutters 32a2, 32b2 to predetermined reference positions.

[0106] When the switching between the first and second gratings is complete, the illumination control unit 72 causes the first light source 32a1 of the first illumination device 32a to emit light and irradiate the first pattern light, and the camera control unit 73 drives and controls the camera 32d to perform a first imaging process under the first pattern light. The image data generated by the imaging process is continuously imported into the image acquisition unit 74 (and so on). This allows three-dimensional data of the inspection area including the multiple lands 3 and multiple cream solders 5 to be acquired.

[0107] Thereafter, upon completion of the first imaging process under the first pattern light, the illumination control unit 72 turns off the first light source 32a1 of the first illumination device 32a and executes a switching process for the first liquid crystal shutter 32a2. Specifically, the illumination control unit 72 switches the position of the first grating formed on the first liquid crystal shutter 32a2 from the reference position to a second position where the phase of the first pattern light is shifted by a quarter pitch (90°).

[0108] When the switching setting of the first grating is completed, the illumination control unit 72 causes the light source 32a1 of the first illumination device 32a to emit light to irradiate the first pattern light, and the camera control unit 73 drives and controls the camera 32d to perform a second imaging process under the first pattern light. Thereafter, by repeating the same process, four sets of three-dimensional data are acquired under the first pattern light, each with a phase difference of 90°.

[0109] Next, the lighting control unit 72 causes the second light source 32b1 of the second lighting device 32b to emit light and irradiate the second pattern light, and the camera control unit 73 drives and controls the camera 32d to perform the first imaging process under the second pattern light.

[0110] Thereafter, at the same time as the end of the first imaging process under the second pattern light, the illumination control unit 72 turns off the second light source 32b1 of the second illumination device 32b and executes a switching process for the second liquid crystal shutter 32b2. Specifically, the position of the second grating formed on the second liquid crystal shutter 32b2 is switched from the reference position to a second position where the phase of the second pattern light is shifted by a quarter pitch (90°).

[0111] When the second grating setting is complete, the illumination control unit 72 causes the light source 32b1 of the second illumination device 32b to emit light to irradiate the second pattern light, and the camera control unit 73 drives and controls the camera 32d to perform a second imaging process under the second pattern light. Thereafter, by repeating the same process, four sets of three-dimensional data are acquired under the second pattern light, each with a phase difference of 90°.

[0112] Next, the two-dimensional data acquisition process will be described. In this process, based on a command from the main control unit 71, the illumination control unit 72 causes the third illumination device 32c to emit light, irradiating a predetermined inspection area with uniform light, while the camera control unit 73 drives and controls the camera 32d to perform imaging processing under the uniform light. As a result, the predetermined inspection area on the printed circuit board 1 is imaged, and two-dimensional data related to the inspection area is acquired.

[0113] The acquired original inspection image data Ik (three-dimensional data and two-dimensional data) is stored in the storage unit 57.

[0114] The original image data for inspection Ik may be image data obtained by the camera 32d without any special processing (for example, monochrome luminance image data or RGB luminance image data), or may be image data obtained by performing predetermined processing on the image data obtained by the camera 32d (for example, HLS image data obtained by converting RGB image data, or height image data obtained by converting image data).

[0115] Next, in step S302, a test image data acquisition process is executed. In the test image data acquisition process, test image data Ka to Kd (see FIGS. 27 to 30) are acquired based on the original test image data Ik obtained in the image data acquisition process. The method for acquiring the test image data Ka to Kd is the same as the method for acquiring the learning data Ga to Gd described above.

[0116] That is, first, the solder image Ihk (see FIG. 22) in the acquired original image data for inspection Ik is identified. This solder image Ihk corresponds to the cream solder 5 corresponding to land 3 No. 1 and its surrounding area in the original image data for inspection Ik. The solder image Ihk is identified in the same way as the solder image Ihg in the original image data for learning Ig is identified.

[0117] Next, four extracted images Cak, Cbk, Cck, and Cdk (hereinafter, sometimes simply referred to as "extracted images Cak-Cdk") are extracted from the identified solder image Ihk (see the left and center diagrams of FIGS. 23-26). In this embodiment, the four extracted images Cak-Cdk are extracted from the solder image Ihk using a method similar to the method used to extract the extracted images Cag-Cdg in the learning process. That is, as the extracted images Cak-Cdk, areas of the solder image Ihk that are located within an extraction frame Wc that has a fixed positional relationship with the outer edge of the reference cream solder 5 are extracted. The extraction frame Wc is the same as that used in the learning process, and, for example, the length W along the short side directions X1 and X2 is set to a fixed value (for example, a fixed value greater than 0.1 mm). Furthermore, when obtaining the extracted images Cak to Cdk, the extraction frame Wc is positioned so that the central long side Y1 and the outer long side Y2 are positioned on either side of the outer edge of the cream solder 5, as in the learning process, and so that the outer edge of the cream solder 5 and the long sides Y1 and Y2 are parallel, and the outer long side Y2 is positioned a distance W0 away from the outer edge of the cream solder 5 in the direction of the short sides X1 and X2.

[0118] Next, rotation processing is performed on the images among the extracted images Cak to Cdk that require rotation processing (in this embodiment, extracted images Cbk, Cck, and Cdk) (see the right diagrams in each of FIGS. 24 to 26). As a result, the extracted images Cak to Cdk are set so that the outer edge side of the cream solder 5 faces a predetermined direction and the long sides or short sides extend along the predetermined direction. In this embodiment, the outer edge side of the cream solder 5 in the extracted images Cak to Cdk faces downward and the long sides of the extracted images Cak to Cdk extend in the left-right direction.

[0119] Next, the extracted images Cak-Cdk are pasted into the image frame W1, respectively, to obtain test image data Ka-Kd in which the extracted images Cak-Cdk are provided in the image frame W1 (see FIGS. 27-30). Since the image frame W1 of the test image data Ka-Kd is the same size as the image frame W1 of the training data Ga-Gd, the size (width and height) of the test image data Ka-Kd is the same as the size of the training data Ga-Gd.

[0120] Then, by repeatedly performing the above processes, such as identifying the solder image Ihk, extracting the extracted images Cak-Cdk, rotating the extracted images Cak-Cdk, and pasting the extracted images Cak-Cdk into the image frame W1, multiple test image data Ka-Kd are acquired from one original test image data Ik. At this time, four test image data Ka-Kd are acquired for each land 3 that is rectangular in plan view and included in the original test image data Ik. In this embodiment, the test image data Ka-Kd include data acquired based on two-dimensional data and data acquired based on three-dimensional data. In this embodiment, the inspection unit 78 that acquires the test image data Ka-Kd constitutes the "test image data acquisition means."

[0121] In the following step S303, a reconstructed image data acquisition step is executed. Specifically, based on a command from the main control unit 71, the inspection unit 78 inputs the test image data Ka-Kd acquired in step S302 to the input layer of the AI ​​model 101 corresponding to the type of the test image data Ka-Kd. Therefore, the test image data Ka-Kd acquired based on two-dimensional data is input to the AI ​​model 101 corresponding to the two-dimensional data, and the test image data Ka-Kd acquired based on three-dimensional data is input to the AI ​​model 101 corresponding to the three-dimensional data. Then, the image data reconstructed by the AI ​​model 101 and output from the output layer is acquired as reconstructed image data. The acquired reconstructed image data is stored in association with the test image data Ka-Kd from which the reconstructed image data was derived.

[0122] Here, when the AI ​​model 101 receives, for example, inspection image data Ka (see Figure 27) relating to cream solder 5 with foreign matter F1 attached, it outputs, as reconstructed image data S, image data relating to good cream solder 5 from which the foreign matter F1 has been removed, by learning as described above (see, for example, Figure 31).

[0123] On the other hand, when the AI ​​model 101 receives, for example, inspection image data Kd relating to a non-defective cream solder 5, it outputs, as reconstructed image data S, image data relating to the non-defective cream solder 5 that is substantially identical to the inspection image data Kd (see FIG. 32). The size (width and height) of the reconstructed image data S is the same as the size of the original inspection image data Ka to Kd. In this embodiment, the inspection unit 78 that acquires the reconstructed image data S constitutes the "reconstructed image data acquisition means."

[0124] In step S304, a pass / fail judgment process is performed based on the acquired reconstructed image data S. In the pass / fail judgment process, based on a command from the main control unit 71, the inspection unit 78 compares the entire test image data Ka-Kd acquired in step S302 with the entire reconstructed image data S acquired in step S303 using the test image data Ka-Kd, and calculates the difference between the two sets of image data Ka-Kd, S. For example, dots (pixels) at the same coordinates in the two sets of image data Ka-Kd, S are compared, and the area (number of dots) of dot clusters where the difference in brightness is equal to or greater than a predetermined value is calculated. Note that there is a relatively large difference between the test image data Ka in which the foreign substance F1 is present and the reconstructed image data S constructed from the test image data Ka. In this embodiment, the inspection unit 78, which compares the test image data Ka-Kd and the reconstructed image data S, constitutes the "comparison means."

[0125] Next, the inspection unit 78 determines whether the calculated difference is smaller than a predetermined threshold value. If the calculated difference is smaller than the predetermined threshold value, the inspection unit 78 determines the product as a "good product," whereas if the difference is larger than the predetermined threshold value, the inspection unit 78 determines the product as a "defective product."

[0126] Furthermore, the inspection unit 78 makes the above-mentioned judgment for all the test image data Ka to Kd relating to the inspection area of ​​the printed circuit board 1, and if the inspection unit 78 judges all the test image data Ka to Kd as "good", it judges the inspection area as "good" and stores this result in the storage unit 57. On the other hand, if the inspection unit 78 makes the above-mentioned judgment for all the test image data Ka to Kd relating to the inspection area and judges at least one of the test image data Ka to Kd as "defective", it judges the inspection area as "defective" and stores this result in the storage unit 57.

[0127] Then, if the above inspection process is performed on all inspected areas of the printed circuit board 1 and all inspected areas are judged to be "good," the solder inspection device 13 judges that the printed circuit board 1 has no abnormalities in the cream solder 5 (pass judgment) and stores this result in the memory unit 57.

[0128] On the other hand, if there is even one inspected area that is judged to be "defective," the solder inspection device 13 judges that the printed circuit board 1 has an abnormality in the cream solder 5 (failure judgment), stores this result in the memory unit 57, and notifies the outside world of this fact via the display unit 56, communication unit 58, etc.

[0129] As described above in detail, according to this embodiment, the test image data Ka to Kd are formed by providing the extracted images Cak to Cdk in the image frame W1. Therefore, the size (width and height) of the test image data Ka to Kd matches the size of the image frame W1 and is constant without varying slightly depending on the size of the land 3. This eliminates the need to prepare multiple different identification means (AI models) for each size of the land 3, thereby reducing the effort and time required to obtain the AI ​​model 101. Furthermore, the AI ​​model 101 can be commonly used even when the size of the land 3 is different.

[0130] Furthermore, the image frame W1 of the learning data Ga-Gd and the image frame W1 of the test image data Ka-Kd are the same size, and the learning data Ga-Gd and the test image data Ka-Kd are the same size. Therefore, when the test image data Ka-Kd are input to the AI ​​model 101, appropriate reconstructed image data S corresponding to the test image data Ka-Kd can be more reliably output, and therefore the presence or absence of an abnormality in the cream solder 5 can be more accurately determined. This makes it possible to more reliably obtain good test accuracy.

[0131] The training data Ga-Gd are each formed by placing extracted images Cag-Cdg corresponding to the four sides of the land 3 in an image frame W1. Therefore, compared to using image data relating to the entire cream solder 5 as training data, an AI model 101 with desired accuracy can be obtained with fewer training data Ga-Gd. Furthermore, since four training data Ga-Gd can be obtained from one solder image Ihg, the required number of training data Ga-Gd can be easily obtained. These combined advantages can very effectively reduce the effort and time required to obtain the AI ​​model 101.

[0132] In addition, the learning data Ga-Gd and the inspection image data Ka-Kd are each formed by placing four extracted images Cag-Cdg and Cak-Cdk corresponding to the four sides of the land 3 in the image frame W1, so the size of both data Ga-Gd and Ka-Kd can be made relatively small compared to when the image of the entire cream solder 5 is placed in the image frame W1. This reduces the processing load related to learning and inspection, and enables learning and inspection to be performed more quickly.

[0133] Furthermore, the extracted images Cag-Cdg, Cak-Cdk in the learning data Ga-Gd and the inspection image data Ka-Kd are set so that the outer edge side of the cream solder 5 (i.e., the side opposite the center of the cream solder 5) is oriented in a predetermined direction. This improves learning efficiency and reduces the effort and labor required to obtain the AI ​​model 101. Furthermore, the accuracy of the reconstructed image data S output from the AI ​​model 101 is improved, making it possible to further increase inspection accuracy.

[0134] Furthermore, the extracted images Cag-Cdg, Cak-Cdk in the training data Ga-Gd and the test image data Ka-Kd are set so that their long or short sides extend in a predetermined direction. This further improves the learning efficiency and reduces the labor and time required to obtain the AI ​​model 101. Furthermore, the accuracy of the reconstructed image data S can be further improved, thereby further improving the test accuracy.

[0135] In addition, the extraction frame Wc for the learning data Ga-Gd and the extraction frame Wc for the test image data Ka-Kd have the same length W in the short side direction, and are positioned at a fixed position relative to the reference (the outer edge of the cream solder 5) when extracting the extracted images Cag-Cdg, Cak-Cdk. Therefore, the position and range of the cream solder 5 are less likely to vary significantly among the extracted images Cag-Cdg, Cak-Cdk. This can further improve learning efficiency and further increase test accuracy.

[0136] The present invention is not limited to the above-described embodiment, and may be implemented as follows: Of course, other applications and modifications not exemplified below are also possible.

[0137] (a) In the above embodiment, the extraction frame Wc is positioned so as to have a fixed positional relationship with the outer edge of the cream solder 5, which serves as the reference. Alternatively, the extraction frame Wc may be positioned so as to have a fixed positional relationship with the outer edge of the land 3, which serves as the reference (see FIG. 33 ; the cream solder 5 is not shown in FIG. 33). In this case, the numerical range of the distance W1 may be wider than the numerical range of the distance W1 in the above embodiment (e.g., 0.1 mm≦W1≦0.3 mm), taking into account dimensional errors. Furthermore, considering that the outer edge of the land 3 may be hidden by the cream solder 5, the position of the outer edge of the land 3, which serves as the reference, may be determined using Gerber data (design data) of the printed circuit board 1. Furthermore, the distance Lp, which indicates the width of the land 3, may be used instead of the distance Ls, which indicates the width of the cream solder 5.

[0138] (b) In the above embodiment, an example is given in which the solder inspection device 13 inspects the cream solder 5 in a state in which a foreign matter F1 protrudes from the outer edge, but the solder inspection device 13 can also inspect cream solder 5 in other states. For example, the solder inspection device 13 may inspect cream solder 5 that is partially bulging and covering a foreign matter F2 on a land 3, as shown in FIG. 34, or cream solder 5 that has an abnormal size or shape.

[0139] In this case, similar to the above embodiment, the inspection image data Ka, Kb, and Kc (see FIGS. 38 to 40) are obtained by performing processes such as identifying the solder image Ihk, extracting the extracted images Cak, Cbk, and Cck using the extraction frame Wc (see the left and center diagrams of FIGS. 35 to 37), rotating the extracted images Cbk and Cck (see the right diagrams of FIGS. 35 and 36), and pasting the extracted images Cak, Cbk, and Cck into the image frame W1. Note that, for ease of explanation, only a portion of the extracted images and inspection image data, four of which are obtained for each land 3, are listed here as representative examples.

[0140] Then, the entirety of the inspection image data Ka, Kb, Kc is compared with the entirety of the reconstructed image data S obtained using the inspection image data Ka, Kb, Kc to determine whether or not there is an abnormality in the cream solder 5. When the inspection image data Kb, Kc relating to cream solder 5 having an abnormality in shape or size is input to the AI ​​model 101, image data relating to a non-defective cream solder 5 whose shape and size are determined to be normal is output as the reconstructed image data S (see FIGS. 41 and 42). When the inspection image data Ka relating to cream solder 5 covering a foreign object F2 is input to the AI ​​model 101, image data relating to a non-defective cream solder 5 whose height and shape of the portion relating to the foreign object F2 are determined to be normal is output as the reconstructed image data S (see FIG. 43).

[0141] (c) In the above embodiment, the learning data Ga to Gd are obtained using the original learning image data Ig of the printed circuit board 1 that passed the post-reflow inspection when training the neural network 90. ​​However, the learning data Ga to Gd may also be obtained using original learning image data of good cream solder 5 that has been visually selected by an operator after printing the cream solder 5, for example.

[0142] The learning unit 77 may also acquire the learning data Ga to Gd using image data of a virtually generated non-defective cream solder 5.

[0143] (d) In the above embodiment, separate AI models 101 are provided for two-dimensional data and three-dimensional data, but it is also possible to provide a common AI model that corresponds to both two-dimensional data and three-dimensional data.

[0144] (e) The configuration of the AI ​​model 101 (neural network 90) and its learning method are not limited to those described in the above embodiment. For example, the neural network 90 may be configured to perform normalization or other processing on various data as needed during the learning process or the reconstructed image data acquisition process. Furthermore, the structure of the neural network 90 is not limited to that shown in FIG. 6 , and may include, for example, a pooling layer after the convolution layer 93. Of course, the number of layers of the neural network 90, the number of nodes in each layer, and the connection structure of each node may be different.

[0145] Furthermore, in the above embodiment, the AI ​​model 101 (neural network 90) is a generative model having the structure of a convolutional autoencoder (CAE), but it is not limited to this and may be a generative model having the structure of a different type of autoencoder, such as a variational autoencoder (VAE).

[0146] Furthermore, in the above embodiment, the neural network 90 is configured to learn using the error backpropagation method, but this is not limiting, and the neural network 90 may be configured to learn using various other learning algorithms.

[0147] Additionally, neural network 90 may be configured by a dedicated AI processing circuit such as an AI chip. In this case, only learning information such as parameters may be stored in storage unit 57, and the dedicated AI processing circuit may read this information and set it in neural network 90, thereby configuring AI model 101.

[0148] In addition, in the above embodiment, the control device 33 includes a learning unit 77 and is configured to perform learning of the neural network 90 within the control device 33. However, the present invention is not limited thereto. For example, the learning unit 77 may be omitted, and the learning of the neural network 90 may be performed outside the control device 33. An AI model 101 (a learned neural network 90) that has been learned externally may be stored in the storage unit 57.

[0149] (f) In the above embodiment, two-dimensional data and three-dimensional data are acquired as the inspection original image data Ik. However, a configuration may be adopted in which only one of the two-dimensional data and the three-dimensional data is acquired. Further, in accordance with the data to be acquired, as the AI model 101, only one corresponding to one of the two-dimensional data and the three-dimensional data may be provided.

[0150] (g) In the above embodiment, the image frame W1 is rectangular and satisfies m < n. However, the image frame W1 may be square.

[0151] (h) The size and arrangement position of the extraction frame Wc may be appropriately changed. For example, by making the size of the extraction frame Wc larger, the cream solder 5 located on the center side may be included in the extraction image.

[0152] (i) When a cream solder 5 other than the target cream solder 5 (non-target cream solder 5) enters the extraction frame Wc, a process of erasing the non-target cream solder 5 from the image data may be performed.

Description of Reference Numerals

[0153] 1...printed circuit board, 3...land, 5...cream solder, 13...solder inspection device, 32d...camera (image data acquisition means), 78...inspection unit (inspection image data acquisition means, reconstructed image data acquisition means, comparison means), 101...AI model (identification means), Cag, Cbg, Ccg, Cdg, Cak, Cbk, Cck, Cdk...extracted image, Ga, Gb, Gc, Gd...learning data, Ka, Kb, Kc, Kd...inspection image data, W1...image frame, Wc...extraction frame, X1, X2...short side, Y1...center long side, Y2...outer long side.

Claims

1. A solder inspection device for inspecting abnormalities related to cream solder printed on a printed circuit board, an image data acquisition means for acquiring image data of a predetermined inspection area on the printed circuit board including cream solder printed on rectangular lands; a recognition means for generating the image data by training only image data relating to non-defective cream solders in a neural network having an encoding unit for extracting feature values ​​from input image data and a decoding unit for reconstructing image data from the feature values; an inspection image data acquisition means for acquiring inspection image data including an image of the cream solder to be inspected based on the image data acquired by the image data acquisition means; a reconstructed image data acquisition means for inputting the inspection image data to the identification means and acquiring reconstructed image data as reconstructed image data; a comparison means for comparing the inspection image data and the reconstructed image data; The apparatus is configured to be able to determine whether or not there is an abnormality in the cream solder based on the comparison result by the comparison means, The learning data is composed of four extracted images extracted from a solder image that is an image of cream solder corresponding to one land, the extracted images including the outer edge of the cream solder and corresponding to the four sides of the land, each of which is arranged in an image frame that is larger than the size of the extracted image, A solder inspection device characterized in that the inspection image data acquisition means acquires the inspection image data in which each of the four extracted images extracted from the solder image in the image data acquired by the image data acquisition means is placed in an image frame of the same size as the image frame of the learning data.

2. 2. The solder inspection device according to claim 1, wherein the learning data and the extracted image in the inspection image data are set so that the outer edge side of the cream solder is oriented in a predetermined direction.

3. 2. The solder inspection device according to claim 1, wherein the extracted images in the learning data and the inspection image data are set so that their long sides or short sides extend in a predetermined direction.

4. The extracted image is configured to extract an area within an extraction frame that has a certain positional relationship with the outer edge of the solder paste or the outer edge of the land, which is a reference, from the solder image; the extraction frame is rectangular in shape having two short sides, a central long side disposed on the center side of the cream solder, and an outer long side corresponding to the side opposite the central long side, The solder inspection device of claim 1, characterized in that the extraction frame for extracting the extracted images that constitute the learning data and the extraction frame for extracting the extracted images that constitute the inspection image data each have the same length W in the short side direction, and when extracting the extracted images, the central long side and the outer long side are positioned so as to sandwich the outer edge of the cream solder, and the outer long side is positioned at a distance W0 in the short side direction from the reference.

Images