Solder ball inspection system, image processing system, image processing method, and program

Abstract

To provide an inspection system, an image processing system, an image processing method and a program with which it is possible to improve the accuracy of detecting an object having a curved surface.SOLUTION: In an inspection system IS1, an image acquisition unit 41 acquires the data of a first captured image having been generated when first light L1 is being irradiated and the data of a second captured image having been generated when second light L2 is being irradiated. A determination unit 42 determines, on the basis of the first captured image, whether or not a solder ball B1 exists on a circuit board 9. A position detection unit 43 detects the position of a region where the solder ball B1 is seen in the first captured image as a detection position. An extraction unit 44 extracts, on the basis of the second captured image, an inspection image from the second captured image that includes the region in which the solder ball B1 is seen. An inspection unit 45 detects the size of the solder ball B1 on the basis of the inspection image and outputs the detection result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure provides: Solder ball The present invention relates to an inspection system, an image processing system, an image processing method, and a program. [Background technology]

[0002] Patent Document 1 proposes a solder ball inspection device (hereinafter referred to as the prior art) that detects solder balls on a printed circuit board. The prior art includes an imaging device and a data processing device. The imaging device obtains an image of a predetermined area on the printed circuit board. The data processing device identifies consecutive pixel groups having a predetermined brightness value from the image obtained by the imaging device. The data processing device then calculates the area of ​​each identified pixel group and calculates the circularity of pixel groups whose area is equal to or less than the predetermined value. The data processing device then determines that pixel groups whose circularity is equal to or greater than the predetermined value are solder balls. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-108850 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been a demand for improved detection accuracy when detecting objects having curved surfaces, such as solder balls.

[0005] The object of the present disclosure is to improve the detection accuracy of an object having a curved surface. Solder ball The present invention provides an inspection system, an image processing system, an image processing method, and a program. [Means for solving the problem]

[0006] According to one aspect of the present disclosure Solder ballThe inspection system includes an imaging device, a first illumination device, a second illumination device, and an image processing system. Circuit board The first illumination device irradiates the inspection object with a first light that is coaxial with the optical axis of the imaging device. The second illumination device irradiates the inspection object with a second light from a direction intersecting the optical axis. The image processing system detects an object having a curved surface in the inspection object based on the captured image. The solder balls The image processing system includes an image acquisition unit, a determination unit, a position detection unit, an extraction unit, and an inspection unit. The image acquisition unit acquires data of a first captured image, which is the captured image generated when the first light is irradiated, and data of a second captured image, which is the captured image generated when the second light is irradiated. The determination unit determines whether the object is present in the inspection target based on the first captured image. If the object is present in the inspection target, the position detection unit detects the position of an area in the first captured image in which the object is captured as a detection position. The extraction unit extracts an inspection image including an area in which the object is captured from the second captured image based on the detection position. The inspection unit detects the size of the object based on the inspection image and outputs the detection result. The inspection unit detects the size of the object by inputting the inspection image into an inspection learning model constructed by machine learning using a neural network.

[0007] An image processing system according to an aspect of the present disclosure includes: an imaging device; Circuit board Based on the captured image, an object having a curved surface in the inspection target is The solder ballsThe image processing system includes an image acquisition unit, a determination unit, a position detection unit, an extraction unit, and an inspection unit. The image acquisition unit acquires data of a first captured image, which is the captured image generated when a first light coaxial with the optical axis of the imaging device is irradiated onto the inspection object, and data of a second captured image, which is the captured image generated when a second light is irradiated onto the inspection object from a direction intersecting the optical axis. The determination unit determines whether the object is present in the inspection object based on the first captured image. If the object is present in the inspection object, the position detection unit detects the position of the area in which the object is captured in the first captured image as a detection position. The extraction unit extracts an inspection image including the area in which the object is captured from the second captured image based on the detection position. The inspection unit detects the size of the object based on the inspection image and outputs the detection result. The inspection unit inputs the inspection image into an inspection learning model constructed by machine learning using a neural network, and the size of the object determined by the inspection learning model is used as the detection result.

[0008] An image processing method according to an aspect of the present disclosure includes: Circuit board Based on the captured image, an object having a curved surface in the inspection target is The solder balls The image processing method includes an image acquisition step, a determination step, a position detection step, an extraction step, and an inspection step. The image acquisition step acquires data of a first captured image, which is the captured image generated when a first light coaxial with the optical axis of the imaging device is irradiated onto the inspection object, and data of a second captured image, which is the captured image generated when a second light is irradiated onto the inspection object from a direction intersecting the optical axis. The determination step determines whether the object is present in the inspection object based on the first captured image. If the object is present in the inspection object, the position detection step detects the position of an area in the first captured image where the object is captured as a detection position. The extraction step extracts an inspection image including an area in which the object is captured from the second captured image based on the detection position. The inspection step detects the size of the object based on the inspection image and outputs the detection result. In the inspection step, the inspection image is input into an inspection learning model constructed by machine learning using a neural network, and the size of the object determined by the inspection learning model is used as the detection result.

[0009] A program according to one aspect of the present invention causes a computer system to execute the image processing method described above. [Effects of the Invention]

[0010] As described above, the present disclosure has an effect of improving the detection accuracy of an object having a curved surface. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of an inspection system including an image processing system according to an embodiment. [Figure 2] FIG. 2 is a partially cutaway side view showing a first lighting device of the inspection system. [Figure 3] FIG. 3 is a partially cutaway side view showing a second illumination device of the inspection system. [Figure 4] FIG. 4 is a diagram showing a first captured image of the inspection system. [Figure 5] FIG. 5 is a side view showing the first light emitted by the first lighting device of the inspection system. [Figure 6] FIG. 6 is a diagram showing an extracted image obtained by extracting a part of the first captured image of the inspection system. [Figure 7] FIG. 7 is a diagram showing a second captured image of the inspection system. [Figure 8] FIG. 8 is a side view showing the second light emitted by the second illumination device of the inspection system. [Figure 9] FIG. 9 is a diagram showing an inspection image of the above inspection system. [Figure 10] FIG. 10 is a diagram showing a binarized image of the above inspection system. [Figure 11] 11A and 11B are diagrams showing a part of a first image captured by the inspection system, in which dust is visible, and a part of a second image captured by the inspection system, in which dust is visible. [Figure 12]12A and 12B are diagrams showing a part of a first image captured by the inspection system, in which the lead terminal is captured, and a part of a second image captured by the inspection system, in which the lead terminal is captured. [Figure 13] FIG. 13 is a flowchart showing the image processing method of the above embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following embodiments relate to an inspection system, an image processing system, an image processing method, and a program. More particularly, the following embodiments relate to an inspection system, an image processing system, an image processing method, and a program that detect an object having a curved surface in an inspection target based on a captured image.

[0013] The embodiment described below is merely an example of an embodiment of the present disclosure. The present disclosure is not limited to the following embodiment, and various modifications are possible depending on the design, etc., as long as the effects of the present disclosure can be achieved.

[0014] (Embodiment) (1) Overview of the inspection system The inspection system of this embodiment detects an object having a curved surface among the objects to be inspected by using image processing technology with a computer system. In this embodiment, the object to be inspected is a circuit board, and the object to be detected is a solder ball.

[0015] Conductor patterns are formed on circuit boards, and various electric and electronic components are mounted on them. Each component is mechanically fixed to the board and electrically connected to the conductor patterns by soldering it to the board. However, during the soldering process, molten solder can splash onto the board and form solder balls. If solder balls adhere to the lead terminals of electric and electronic components mounted on the board, or to the conductor patterns, they can cause defects in the circuit board. Therefore, inspection systems inspect circuit boards by detecting solder balls on the circuit board.

[0016] 1 shows a block diagram of an inspection system IS1 of this embodiment. The inspection system IS1 inspects a circuit board 9 and detects solder balls B1 on the circuit board 9 as objects to be detected. In other words, it is possible to inspect solder balls B1, which may cause defects in the circuit board 9.

[0017] Solder balls B1 are formed when molten solder splashes onto a circuit board 9 during the soldering process performed when mounting components (electrical components, electronic components, etc.) on the circuit board 9. The surface of solder ball B1 has a spherical, hemispherical, or partially spherical curved surface. In FIG. 1, the surface of solder ball B1 is shown as a spherical curved surface B11.

[0018] The inspection system IS1 includes an imaging device 3, a first lighting device 1, a second lighting device 2, and an image processing system 4. The imaging device 3 generates a captured image of a circuit board 9 that is an inspection target. The first lighting device 1 irradiates the circuit board 9 with a first light L1 that is coaxial with an optical axis OA1 of the imaging device 3. The second lighting device 2 irradiates the circuit board 9 with a second light L2 from a direction intersecting the optical axis OA1. The image processing system 4 detects solder balls B1, which are objects having curved surfaces B11, on the circuit board 9 based on the captured image.

[0019] The image processing system 4 includes an image acquisition unit 41, a determination unit 42, a position detection unit 43, an extraction unit 44, and an inspection unit 45. The image acquisition unit 41 acquires data of a first captured image, which is an image captured when the first light L1 is irradiated, and data of a second captured image, which is an image captured when the second light L2 is irradiated. The determination unit 42 determines whether or not a solder ball B1 is present on the circuit board 9 based on the first captured image. If a solder ball B1 is present on the circuit board 9, the position detection unit 43 detects the position of the area in which the solder ball B1 is captured in the first captured image as the detection position. The extraction unit 44 extracts an inspection image including the area in which the solder ball B1 is captured from the second captured image based on the detection position. The inspection unit 45 detects the size of the solder ball B1 based on the inspection image and outputs the detection result.

[0020] The inspection system IS1 and the image processing system 4 are provided with the above-described configuration, and thus can improve the detection accuracy of the solder ball B1, which is an object having a curved surface B11.

[0021] (2) Details of the inspection system As shown in FIG. 1, the inspection system IS1 includes an imaging device 3, a first illumination device 1, a second illumination device 2, and an image processing system 4.

[0022] The circuit board 9 is carried into the inspection system IS1 by a conveying device or the like (not shown). Then, after the inspection by the inspection system IS1 is completed, the circuit board 9 is carried out from the inspection system IS1 by the conveying device or the like. The circuit board 9 has a rectangular plate shape and has a first surface 91 and a second surface 92 that face each other in the thickness direction of the circuit board 9. In FIG. 1, mutually orthogonal X-axis, Y-axis, and Z-axis are defined, and the circuit board 9 carried into the inspection system IS1 is arranged so that the first surface 91 and the second surface 92 are along the XY plane (the plane defined by the X-axis and the Y-axis). In other words, the normals of the first surface 91 and the second surface 92 extend along the Z-axis. In FIG. 1, the inspection system IS1 detects solder balls B1 on the first surface 91.

[0023] (2.1) Imaging device The imaging device 3 has a lens and a plurality of imaging elements. The imaging elements are, for example, CCDs (Charged Coupled Devices) or CMOSs ​​(Complementary Metal Oxide Semiconductors). The imaging elements receive light collected by the lenses, allowing the imaging device 3 to capture a black-and-white (or color) still image. The image captured by the imaging device 3 (captured image) is a grayscale image in which the grayscale value of each pixel of the captured image is set to, for example, 256 levels (any grayscale value from 0 to 255). The imaging device 3 transmits data of the grayscale image to the image processing system 4 as data of the captured image. In this embodiment, dark pixels have low grayscale values, and bright pixels have high grayscale values.

[0024] If the optical axis of the imaging device 3 is OA1, the imaging device 3 is installed facing the first surface 91 of the circuit board 9 so that the optical axis OA1 is along the Z axis. In other words, the optical axis OA1 is parallel or approximately parallel to the Z axis.

[0025] (2.2) First lighting device As shown in FIG. 2, the first lighting device 1 is disposed between the imaging device 3 and the circuit board 9, which face each other along the Z axis. When the first lighting device 1 is turned on, the second lighting device 2 is removed from between the imaging device 3 and the circuit board 9. That is, the imaging device 3, the first lighting device 1, and the circuit board 9 are disposed in this order along the Z axis. The first lighting device 1 irradiates the circuit board 9, which is the inspection target, with first light L1 that is coaxial with the optical axis OA1 of the imaging device 3. That is, the first lighting device 1 functions as a coaxial illumination for the imaging device 3. In FIG. 1, the first light L1 is incident on the first surface 91 of the circuit board 9 from the normal direction of the first surface 91.

[0026] Specifically, as shown in Fig. 2, the first lighting device 1 includes a light source 11 and a half mirror 12. The light source 11 and the half mirror 12 face each other along the X axis.

[0027] The light source 11 includes a case 111 , a plurality of solid-state light emitting elements 112 , and a diffusion plate 113 .

[0028] The case 111 has a cylindrical shape with one end closed. The axial direction of the case 111 is along the X-axis. Inside the case 111, a plurality of solid-state light-emitting elements 112 are arranged on the bottom surface of one end of the case 111. The solid-state light-emitting elements 112 are LEDs (Light Emitting Diodes) or organic EL (Organic Electro Luminescence, OEL), etc., and emit light L100 of a predetermined light color. The light color of the light L100 is, for example, white, yellow, blue, red, or a mixture of two or more colors.

[0029] The case 111 is provided with a diffusion plate 113 fitted into an opening facing the bottom surface, and light L100 emitted by the solid-state light emitting elements 112 becomes diffused light by passing through the diffusion plate 113. The light L100 that has passed through the diffusion plate 113 travels toward the half mirror 12 along the X axis.

[0030] The half mirror 12 is disposed so that the optical axis OA1 passes through the center of the half mirror 12. Furthermore, the half mirror 12 is disposed so that it intersects with the XY plane at an angle of 45 degrees. That is, the half mirror 12 is disposed so that it intersects with the optical axis OA1 of the imaging device 3 at an angle of 45 degrees. The imaging device 3 captures an image of the first surface 91 of the circuit board 9 via the half mirror 12.

[0031] Then, when the light L100 emitted from the light source 11 reaches the half mirror 12, it is split into a first light L1 reflected by the half mirror 12 and a transmitted light L101 that passes through the half mirror 12. The first light L1 reflected by the half mirror 12 travels along the Z axis toward the circuit board 9 and is irradiated onto a first surface 91 of the circuit board 9. In other words, the first light L1 becomes light coaxial with the optical axis OA1 of the imaging device 3 and is irradiated onto the first surface 91.

[0032] The above-described first lighting device 1 is equivalent to the virtual light-emitting surface 10 shown in Fig. 1. The virtual light-emitting surface 10 faces the first surface 91 of the circuit board 9 along the Z axis, and is a virtual light source that irradiates the first surface 91 of the circuit board 9 with first light L1, which is coaxial with the optical axis OA1.

[0033] (2.3) Second lighting device 3, the second lighting device 2 is disposed between the imaging device 3 and the circuit board 9, which face each other along the Z axis. When the second lighting device 2 is turned on, the first lighting device 1 is removed from between the imaging device 3 and the circuit board 9. That is, the imaging device 3, the second lighting device 2, and the circuit board 9 are disposed in this order along the Z axis. The second lighting device 2 irradiates the circuit board 9 with the second light L2 from a direction intersecting the optical axis OA1 of the imaging device 3. That is, the second lighting device 2 irradiates the circuit board 9 with the second light L2 so that the second light L2 is obliquely incident on the first surface 91 of the circuit board 9.

[0034] Specifically, the second lighting device 2 is a ring lighting device including a case 21 and a plurality of solid-state light emitting elements 22, as shown in FIG.

[0035] The case 21 includes a circular ring-shaped case body 211, and a cylindrical insertion hole 23 is formed on the inner periphery of the case body 211. The case body 211 has a hollow structure, and houses a plurality of solid-state light-emitting elements 22 therein.

[0036] The plurality of solid-state light-emitting elements 22 are arranged in a circular ring shape inside the case body 211. The solid-state light-emitting elements 22 are LEDs, organic ELs, or the like, and emit second light L2 of a predetermined light color. The light color of the second light L2 is, for example, white, yellow, blue, red, or a mixture of two or more colors.

[0037] The second light L2 emitted from the solid-state light-emitting element 22 is emitted from the inner peripheral side surface of the case 21 (the side surface of the insertion hole 23) and travels toward the first surface 91 of the circuit board 9. At this time, the second light L2 travels in a direction intersecting the optical axis OA1 of the imaging device 3 and is irradiated onto the first surface 91 of the circuit board 9. In other words, the second light L2 is irradiated onto the first surface 91 from an oblique direction that intersects with the optical axis OA1.

[0038] The optical axis OA1 of the imaging device 3 passes through the central axis of the cylindrical insertion hole , and the imaging device 3 captures an image of the first surface 91 of the circuit board 9 through the insertion hole .

[0039] (2.4) Image Processing System As shown in FIG. 1, the image processing system 4 includes an image acquisition unit 41, a determination unit 42, a position detection unit 43, an extraction unit 44, and an inspection unit 45.

[0040] The image processing system 4 preferably includes a computer system. Specifically, in the image processing system 4, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) reads and executes programs stored in memory, thereby realizing some or all of the functions of the image processing system 4. The image processing system 4 primarily includes a processor that operates according to a program. The type of processor is not important as long as it can realize functions by executing a program. The processor may be composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or an LSI (Large Scale Integration). While ICs and LSIs are used here, the names may vary depending on the degree of integration, and may also be called system LSIs, VLSIs (Very Large Scale Integration), or ULSIs (Ultra Large Scale Integration). Field-programmable gate arrays (FPGAs), which are programmable after LSI fabrication, or reconfigurable logic devices that can reconfigure the connections within the LSI or set up circuit partitions within the LSI, can also be used for the same purpose. Multiple electronic circuits may be integrated on a single chip or on multiple chips. The plurality of chips may be arranged in a concentrated manner or in a dispersed manner.

[0041] (2.4.1) Image acquisition unit The image acquisition unit 41 acquires data of the captured image generated by the imaging device 3 by performing wired or wireless communication with the imaging device 3. In this embodiment, the captured image data is grayscale image data. The wired communication is, for example, wired communication via a twisted pair cable, a dedicated communication line, or a LAN (Local Area Network) cable. The wireless communication is, for example, wireless communication conforming to standards such as Wi-Fi (registered trademark) or low-power wireless (specified low-power wireless) that does not require a license.

[0042] Specifically, the image acquisition unit 41 acquires data of a first captured image G1 shown in FIG. 4 and data of a second captured image G2 shown in FIG.

[0043] (First captured image) 4 is a grayscale image captured by the imaging device 3 of the circuit board 9 when the first lighting device 1 is located between the imaging device 3 and the circuit board 9 and turned on, and the second lighting device 2 is removed from between the imaging device 3 and the circuit board 9 and turned off. In other words, the data of the first captured image G1 is data of a grayscale image captured by the imaging device 3 when the first light L1 is irradiated onto the circuit board 9 and the second light L2 is not irradiated onto the circuit board 9.

[0044] The grayscale value of each pixel constituting the first captured image G1 is determined by the amount of reflected light of the first light L1 on the first surface 91 of the circuit board 9. Bright pixels (pixels with high grayscale values) in the first captured image G1 correspond to areas on the first surface 91 of the circuit board 9 where a lot of reflected light of the first light L1 is present. Dark pixels (pixels with low grayscale values) in the first captured image G1 correspond to areas on the first surface 91 of the circuit board 9 where a little reflected light of the first light L1 is present. In other words, the first surface 91 of the circuit board 9 is represented by brighter pixels in the first captured image G1 as the amount of reflected light of the first light L1 increases. In FIG. 4, the area in which the solder ball B1 is photographed is designated as the target area R1.

[0045] In FIG. 5, a first light L1 is irradiated onto a solder ball B1. The first light L1 is coaxial with the optical axis OA1 of the imaging device 3 and is irradiated onto the spherical curved surface B11 of the solder ball B1 from a direction along the Z axis. Of the first light L1 irradiated onto the solder ball B1, the first light L1 irradiated near the vertex of the curved surface B11 (the point on the spherical curved surface B11 closest to the imaging device 3) is reflected near the vertex of the curved surface B11 to become reflected light L11. The reflected light L11 travels along the Z axis (optical axis OA1) toward the imaging device 3, passes through the half mirror 12, and then enters the imaging device 3. Of the first light L1 irradiated onto the ball B1, the first light L1 irradiated onto an area surrounding the vertex of the curved surface B11 (an area offset from the vertex) is reflected by the area surrounding the vertex of the curved surface B11 to become reflected light L12. The reflected light L12 travels in a direction intersecting the Z axis (optical axis OA1) rather than toward the imaging device 3. That is, the reflected light L11 is incident on the imaging device 3, but the reflected light L12 is not incident on the imaging device 3.

[0046] As a result, the target region R1, which is the region in which the solder ball B1 appears in the first captured image G1, is made up of the first bright region R11 and the first dark region R12 in Fig. 6. Fig. 6 shows an extracted image G11 obtained by extracting a portion of the first captured image G1.

[0047] The first bright region R11 is a dot-like region having pixels that are brighter than the surrounding area due to the reflected light L11. The first dark region R12 is a region having pixels that are darker than the first bright region R11 because the amount of reflected light L11 is less than that of the first bright region R11, and surrounds the first bright region R11 in a circular ring shape. In other words, the first bright region R11 corresponds to the vicinity of the vertex of the curved surface B11 of the solder ball B1, and the first dark region R12 corresponds to the region surrounding the vicinity of the vertex of the curved surface B11 of the solder ball B1.

[0048] (Second captured image) 7 is a grayscale image captured by the imaging device 3 of the circuit board 9 when the second lighting device 2 is located between the imaging device 3 and the circuit board 9 and turned on, and the first lighting device 1 is removed from between the imaging device 3 and the circuit board 9 and turned off. That is, the data of the second captured image G2 is data of a grayscale image captured by the imaging device 3 when the second light L2 is irradiated onto the circuit board 9 and the first light L1 is not irradiated onto the circuit board 9.

[0049] The grayscale value of each pixel constituting the second captured image G2 is determined by the amount of reflected light of the second light L2 on the first surface 91 of the circuit board 9. Bright pixels (pixels with high grayscale values) in the second captured image G2 correspond to areas on the first surface 91 of the circuit board 9 where a lot of reflected light of the second light L2 is present. Dark pixels (pixels with low grayscale values) in the second captured image G2 correspond to areas on the first surface 91 of the circuit board 9 where a little reflected light of the second light L2 is present. In other words, the first surface 91 of the circuit board 9 is represented by brighter pixels in the second captured image G2 as the amount of reflected light of the second light L2 increases.

[0050] 8, the second light L2 is irradiated onto the solder ball B1. The second light L2 is irradiated onto the spherical curved surface B11 of the solder ball B1 from a direction (diagonal direction) intersecting the optical axis OA1 of the imaging device 3. That is, the second light L2 is irradiated onto a hemispherical area centered on the vertex of the curved surface B11 from a direction intersecting the Z axis. The second light L2 is then reflected by the hemispherical area centered on the vertex of the curved surface B11 to become reflected light L21. The reflected light L21 travels towards the imaging device 3 and is incident on the imaging device 3.

[0051] As a result, the area in the second captured image G2 where the solder ball B1 is captured is composed of the second bright region R2 in Figures 7 and 9. Figure 9 shows an inspection image G21 obtained by extracting a portion of the second captured image G2. The second bright region R2 is a circular region having pixels that are brighter than their surroundings due to reflected light L21. In other words, the second bright region R2 corresponds to a hemispherical area centered on the vertex of the curved surface B11 of the solder ball B1, and is wider than the first bright region R11 in the first captured image G1, with approximately the same shape and size as the target region R1.

[0052] (2.4.2) Judgment part The determination unit 42 determines whether or not the solder balls B1 are present on the circuit board 9 based on the first captured image G1.

[0053] Specifically, if there is an area in the first captured image G1 where a point-like bright area is surrounded by a circular dark area, such as the first bright area R11 and the first dark area R12 (see Figure 6), then that area is likely to be an area where a solder ball B1 is captured.

[0054] Therefore, the judgment unit 42 uses a judgment learning model constructed by machine learning such as deep learning to judge whether or not there is an area in the first captured image G1 that shows the solder ball B1 (to judge whether or not the solder ball B1 is present on the circuit board 9).

[0055] The learning model for determination is preferably constructed by deep learning using FCN (Fully Convolutional Networks). The machine learning uses the first captured image and a segmented image of the first captured image as training data. The segmented image of the first captured image is an image obtained by performing semantic segmentation processing to associate labels with all pixels constituting the first captured image and divide the first captured image into multiple classes. In this embodiment, "solder ball" is included in the multiple classes divided by semantic segmentation. As a result, when the first captured image G1 is input, the learning model for determination can divide the first captured image G1 into multiple classes by performing semantic segmentation processing on the first captured image G1.

[0056] Then, if the "solder ball" class is present in the first captured image G1, the judgment learning model determines that a solder ball B1 is present on the circuit board 9 and outputs the judgment result. Also, if the "solder ball" class is not present in the first captured image G1, the judgment learning model determines that a solder ball B1 is not present on the circuit board 9 and outputs the judgment result.

[0057] Image G101 in FIG. 11A is a part of the first captured image G1, and dust is captured in the dark region R101. The dark region R101 is not a region in which a dot-like bright region is surrounded by an annular dark region, as in the first bright region R11 and the first dark region R12 (see FIG. 6). Therefore, the determination unit 42 does not determine that the dark region R101 is a region in which a solder ball B1 is captured. Note that image G201 in FIG. 11B is a part of the second captured image G2, and dust is captured in the bright region R201.

[0058] Image G102 in FIG. 12A is a part of the first captured image G1, and a lead terminal (metal terminal) is captured in region R102. Region R102 has a dark region R102a representing the shape of the lead terminal, and two flat bright regions R102b exist within dark region R102a. Bright region R102b is not a region in which a dot-like bright region is surrounded by an annular dark region, as in the first bright region R11 and the first dark region R12 (see FIG. 6). Therefore, the determination unit 42 does not determine that bright region R102b is a region in which a solder ball B1 is captured. Image G202 in FIG. 12B is a part of the second captured image G2, and a lead terminal is captured in region R202. Region R202 has an irregularly shaped dark region R202a and an irregularly shaped bright region R202b.

[0059] The machine learning system that constructs the learning model for determination may be either a configuration provided in the inspection system IS1 or a configuration provided outside the inspection system IS1.

[0060] (2.4.3) Position detection unit If a solder ball B1 is present on the circuit board 9, the position detection unit 43 detects the position of the area in which the solder ball B1 is captured in the first captured image G1 as the detected position.

[0061] Specifically, when the determination unit 42 determines that a solder ball B1 is present on the circuit board 9, the position detection unit 43 obtains the coordinates (detection coordinates) of the target region R1 in the first captured image G1 as the detection position. The target region R1 in the first captured image G1 is made up of a first bright region R11 and a first dark region R12 (see FIG. 6), and the detection coordinates are represented by the coordinates of the center of the first bright region R11. The detection coordinates are represented in an image coordinate system set in the first captured image G1.

[0062] (2.4.4) Extraction part The extraction unit 44 extracts an inspection image G21 (see FIG. 9) including an area in which the solder ball B1 is captured from the second captured image G2 based on the detected position. At this time, it is preferable that the extraction unit 44 extracts an area of ​​a predetermined size including the area in which the solder ball B1 is captured from the second captured image G2 as the inspection image G21.

[0063] Specifically, when the position detection unit 43 detects detection coordinates, which are the coordinates of the target region R1, from the first captured image G1, the extraction unit 44 applies the detection coordinates to the second captured image G2. Because the image coordinate system of the first captured image G1 and the image coordinate system of the second captured image G2 are common, the detection coordinates in the second captured image G2 include a second bright region R2 (see FIGS. 7 and 9), which is an area where the solder ball B1 is captured.

[0064] Therefore, the extraction unit 44 extracts a rectangular area centered on the detection coordinates (corresponding not only to the target area R1 but also to the coordinates of the second bright area R2) from the second captured image G2 as the inspection image G21. The vertical and horizontal sizes of the rectangular inspection image G21 are predetermined, and are set so that the entire second bright area R2 (the area in which the solder ball B1 is captured) is included in the inspection image G21.

[0065] The extraction unit 44 sets the size of the inspection image G21 according to the resolution of the second captured image G2, thereby improving the detection accuracy of the second bright region R2 in the inspection image G21 by the inspection unit 45, which will be described later. Generally, the smaller the size of the inspection image G21, the higher the detection accuracy by the inspection unit 45.

[0066] (2.4.5) Inspection Department The inspection unit 45 detects the size of the solder ball B1 based on the inspection image G21 and outputs the detection result.

[0067] Specifically, the inspection unit 45 performs a binarization process on the inspection image G21, which is a grayscale image, to generate the binarized image G3 shown in FIG. 10. The inspection unit 45 compares the grayscale value of each pixel in the inspection image G21 with a binarization threshold, and classifies pixels whose grayscale values ​​are equal to or greater than the binarization threshold as bright pixels (grayscale value = 255) and pixels whose grayscale values ​​are less than the binarization threshold as dark pixels (grayscale value = 0). The inspection unit 45 preferably uses a predetermined constant value or a value calculated from the grayscale values ​​of each pixel in the inspection image G21 as the binarization threshold. The binarized image G3 includes an inspection region R31 consisting of a plurality of bright pixels corresponding to the second bright region R2, and a third dark region R32 consisting of a plurality of dark pixels corresponding to the region other than the second bright region R2. The inspection region R31 is surrounded by the third dark region R32.

[0068] The inspection unit 45 then detects the size of the solder ball B1 based on the size of the inspection area R31. The inspection area R31 is substantially circular, and the inspection unit 45 can determine the diameter of the solder ball B1 based on the number of pixels corresponding to the diameter D1 of the inspection area R31. The inspection unit 45 may also determine the diameter of the solder ball B1 at multiple locations in the inspection area R31, and then determine the diameter of the solder ball B1 based on the maximum, minimum, or average of the multiple diameters.

[0069] Then, the inspection unit 45 outputs data on the diameter of the solder ball B1 as the detection result to the host system HS1.

[0070] Furthermore, when the determining unit 42 determines that there are no solder balls B1 on the circuit board 9, the inspecting unit 45 outputs the absence of solder balls B1 as a detection result to the upper system HS1.

[0071] (2.5) Upper System The host system HS1 is a system that manages the manufacturing process of the circuit board 9.

[0072] The host system HS1 receives the detection results from the inspection unit 45. When the host system HS1 receives data on the diameter of the solder ball B1 as the detection result, it compares the diameter of the solder ball B1 with a predetermined inspection threshold. In this case, if the diameter of the solder ball B1 is equal to or greater than the inspection threshold, the host system HS1 determines the circuit board 9 to be defective, and if the diameter of the solder ball B1 is less than the inspection threshold, the host system HS1 determines the circuit board 9 to be pass.

[0073] Even if the host system HS1 receives a detection result indicating that there are no solder balls B1, it still determines that the circuit board 9 is good.

[0074] The host system HS1 then removes the circuit boards 9 that it has determined to be good from the inspection system IS1 using a conveying device or the like, and sends them to a subsequent process. The host system HS1 then removes the circuit boards 9 that it has determined to be defective from the inspection system IS1 using a conveying device or the like, and sends them to a rework process in which solder balls B1 are removed from the circuit boards 9.

[0075] (2.6) Advantages As described above, the inspection system IS1 detects the size of the solder balls B1 present on the circuit board 9 based on the first captured image G1 captured using the first light L1 emitted by the first lighting device 1 and the second captured image G2 captured using the second light L2 emitted by the second lighting device 2. Such an inspection system IS1 can accurately determine the presence and position of solder balls B1 having a curved surface (particularly, a spherical curved surface B11) based on the first captured image G1. Furthermore, the inspection system IS1 can accurately detect the size of solder balls B1 having a curved surface (particularly, a spherical curved surface B11) based on the second captured image G2. That is, the inspection system IS1 can improve the detection accuracy of solder balls B1, which are objects having a curved surface B11.

[0076] (3) First Modification The inspection unit 45 may detect the size of the solder ball B1 by inputting the inspection image G21 into an inspection learning model constructed by machine learning using a neural network. By using the inspection learning model, the inspection unit 45 can improve the detection accuracy of the solder ball B1.

[0077] For example, the inspection learning model generates a segmentation image from the inspection image G21 and detects the size of the solder ball B1 based on the segmentation image. By using the segmentation image, the inspection unit 45 can improve the detection accuracy of the solder ball B1.

[0078] Specifically, the inspection learning model is preferably constructed by deep learning using FCN. Machine learning uses a large number of teacher images of the circuit board 9 and segmentation images of each teacher image as training data. The size of the teacher images is the same as the size of the inspection image G21. The segmentation image of the teacher image is an image obtained by performing semantic segmentation processing to associate a label with every pixel constituting the teacher image and divide the teacher image into multiple classes. In this embodiment, "solder ball" is included in the multiple classes divided by semantic segmentation. As a result, when the inspection learning model receives the inspection image G21, it can perform semantic segmentation processing on the inspection image G21 to generate a segmentation inspection image in which the inspection image G21 is divided into multiple classes.

[0079] The inspection learning model extracts regions of the "solder ball" class from the segmentation inspection image and detects the size (e.g., diameter) of the regions of the "solder ball" class.

[0080] The inspection unit 45 outputs the size of the area of ​​the "solder ball" class detected by the inspection learning model to the upper system HS1 as the detection result of the size of the solder ball B1.

[0081] The machine learning system that constructs the learning model for inspection may be either a configuration provided in the inspection system IS1 or a configuration provided outside the inspection system IS1.

[0082] (4) Second Modification The inspection unit 45 may detect the size of the solder ball B1 by inputting the inspection image G21 into an inspection learning model constructed by machine learning using a neural network.

[0083] The inspection learning model recognizes the area in the inspection image G21 where the solder ball B1 appears, and the inspection unit 45 detects the size of the solder ball B1 based on the recognition result of the inspection learning model.

[0084] Specifically, as in the first variant, when a test image G21 is input, the test learning model performs semantic segmentation processing on the test image G21 to generate a segmented test image that divides the test image G21 into multiple classes.

[0085] In the second modified example, unlike the first modified example, the inspection unit 45 extracts a region of the "solder ball" class from the segmentation inspection image and detects the size (e.g., diameter) of the region of the "solder ball" class. That is, in the first modified example, the inspection learning model detects the size of the region of the "solder ball" class, whereas in the second modified example, the inspection unit 45 detects the size of the region of the "solder ball" class by applying image processing to the segmentation inspection image.

[0086] The inspection unit 45 outputs the size of the area of ​​the "solder ball" class to the host system HS1 as the result of detecting the size of the solder ball B1.

[0087] The machine learning system that constructs the learning model for inspection may be either a configuration provided in the inspection system IS1 or a configuration provided outside the inspection system IS1.

[0088] (5) Third Modification The inspection unit 45 may determine the size of the solder ball B1 after performing edge extraction processing on the inspection image G21.

[0089] Specifically, the inspection unit 45 extracts edges from the inspection image G21 by performing at least one process such as binarization, differential filtering, and Sobel filtering on the inspection image G21. Then, the inspection unit 45 outputs the size (e.g., diameter) of a circular or nearly circular area surrounded by the edges to the upper system HS1 as the detection result of the size of the solder ball B1.

[0090] (6) Fourth Modification The inspection unit 45 may compare the diameter of the solder balls B1 obtained as described above with a predetermined inspection threshold. In this case, the inspection unit 45 determines the circuit board 9 as defective if the diameter of the solder balls B1 is equal to or greater than the inspection threshold, and determines the circuit board 9 as pass if the diameter of the solder balls B1 is less than the inspection threshold. The inspection unit 45 also determines the circuit board 9 as pass when no solder balls B1 are detected.

[0091] The inspection unit 45 then outputs not only the result of detecting the size of the solder balls B1 but also the result of determining whether the circuit board 9 is good or bad to the host system HS1.

[0092] (7) Fifth Modification The above-mentioned learning model for determination and learning model for inspection are not limited to learning models constructed by deep learning using FCN. The learning model for determination and learning model for inspection may be constructed using, for example, a convolutional neural network (CNN). Furthermore, the learning model for determination and learning model for inspection may use other algorithms such as a support vector machine.

[0093] (8) Inspection method The image processing method executed by the image processing system 4 described above is shown in the flowchart of FIG.

[0094] The inspection method is an image processing method for detecting solder balls B1 on the circuit board 9 based on an image of the circuit board 9 captured by the imaging device 3. The inspection method includes an image acquisition step S1, a determination step S2, a position detection step S3, an extraction step S4, and an inspection step S5.

[0095] In the image acquisition step S1, the image acquisition unit 41 acquires data of a first captured image G1 and data of a second captured image G2. The first captured image G1 is an image of the circuit board 9 captured by the imaging device 3 when the first lighting device 1 is located between the imaging device 3 and the circuit board 9 and turned on, and the second lighting device 2 is removed from between the imaging device 3 and the circuit board 9 and turned off. The second captured image G2 is an image of the circuit board 9 captured by the imaging device 3 when the second lighting device 2 is located between the imaging device 3 and the circuit board 9 and turned on, and the first lighting device 1 is removed from between the imaging device 3 and the circuit board 9 and turned off. That is, the first captured image G1 is an image generated when the circuit board 9 is irradiated with a first light L1 that is coaxial with the optical axis OA1 of the imaging device 3. The second captured image G2 is an image generated when the circuit board 9 is irradiated with a second light L2 from a direction intersecting the optical axis OA1.

[0096] In the determination step S2, the determination unit 42 determines whether or not the solder balls B1 are present on the circuit board 9 based on the first captured image G1.

[0097] In the position detection step S3, if the solder ball B1 is present on the circuit board 9, the position detection unit 43 detects the position of the area in the first captured image G1 where the solder ball B1 is shown as the detected position.

[0098] In the extraction step S4, the extraction unit 44 extracts an inspection image G21 including an area in which the solder ball B1 is captured from the second captured image G2 based on the detected position.

[0099] In the inspection step S5, the inspection unit 45 detects the size of the solder ball B1 based on the inspection image G21 and outputs the detection result.

[0100] By including the above-described steps, the image processing method can improve the detection accuracy of the solder ball B1, which is an object having a curved surface B11.

[0101] (9) Other variations The object detected by the image processing system 4 is not limited to the solder ball B1, but may be a steel ball, a resin ball, etc. In other words, the object detected by the image processing system 4 may be any object having a curved surface, and is preferably an object having a spherical or hemispherical curved surface.

[0102] The inspection unit 45 is not limited to a configuration that detects the diameter of an object as the size of an object such as a solder ball B1. The inspection unit 45 may also detect the area, length, or width of the object as the size of the object.

[0103] The inspection system IS1 may be configured such that both the first lighting device 1 and the second lighting device 2 are positioned between the imaging device 3 and the circuit board 9, and may be capable of capturing the first captured image G1 and the second captured image G2.

[0104] The functions of the image processing system 4 may be distributed among a plurality of devices, and the plurality of devices may constitute the image processing system 4. For example, the image processing system 4 may be realized by cloud computing technology.

[0105] Furthermore, the image processing system 4 may be configured as a single device such as a personal computer.

[0106] (10) Summary An inspection system (IS1) according to a first aspect of the embodiment includes an imaging device (3), a first illumination device (1), a second illumination device (2), and an image processing system (4). The imaging device (3) generates a captured image of an inspection object (9). The first illumination device (1) irradiates the inspection object (9) with a first light (L1) that is coaxial with an optical axis (OA1) of the imaging device (3). The second illumination device (2) irradiates the inspection object (9) with a second light (L2) from a direction intersecting the optical axis (OA1). The image processing system (4) detects an object (B1) having a curved surface (B11) in the inspection object (9) based on the captured image. The image processing system (4) includes an image acquisition unit (41), a determination unit (42), a position detection unit (43), an extraction unit (44), and an inspection unit (45). The image acquisition unit (41) acquires data of a first captured image (G1), which is a captured image generated when the first light (L1) is irradiated, and data of a second captured image (G2), which is a captured image generated when the second light (L2) is irradiated. The determination unit (42) determines whether an object (B1) is present in the inspection target (9) based on the first captured image (G1). If the object (B1) is present in the inspection target (9), the position detection unit (43) detects the position of an area in the first captured image (G1) where the object (B1) is captured as a detection position. The extraction unit (44) extracts an inspection image (G21) including an area in which the object (B1) is captured from the second captured image (G2) based on the detection position. The inspection unit (45) detects the size of the object (B1) based on the inspection image (G21) and outputs the detection result.

[0107] The above-described inspection system (IS1) can improve the detection accuracy of an object (B1) having a curved surface (B11).

[0108] In the inspection system (IS1) of the second aspect of the embodiment, in the first aspect, it is preferable that the extraction unit (44) extracts an area of ​​a predetermined size including an area in which the object (B1) is captured from the second captured image (G2) as the inspection image (G21).

[0109] The inspection system (IS1) described above can improve the detection accuracy of the inspection unit (45) by setting the size of the inspection image (G21) according to the resolution of the second captured image (G2).

[0110] In the inspection system (IS1) of the third aspect of the embodiment, in the first or second aspect, the inspection unit (45) preferably detects the size of the object (B1) by inputting the inspection image (G21) into an inspection learning model constructed by machine learning using a neural network.

[0111] The inspection system (IS1) described above can improve the detection accuracy of the object (B1) by the inspection unit (45) by using the inspection learning model.

[0112] In the inspection system (IS1) of the fourth aspect of the embodiment, in the third aspect, it is preferable that the inspection learning model generates a segmentation image from the inspection image (G21) and detects the size of the object (B1) based on the segmentation image.

[0113] The inspection system (IS1) described above can improve the detection accuracy of the solder balls (B1) by the inspection unit (45) by using a segmentation image.

[0114] In the inspection system (IS1) of the fifth aspect of the embodiment, in the third aspect, it is preferable that the inspection learning model recognizes the area in the inspection image (G21) where the object (B1) is captured, and the inspection unit (45) detects the size of the object (B1) based on the recognition result of the inspection learning model.

[0115] The inspection system (IS1) described above can improve the detection accuracy of the object (B1) by the inspection unit (45) by using the inspection learning model.

[0116] In the inspection system (IS1) of the sixth aspect of the embodiment, in the first or second aspect, it is preferable that the inspection unit (45) determines the size of the object (B1) after performing edge extraction processing on the inspection image (G21).

[0117] The inspection system (IS1) described above can improve the detection accuracy of the object (B1) by the inspection unit (45) by performing edge extraction processing.

[0118] In the inspection system (IS1) of the seventh aspect according to the embodiment, in any one of the first to sixth aspects, it is preferable that the inspection object is a circuit board (9) and the object is a solder ball (B1).

[0119] The above-mentioned inspection system (IS1) can inspect the solder balls (B1) that are the cause of defects in the circuit board (9).

[0120] An image processing system (4) according to an eighth aspect of the embodiment detects an object (B1) having a curved surface (B11) in an inspection object (9) based on a captured image of the inspection object (9) captured by an imaging device (3). The image processing system (4) includes an image acquisition unit (41), a determination unit (42), a position detection unit (43), an extraction unit (44), and an inspection unit (45). The image acquisition unit (41) acquires data on a first captured image (G1), which is a captured image generated when the inspection object (9) is irradiated with a first light (L1) coaxial with an optical axis (OA1) of the imaging device (3), and data on a second captured image (G2), which is a captured image generated when the inspection object (9) is irradiated with a second light (L2) from a direction intersecting the optical axis (OA1). The determination unit (42) determines whether the object (B1) is present in the inspection object (9) based on the first captured image (G1). If an object (B1) is present in the inspection target (9), the position detection unit (43) detects the position of an area in the first captured image (G1) where the object (B1) is captured as a detected position. The extraction unit (44) extracts an inspection image (G21) including an area in which the object (B1) is captured from the second captured image (G2) based on the detected position. The inspection unit (45) detects the size of the object (B1) based on the inspection image (G21) and outputs the detection result.

[0121] The image processing system (4) described above can improve the detection accuracy of an object (B1) having a curved surface (B11).

[0122] In the image processing system (4) of the ninth aspect of the embodiment, in the eighth aspect, it is preferable that the extraction unit (44) extracts an area of ​​a predetermined size including an area in which the object (B1) is captured from the second captured image (G2) as the inspection image (G21).

[0123] The image processing system (4) described above can improve the detection accuracy of the inspection unit (45) by setting the size of the inspection image (G21) according to the resolution of the second captured image (G2).

[0124] In the image processing system (4) of the tenth aspect of the embodiment, in the eighth or ninth aspect, the inspection unit (45) preferably inputs the inspection image (G21) to an inspection learning model constructed by machine learning using a neural network, and sets the size of the object (B1) determined by the inspection learning model as the detection result.

[0125] The image processing system (4) described above can improve the accuracy of detecting the object (B1) by the inspection unit (45) by using the inspection learning model.

[0126] In the image processing system (4) of the eleventh aspect of the embodiment, in the tenth aspect, it is preferable that the inspection learning model generates a segmentation image from the inspection image (G21) and determines the size of the object (B1) based on the segmentation image.

[0127] The image processing system (4) described above can improve the detection accuracy of the solder balls (B1) by the inspection unit (45) by using the segmentation image.

[0128] In the image processing system (4) of the 12th aspect of the embodiment, in the 10th aspect, it is preferable that the inspection learning model recognizes the area in the inspection image (G21) where the object (B1) is captured, and determines the size of the object (B1) based on the recognition result.

[0129] The image processing system (4) described above can improve the accuracy of detecting the object (B1) by the inspection unit (45) by using the inspection learning model.

[0130] In the image processing system (4) of the thirteenth aspect of the embodiment, in the eighth or ninth aspect, it is preferable that the inspection unit (45) determines the size of the object (B1) after performing edge extraction processing on the inspection image (G21).

[0131] The image processing system (4) described above can improve the accuracy of detecting the object (B1) by the inspection unit (45) by performing edge extraction processing.

[0132] In the image processing system (4) of the 14th aspect of the embodiment, in any one of the 8th to 13th aspects, it is preferable that the object to be inspected is a circuit board (9) and the object is a solder ball (B1).

[0133] The image processing system (4) described above can inspect the solder balls (B1) that can cause defects in the circuit board (9).

[0134] An image processing method according to a fifteenth aspect of the present embodiment detects an object (B1) having a curved surface (B11) in an inspection object (9) based on a captured image of the inspection object (9) captured by an imaging device (3). The image processing method includes an image acquisition step (S1), a determination step (S2), a position detection step (S3), an extraction step (S4), and an inspection step (S5). The image acquisition step (S1) acquires data on a first captured image (G1), which is an image captured when a first light (L1) coaxial with an optical axis (OA1) of the imaging device (3) is irradiated onto the inspection object (9), and data on a second captured image (G2), which is an image captured when a second light (L2) is irradiated onto the inspection object (9) from a direction intersecting the optical axis (OA1). The determination step (S2) determines whether the object (B1) is present in the inspection object (9) based on the first captured image (G1). In the position detection step (S3), if an object (B1) is present in the inspection target (9), the position of the area in which the object (B1) appears in the first captured image (G1) is detected as a detection position. In the extraction step (S4), an inspection image (G21) including the area in which the object (B1) appears is extracted from the second captured image (G2) based on the detection position. In the inspection step (S5), the size of the object (B1) is detected based on the inspection image (G21) and the detection result is output.

[0135] The image processing method described above can improve the accuracy of detecting an object (B1) having a curved surface (B11).

[0136] A program according to a sixteenth aspect of the present embodiment causes a computer system to execute the image processing method according to the fifteenth aspect.

[0137] The above program can improve the detection accuracy of an object (B1) having a curved surface (B11). [Explanation of symbols]

[0138] IS1 Inspection System 1. First lighting device 2. Second lighting device 3. Imaging device 4. Image Processing System 41 Image acquisition unit 42 Judgment section 43 Position detection unit 44 Extraction part 45 Inspection Department 9 Circuit board (test object) OA1 optical axis L1 1st light L2 2nd light B1 Solder ball (object) B11 Curved surface G1 First captured image G2 Second captured image G21 Inspection Image S1 Image acquisition step S2 Judgment step S3 Position detection step S4 Extraction step S5 Inspection step

Claims

1. an imaging device that generates an image of a circuit board to be inspected; a first illumination device that irradiates the inspection object with first light that is coaxial with an optical axis of the imaging device; a second illumination device that irradiates the inspection object with second light from a direction intersecting the optical axis; an image processing system that detects solder balls, which are objects having curved surfaces, in the inspection object based on the captured image; The image processing system includes: an image acquisition unit that acquires data of a first captured image that is the captured image generated when the first light is irradiated and data of a second captured image that is the captured image generated when the second light is irradiated; a determination unit that determines whether the object is present in the inspection target based on the first captured image; a position detection unit that detects, if the object is present in the inspection target, a position of an area in which the object is captured in the first captured image as a detection position; an extracting unit that extracts an inspection image including an area in which the object is captured from the second captured image based on the detected position; an inspection unit that detects the size of the object based on the inspection image and outputs the detection result; The inspection unit detects the size of the object by inputting the inspection image into an inspection learning model constructed by machine learning using a neural network. Solder ball inspection system.

2. The extraction unit extracts, from the second captured image, an area of ​​a predetermined size including an area in which the object is captured, as the inspection image. The solder ball inspection system of claim 1 .

3. The inspection learning model generates a segmentation image from the inspection image and detects the size of the object based on the segmentation image.

3. The solder ball inspection system according to claim 1 or 2.

4. The inspection learning model recognizes an area in the inspection image in which the object is captured, The inspection unit detects the size of the object based on the recognition result of the inspection learning model.

3. The solder ball inspection system according to claim 1 or 2.

5. An image processing system for detecting solder balls, which are objects having curved surfaces, on a circuit board that is an object to be inspected, based on an image captured by an imaging device of the circuit board, comprising: an image acquisition unit that acquires data of a first captured image, which is the captured image generated when a first light that is coaxial with an optical axis of the imaging device is irradiated onto the inspection object, and data of a second captured image, which is the captured image generated when a second light is irradiated onto the inspection object from a direction intersecting the optical axis; a determination unit that determines whether the object is present in the inspection target based on the first captured image; a position detection unit that detects, if the object is present in the inspection target, a position of an area in which the object is captured in the first captured image as a detection position; an extracting unit that extracts an inspection image including an area in which the object is captured from the second captured image based on the detected position; an inspection unit that detects the size of the object based on the inspection image and outputs the detection result; The inspection unit inputs the inspection image into an inspection learning model constructed by machine learning using a neural network, and sets the size of the object determined by the inspection learning model as the detection result. Image processing system.

6. The extraction unit extracts an area of ​​a predetermined size including an area in which the object is captured from the second captured image as the inspection image. The image processing system of claim 5.

7. The inspection learning model generates a segmentation image from the inspection image and determines the size of the object based on the segmentation image.

7. The image processing system according to claim 5 or 6.

8. The inspection learning model recognizes the area in the inspection image where the object is captured, and determines the size of the object based on the recognition result.

7. The image processing system according to claim 5 or 6.

9. An image processing method for detecting solder balls, which are objects having curved surfaces, on an inspection target circuit board based on an image captured by an imaging device of the inspection target circuit board, comprising: an image acquiring step of acquiring data of a first captured image, which is the captured image generated when a first light that is coaxial with an optical axis of the imaging device is irradiated onto the inspection object, and data of a second captured image, which is the captured image generated when a second light is irradiated onto the inspection object from a direction intersecting the optical axis; a determining step of determining whether or not the object is present in the inspection target based on the first captured image; a position detection step of detecting, as a detection position, a position of an area in which the object is captured in the first captured image if the object is present in the inspection target; an extraction step of extracting an inspection image including an area in which the object is captured from the second captured image based on the detected position; an inspection step of detecting the size of the object based on the inspection image and outputting a detection result; In the inspection step, the inspection image is input to an inspection learning model constructed by machine learning using a neural network, and the size of the object determined by the inspection learning model is used as the detection result. Image processing methods.

10. A program that causes a computer system to execute the image processing method described in claim 9.

Images