Inspection device

Abstract

Provided is an inspection device in which determination accuracy is improved by combining a plurality of inspection methods.　An inspection device 10 includes an imaging unit 20 that images an object 12 being inspected and outputs image data, and a control unit 30 that performs inspection on the basis of the image data. At least one inspection region from among inspection regions set in the object 12 being inspected has a first inspection method and a second inspection method set thereto. The control unit 30 executes: a first step for selecting first inspection information on the basis of information set in the inspection region; a second step for applying the first inspection method to the image data for the inspection region and determining, on the basis of the first inspection information, whether the image data is satisfactory or unsatisfactory; a third step for selecting one or more items of second inspection information on the basis of the first inspection information and an "unsatisfactory" classification; and a fourth step for applying the second inspection method to the image data for the inspection region and determining, on the basis of the second inspection information, whether the image data is satisfactory or unsatisfactory.

Description

Inspection Equipment

[0001] The present invention relates to an inspection device.

[0002] Inspection devices for inspecting electronic circuit boards (hereinafter simply referred to as "boards") generally perform inspections using pre-set inspection information (inspection rules) on image data obtained by capturing an image of the board (this inspection method and inspection are referred to as a "rule-based inspection method" or "rule-based inspection"). This inspection information is created and managed for each type of board (see, for example, Patent Document 1).

[0003] When conducting rigorous inspections using rule-based inspection methods, strict inspection conditions must be set based on the inspection information. However, due to the wide variation in the shapes and brightness of components, solder, etc. that occurs during the mass production process of PCBs, numerous inspection conditions and logical expressions are often used, and setting this inspection information requires advanced skills. In rule-based inspections, if the inspection conditions are not set appropriately, there will be an increase in overdetection, in which boards that should have been judged as good are judged as defective. Boards judged as defective by the inspection system are then re-evaluated by a visual inspector, but the increase in overdetection increases the workload of the visual inspector. As a result, the visual inspector's attention will decrease, and defective products will be judged as good, leading to the outflow of defective boards.

[0004] Meanwhile, as a method for addressing the above-mentioned issues in rule-based inspection, a technique has been proposed in which an AI-based inspection method is applied to image data obtained by imaging a board. This AI-based inspection method creates an inference model using image data of good and bad boards acquired by an inspection device, and then inspects the board using this inference model. While inspections using AI-based inspection methods (hereinafter referred to as "AI-based inspections") are effective in reducing the number of defective product identifications, variations in the mass production of boards can lead to sudden overdetection of untrained good components. Furthermore, since performing AI-based inspections requires creating inference models for each inspection item and target component, the process of registering inference models for target components becomes complicated and time-consuming when there are many inspection targets.

[0005] JP 2013-174450 A

[0006] As described above, both rule-based inspection methods and inspection methods using AI have the potential for overdetection, which poses the problem of not being able to reduce the assessment work of visual assessors.

[0007] The present invention has been made in consideration of these problems, and aims to provide an inspection device that improves judgment accuracy by combining multiple inspection methods, such as rule-based inspection methods and inspection methods using AI, and determining whether an inspection area set on a board is good or bad using one inspection method.If an inspection area is determined to be bad, the other inspection method is then used to again determine whether the inspection area is good or bad.

[0008] In order to solve the above problem, an inspection apparatus according to the present invention includes an imaging unit that images an object to be inspected and outputs image data, and a control unit that performs an inspection based on the image data output from the imaging unit, wherein a first inspection method and a second inspection method are set for at least one of the inspection areas set on the object to be inspected, and the control unit executes the following steps when selecting the inspection area: a first step of selecting first inspection information based on the information set for the inspection area; a second step of applying the first inspection method to the image data of the inspection area based on the first inspection information selected in the first step to determine whether the object is good or bad, and outputting a defect type if the object is determined to be bad; a third step of selecting one or more second inspection information based on the first inspection information and the defect type when the object is determined to be bad in the second step; and a fourth step of applying the second inspection method to the image data of the inspection area based on the second inspection information selected in the third step to determine whether the object is good or bad.

[0009] In the inspection device of the present invention, in the fourth step, if the inspection results for each inspection area using the second inspection method based on one or more pieces of second inspection information are all good, it is desirable that the control unit judges the inspection area to be good even if the inspection results using the first inspection method are bad.

[0010] Furthermore, in the inspection device according to the present invention, it is desirable that the first inspection information is an inspection rule for determining whether the image data is good or bad, the first inspection method is a rule-based inspection method for performing inspection based on the inspection rule, the second inspection information is an inference model obtained by prior learning, and the second inspection method is an inspection method using AI for determining whether the image data is good or bad based on the inference model.

[0011] Furthermore, in the inspection device according to the present invention, it is desirable that the first inspection information is an inference model obtained by prior learning, the first inspection method is an inspection method using AI that judges pass / fail from the image data based on the inference model, the second inspection information is an inspection rule that judges pass / fail from the image data, and the second inspection method is a rule-based inspection method that performs inspection based on the inspection rule.

[0012] In addition, in the inspection device according to the present invention, it is desirable that the control unit outputs, for each inspection area, a pass / fail judgment result obtained by the first inspection method and a pass / fail judgment result obtained by the second inspection method in association with each other.

[0013] Furthermore, in the inspection device according to the present invention, it is desirable that the control unit outputs, for each piece of information set in the inspection area, the proportion of pass / fail judgment results obtained by the first inspection method and the proportion of pass / fail judgment results obtained by the second inspection method in correspondence with each other.

[0014] According to the present invention, an inspection device with improved judgment accuracy can be provided by combining multiple inspection methods, such as a rule-based inspection method and an inspection method using AI, and when a pass / fail judgment is made by one inspection method in an inspection area set on an object to be inspected (substrate), the other inspection method is executed on the inspection area that is judged to be faulty, and the pass / fail judgment is made again.

[0015] FIG. 1 is an explanatory diagram for explaining the configuration of an inspection device. FIG. 2 is an explanatory diagram showing an example of an electronic component attached to an object under inspection (substrate), where (a) shows the relationship between the electronic component and the inspection window, and (b) is an explanatory diagram for explaining information on the leads of the electronic component in image data used to create an inference model. FIG. 3 is an explanatory diagram for explaining inspection information, where (a) shows the configuration of an inspection information table used in a rule-based inspection method, and (b) shows the configuration of an inspection information table used in an inspection method using AI. FIG. 4 is an explanatory diagram showing the configuration of an inference model creation image information table that manages information when creating an inference model used in inspection using AI. FIG. 5 is a flowchart for explaining main processing for inspecting an object under inspection. FIG. 6 is a flowchart for explaining inspection processing executed within the main processing.

[0016] A preferred embodiment of the present invention will now be described with reference to the drawings. First, the configuration of an inspection device 10 according to this embodiment will be described with reference to Fig. 1. This inspection device 10 inspects an object 12 under inspection using image data (two-dimensional image data or pattern image data) of the object 12 under inspection obtained by imaging the object 12. The object 12 under inspection is, for example, an electronic circuit board (substrate) on which components are mounted and solder is applied.

[0017] The inspection device 10 includes an inspection table 14 that holds the object under inspection 12, an imaging unit 20 that is an imaging section that illuminates and images the object under inspection 12, an XY stage 16 that moves the imaging unit 20 relatively to the inspection table 14, and a control unit 30 that is a control section that controls the operation of the imaging unit 20 and the XY stage 16 and inspects the object under inspection 12. For ease of explanation, as shown in Fig. 1, the surface of the inspection table 14 on which the object under inspection is placed is defined as the XY plane (orthogonal X and Y directions), and the direction perpendicular to the placement surface (i.e., the imaging direction of a camera unit 21 that constitutes the imaging unit 20 (the optical axis direction of the optical system of the camera unit 21)) is defined as the Z direction.

[0018] The imaging unit 20 is attached to a moving table (not shown) of the XY stage 16 and can be moved in both the X and Y directions by the XY stage 16. The XY stage 16 is, for example, a so-called H-shaped XY stage. The XY stage 16 includes a Y drive unit that moves the moving table in the Y direction along a Y-direction guide extending in the Y direction, and two X-direction guides and an X drive unit that support the Y-direction guide at both ends and are configured to move the moving table and the Y-direction guide in the X direction. The XY stage 16 also includes a Z-movement mechanism that moves the imaging unit 20 in the Z direction. It may also include a rotation mechanism that rotates the imaging unit 20 around the optical axis of the optical system of the camera unit 21. The inspection device 10 may further include an XY stage that moves the inspection table 14. In this case, the XY stage 16 that moves the imaging unit 20 may be omitted. The X drive unit and the Y drive unit may use linear motors or ball screws. Also, instead of moving the imaging unit 20 in the Z direction, the XY stage 16 may be configured to move in the Z direction.

[0019] The imaging unit 20 includes a camera unit 21 that captures an image from a direction perpendicular (Z direction) to the inspection surface (substrate surface) of the inspected object 12, an illumination unit 22, and a projection unit 23. In the inspection device 10 according to this embodiment, the camera unit 21, the illumination unit 22, and the projection unit 23 may be configured as an integrated imaging unit 20. In this integrated imaging unit 20, the relative positions of the camera unit 21, the illumination unit 22, and the projection unit 23 may be fixed, or each unit may be configured to be relatively movable. Alternatively, the camera unit 21, the illumination unit 22, and the projection unit 23 may be separate units configured to be independently movable.

[0020] The camera unit 21 includes an imaging element that generates a two-dimensional image of the object, and an optical system (e.g., a lens) for focusing the image on the imaging element. The camera unit 21 is, for example, a CCD camera. The maximum field of view of the camera unit 21 may be smaller than the area on the inspection table 14 where the object under test 12 is placed. In this case, the camera unit 21 captures an image of the entire object under test 12 by dividing the image into multiple partial images. The control unit 30 controls the XY stage 16 so that the camera unit 21 moves to the next imaging position each time the camera unit 21 captures a partial image. The control unit 30 combines the partial images to generate an entire image of the object under test 12.

[0021] Note that the camera unit 21 may include an imaging element that generates a one-dimensional image, instead of a two-dimensional imaging element. In this case, the entire image of the object under inspection 12 can be acquired by scanning the object under inspection 12 with the camera unit 21. Furthermore, the imaging unit 20 may include, in addition to the camera unit 21, a plurality of camera units that image the object under inspection 12 at angles different from that of the camera unit 21. By imaging the object under inspection 12 at angles different from that of the camera unit 21, areas that are shaded by components attached to the object under inspection 12 and cannot be imaged by the camera unit 21 can be inspected using image data captured at angles different from that of the camera unit 21.

[0022] The illumination unit 22 is configured to project illumination light onto the surface of the inspection object 12 for imaging by the camera unit 21. The illumination unit 22 includes one or more light sources that emit light of a wavelength or wavelength range selected from the wavelength range detectable by the imaging element of the camera unit 21. The illumination light is not limited to visible light, and ultraviolet light, X-rays, etc. may also be used. When multiple light sources are provided, each light source is configured to project light of a different wavelength (e.g., red, blue, and green) onto the surface of the inspection object 12 at a different projection angle.

[0023] The object under inspection 12 illuminated by the illumination unit 22 is imaged by the camera unit 21. The inspection device 10 determines the presence or absence of defects on the board of the object under inspection 12 (for example, whether components are present or not, whether their placement is appropriate, and whether the state of solder application is good or bad) based on image data of the object under inspection 12 (this image data is referred to as "two-dimensional image data") obtained by illuminating and imaging the object under inspection 12 with the illumination by the illumination unit 22 and a height map, which will be described later.

[0024] In the inspection device 10 according to this embodiment, the illumination unit 22 is a side illumination source that projects illumination light obliquely onto the inspection surface of the object under inspection 12, and in this embodiment, includes an upper light source 22a, a middle light source 22b, and a lower light source 22c. In the inspection device 10 according to this embodiment, the side illumination sources 22a, 22b, and 22c are each a ring illumination source that surrounds the optical axis of the camera unit 21 and is configured to project illumination light obliquely onto the inspection surface of the object under inspection 12. Each of these side illumination sources 22a, 22b, and 22c may be configured with multiple light sources arranged in a circular ring shape. Furthermore, the upper light source 22a, the middle light source 22b, and the lower light source 22c, which are side illumination sources, are each configured to project illumination light at different angles onto the inspection surface.

[0025] The projection unit 23 projects a pattern (e.g., illumination light with varying light intensity) onto the inspection surface of the object under test 12. The object under test 12 onto which the pattern is projected is imaged by the camera unit 21. The inspection device 10 creates a height map of the inspection surface of the object under test based on image data of the object under test 12 obtained by imaging (this image data is referred to as "pattern image data"). Here, the height map is data that contains height information of the object under test 12 for each pixel of the pattern image data. The control unit 30 detects local mismatches between the projected pattern and the pattern image data, and acquires height information for that portion based on the local mismatches. In other words, changes in the imaged pattern relative to the projected pattern correspond to changes in height on the inspection surface.

[0026] The projection pattern is preferably a one-dimensional stripe pattern in which bright and dark lines alternate and repeat periodically. The projection unit 23 is arranged to project the stripe pattern obliquely onto the inspection surface of the inspection object 12. Discontinuities in height on the inspection surface of the inspection object 12 appear as pattern shifts in the stripe pattern image. Therefore, the height difference can be determined from the amount of pattern shift. For example, the control unit 30 creates a height map using the PMP (Phase Measurement Profilometry) method, which uses a stripe pattern whose brightness changes according to a sine curve. In the PMP method, the amount of shift in the stripe pattern corresponds to the phase difference of the sine curve.

[0027] The projection unit 23 includes a pattern forming device, a light source device for illuminating the pattern forming device, and an optical system for projecting a pattern (light transmitted through the pattern forming device) onto the inspection surface of the object under test 12. The pattern forming device may be, for example, a variable patterning device capable of dynamically generating a desired pattern, such as a liquid crystal display, or a fixed patterning device in which a pattern is fixedly formed on a substrate, such as a glass plate. When the pattern forming device is a fixed patterning device, it is preferable to provide a movement mechanism for moving the fixed patterning device or an adjustment mechanism in the optical system for pattern projection, thereby making the projection position of the pattern variable. Furthermore, the projection unit 23 may be configured to be able to switch between multiple fixed patterning devices having different patterns.

[0028] A plurality of projection units 23 may be provided around the camera unit 21. The plurality of projection units 23 are arranged so as to project patterns onto the inspection object 12 from different projection directions, respectively. In this way, it is possible to reduce the area on the inspection surface that is shaded due to height differences and onto which the pattern is not projected.

[0029] 1, which controls the entire device, is realized as hardware by the CPU, memory, and other LSIs of any computer, and as software by programs loaded into memory, but here the diagram shows functional blocks realized by the cooperation of these. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by hardware alone, software alone, or a combination of both.

[0030] FIG. 1 shows an example of the configuration of the control unit 30. The control unit 30 includes an inspection control unit 31 and a memory 35, which is a storage unit. The inspection control unit 31 includes an image processing unit 32, an inspection information processing unit 33, and an inspection unit 34. The image processing unit 32 includes an image capturing processing unit 32a and a height measurement unit 32b. The inspection device 10 also includes an input unit 36 ​​for receiving input from a user or another device, and an output unit 37 for outputting information related to the inspection. The input unit 36 ​​and the output unit 37 are each connected to the control unit 30. The input unit 36 ​​includes, for example, input means such as a mouse or keyboard for receiving input from a user, and communication means for communicating with other devices. The output unit 37 includes known output means such as a display or printer.

[0031] As a preprocessing step for creating a height map, the inspection control unit 31 controls the relative movement of the imaging unit 20 and the inspection table 14 while projecting a pattern onto the object under inspection 12 from the projection unit 23 using the imaging processing unit 32a of the image processing unit 32, thereby dividing and sequentially capturing pattern images of the object under inspection 12. The projected pattern is preferably a stripe pattern whose brightness changes according to a sine curve based on the PMP method. The inspection control unit 31 combines the captured divided images to generate pattern image data for the entire inspection surface of the object under inspection 12. The inspection control unit 31 stores the pattern image data in the memory 35. Note that pattern image data may be generated for only a portion of the inspection surface of the object under inspection 12, rather than for the entire inspection surface.

[0032] The height measurement unit 32b creates a height map of the entire inspection surface of the object under inspection 12 based on the image of the pattern in the pattern image data. The height measurement unit 32b first calculates the local phase difference between the pattern image data and the reference pattern image data for the entire image data, thereby calculating a phase difference map of the inspection surface of the object under inspection 12. Here, the "reference pattern image data" refers to image data in which a pattern is projected onto a reference plane by the projection unit 23 (i.e., image data acquired by the camera unit 21 in a state in which a pattern generated by a pattern forming device built into the projection unit 23 is projected onto the reference plane). The height measurement unit 32b creates a height map of the object under inspection 12 based on a reference plane serving as a reference for height measurement and the phase difference map. The reference plane is, for example, the surface of the electronic circuit board to be inspected. The reference plane does not necessarily have to be flat, but may be a curved surface that reflects deformation such as warpage of the board.

[0033] Specifically, the height measurement unit 32b determines the phase difference of the stripe pattern between each pixel of the pattern image data and the corresponding pixel of the reference pattern image data. The height measurement unit 32b converts the phase difference into height information. This is because the distance from the projection unit 23 varies depending on the position on the inspection surface, so even if the stripe width of the reference pattern is constant, the stripe width changes from one end of the pattern projection area on the inspection surface to the other. The height measurement unit 32b obtains height information from the reference plane based on the converted height information and the reference plane, and creates a height map of the inspection object 12.

[0034] The image processing unit 32 of the inspection control unit 31 may create an image of the object under test 12 having a height distribution by associating height information contained in the height map of the object under test 12 with each pixel of the two-dimensional image data of the object under test 12. The image processing unit 32 may also perform a three-dimensional modeling display of the object under test 12 based on the object under test image data with height distribution. The image processing unit 32 may also superimpose the height distribution on the two-dimensional image data of the object under test 12 and display it on the output unit 37. For example, the two-dimensional image data may be displayed in different colors depending on the height distribution.

[0035] The inspection control unit 31 is configured to execute various control processes for inspection based on input from the input unit 36 ​​and inspection-related information stored in the memory 35. The inspection-related information includes inspection information such as two-dimensional image data of the object 12 to be inspected, a height map of the object 12 to be inspected (calculated from the pattern image data as described above), and substrate inspection information and an inference model. The inspection unit 34 executes inspection based on the created substrate inspection information and inference model, and the two-dimensional image data and height map of the object 12 to be inspected.

[0036] Board inspection information is created for each type of board and is used in rule-based inspections. Board inspection information is a collection of inspection information for components arranged on the board, their positions, and each solder applied to the board. The inspection information for each component or solder includes the inspection items required for that component or solder, the inspection window, which is the inspection area on the image for each inspection item, and the inspection criteria that serve as the basis for determining the placement and pass / fail for each inspection item. One or more inspection windows are set for each inspection item. For example, in an inspection item that determines the pass / fail of solder application, typically, the same number of inspection windows as the number of solder application areas on the component are set in a layout that corresponds to the layout of the solder application areas. Furthermore, for inspection items that use image data that has undergone predetermined image processing on two-dimensional image data of the inspected object 12, the details of that image processing are also included in the inspection information.

[0037] 2(a) shows an example of an electronic component mounted on a substrate, which is the device under test 12. This electronic component 60 has a housing (package) 61 to which multiple leads (pins) 62 are attached, and the underside of each lead tip is electrically joined to the upper surface of the substrate by solder. Therefore, an inspection window W indicating a search area is set at each tip of the leads 62 (only one inspection window W is shown in FIG. 2(a)).

[0038] 3A shows an example of inspection information (inspection information table 70) in which inspection rules are set for rule-based inspection of the condition of solder that joins electronic components to a circuit board. In this inspection information table 70 for loose-solder inspection, information on a component type 72, component shape information 73, and an inspection rule definition 74 is set for a component number 71 for identifying an electronic component, and as described above, these are stored in memory 35 as inspection information. For example, for component number "Chip_0603," four inspection rules, "position shift inspection," "floating inspection," "polarity inspection," and "solder inspection," are set in the inspection rule definition 74. For component number "SOP_8pin," five inspection rules, "position shift inspection," "floating inspection," "polarity inspection," "solder inspection," and "bridge inspection," are set in the inspection rule definition 74.

[0039] Here, one inspection window is set for each inspection rule. Therefore, if the electronic component number 71 is "Chip_0603," four inspection windows corresponding to the four inspection rules (the settings of the inspection rule definition 74), "position shift inspection," "lift inspection," "polarity inspection," and "solder inspection," are set at the tips of the pins 62 of the electronic component 60 in FIG. 2( a). Therefore, for each board, which is the object under test 12, the electronic components attached to the board are identified, and inspection windows corresponding to the number in the inspection rule definition 74 are set at the locations (inspection zones) of each electronic component to be inspected. The inspection rules required for each inspection window are set according to the type (component number) of the electronic component.

[0040] In FIG. 3A, up to five inspection rule definitions are set for each part number, but this is just an example, and the number of inspection rule definitions is not limited to this example.

[0041] An inference model is created for each component attached to the board and is inspection information used in AI-based inspection. Image data and height maps of the inspected objects 12, both good and bad, captured by the inspection device 10 are used as training data and learning data to generate an inference model through machine learning (AI learning) such as deep learning, and inspection is then performed using the inference model thus generated. The inspection area where AI-based inspection is performed is defined by an inspection window, similar to the board inspection information described above.

[0042] Specifically, as shown in FIG. 4, the inspection information used for the AI-based inspection is stored as an inference model creation image information table 80, in which the type of electronic component is associated with an inference model 81 and stored as a component type 82. For example, when inspecting the solder condition of a lead, the inference model creation image information table 80 stores information 83 on the lead width A of the lead 62 and information 84 on the distance B from the lead end of the lead 62 to the land end. The lead width A and the distance B from the lead end to the land end are shown in FIG. 2B. The inference model creation image information table 80 also stores information on the content to be determined in the AI-based inspection, such as a defect type 85, information on the image area to be inspected using AI (AI inspection image area) 86, and an inspection image type 87. The defect type 85 corresponds to the inspection rule definition 74 of the rule-based inspection method described above. The configuration of the inference model creation image information table 80 shown in FIG. 4 is merely an example, and different information can be stored depending on the inspection target.

[0043] 3B, an inference model 79, which is inspection information used during inspection using AI, is stored in association with a component number 77 and an inspection item 78 as an inspection information table 75 for inspection using AI. The inspection information table 75 for inspection using AI and an inference model creation image information table 80 are also stored in the memory 35. For example, for an electronic component with component number 77 "Chip_0603," information "solder inspection" is set for the inspection item 78 and information "inference model 1" is set for the inference model 79. An inspection window using AI is then set at a location to be inspected (an inspection area, for example, the tips of each of the pins 62 of the electronic component 60 in FIG. 2A) of the electronic component to be inspected based on information corresponding to the component number of the electronic component (information in the inspection information table 75 for inspection using AI).

[0044] The inspection information processing unit 33 sets each item of inspection information to suit the board in the board inspection information creation process. For example, the inspection information processing unit 33 automatically sets the position and size of each inspection window for each inspection item so that it matches the solder layout of the board. The inspection information processing unit 33 may accept user input for some items of inspection information. For example, the inspection information processing unit 33 may accept tuning of inspection criteria by the user. The inspection criteria may be set using height information. Similarly, the inspection information processing unit 33 generates the above-mentioned inference model and associates the inspection window with the inference model.

[0045] The inspection control unit 31 performs imaging processing of the object under inspection 12 using the image processing unit 32 as preprocessing for creating the board inspection information and generating the inference model. Here, the image data (two-dimensional image data, which is full-surface image data of the board, and pattern image data) used to create the board inspection information are those of the object under inspection 12 that pass all inspection items. As described above, the imaging processing is performed by controlling the relative movement of the imaging unit 20 and the inspection table 14 while illuminating the object under inspection 12 with the illumination unit 22, and sequentially capturing partial images of the object under inspection 12. Multiple partial images are captured so that the entire object under inspection 12 is covered. The inspection control unit 31 combines these multiple partial images to generate full-surface image data (two-dimensional image data) including the entire inspection surface of the object under inspection 12. Furthermore, the image data (full-surface image data) used to create the inference model are those of the object under inspection 12 that pass all inspection items (all inspection areas judged as good) and those of the object under inspection 12 that have some search areas judged as bad. The inspection control unit 31 stores the full-surface image data in memory 35.

[0046] In addition, prior to inspection, the inspection information processing unit 33 creates substrate inspection information and an inference model using the two-dimensional image data of the object to be inspected 12, which is the above-mentioned entire substrate image data, and a height map (created from pattern image data).

[0047] As described above, the inspection unit 34 inspects the object under inspection 12 based on the created substrate inspection information and inference model (inspection information) and the two-dimensional image data and height map of the object under inspection 12 to be inspected. In this inspection device 10, the inspection unit 34 can inspect the object under inspection 12 using a preselected method (either rule-based inspection or inspection using AI), but here we will explain a configuration in which the two methods are used in combination depending on the situation, using Figures 5 and 6.

[0048] The inspection method of the inspection device 10, which will be described below, is configured to inspect each inspection area set on the object under test 12 using a first inspection method out of multiple inspection methods, and if the inspection results in a pass, the inspection area is determined to be pass. On the other hand, if the first inspection method results in a fail, the inspection area is inspected using a second inspection method, and if the second inspection method results in a pass, the judgment result for the inspection area is replaced with pass, and if the second inspection method also results in a fail, the inspection area is determined to be fail. In the following description, a case will be described in which the first inspection method is a rule-based inspection method and the second inspection method is an inspection method using AI.

[0049] As shown in FIG. 5 , when an inspection starts, the inspection control unit 31 of the control unit 30 executes main processing and transports a substrate, which is the object 12 to be inspected, into the inspection area of ​​the inspection device 10 (step S100). Then, the inspection control unit 31 causes the imaging processing unit 32a of the image processing unit 32 to image the substrate, which is the object 12, transported into the inspection area, and acquires two-dimensional image data and pattern image data of the object 12 (step S102). The acquired two-dimensional image data and pattern image data are stored in the memory 35. The inspection control unit 31 also causes the height measurement unit 32b of the image processing unit 32 to calculate height information using the acquired pattern image data, and creates a height map (step S104). This height map is also stored in the memory 35.

[0050] Next, the inspection control unit 31 uses the two-dimensional image data and height map acquired by the above processing to inspect the object under inspection 12 using the inspection unit 34. Specifically, one of the inspection windows set for the object under inspection 12 is selected (step S106), and inspection processing is performed for the selected inspection window (step S108).

[0051] 6, when the inspection process S108 is started, the inspection unit 34 reads the first inspection information set for the currently selected inspection window, i.e., the board inspection information, from the memory 35 (step S200). Then, the inspection unit 34 performs an inspection of the 2D image data and height map in the currently selected inspection window using the first method (rule-based inspection method) based on the board inspection information (step S202), and determines whether the inspection result is good or bad based on the inspection result (step S204). If the currently selected inspection window is determined to be good (step S204: good), the inspection unit 34 stores the inspection result of the currently selected inspection window as good in the memory 35 (step S220), and ends the inspection process S108.

[0052] On the other hand, if the judgment result for the currently selected inspection window is determined to be defective (step S204: defective), the inspection unit 34 extracts the defect type for the currently selected inspection window (step S206). As described above, since multiple inspection rules are set as different inspection windows for the same location on the device under test 12 (e.g., the tip of a lead of an electronic component), the inspection unit 34 identifies the inspection rule based on which the currently selected inspection window was judged as the defect type. In the example shown in Figure 3(a), one of the position shift amount inspection, lifting inspection, polarity inspection, solder inspection, and bridge inspection set in the inspection rule definition 74 of the inspection information table 70 is extracted as the defect type.

[0053] When the defect type of the currently selected inspection window is extracted, the inspection unit 34 selects second inspection information, i.e., an inference model (information set in the inference model 81 in the inference model creation image information table 80) according to the defect type (step S208). Here, the inference model selection method can use, for example, the inference model creation image information table 80 described above. In addition to the method of comparing the extracted defect type with the information on the defect type 85 in the inference model creation image information table 80 to select a matching inference model, it can also use the method of comparing the component type 72 in the inspection information table 70 set in the currently selected inspection window with the component type (lead type) 82 in the inference model creation image information table 80 to select a partially or fully matching inference model, or the method of comparing the component shape 73 in the inspection information table 70 of the currently selected inspection window with at least one of the information on the lead width (A) 83 and the information on the distance (B) from the lead end to the land end 84 in the inference model creation image information table 80 to select a partially or fully matching inference model. These methods can also be used in combination to narrow down the inference models to be selected. The second inspection method described below is executed on the image area defined in the AI ​​inspection image area 86 of the inference model creation image information table 80 using the second inspection information (inference model) selected in step S208. If multiple pieces of second inspection information (inference models) are selected based on the first inspection information and the defect type in step S208, the second inspection method described below is executed using all of the selected second inspection information (inference models). In addition, the illumination image defined in the inspection image type 87 of the inference model creation image information table 80 is used in the selection of the second inspection information (inference model) and in the second inspection method using this second inspection information.

[0054] The inspection unit 34 determines whether one or more pieces of second inspection information were selected in step S208 (step S210). If it determines that at least one piece of second inspection information corresponding to the defect type has been selected (step S210: Y), the inspection unit 34 selects one piece of the selected second inspection information (step S212) and performs an inspection by applying a second inspection method (an inspection method using AI) to the 2D image data and height map in the selected inspection window based on the selected second inspection information (inference model) (step S214). The inspection unit 34 also determines whether all of the second inspection methods selected in step S208 have been selected and inspected (step S216). If it determines that there are still second inspection methods that have not yet been selected (step S216: N), the inspection unit 34 returns to step S212 and repeats the above-described process. On the other hand, if it is determined that all of the second inspection methods selected in step S208 have been selected and inspections have been performed (step S216: Y), the inspection unit 34 determines whether the inspections using all of the second inspection information have been judged to be good (step S218).

[0055] If it is determined that all the inspections using the second inspection information have been judged to be good (step S218: Y), the inspection unit 34 replaces the inspection result of the currently selected inspection window with good, stores the result in the memory 35 (step S220), and ends the inspection process S108. On the other hand, if it is determined that at least one inspection using the second inspection information has been judged to be bad (step S218: N), the inspection unit 34 stores the inspection result of the currently selected inspection window as bad in the memory 35 (step S222), and ends the inspection process S108. Also, if it is determined in step S208 that second inspection information corresponding to the defect type could not be selected (step S210: N), the inspection unit 34 determines that an inference model corresponding to the currently selected inspection window has not been registered, maintains the inspection result of the currently selected inspection window as bad, and stores it in the memory 35 (step S222), and ends the inspection process S108.

[0056] 5 , when the inspection unit 34 completes the inspection process S108, it determines whether all inspection windows have been selected (step S110). If it determines that there are any unselected inspection windows remaining (step S110: N), it returns to step S106 and repeats the subsequent processes. On the other hand, if it determines that all inspection windows have been selected (step S110: Y), the inspection unit 34 determines whether the inspection results of all inspection windows have been judged to be good (step S112). If it determines that the inspection results of all inspection windows are good (step S112: Y), the inspection unit 34 judges the currently inspected object 12 to be good and stores this in the memory 35 (step S114). On the other hand, if it determines that at least one inspection window has been judged to be bad (step S112: N), the inspection unit 34 judges the currently inspected object 12 to be bad and stores this in the memory 35 together with information on the inspection window judged to be bad (step S116).

[0057] Finally, the inspection control unit 31 removes the object 12 under inspection from the inspection area (step S118), completing the inspection of one object 12 under inspection. Furthermore, when there is a next object 12 to be inspected, the inspection control unit 31 executes the inspection from step S100, repeating the process of carrying in, imaging, inspection, and carrying out. After the imaging of the object 12 under inspection in step S102 is completed, the process from step S104 onward is executed, and the object 12 under inspection is removed from the inspection area, and then the object 12 under inspection is carried into the inspection area and imaged. This allows the inspection of the previous object 12 under inspection and the imaging of the next object 12 under inspection to be performed in parallel, thereby shortening the takt time required for inspection.

[0058] In the above-described inspection method, a rule-based inspection method is adopted as the first inspection method, and an inspection area determined to be defective by the first inspection method is inspected using an AI-based inspection method, which is a second inspection method. However, the first inspection method may be an AI-based inspection method, and the second inspection method may be a rule-based inspection method. In this case, the first inspection information (inference model) can be determined from the inspection information table 76 shown in FIG. 3( b). This inspection information table 76 is composed of a part number 77, an inspection item 78, and an inference model 79, and this information is associated with the inspection window. Therefore, an inference model is selected according to the information in the inspection information table 76 set in the currently inspected (selected) inspection window, and the 2D image data and height map within the inspection window are inspected using this inference model. In addition, at this time, the defect type corresponds to the inspection item 78, and when a defect is determined by the first inspection method (inspection method using AI), information on the inspection item 78 corresponding to the currently selected inspection window is extracted as the defect type, and second inspection information (inspection rule in rule-based inspection) is selected from the inspection information table 70 based on this defect type and the information in the inference model creation image information table 80 for the currently applied inference model, thereby applying the second inspection method (rule-based inspection) to the two-dimensional image data and height map in the currently selected inspection window to perform inspection.

[0059] Furthermore, in the above description, the first and second inspection methods are applied to the two-dimensional image data and height map within the inspection window, but inspection may be performed using only either the two-dimensional image data or the height map. In this case, for example, if the height map is not used for the entire inspection, the inspection device 10 does not need to be provided with the projection unit 23. Furthermore, if a camera unit that captures an image of the inspected object 12 from a direction different from that of the camera unit 21 is provided in addition to the camera unit 21, the inspection may also be performed using the two-dimensional image data captured by this camera unit.

[0060] Furthermore, in order to analyze the inspection performance from the inspection results, the inspection control unit 31 of the control unit 30 may be configured to, after the above-described inspection process for the object 12 under inspection is completed, output to the output unit 37, for each inspection window (inspection area), the pass / fail judgment result for the object 12 under inspection by the first inspection method and the pass / fail judgment result for the object 12 under inspection by the second inspection method in association with each other. Alternatively, the inspection results for the object 12 under inspection inspected from the start of the inspection to the present time may be accumulated, and the pass / fail judgment result for the first inspection method and the pass / fail judgment result for the second inspection method in association with each other. In this case, the judgment result is updated each time the inspection of the object 12 under inspection is completed. Furthermore, the inspection control unit 31 of the control unit 30 may be configured to output to the output unit 37, for each piece of information set in the inspection area (inspection window), the proportion of pass / fail judgment results for the first inspection method and the proportion of pass / fail judgment results for the second inspection method in association with each other.

[0061] Furthermore, in the above explanation, the explanation was based on the inspection device 10 equipped with so-called "automated optical inspection (AOI)" that performs inspection using two-dimensional image data (and a height map generated from pattern image data) obtained by capturing an image of the inspected object 12 with the camera unit 21, but the above-mentioned inspection method may also be applied to an inspection device equipped with so-called "automated X-ray inspection (XOI) that performs inspection using three-dimensional image data (reconstructed image data or cross-sectional image data) generated from transmission image data obtained by irradiating the inspected object 12 with X-rays.

[0062] The features and effects of the inspection device 10 according to this embodiment are summarized below.

[0063] First, the inspection device 10 according to this embodiment uses two inspection methods (for example, a rule-based inspection method and an inspection method using AI) in combination to inspect the object under test 12. Therefore, even if one of the inspection methods determines that the object under test 12 is defective in any of the inspection areas (inspection windows) set on the object under test 12, it may be possible to determine that the object is passable by applying the other inspection method, thereby improving the accuracy of pass / fail determination in this inspection device 10.

[0064] As described above, when inspections using a rule-based inspection method are performed using strict inspection conditions based on inspection information in order to perform rigorous inspections, numerous inspection conditions and logical expressions are often used due to the large variations in the shapes and brightness of components, solder, etc. that occur during the mass production process of printed circuit boards. Setting this inspection information requires advanced skills. However, by setting a rule-based inspection method as the first inspection method and an AI-based inspection method as the second inspection method, even if the inspection conditions are set too loosely in the rule-based inspection method, resulting in an increase in inspection areas that are judged as defective (overdetected), those inspection areas are re-inspected using the AI-based inspection method. If this inspection correctly determines that the inspection area is good, an increase in overdetection can be prevented. This makes it easier to set the inspection information for the rule-based inspection method. Furthermore, reducing overdetection reduces the number of inspected objects 12 judged as defective, thereby reducing the number of inspected objects 12 judged as defective by visual inspectors and reducing the outflow of defective boards.

[0065] Furthermore, when an inspection method using AI is used, if variations occur during mass production of boards, sudden overdetection of unlearned good parts may occur. However, when an inspection method using AI is used as the second inspection method, the first inspection method (rule-based inspection method) has already made a certain degree of pass / fail judgment, and only the inspection areas that were judged to be bad by the first inspection method are inspected by the inspection method using AI, so sudden overdetection can be suppressed.

[0066] On the other hand, when an inspection method using AI is applied to the first inspection method, even if an overdetection occurs suddenly in the inspection method using AI and the number of inspection areas determined to be defective increases, the inspection area is inspected using the second inspection method (rule-based inspection method), and if this inspection can correctly determine that the area is good, an increase in overdetection can be prevented. Alternatively, in the actual production process of electronic circuit boards, for example, the ratio of components contained in solder may be changed or some components may be replaced with other components, and when an inspection method using AI is applied to the first inspection method, an area may be determined to be defective even if it is good according to the already generated inference model. However, by applying a rule-based inspection method as the second inspection method, it becomes possible to correctly determine that the area is good.

[0067] Secondly, in the inspection device 10 according to this embodiment, when an inspection using the first inspection method results in a determination of defective, second inspection information is selected based on the type of defect determined using the first inspection method and the first inspection information. However, when multiple pieces of second inspection information are selected, inspection using the second inspection method is performed based on all of the selected second inspection information. Furthermore, only when all of the inspections using the second inspection method result in a determination of defective is the inspection result of the inspection area where the inspection was performed determined to be defective. This allows for more accurate determination of defective / non-defective.

[0068] Here, when an AI-based inspection method is applied to the second inspection method, the conditions for selecting second inspection information (inference model) from the first inspection information (board inspection information) and defect type are relaxed, and an inference model generated for a component similar to the electronic component (component to be inspected) in the inspection area currently being inspected is also selected. Therefore, even if an inference model for the component to be inspected has not been generated, inspection can be performed using an inference model for a similar component. Therefore, even if all inference models for the electronic component to be inspected have not been generated, inspection using AI can be performed. Furthermore, as described above, inspections are performed using multiple selected inference models, and the inspection area is judged as pass only when all of the inspections result in a pass. Therefore, even if a judgment is made using an inference model for a similar component, an accurate judgment can be made by taking into account the judgment results of the other selected inference models.

[0069] It goes without saying that the above embodiments do not limit the invention described in the claims, and that not all of the combinations of the characteristic features described in the embodiments are necessarily essential features of the solution.

[0070] 10 Inspection device 12 Inspection object 20 Imaging unit (imaging section) 30 Control unit (control section)

Claims

1. An inspection device comprising: an imaging unit which images an object to be inspected and outputs image data; and a control unit which performs an inspection based on the image data output from the imaging unit, wherein a first inspection method and a second inspection method are set for at least one inspection area set on the object to be inspected, and wherein the control unit, when selecting the inspection area, executes the following steps: a first step of selecting first inspection information based on the information set for the inspection area; a second step of applying the first inspection method to the image data of the inspection area based on the first inspection information selected in the first step to determine whether the image is good or bad, and outputting a defect type if the image is determined to be bad; a third step of selecting one or more second inspection information based on the first inspection information and the defect type when the image is determined to be bad in the second step; and a fourth step of applying the second inspection method to the image data of the inspection area based on the second inspection information selected in the third step to determine whether the image is good or bad.

2. The inspection device described in claim 1, wherein in the fourth step, if all inspection results for each inspection area using the second inspection method based on one or more pieces of second inspection information are good, the control unit determines that the inspection area is good even if the inspection result using the first inspection method is bad.

3. The inspection device according to claim 1 or 2, wherein the first inspection information is an inspection rule for determining whether the image data is good or bad, the first inspection method is a rule-based inspection method for performing inspection based on the inspection rule, the second inspection information is an inference model obtained by prior learning, and the second inspection method is an inspection method using AI for determining whether the image data is good or bad based on the inference model.

4. An inspection device as described in claim 1 or 2, wherein the first inspection information is an inference model obtained by prior learning, the first inspection method is an inspection method using AI that judges whether the image data is good or bad based on the inference model, the second inspection information is inspection rules that judge whether the image data is good or bad, and the second inspection method is a rule-based inspection method that performs inspection based on the inspection rules.

5. The inspection device according to claim 1 or 2, wherein the control unit outputs, for each inspection area, a pass / fail judgment result obtained by the first inspection method and a pass / fail judgment result obtained by the second inspection method in association with each other.

6. The inspection device according to claim 1 or 2, wherein the control unit outputs, for each piece of information set in the inspection area, a proportion of pass / fail judgment results obtained by the first inspection method and a proportion of pass / fail judgment results obtained by the second inspection method in association with each other.

Images