Visual inspection support system

Abstract

Provided is a visual inspection support device for maintaining the quality of inspection by displaying visual inspection support information during visual inspection by an inspector. A visual inspection support system 1 comprises: an appearance inspection unit 2 having an imaging device 11, the appearance inspection unit 2 capturing an image of a predetermined inspection portion of an inspection object by means of the imaging device 11, automatically performing appearance inspection using the captured image, performing quantitative evaluation of the quality of the predetermined inspection portion, and generating visual inspection support information based on the quantitative evaluation; and a visual inspection unit 3 having a display part 5 for displaying the image captured by the imaging device 11 and an input part 6 for inputting a visual determination result by an inspector. The visual inspection support information is displayed on the display part 5.

Description

Visual Inspection Support System

[0001] The present technology relates to an inspection apparatus for inspecting various inspection objects including printed circuit boards, mounted substrates, liquid crystal panels, and the like, and more particularly to a visual inspection support system having a support function that allows inspectors to efficiently perform visual inspection. This application claims priority based on Japanese Patent Application No. 2025-028452 filed in Japan on February 25, 2025, which is incorporated herein by reference.

[0002] Conventionally, automatic inspection apparatuses have been provided for inspecting pads, wiring patterns, mounting conditions of electronic components, and the like on various inspection objects including printed circuit boards, mounted substrates, liquid crystal panels, and the like. For inspection objects determined to be defective by the automatic inspection apparatus, visual inspection is subsequently performed by an inspector.

[0003] In visual inspection, an inspector generally re-inspects an inspection image captured by an automatic inspection apparatus visually to determine whether the object is a non-defective product or a defective product.

[0004] There is also a known configuration in which a plurality of inspection lines each provided with an automatic inspection apparatus are provided, inspection images related to inspection objects determined to be defective by the automatic inspection apparatus of each inspection line are sequentially displayed on one inspection terminal, and visual inspection is intensively performed with one inspection terminal.

[0005] Japanese Unexamined Patent Application Publication No. 2008-111687

[0006] The inspection process is provided to prevent defective products from flowing out to subsequent processes. Therefore, in the inspection process, it is necessary to prevent non-defective products from being incorrectly determined as defective as much as possible. On the other hand, when a non-defective product is incorrectly determined as defective, at least the situation where defective products flow out to subsequent processes does not occur.

[0007] Therefore, in the quality determination by the automatic inspection apparatus, any items other than those definitely determined to be non-defective, including those of unknown quality, are determined as defective and subjected to visual inspection by an inspector. As a result, most of the inspection objects subjected to the visual inspection by an inspector are determined to be non-defective.

[0008] Thus, if a single inspector consistently assigns good marks to items during visual inspection, the quality of the inspection may decline, creating a risk that items that should be incorrectly classified as defective may be mistakenly identified as good due to inertia. Reasons for an inspector incorrectly classifying a defective item as good include errors in the inspector's judgment or erroneous input where an item judged as defective is mistakenly recorded as good during the input process.

[0009] Furthermore, while visual inspections can yield different results depending on the inspector's experience and ability, it is necessary to minimize the variability in pass / fail judgments based on differences in experience and ability among inspectors.

[0010] One aspect of the present invention aims to provide a visual inspection support device that can display visual inspection support information during visual inspection by an inspector, thereby maintaining or improving the quality of the inspection.

[0011] Furthermore, the description of this problem does not preclude the existence of other problems. Moreover, one aspect of the present invention does not need to solve all of these problems. It is possible to extract other problems from the description in the specification, drawings, and claims.

[0012] To solve one of the above-mentioned problems, a visual inspection support system according to one aspect of the present invention includes: an appearance inspection unit having an imaging device that captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation; and a visual inspection unit having a display unit that displays the image captured by the imaging device and an input unit that inputs the visual inspection result by the inspector, wherein the display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the appearance inspection unit and is to be inspected by an inspector, and the visual inspection support information.

[0013] Furthermore, in order to solve one of the above-mentioned problems, a visual inspection method according to one aspect of the present invention includes the steps of: performing an automatic visual inspection of an object to be inspected by an inspector using a visual inspection unit and quantitatively evaluating the quality of the inspected area; generating visual inspection support information that serves as an indicator for determining the quality of the inspected area based on the quantitative evaluation; displaying on the display unit of the visual inspection unit a visual inspection image taken by the imaging device of the visual inspection unit of the inspected area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, along with the visual inspection support information; and receiving input of the visual judgment result from the inspector.

[0014] According to one aspect of the present invention, the display unit of the visual inspection unit displays an image of the inspection area and visual inspection support information. This prevents errors in judgment where an inspector mistakenly identifies an object that should be judged as defective as a good product, and prevents errors in input where an inspector mistakenly enters a good product as the result of a judgment that the inspector has judged as defective. Therefore, according to one aspect of the present invention, it is possible to prevent a situation where a defective product is judged as good and flows to a later process.

[0015] Figure 1 shows an example of the overall configuration of the inspection system. Figure 2 shows an example of the overall configuration of the inspection system. Figure 3 shows an example of the connection configuration of the inspection system. Figure 4 shows the main configuration of the visual inspection device (visual inspection unit). Figure 5 is an external perspective view of the visual inspection device. Figure 6 shows the configuration of the imaging camera. Figure 7 shows the states of no warping, upward warping, and downward warping of the substrate. Figure 8 shows a configuration in which the matching score is set to a score (numerical value) on a scale of 100. Figure 9 shows a configuration that divides the products into three categories: high good product rate category, low good product rate category, and medium good product rate category, according to the matching score. Figure 10 shows a configuration in which the inspection targets or visual inspection areas belonging to the high good product rate category and low good product rate category are excluded from the visual inspection, and only the inspection targets or visual inspection areas belonging to the medium good product rate category are subject to visual inspection. Figure 11 is a flowchart showing the operation flow of the visual inspection device. Figure 12 shows the main configuration of the visual inspection terminal (visual inspection unit). Figure 13 is a diagram showing one example of the display unit during visual inspection. Figure 14 is a diagram showing another example of the display unit during visual inspection. Figure 15 shows another example of the display unit during visual inspection. Figure 16 is a flowchart showing the operation flow of the visual inspection terminal. Figure 17 is a diagram showing the data stored in the database. Figure 18 is a flowchart showing an example of the inspection process of the visual inspection support system. Figure 19 is a flowchart showing another example of the inspection process of the visual inspection support system. Figure 20 is a flowchart showing another operation flow of the visual inspection terminal. Figure 21 is a diagram showing an example of the display shown on the display unit when the inspector makes a judgment in the process of displaying visual inspection support information after judgment. Figure 22 is a diagram showing an example of the display unit when judgment input is restricted. Figure 23 is a diagram showing an example of the display with the "Administrator Only" icon displayed in the input unit. Figure 24 is a diagram showing an example of the display with the "NG", "OK", and "Acceptable Item" icons displayed in the input unit. Figure 25 is a flowchart showing the operation flow of the visual inspection terminal that performs additional shooting. Figure 26 is a diagram showing an example of the overall configuration of a visual inspection support system using AOI.Figure 27 shows an example of the connection configuration of a visual inspection support system using AOI. Figure 28 shows an example of the overall configuration of AOI. Figure 29 is a flowchart showing an example of the inspection process of a visual inspection support system using AOI. Figure 30 is a flowchart showing another example of the inspection process of a visual inspection support system using AOI. Figure 31 shows an example of the overall configuration of an inspection system using AOI.

[0016] The following describes in detail a visual inspection support system to which the present invention is applied, with reference to the drawings. It should be noted that the present invention is not limited to the following embodiments, and various modifications are possible without departing from the spirit of the invention. Furthermore, the drawings are schematic, and the proportions of dimensions may differ from those of reality. Specific dimensions should be determined by referring to the following description. It should also be noted that there are parts where the relationships and proportions of dimensions differ between drawings.

[0017] A visual inspection support system 1 according to one aspect of the present invention, as shown in Figure 1, comprises a visual inspection unit 2 having an imaging camera 11 that captures a predetermined inspection area of ​​an object to be inspected with the imaging camera 11, and automatically performs a visual inspection, i.e., quantitative evaluation of the quality of the visually inspected area, and generates visual inspection support information based on the quantitative evaluation, using the captured image; a visual inspection unit 3 having a display unit 5 that displays the image of the visually inspected area captured by the imaging camera 11, and an input unit 6 that inputs the visual judgment result by the inspector, and the display unit 5 displays visual inspection support information based on the quantitative evaluation.

[0018] The visual inspection unit 2 is a unit that photographs the inspection area of ​​the object to be inspected and automatically quantitatively evaluates the quality of the inspection area based on the image data, and is composed of, for example, an automatic inspection device 2A. The automatic inspection device 2A is installed, for example, on the inspection line through which the object to be inspected is transported.

[0019] The visual inspection unit 3 is equipped with a display monitor that constitutes the display unit 5 and various input means that constitute the input unit 6. Based on the visual inspection image displayed on the display unit 5, the quality of the visual inspection part is visually judged by the inspector, and the judgment result is input via the input means. For example, it is composed of a visual inspection terminal 3A.

[0020] The objects to be inspected are not particularly limited as long as they are parts that are subjected to visual inspection. For example, they include printed circuit boards and mounted boards on which various electronic components such as ICs, LSIs, resistors, and capacitors are mounted, or on which circuit patterns and resist patterns are formed, as well as liquid crystal panels and other various objects to be inspected.

[0021] The designated inspection areas refer to parts of the object to be inspected that require inspection, such as the mounting state of electronic components or the formation state of circuit patterns, or parts that have been judged as defective or not judged as "OK" by the automated optical inspection device (AOI) described later. The parts that are automatically visually inspected by this visual inspection unit 2 are called inspection-required areas.

[0022] Visual inspection refers to the quantitative evaluation of the quality of a designated inspection area, performed by the visual inspection unit 2 based on an image of that area. In this specification, it is also called automated visual inspection. The image of the inspection area captured by the visual inspection unit 2 is called an automated inspection image.

[0023] A visual inspection area is a part that is visually inspected by an inspector, and refers to the part that is subject to visual inspection among the parts requiring inspection that have been automatically visually inspected by the visual inspection unit 2. In addition, among the automatically inspected images, the images used for visual inspection are called visual inspection images.

[0024] Visual inspection support information refers to information that serves as an indicator for inspectors to determine whether an item is good or bad. Based on quantitative evaluation of the degree of goodness or badness through automated visual inspection, it displays the degree of goodness or badness using numerical values, probabilities, images, characters, sentences, colors, etc.

[0025] During visual inspection, the display unit 5 of the visual inspection unit 2 displays an image of the area being inspected, captured by the imaging camera 11, along with visual inspection support information. This prevents inspectors from misjudging defective items as good products, and prevents erroneous inputs where an inspector mistakenly enters a good product as a defective product. Therefore, the visual inspection support system 1 according to the present invention prevents defective products from being mistakenly identified as good products and sent to subsequent processes.

[0026] The following describes the various components of the visual inspection support system 1. As mentioned above, the visual inspection support system 1 may be configured with the visual inspection unit 2 and the visual inspection unit 3 as a single device, or with the visual inspection device 2A and the visual inspection terminal 3A as separate terminals. Furthermore, as shown in Figure 2, the visual inspection support system 1 may be configured with multiple inspection lines 9 (for example, inspection line 9A, inspection line 9B, inspection line 9C, etc.) each equipped with a visual inspection device 2A, and one visual inspection terminal 3A, and the single visual inspection terminal 3A may sequentially display the visual inspection images and visual inspection support information captured by the visual inspection devices 2A of each inspection line 9, thereby enabling centralized visual inspection. In the following description, the visual inspection support system 1 equipped with multiple inspection lines 9 and one visual inspection terminal 3A will be used as an example.

[0027] Furthermore, the visual inspection support system 1 is used to inspect various objects, including printed circuit boards, mounted circuit boards, and liquid crystal panels, on which various electronic components such as ICs, LSIs, resistors, and capacitors are mounted, or on which circuit patterns and resist patterns are formed. However, the objects to be inspected are not limited to these. In the following explanation, a printed circuit board 7 on which electronic components are surface-mounted will be used as an example of an object to be inspected.

[0028] Each of the multiple inspection lines 9 has a transport means such as a belt conveyor for transporting the objects to be inspected, and the visual inspection device 2A is positioned on the transport line for the objects to be inspected. As a result, the inspection line 9 can transport the objects to be inspected one after another and perform inspections continuously.

[0029] Furthermore, as shown in Figure 2, the visual inspection devices 2A and visual inspection terminals 3A installed on each inspection line 9 are connected to the database 10 via a bus 8, enabling mutual data communication. The database 10 stores, in an interconnected manner, the following: automated inspection images (visual inspection images) from the visual inspection device 2A, location information of areas requiring inspection (visual inspection areas), visual inspection support information, visual judgment results entered into the visual inspection terminal 3A by the inspector, IDs of the objects to be inspected and parts, and automated inspection results (score, defect name, etc.) from the visual inspection device 2A. Note that the connection configuration of each device 2A, 3A and the database 10 via the bus 8 in the visual inspection support system 1 is illustrative, and of course, network connection configurations other than the bus type may be used (see Figure 3). Also, the database 10 may consist of one or more databases.

[0030] [Visual Inspection Device 2A] The visual inspection device 2A is a device that photographs the objects to be inspected on the inspection line 9 and quantitatively evaluates the quality of the objects based on the image data. The visual inspection device 2 can employ either a manual or automatic method for attaching and detaching the objects to be inspected, but an automatic method is preferable in order to shorten the inspection cycle time.

[0031] As shown in Figures 4 and 5, the visual inspection device 2A includes an imaging camera 11 such as a CCD camera for photographing the object to be inspected, an automated inspection computer 12 that uses the imaging camera 11 to photograph the object to be inspected by referring to predetermined inspection area data (XYθ coordinates), and performs quantitative evaluation of the quality of the inspection area based on the captured image data, and generates visual inspection support information based on the quantitative evaluation, a communication unit 13 connected to the bus 8 and communicating with a database 10, etc., and a support unit 14 for supporting the object to be inspected.

[0032] [Imaging Camera 11] The imaging camera 11 is positioned on the upper side of the device housing and includes an oblique camera 11a that photographs a predetermined inspection area of ​​the object to be inspected from an oblique angle above. The imaging camera 11 may also include a planar camera 11b that photographs the inspection area of ​​the object to be inspected from above. In the visual inspection device 2A, the imaging camera 11 is positioned on the upper side of the device housing, and a movement mechanism in the XYθ direction is provided on the support part 14 that supports the object to be inspected or on the imaging camera 11, so that the imaging camera 11 can move relative to the object to be inspected supported by the support part 14, and imaging can be taken from any position on the object to be inspected. In addition, in the visual inspection device 2A, a rotation mechanism is provided on the support part 14 that supports the object to be inspected or on the imaging camera 11, so that the imaging camera 11 can rotate relative to the object to be inspected supported by the support part 14, and imaging can be taken from any direction on the object to be inspected.

[0033] The oblique camera 11a is supported with its camera optical axis α1 tilted at an angle to the object being inspected, and it photographs the object being inspected from an oblique angle above. As shown in Figure 6, the oblique camera 11a is positioned so that its optical axis is at an angle of 45° with respect to the printed circuit board 7, which is the object being inspected. The planar camera 11b is supported with its camera optical axis α2 aligned with the rotation axis of the rotation mechanism, and it photographs the object being inspected from directly above.

[0034] The imaging camera 11 has predetermined imaging lenses and image sensors incorporated into the lens barrels of the oblique camera 11a and the planar camera 11b. The captured image data is stored in the database 10 and reproduced and displayed on the display unit 5 of the visual inspection terminal 3A during visual inspection. Alternatively, during visual inspection at the visual inspection terminal 3A, the images captured by the oblique camera 11a and the planar camera 11b of the imaging camera 11 may be displayed as real-time images on the display unit 5 of the visual inspection terminal 3A.

[0035] The imaging camera 11 may be equipped with focus and zoom functions for the oblique camera 11a and the planar camera 11b, or it may not be equipped with these functions, but may use a camera that has a focal length and aperture to achieve a deep depth of field and is capable of pan-focus shooting, and the image may be displayed at an appropriate size by processing on the automatic inspection computer 12 or the visual inspection terminal 3A.

[0036] Furthermore, the imaging camera 11 may be equipped with multiple oblique cameras 11a. For example, the imaging camera 11 can be equipped with a pair of oblique cameras 11a, consisting of a high-magnification camera and a low-magnification camera, at opposite positions via the rotation axis of the rotation mechanism, allowing them to be used interchangeably depending on the size of the area to be inspected. Alternatively, the imaging camera 11 may be equipped with multiple oblique cameras 11a arranged concentrically around the rotation axis of the rotation mechanism, or the multiple oblique cameras 11a may be arranged at equal intervals or at unequal intervals.

[0037] When an object to be inspected is brought in, the imaging camera 11 refers to the inspection area data (XYθ coordinate data) stored in the database 10 and takes a photograph of the area to be inspected at a predetermined angle and magnification. The captured image is stored in the database 10 and read out according to the inspector's operation, etc., and displayed on the display unit 5 of the visual inspection terminal 3A. In addition, when additional shooting is performed as described later, the captured image is displayed as a real-time image. The inspector operating the visual inspection terminal 3A makes a pass / fail judgment by looking at the visual inspection image and real-time image displayed on the display unit 5. The visual inspection image of the area to be inspected displayed on the display unit 5 during visual inspection is not limited to one image; multiple visual inspection images with different angles and magnifications may be displayed.

[0038] [Support section 14] The support section 14 can be any mechanism that detachably fixes and supports the object to be inspected. For example, the support section 14 may have a pair of long-side frames with support surfaces that protrude to support the lower surface of the outer edge of the printed circuit board 7, a pair of connecting bars that connect both ends of the long-side frames, and fixing pins that slide along the longitudinal direction of the long-side frames to fix the printed circuit board 7.

[0039] [Z-axis correction value] The visual inspection device 2A may also be provided with a vertical movement mechanism (in the Z-axis direction) and a correction means in the Z-axis direction on the imaging camera 11 or the support part 14 that supports the object to be inspected, so as to vary the relative distance in the Z-axis direction between the imaging camera 11 and the object to be inspected. This makes it possible to reliably capture the area to be inspected even if the plate-shaped object to be inspected, such as the printed circuit board 7, is warped, by correcting the imaging position of the imaging camera 11 in the Z-axis direction.

[0040] In other words, as shown in Figure 7(A), if there is no warping in the printed circuit board 7, the inspection area identified based on the position data of the area to be inspected will coincide with the area to be subjected to automatic visual inspection or displayed on the display unit 5, and the area to be actually inspected by automatic visual inspection or visual inspection. Therefore, by setting the relative position between the printed circuit board 7 and the imaging camera 11 based on the position data (XYθ coordinates) of the area to be inspected, the optical axis of the oblique camera 11a will coincide with the area to be inspected of the electronic component set on the printed circuit board 7, and the area to be inspected can be captured by the oblique camera 11a.

[0041] However, as shown in Figure 7(B), if the printed circuit board 7 warps downward, or as shown in Figure 7(C), if the printed circuit board 7 warps upward, setting the relative position between the printed circuit board 7 and the oblique camera 11a of the imaging camera 11 based on the position data of the area to be inspected set on the printed circuit board 7 may result in a discrepancy between the area to be inspected identified based on the position data and the image from the oblique camera 11a. This could prevent the oblique camera 11a from capturing the area that needs inspection, or it may result in an out-of-focus image.

[0042] In Fig. 7(c), because the printed circuit board 5 is warped upward, the optical axis α1 of the oblique camera 11a captures an area before the inspection site. In Fig. 7(b), because the printed circuit board 5 is warped downward, the optical axis α1 of the oblique camera 11a captures an area beyond the site to be inspected. If the relative position between the printed circuit board 7 and the imaging camera 11 is set based on the position data of the site to be inspected as described above, it is impossible to capture the site required for automatic visual inspection or manual visual inspection, so a proper quantitative evaluation of the quality cannot be performed, and an inspector cannot know which position of which surface-mounted component should be visually checked. Particularly in recent years, as surface-mounted components, extremely small chip components with a side length of less than 1 mm, such as 0603 chips, have been used, and these extremely small chips are mounted at high density, so it is required to accurately capture the inspection site.

[0043] Therefore, in the visual inspection apparatus 2A, it is preferable that a moving mechanism in the vertical direction (Z-axis direction) and a correction means in the Z-axis direction are provided on the imaging camera 11 or the support portion 14 that supports the inspection object, so as to change the relative distance between the imaging camera 11 and the inspection object in the Z-axis direction.

[0044] As the moving mechanism in the vertical direction for the imaging camera 11 or the support portion 14, a known moving mechanism can be employed.

[0045] As the correction means in the Z-axis direction, for example, a method can be adopted in which the deflection amount of the inspection object (printed circuit board 7) is measured using a laser displacement meter or the like, and the relative distance (height direction: Z-axis direction) between the imaging camera 11 and the support portion 14 is corrected based on the measured value. In order to grasp the deflection amount, the height of the reference point of the printed circuit board 7 is measured in advance. For example, measurement is performed at points where the structural deflection amount can be assumed to be approximately zero, such as the vicinity of the origin (X:Y=0:0) at the lower left, the vicinity of the upper left, the vicinity of the upper right, and the vicinity of the lower right on the outer periphery of the rectangular substrate in a plan view, and other such points. The number of measurement points may be one, but a plurality of measurement points is preferable. When there is one measurement point, the height of that measurement point is used as a reference value, and when there are a plurality of measurement points, the average of the heights of the respective measurement points is used as a reference value.

[0046] Next, the substrate height of the inspection-required portion of the inspection object (printed circuit board 7) is measured using a laser displacement meter or the like, and a difference value from a reference value is calculated. This difference value serves as the correction amount in the Z-axis direction. The correction amount in the Z-axis direction is measured in advance prior to the inspection process, and stored in the database 10 in association with a substrate ID or the like. Note that the correction means in the Z-axis direction is not limited to the method described above, and any publicly known method can be used.

[0047] When photographing an inspection-required portion with the imaging camera 11 of the visual inspection apparatus 2A, the relative height between the imaging camera 11 and the support portion 14 is corrected by referencing the correction amount in the Z-axis direction of the inspection-required portion on the substrate. As a result, in the visual inspection process, a visual inspection image corrected in advance in the Z-axis direction is displayed, and additional photographing such as manually correcting the imaging camera 11 in the Z-axis direction becomes unnecessary, whereby the tact time of the inspection process can be shortened.

[0048] Note that the correction amount in the Z-axis direction is referenced not only for photographing in automatic visual inspection but also during additional photographing. Further, the correction amount in the Z-axis direction may be measured in the inspection of the first inspection object (printed circuit board 7) in the inspection process, stored in the database 10 in association with a substrate type ID or the like, and referenced in subsequent automatic visual inspection and additional photographing of inspection objects of the same substrate type. Alternatively, the correction amount in the Z-axis direction may be measured each time an inspection object is carried into the visual inspection apparatus 2A, and used for photographing and additional photographing in automatic visual inspection.

[0049] Note that the apparatus housing may be provided with an unillustrated illumination means that illuminates the interior of the housing.

[0050] [Automatic inspection computer 12] The automatic inspection computer 12 photographs a predetermined inspection-required portion of an inspection object with the imaging camera 11 based on an automatic inspection processing program, and performs quantitative evaluation of the quality status of the predetermined inspection-required portion using the photographed image (also referred to as an "automatic inspection image"). Specifically, the visual inspection apparatus 2A has or acquires from the database 10 inspection site data indicating the positions of inspection-required portions of the printed circuit board 7 that is to be inspected in advance, and according to a program, sequentially photographs inspection images such as planar images and oblique images at positions indicated by the inspection site data, for example in order of proximity.

[0051] The position data of the area to be inspected is determined in an XYθ coordinate system specific to the imaging camera 11. Of these, XY are the two orthogonal coordinates within the inspection surface of the object to be inspected, and define the imaging position of the imaging camera 11. θ is the angle obtained when the object to be inspected is rotated counterclockwise around the X-axis, with the center point of the XY coordinate as the rotation center, and defines the imaging direction (θ direction) by the oblique camera 11a.

[0052] Furthermore, the automated inspection computer 12 may classify the defect of the inspected part against a "defect name database" which contains pre-categorized types of defects (e.g., bridging, misalignment, electrode detachment, missing electrode, foreign matter, dirt, insufficient soldering, etc.). Defect names such as bridging may be displayed on the display unit 5 along with visual inspection support information.

[0053] The inspection results obtained by the automated inspection computer 12 are called automated inspection results. The automated inspection results include quantitative evaluation values ​​(scores) generated based on the quantitative evaluation described below, and information on the name of the defect (e.g., bridge, misalignment, electrode floating, missing electrode, foreign matter, dirt, insufficient soldering, etc.).

[0054] Furthermore, the automated inspection computer 12 quantitatively evaluates the quality of the areas requiring inspection based on the automated inspection images and generates visual inspection support information based on the quantitative evaluation.

[0055] Furthermore, areas suspected of having mounting defects, identified by the visual inspection device 2A, that require visual inspection by an inspector are referred to as "visual inspection areas." If the entire assembly is subject to visual inspection, all areas requiring inspection will be considered visual inspection areas. Also, as described later, if the inspection target is limited according to quantitative evaluation values ​​(scores), etc., the areas among the areas requiring inspection that are subject to visual inspection will be considered visual inspection areas.

[0056] The quantitative evaluation of the quality of the areas requiring inspection, the generation of quantitative evaluation values ​​(scores), and the generation of visual inspection support information are performed by an automated inspection processing program using a known image processing program.

[0057] [Quantitative Evaluation] The quantitative evaluation of the quality is performed by using known processes such as pattern matching and artificial intelligence (AI) to calculate a matching score that evaluates the similarity representing the matching state using a predetermined calculation formula. Known methods such as template matching and feature-based matching can be used for pattern matching.

[0058] As shown in Figure 8, the matching score is set as a numerical score on a scale of, for example, 100, with 100 representing the highest value for the good product rate and 0 representing the lowest value. In other words, the closer the score is to 100, the higher the probability that the product is good, and the lower the score, the higher the probability that the product is defective.

[0059] [Visual Inspection Support Information] Visual inspection support information is information that serves as an indicator for inspectors to determine whether an item is good or bad. It is generated based on a quantitative evaluation of the quality by the automated inspection computer 12, and displays the quality using numbers, probabilities, characters, colors, etc. In other words, visual inspection support information is a display generated based on a score calculated by quantitative evaluation. For example, it can be used to display the score itself, characters such as "high good product rate" or "low good product rate" corresponding to the score, warnings such as "caution," the probability of a good product corresponding to the score, a blue mark indicating a high good product rate or a red mark indicating a low good product rate according to the score, or a combination of these.

[0060] During visual inspection, visual inspection support information is displayed on the display unit 5 along with the visual inspection image. This helps to prevent errors in judgment, such as the inspector mistakenly identifying an item that should be judged as defective as a good product, or the inspector mistakenly entering a good product as the result of a defect.

[0061] Furthermore, the visual inspection support information may be divided into multiple categories based on the score, each within a predetermined range, and the information may be identifiable to each category. For example, as shown in Figure 9, if the score is set on a scale of 100, with 100 being the highest value for the high good product rate and 0 being the lowest value, two thresholds are set to divide the products into three categories: a high good product rate category with a score of 80 or higher, a low good product rate category with a score of 40 or lower, and a medium good product rate category between the high and low good product rate categories. The visual inspection support information may then display text such as "high good product rate," "medium good product rate," or "low good product rate," a blue mark indicating the high good product rate category, a gray mark indicating the medium good product rate category, a red mark indicating the low good product rate category, or a combination of these.

[0062] Alternatively, the display format of the visual inspection support information may be changed according to each category. For example, for items classified in the medium good product rate category, a message such as "Please check carefully" may be displayed, while for items classified in the high good product rate category, the visual inspection support information may not be displayed. By hiding the visual inspection support information, the time required for visual inspection will be reduced.

[0063] The score threshold may be one or more than three.

[0064] Furthermore, the threshold scores used to categorize each category may be fixed, arbitrarily set, or changeable. By changing the thresholds according to the type of substrate or component, for example, the lower limit of the threshold indicating a high good product rate can be lowered for components with a relatively high good product rate, or the upper limit of the threshold indicating a low good product rate category can be raised for components that require particularly careful visual inspection, thereby allowing for the emphasis placed on visual inspection. Also, for example, setting the high good product rate category to a score of 90 or higher broadens the categories that inspectors should focus on. Also, for example, setting the low good product rate category to a score of 50 or lower reduces the possibility of defective products being judged as good products. In other words, inspectors can relatively easily determine whether an object or visual inspection area belonging to the high good product rate category or the low good product rate category is good or bad, and by focusing on objects or visual inspection areas belonging to the intermediate good product rate category between these two, they can perform visual inspections efficiently and prevent misjudgments and incorrect inputs.

[0065] Furthermore, the threshold scores used to categorize each item may be set for each defect name. For example, in the case of a defect name "missing parts," it can be divided into four categories: score less than 10, score 10 or more but less than 70, score 70 or more but less than 90, and score 90 or more. In the case of a defect name "incorrect polarity," it can be divided into four categories: score less than 10, score 10 or more but less than 50, score 50 or more but less than 90, and score 90 or more. Since the reliability of automated inspection results and visual inspection results by inspectors differs depending on the defect name, i.e., the nature of the defect, setting a threshold for each defect name can increase the reliability of the visual inspection support system according to the present invention.

[0066] Furthermore, the visual inspection support system 1 may be configured to limit the visual inspection to only those objects belonging to a predetermined category, thereby reducing the number of items to be inspected and easing the burden on inspectors, thus preventing misjudgments and incorrect inputs. For example, as shown in Figure 10, by excluding objects or parts to be visually inspected belonging to the high-quality-rate category and the low-quality-rate category from the visual inspection, only the medium-quality-rate category between the high-quality-rate and low-quality-rate categories is visually inspected. This reduces the number of visual inspections performed by inspectors, allowing them to concentrate their visual inspection efforts. By setting a high threshold score for the high-quality-rate category (for example, 95 or higher), it is possible to prevent defective products from being released without undergoing visual inspection by inspectors. Also, by setting a low threshold score for the low-quality-rate category (for example, 40 or lower), defective products can be removed from the line without undergoing visual inspection by inspectors, preventing both defective products from being released and good products from being removed from the line.

[0067] [Automated Inspection Data] The automated inspection results from the automated inspection computer 12 and the visual inspection support information are stored in the database 10 along with the automated inspection images, which are the imaging data used for automated judgment. The database 10 manages automated inspection data such as automated inspection results, visual inspection support information, automated inspection images, automated shooting condition data (angle, magnification, etc.), substrate ID, inspection area data (XYθ coordinate data), defect name, and quantitative evaluation value (score). Note that the automated inspection data is not limited to those described above.

[0068] [Operation Flow] Next, the operation of the visual inspection device 2A will be described. As shown in Figure 11, first, in step S1, the object to be inspected, such as the printed circuit board 7, is brought into the visual inspection device 2A. Then, in step S2, the imaging camera 11 takes a photograph of a predetermined inspection area from diagonally above and / or from above.

[0069] Then, in step S3, an automated visual inspection is performed. Specifically, the automated inspection processing program uses the automated inspection image to quantitatively evaluate the quality of the product at a predetermined inspection location and generate visual inspection support information. In step S4, the visual inspection device 2A transmits the automated inspection results, visual inspection support information, automated inspection image used for the automated visual inspection, inspection location data (XYθ coordinate data), and various automated inspection data such as defect names to the database 10. Each transmitted data is stored in the database 10.

[0070] Of this automated inspection data, the data obtained by adding reference image data indicating the correct implementation state of the parts to be inspected by an inspector to the automated inspection data relating to the objects to be inspected by an inspector is called visual inspection data. For example, if objects or parts to be inspected that belong to the high good product rate category or the low good product rate category are excluded from the visual inspection, the data obtained by adding reference image data to the automated inspection data of objects or parts to be inspected that are classified as the medium good product rate category will be called visual inspection data. Also, if all objects to be inspected that have undergone automated visual inspection are to be subjected to visual inspection, the data obtained by adding reference image data to the fully automated inspection data will be called visual inspection data. Note that not all of the visual inspection data is necessarily displayed on the display unit 5 of the visual inspection terminal 3A. For example, the reference image may or may not be displayed.

[0071] Subsequently, in step S5, the object to be inspected is removed, and the next object to be inspected is brought in (step S1). In this way, the visual inspection device 2A receives the objects to be inspected as they are transported along the inspection line 9 and performs inspections continuously.

[0072] [Visual Inspection Terminal 3A] The visual inspection terminal 3A is a device that sequentially displays visual inspection images taken by the visual inspection devices 2A of multiple inspection lines 9, and performs centralized visual inspection of each object being transported on multiple inspection lines 9 in one location. It is connected to the visual inspection devices 2A and the database 10 via the bus 8.

[0073] As shown in Figure 12, the visual inspection terminal 3A includes an input unit 6 into which the inspector inputs the results of the visual inspection; a display unit 5 that displays visual inspection images (automatically captured images) taken by the visual inspection device 2A and visual inspection support information for the visual inspection images; a communication unit 17 that communicates with the database 10 and the visual inspection device 2A; and a control unit 18 that controls the operation of the entire visual inspection support system 1, including switching images on the display unit 5 and communicating with the database 10 and the visual inspection device 2A.

[0074] The input unit 6 receives instructions from the inspector to execute the inspection process of the visual inspection support system 1, such as inputting the results of visual inspection, instructing the capture of additional visual inspection images, and instructing the display unit 5 to switch between visual inspection areas and objects being inspected. Known input methods can be used, such as an operation panel with input buttons and levers, or a mouse or keyboard used to input instructions on the operation screen displayed on the display unit 5.

[0075] The inspector operating the visual inspection terminal 3A can perform any operation on the visual inspection device 2A and the database 10 via the input unit 6, such as inputting the visual judgment results, switching the visual inspection image of the object being inspected to a visual inspection image of another object, or issuing a screen display switching instruction to switch to a different visual inspection image of the same object.

[0076] The visual inspection results entered via the input unit 6 may include not only the pass / fail judgment result but also the name of the defect. The name of the defect may be displayed on the display unit 5 from a pre-categorized list of defect names, allowing the user to select one or more of them, or a free-form field may be provided where the name of the defect can be entered.

[0077] The screen display switching instruction displays a list of inspection targets that have been automatically inspected by the visual inspection device 2A and are subject to visual inspection by an inspector on the display unit 5, and switches the visual inspection images in the order they are displayed according to the switching instruction. Alternatively, the inspector may select any one of the displayed inspection targets from the list, thereby switching the visual inspection image displayed on the display unit 5.

[0078] The display unit 5 has one or more monitors and displays visual inspection images (automatically captured images) taken by the visual inspection device 2A, reference images showing the correct mounting state to be compared with the visual inspection images, a list of visual inspection devices 2A on the inspection line 9 that are performing visual inspection or are in standby mode for visual inspection, operation screens, etc.

[0079] [Visual Inspection Data] As described above, visual inspection data is displayed on the display unit 5 of the visual inspection terminal 3A. The visual inspection data displayed on the display unit 5 includes data necessary for visual inspection by the inspector, and includes at least a visual inspection image (automatic inspection image), visual inspection support information, and a reference image showing the correct implementation state of the parts to be inspected. The inspector compares the visual inspection image with the reference image to make a pass / fail judgment. The inspector inputs the visual judgment result from the input unit 6. The input visual judgment result is associated with other automatic inspection data of the object to be inspected or the parts to be inspected, and is stored in the database 10 via the communication unit 17.

[0080] The control unit 18 enables centralized inspection across multiple inspection lines 9 by controlling each part of the visual inspection terminal 3A in response to the inspector's operation, and is configured, for example, by a computer. The control unit 18 can remotely operate the visual inspection device 2A and the database 10 via the bus 8 or other network and have them perform the necessary processing.

[0081] Figure 13 shows an example of the display on the display unit 5 during visual inspection. As a visual inspection image, a front view image 30 of the area being inspected is displayed in the center, and oblique images 31a and 31b are displayed on the left and right. A reference image 32 is displayed in the upper right. In the lower right, "NG (defective)" and "OK (good)" icons, which are input units 6 for entering the visual inspection result, are displayed. The inspector can input the visual inspection result by clicking the "NG" or "OK" icon with a cursor displayed on the display unit 5 using an input means such as a mouse.

[0082] Visual inspection support information 33 is displayed in the upper left of the display unit 5. In the display example shown in Figure 13, the defect names "Foreign matter / dirt," "Insufficient soldering," and "Solder bridging between pins" are displayed, and next to each defect name, the defect probability (%) calculated based on the quantitative evaluation value is displayed. In addition, a graph showing the categories and thresholds is displayed. In the display example shown in Figure 13, it is divided into four categories, and the thresholds are 30%, 70%, and 90%. By referring to the visual inspection support information 33 in addition to the visual inspection images 30, 31a, and 31b, inspectors can prevent judgment errors such as misjudging an inspected item that should be judged as defective as a good product, or misinputting an item that the inspector judged as defective as a good product when inputting the judgment result.

[0083] Visual inspection support information is displayed in a manner appropriate to the category. Figure 14 shows another example of display on the display unit 5 during a visual inspection. The difference from Figure 13 is that the text "Please check carefully. There is a possibility of a solder bridge between pins." is displayed in the upper center as visual inspection support information 33. Among the visual inspection support information 33 displayed in the upper left, "solder bridge between pins" has an 86% failure rate. In this way, when a product is categorized into a category with a high failure rate, additional messages or other displays may be added as visual inspection support information to draw more attention.

[0084] Figure 15 shows another example of the display unit 5 during visual inspection. The difference from Figure 14 is that a frame is displayed surrounding the area suspected of being defective in the front image 30. When quantitatively evaluating the quality based on the automated inspection image (visual inspection image), the area suspected of being defective can also be identified. The frame may be colored, flashed, or otherwise used to draw attention to the area.

[0085] [Operation Flow] The operation of the visual inspection terminal 3A will be explained. As shown in Figure 16, in step S11, the inspector operates the input unit 6 to input a screen display switching instruction to the visual inspection terminal 3A. Next, in step S12, the visual inspection terminal 3A retrieves visual inspection data of the object to be inspected (printed circuit board 7) to be displayed on the display unit 5 from the database 10.

[0086] Then, in step S13, the visual inspection terminal 3A displays the visual inspection image (automatic inspection image), reference image, and visual inspection support information of the object to be inspected that has been switched on the display unit 5, and the inspector performs a visual inspection. The visual inspection terminal 3A may also display other visual inspection data. In addition, the visual inspection image of the part to be inspected displayed on the display unit 5 during the visual inspection is not limited to one image, but may display multiple visual inspection images with different angles and magnifications. In step S14, the inspector performs a visual inspection based on the visual inspection data and inputs the visual judgment result via the input unit 6. In step S15, the visual inspection support terminal 3A stores the visual judgment result in the database 10 in association with the visual inspection data.

[0087] Then, when the visual inspection terminal 3A receives an input (step S11) for switching the screen display to switch to the visual inspection image of another inspection line 9, it reads the visual inspection image (automatic inspection image) and other visual inspection data of the specified inspection line 9 from the database 10 and displays it on the display unit 5. In this way, the visual inspection terminal 3A can display visual inspection images taken by the visual inspection devices 2A of multiple inspection lines 9, and one inspector can perform visual inspections of multiple inspection lines 9.

[0088] [Database 10] As shown in Figure 17, database 10 contains automated inspection data, visual inspection data, and inspection line data. Automated inspection data stores data related to the object to be inspected (printed circuit board 7) and automated photography, as well as data related to quantitative evaluation and visual inspection support information. Data related to the object to be inspected includes, for example, board type ID, board ID, component ID, inspection area data (XYθ coordinates), and automated inspection images. Data related to automated photography includes, for example, shooting condition data (angle, magnification, etc.) and Z-axis correction value. Data related to quantitative evaluation and visual inspection support information includes automated inspection results (defect name, quantitative evaluation value (score)) and visual inspection support information.

[0089] Furthermore, data related to the inspected object is recorded after the visual inspection, along with the visual judgment results made by the inspector. Other data recorded after the visual inspection of the inspected object may include additional shooting condition data (position, angle, magnification, etc.), defect name (after visual inspection), and inspection time (time taken from displaying the visual inspection image to inputting the visual judgment results).

[0090] The visual inspection data consists of automated inspection data related to the object being inspected, plus reference images. During the visual inspection, the visual inspection data is read by the visual inspection terminal 3A and displayed on the display unit 5 as appropriate.

[0091] The inspection line data manages data such as the inspection line ID, which is an identification ID assigned to each inspection line; the visual inspection device ID, which is an identification ID assigned to each visual inspection device 2A; the visual inspection terminal ID, which is an identification ID assigned to each inspector operating the visual inspection terminal 3A; the date and time of inspection; and the quantity inspected.

[0092] The data managed in database 10 is not limited to these.

[0093] Each data point is managed in an interconnected manner and referenced according to the operation of the visual inspection terminal 3A. Furthermore, each data point is managed in an interconnected manner and used for verifying judgment accuracy, improving automatic judgment accuracy, and evaluating and improving the inspection capabilities of inspectors.

[0094] The database 10 may be integrated with the visual inspection device 2A or the visual inspection terminal 3A, or it may be connected to the visual inspection device 2A and the visual inspection terminal 3A via a bus 8 or the internet.

[0095] Furthermore, while database 10 manages automated inspection data, visual inspection data, and inspection line data in a single database, it may also be composed of multiple databases depending on the type and purpose of the data, such as data on the object being inspected, data on automated photography, quantitative evaluation, and visual inspection support information.

[0096] [Inspection Flow] Next, the inspection process of the entire visual inspection support system 1 will be explained with reference to Figure 18. In the following inspection flow steps, the visual inspection support system 1 using the visual inspection device 2A as the visual inspection unit 2 and the visual inspection terminal 3A as the visual inspection unit 3 will be explained as an example.

[0097] First, an automated visual inspection is performed by the visual inspection device 2A (step S31). The automated inspection flow is carried out according to the flow shown in Figure 11, as described above. That is, when the object to be inspected is brought in, the visual inspection device 2A photographs the predetermined areas to be inspected and performs an automated visual inspection based on the obtained automated inspection images. This generates automated inspection data, which includes visual inspection support information. The automated inspection data is stored in the database 10.

[0098] Next, a visual inspection is performed by an inspector using the visual inspection terminal 3A (step S32). The visual inspection flow is carried out according to the flow shown in Figure 16, as described above. That is, when the inspector inputs a screen display switching instruction, the visual inspection terminal 3A retrieves visual inspection data from the database 10 and displays it on the display unit 5. The display unit 5 displays the visual inspection image (automatic inspection image) of the object to be inspected, a reference image, visual inspection support information, and other visual inspection data. Based on this visual inspection data, the inspector makes a judgment on the quality of the visually inspected parts.

[0099] Here, the display unit 5 of the visual inspection terminal 3A displays visual inspection support information along with the visual inspection image. This allows inspectors to perform visual inspections while referring to the visual inspection support information, thereby reducing the variability in pass / fail judgments based on differences in experience and ability among inspectors. In other words, by referring to the visual inspection support information as an indicator for pass / fail judgment, it is possible to reduce the variability in judgments among inspectors, such as when one inspector judges an item as defective but another inspector judges it as good.

[0100] Furthermore, if a single inspector consistently assigns good results to inspections during visual inspection, the quality of the inspection may decline due to inertia or fatigue, increasing the risk of inconsistent judgments, such as incorrectly classifying defective items as good. Besides errors in judgment, inspectors may also mistakenly classify defective items as good due to input errors. However, by referencing visual inspection support information as an indicator for quality judgments, the risk of such misjudgments and input errors can be reduced.

[0101] Furthermore, visual inspection support information is divided into multiple categories based on the score, and by making the category identifiable information, inspectors can allocate inspection time and concentration according to the category. For example, if the inspected items are classified into a high-quality category, a low-quality category, and an intermediate-quality category according to the score, the quality of the inspected items or visual inspection parts belonging to the high-quality or low-quality category can be judged relatively easily. By focusing on the inspected items or visual inspection parts belonging to the intermediate-quality category, visual judgment can be performed efficiently, and this can prevent misjudgments and incorrect input.

[0102] [Selection of Inspection Objects] In Figure 18, all inspection objects that underwent automatic visual inspection by the visual inspection device 2A were subject to visual inspection. However, as shown in Figure 19, the number of inspections may be reduced by selecting which objects to visually inspect. As described above, by setting thresholds for quantitative evaluation values ​​(scores), inspection objects can be classified into multiple categories, and inspection objects belonging to a predetermined category can be excluded from visual inspection, or only inspection objects belonging to a predetermined category can be subject to visual inspection.

[0103] Specifically, after an automatic visual inspection by the visual inspection device 2A (step S33), the control unit 18 of the visual inspection terminal 3A selects only the visual inspection data of the inspected objects belonging to a predetermined category (step S34). For example, inspected objects or visual inspection parts belonging to the high-quality-rate category and the low-quality-rate category are excluded from the visual inspection. Then, only the visual inspection data of inspected objects belonging to the medium-quality-rate category, which is between the high-quality-rate category and the low-quality-rate category, is displayed on the display unit 5 (step S35).

[0104] When inspectors perform an excessive number of judgment tasks, fatigue and complacency can lead to misjudgments and incorrect data entry. By focusing on inspection items where pass / fail judgment is ambiguous, the number of visual inspections performed by inspectors can be reduced, allowing them to concentrate on the visual inspection.

[0105] [Display after judgment] In the above inspection flow, visual inspection support information was displayed during the visual inspection by the inspector. However, the visual inspection support system 1 may also be configured to display the visual inspection support information after the inspector has entered the judgment result. This allows the inspector to compare their own judgment result with the judgment made by the automated visual inspection to confirm and verify the accuracy of their own judgment.

[0106] Figure 20 shows the process of displaying visual inspection support information after this determination. Similar to the flow shown in Figure 16, when a screen display switching instruction is input (step S11), the visual inspection terminal 3A retrieves the visual inspection data of the object to be inspected (printed circuit board 7) to be displayed on the display unit 5 from the database 10 (step S12).

[0107] In step S16, the visual inspection terminal 3A displays the visual inspection image (automatic inspection image) and reference image of the object to be inspected that was switched on the display unit 5, and the inspector performs the visual inspection. The visual inspection terminal 3A may also display other visual inspection data in addition to the visual inspection support information.

[0108] Figure 21 shows an example of the display shown on the display unit 5 during the process of displaying visual inspection support information after the judgment is made. As a visual inspection image, a frontal image 30 of the visual inspection area is displayed in the center, and oblique images 31a and 31b are displayed on the left and right. A reference image 32 is displayed in the upper right. "NG" and "OK" icons, which are input units 6 for entering the visual judgment result, are displayed in the lower right. However, the visual inspection support information 33 is not displayed in the upper left of the display unit 5. The inspector can input the visual judgment result by clicking the "NG" and "OK" icons with a cursor displayed on the display unit 5 using an input means such as a mouse.

[0109] In step S17, the inspector performs a visual inspection based on the visual inspection data and inputs the visual judgment result via the input unit 6. In step S18, the visual inspection support terminal 3A displays the visual inspection support information for the visual inspection area of ​​the object being inspected on the display unit 5 (see Figures 14 and 13). The automatic inspection results of the visual inspection device 2A may also be displayed. This allows the inspector to refer to the visual inspection support information and the automatic inspection results of the visual inspection device 2A to confirm and verify their own judgment result. Subsequently, in step S19, the visual judgment result is re-input as necessary. If there is no need to change the visual judgment result, no input operation is performed, or input indicating that no change is necessary is made. In step S20, the visual inspection support terminal 3A saves the visual judgment result in the database 10 in association with the visual inspection data.

[0110] Furthermore, the step of displaying visual inspection support information after the inspector has entered the judgment result may be executed if certain conditions are met, for example, if there is a discrepancy between the visual inspection support information or the automated inspection result and the judgment result entered by the inspector.

[0111] For example, visual inspection support information may be displayed on the display unit 5 when an inspector determines that an object or part of an object classified into a low good product category in an automated visual inspection is a good product. In this case, if the inspector determines that an object is defective, the visual inspection support information will not be displayed, thus shortening the inspection time.

[0112] Furthermore, there are no particular restrictions on the conditions for displaying visual inspection support information after inputting the judgment result. In addition to the above, it can be set as appropriate, such as when an inspector judges an object or visual inspection area classified in the high good product rate category as defective in automated visual inspection, or when an inspector judges an object or visual inspection area classified in the low good product rate category as good in automated visual inspection.

[0113] Furthermore, the visual inspection support system 1 may display visual inspection support information along with a notation indicating a match when the judgment result of the automated visual inspection matches the visual judgment result. In addition, the visual inspection support system 1 may display the visual inspection support information when the inspector makes a visual judgment and may redisplay it after the visual judgment result has been entered.

[0114] [Restriction on Judgment Input] In addition, the visual inspection support system 1 may, if certain conditions are met, for example, if the judgment result entered by the inspector does not match the visual inspection support information or automatic inspection result, prevent the inspector from entering a judgment of good or bad product. For example, the visual inspection support system 1 may restrict the inspector from entering a judgment of good product for an object or part of a visual inspection that has been classified into a low good product rate category in the automatic visual inspection of the visual inspection device 2A, and may also display a warning. This can suppress misjudgments or incorrect inputs by inexperienced inspectors or inspectors who have accumulated fatigue.

[0115] There are no particular restrictions on the conditions for restricting input for good product judgment. In addition to the above, it is possible to set conditions as appropriate, such as restricting inspectors from judging inspected items or visually inspected areas that are classified into the high good product rate category as defective in automated visual inspection, or restricting inspectors from judging inspected items or visually inspected areas that are classified into the low good product rate category as good products in automated visual inspection.

[0116] Figure 22 shows an example of the display on the display unit 5 when judgment input is restricted. Of the "NG (defective judgment)" and "OK (good product judgment)" icons in the input unit 6, the "OK" icon is grayed out and cannot be clicked.

[0117] [Multiple Person Judgments] Furthermore, if certain conditions are met, for example, if the judgment result entered by the inspector does not match the judgment information or automated inspection result, the visual inspection support system 1 may require the input of judgment results from other inspectors in addition to the inspector's judgment result in order for the inspector to accept the judgment input of good or defective. In this case, the input will only be accepted if the judgment results of the inspector and the other inspectors are the same. Other inspectors may be inspectors with more experience than the inspector, supervisors, managers, etc. The number of other inspectors may be one or two or more. This allows for accurate visual judgment, and the inspector can compare their own judgment results with the judgment from automated visual inspection to confirm and verify the accuracy of their own judgment.

[0118] There are no particular restrictions on the conditions requiring judgment by multiple people. For example, they can be set as appropriate, such as when an inspector judges an object or visually inspected area classified in the low good product rate category as a good product in an automated visual inspection, or when an inspector judges an object or visually inspected area classified in the high good product rate category as a defective product in an automated visual inspection.

[0119] [Administrator's Judgment] Furthermore, if certain conditions are met, the visual inspection support system 1 may require the administrator's visual judgment and input of the visual judgment result when an inspector judges an object or part of an object that has been classified into a category that is difficult to judge even within the medium-good product rate category in an automated visual inspection. Figure 23 shows an example of a display where the "Administrator Only" icon is displayed on the input unit 6. As shown in Figure 23, the display unit 5 does not display the "NG" or "OK" icons that the administrator would use for the input unit 6, but instead displays the "Administrator Only" icon. Then, by entering a password known only to the administrator, the "NG" or "OK" icons are displayed, and the visual judgment result can be entered. This makes it possible to accurately perform visual judgments on objects or parts of an object that have been classified into a category that is difficult to judge.

[0120] [Acceptable Product Icon] In addition, as shown in Figure 24, the visual inspection support system 1 may display an "acceptable product" icon in addition to the "NG (defective product)" and "OK (good product)" icons as the input unit 6 for inputting the visual judgment results. The "acceptable product" icon is entered for the inspection target or visual inspection part that cannot be clearly called a good product, but is implemented at a level that does not cause practical problems even if it is distributed in the market.

[0121] This allows us to collect data on how inspectors visually judged the results of the automated inspection by the visual inspection device 2A, and statistically analyze this data to help, for example, set thresholds for each category.

[0122] Furthermore, this "acceptable item" icon may be displayed in all visual inspections, or it may be displayed only in visual inspections of a specified defect name (defect description). In other words, some defects may require a strict pass / fail judgment, while others may allow for a more flexible pass / fail judgment. By providing inspectors with options for inputting the pass / fail result in the latter type of visual inspection, the results of the visual inspection can be appropriately reflected.

[0123] Table 1 shows an example of setting defect names, categories, and thresholds. The "Defect Name" column lists the defect names that describe the nature of the defect. In the settings shown in Table 1, categories 1 through 4 are set, and the threshold for each category differs for each defect name. The threshold uses the defect probability (%) calculated based on a quantitative evaluation value of the quality. In the settings shown in Table 1, the higher the defect probability, the higher the category number. That is, Category 1 is a high-quality product category, Category 4 is a low-quality product category, Categories 2 and 3 are medium-quality product categories, and Category 3 has a higher defect probability than Category 2.

[0124]

[0125] Each category has defined control content for visual inspection. Examples of control content are as follows: As shown in Table 1, visual inspection support information is displayed in a manner corresponding to the defect name and / or category. Note that the present invention is not limited to the defect names, categories and thresholds shown in Table 1. Automatic OK: Automatically determines OK. The inspector does nothing. (If the initial setting is to display after judgment, it behaves the same as "OK only") OK only: Only the OK button is enabled. The NG button is disabled. The worker has no choice but to press the OK button to determine OK. —: Visual inspection support information is displayed, but no warning message is displayed at the top of the display unit 5. Standard inspection warning: In addition to visual inspection support information, a warning message is displayed at the top of the display unit 5. Administrator call: Requires input of a password known only to the administrator for OK / NG judgment NG only: Only the NG button is enabled. The OK button is disabled. The inspector has no choice but to press the NG button to determine NG. Automatic NG: Automatically determines NG. The inspector does nothing. (If the default setting is to display after judgment, it will behave the same as "NG only".)

[0126] [Feedback] In addition, the visual inspection support system 1 may display the visual inspection support information and visual judgment results on the display unit as reference information when visually inspecting other objects, along with the visual inspection support information for other objects. That is, in the inspection of the same type of object or the same part requiring inspection on the same type of object, the judgment results of the visual inspection performed previously may be displayed in addition to the visual inspection support information.

[0127] The visual inspection results from previous visual inspections may include, for example, the number of good and defective items. In addition, the visual inspection images used for the judgment, the name of the inspector who made the judgment, or the inspector's ID may also be displayed.

[0128] [Batch Inspection of Inspection Targets] Alternatively, the visual inspection support system 1 may perform sequential visual inspections of multiple inspection targets after multiple sets of automatic inspection data for inspection targets requiring visual inspection have been recorded in the database 10. This control is performed by the control unit 18 of the visual inspection terminal 3A. After multiple sets of automatic inspection data have been recorded, an inspection order is assigned to these visual inspection data, and the display unit 5 is displayed, a visual inspection is performed, and the visual inspection results are input sequentially (see Figure 16, etc.).

[0129] [Proactive Photography] While the visual inspection support system 1 is performing a visual inspection of an object being transported on one inspection line 9 using the visual inspection terminal 3A, the visual inspection device 2A on another inspection line 9 may perform an automatic visual inspection of the parts of the object to be inspected that are being transported on the other inspection line 9, and save the automatic inspection data in the database 10. After the other inspection line 9 has performed an automatic visual inspection of an object, it can perform an automatic visual inspection of the next object without waiting for the visual inspection terminal 3A to finish, thereby shortening the cycle time of the automatic visual inspection process without any waiting time for visual inspection.

[0130] When the visual inspection terminal 3A completes the visual inspection of an object being transported on one inspection line 9, it switches its display screen to read the visual inspection data from the other inspection line 9 from the database 10 and displays it on the display unit 5, and then performs the visual inspection. In this way, the visual inspection support system 1 can shorten the cycle time of the visual inspection process by having the visual inspection device 2A installed on each inspection line 9 perform the automatic visual inspection in advance, and then having the visual inspection terminal 3A sequentially read the visual inspection data stored in the database 10 and perform the visual inspection.

[0131] In other words, the visual inspection support system 1 is such that each inspection line 9 is equipped with an appearance inspection device 2A. Visual inspection of the objects to be inspected transported on each inspection line 9 is performed centrally by displaying the visual inspection images on a display unit 5 such as a monitor provided on the visual inspection terminal 3A. Furthermore, the visual inspection by the visual inspection terminal 3A is performed using images captured by the appearance inspection device 2A.

[0132] Therefore, if visual inspection is required simultaneously on multiple inspection lines 9, the other inspection lines will be on standby while the visual inspection of the object being inspected is being performed on one inspection line 9. Then, after the visual inspection on one inspection line 9 is completed, if the visual inspection device 2A on another inspection line moves its imaging camera to the part of the object that needs to be inspected, time is required for the camera to move and focus in order to photograph the part of the object that needs to be inspected at a predetermined angle and depth of field. During this time, the inspector is in an inspection waiting state and cannot proceed with the inspection using the visual inspection terminal 3A, resulting in a loss of cycle time for the inspection process. If there are multiple parts that need to be inspected on a single object, the time required to move the imaging camera will increase further, extending the cycle time of the inspection process.

[0133] Therefore, the visual inspection support system 1 according to this technology performs an inspection of one inspection line 9 using a visual inspection terminal 3A, while the visual inspection device 2A of the other inspection line 9 takes photographs of the parts of the object to be inspected on that other inspection line 9 under predetermined shooting conditions. In this technology, this is called pre-emptive shooting. The automated inspection images acquired by pre-emptive shooting are stored in the database 10 along with other automated inspection data. When the inspector finishes inspecting one inspection line 9, the display on the display unit 5 of the visual inspection terminal 3A is switched to the pre-emptively shot visual inspection images of the other inspection line 9. The pre-emptively shot visual inspection image data of the other inspection line 9 is read from the database 10 and displayed on the display unit of the visual inspection terminal 3A. This allows the inspector to quickly perform a visual inspection of the object to be inspected on that other inspection line 9.

[0134] In this way, the visual inspection support system 1 can shorten the cycle time of the visual inspection process by displaying the visual inspection images captured in advance by the visual appearance inspection devices 3 installed on each inspection line 9 on the visual inspection terminal 3A.

[0135] [Additional Photography] The visual inspection support system 1 may also perform additional photography using the visual inspection device 2A while a visual inspection is being performed using the visual inspection terminal 3A. When the visual inspection device 2A receives an instruction to take additional photographs from the visual inspection terminal 3A during a visual inspection using the visual inspection image, the imaging camera 11 is driven in accordance with the operation of the visual inspection terminal 3A and photographs the object to be inspected. The additionally photographed image is displayed as a real-time image on the display unit 5 of the visual inspection terminal 3A. This allows the inspector operating the visual inspection terminal 3A to display the object to be inspected at a desired angle and magnification, and to observe areas requiring further inspection in more detail, such as when it is difficult to make a judgment using the predetermined angle and magnification of the visual inspection image.

[0136] Furthermore, real-time images additionally captured by the visual inspection device 2A may be converted into data in response to the operation of the visual inspection terminal 3A, and this additionally captured video data may be stored in the database 10 as the visual inspection image used for judgment, associated with the judgment result and other information. Alternatively, the visual inspection device 2A may capture one or more still images of the real-time images obtained through additional capture in response to the operation of the visual inspection terminal 3A, and store the additionally captured image data in the database 10.

[0137] Furthermore, additional shooting condition data such as angle and magnification set by the inspector can be stored in the database 10 as visual inspection data in association with the substrate type ID, etc. Subsequent automatic shooting for the same substrate type, etc., can be performed based on this additional shooting condition data, or subsequent automatic shooting can be performed based on the additional shooting condition data in addition to the predetermined shooting conditions. This allows for automatic shooting with desired shooting conditions for inspection targets of the same substrate type, etc., enabling efficient visual inspection by the inspector.

[0138] Furthermore, when the visual inspection image is displayed on the display unit 5 and the inspector performs a visual inspection, the visual inspection device 2A may be configured to wait at the position where the imaging camera 11, which performed the automatic shooting, has captured the first inspection area that the inspector will visually inspect first using the visual inspection image. If there are multiple inspection areas for a single object to be inspected, that is, if there are multiple visual inspection images that are the subject of the visual inspection, the visual inspection images are displayed on the display unit 5 in a predetermined order and used for the inspector's visual inspection.

[0139] Here, the visual inspection device 2A positions the imaging camera 11 to capture the first inspection area in preparation for the inspector's instruction to take additional images. The first inspection area refers to the area of ​​the object being inspected during visual inspection that is first displayed on the display unit 5 as a visual inspection image. As a result, when the inspector instructs for additional images to be taken, the imaging camera 11 can quickly take additional images, eliminating the time required to move the imaging camera 11 and shortening the cycle time of the inspection process.

[0140] Alternatively, the visual inspection device 2A may move the imaging camera 11 to the area corresponding to the visual inspection image sequentially displayed on the display unit 5. As a result, when the inspector instructs the imaging camera 11 to take additional shots, the imaging camera 11 can quickly take additional shots as the imaging camera 11 moves to the area displayed on the display unit 5, thereby reducing the time required to move the imaging camera 11 and shortening the cycle time of the inspection process.

[0141] Figure 25 is a flowchart showing the operation flow of the visual inspection terminal 3A when an additional shooting instruction is given. Similar to the flow shown in Figure 16, when a screen display switching instruction is input (step S11), the visual inspection terminal 3A retrieves visual inspection data of the object to be inspected (printed circuit board 7) to be displayed on the display unit 5 from the database 10 (step S12). Then, the visual inspection terminal 3A displays the visual inspection image (automatic inspection image), reference image, and visual inspection support information of the object to be inspected that was switched on the display unit 5, and the inspector performs a visual inspection (step S13).

[0142] Next, the inspector inputs an instruction to take additional images via the input unit 6 (step S21). The visual inspection device 2A drives the imaging camera 11 in response to the operation signal from the input unit 6 and photographs the object to be inspected at the angle and magnification corresponding to the operation. The captured images are displayed as real-time images on the display unit 5, and the inspector performs a visual inspection. The display unit 5 may also display the visual inspection images along with the additional captured images.

[0143] Next, the inspector inputs the judgment result (step S22). At this time, if any corrections are made to the automated inspection data, such as the defect name, the correction details are also input. Then, the visual inspection terminal 3A stores the judgment result in the database 10 in association with visual inspection data such as the board ID, via the control unit 18 (step S23). In addition, the visual inspection images (additional images) used for the judgment are converted into data and stored in the database 10 in association with the judgment result and visual inspection data.

[0144] The digitization of visual inspection images (additional images) and their storage in database 10 is the operation of storing the visual inspection images used as the basis for the judgment as image data in the first database 10A. The images additionally captured by the visual appearance inspection device 3 based on the inspector's instructions are real-time images and are not replayed images recorded in database 10. Therefore, the visual inspection images used as the basis for the pass / fail judgment are stored in database 10 as video data and / or image data, associated with data such as the pass / fail judgment result, defect name, and board ID.

[0145] [AOI] The visual inspection support system 1 may also perform an automated inspection using an automated optical inspection device such as an automated optical inspection device (AOI) 20 before the automated visual inspection by the visual inspection unit 2 or the visual inspection device 2A. As shown in Figure 26, the AOI 20 is connected via a bus 8 to the visual inspection devices 2A, the visual inspection terminals 3A, and the database 10 installed on each inspection line 9, and is capable of mutual data communication. Of course, the visual inspection support system 1 may also use a network connection configuration other than the bus type (see Figure 27).

[0146] AOI20 automatically performs visual inspection of the parts of an object that need to be inspected. It photographs the object on the inspection line 9 and automatically determines the quality of the object and the parts that need to be inspected based on the image data. AOI20 is installed on the inspection line 9 upstream of the visual inspection device 2A in the direction of transport of the object.

[0147] The AOI 20 has a configuration similar to that of known AOIs, and for example, as shown in Figure 28, it comprises an imaging camera 21 such as a CCD camera that photographs the object to be inspected, an AOI computer 22 that performs pass / fail judgment based on the image data captured by the imaging camera 21, a communication unit 23 connected to the bus 8 and communicating with a database 10, etc., and a control unit 24 that controls the operation of the entire AOI.

[0148] The AOI computer 22, based on an automatic judgment processing program, uses the imaging camera 21 to photograph predetermined inspection points on the object to be inspected and uses the captured image data to determine whether the object is good or bad at those predetermined inspection points. Specifically, the automatic visual inspection device 2 has pre-existing position data (XYθ coordinates) indicating the inspection points of the printed circuit board 7 to be inspected, and, according to the program, sequentially captures inspection images of the positions indicated by the position data, for example, in order of proximity. The automatic judgment result can be arbitrarily set to, for example, "OK," "NG," or "Undeterminable." A known image processing program can be used for the good or bad judgment.

[0149] As a result, the inspection system 1 can use the AOI 20 to automatically determine the quality of the soldering of electronic components mounted on the printed circuit board 7, and save the automatic determination results, along with the location data of the inspection area, to the database 10 via the communication unit 23.

[0150] [Inspection Flow] Here, the visual inspection support system 1 may perform automatic visual inspection by the visual inspection device 2A on all items, similar to the AOI 20, or it may be limited to items that have been judged "OK" by the AOI 20 or to parts that require inspection. Specifically, as shown in Figure 29, in step S41, an automatic inspection is performed by the AOI 20. This performs a visual inspection on all items. Next, in step S42, an automatic visual inspection is performed by the visual inspection device 2A. The target of this automatic visual inspection is limited to items that have been judged "NG" or "undeterminable" by the AOI 20, or to parts that require inspection, excluding items that have been judged "OK" or to parts that require inspection. The position data (XYθ coordinates) of the parts that require inspection to be performed by the visual inspection device 2A are read from the database 10 along with the automatic judgment result of the AOI 20. After that, in step S43, a visual inspection is performed using the visual inspection terminal 3A.

[0151] This reduces the number of inspections performed by the automated visual inspection device 2A and the visual inspection terminal 3A, thereby shortening the cycle time for visual inspection and reducing the burden on inspectors, thus preventing misjudgments and incorrect inputs.

[0152] In Figure 29, all objects that underwent automatic visual inspection by the visual inspection device 2A were subject to visual inspection. However, as shown in Figure 30, the number of objects to be visually inspected may be reduced by selecting which objects to inspect visually. As described above, by setting thresholds for quantitative evaluation values ​​(scores), objects to be inspected can be classified into multiple categories, and objects belonging to a predetermined category can be excluded from visual inspection, or only objects belonging to a predetermined category can be subject to visual inspection.

[0153] Specifically, an automated inspection by AOI 20 (step S44) and an automated visual inspection by the visual inspection device 2A (step S45) are performed. The objects to be inspected by the visual inspection device 2A may be all the objects that AOI 20 has targeted for inspection, or they may be objects other than those that AOI 20 has judged as "OK". Subsequently, the control unit 18 of the visual inspection terminal 3A selects only the visual inspection data of objects belonging to a predetermined category (step S46). For example, objects or visual inspection parts belonging to the high-quality-rate category and the low-quality-rate category are excluded from the visual inspection. Then, only the visual inspection data of objects belonging to the medium-quality-rate category between the high-quality-rate category and the low-quality-rate category is displayed on the display unit 5 (step S47). This reduces the number of visual inspections performed by the inspector and allows for focused visual inspection.

[0154] [Changing Shooting Conditions] The imaging conditions for the object to be inspected by the visual inspection device 2A may be different from the imaging conditions for the object to be inspected by the AOI 20. For example, when the visual inspection device 2A is used to photograph areas that need to be inspected, the angle and magnification of the imaging camera 11 (oblique camera 11a and planar camera 11b) may be set differently from the angle and magnification of the imaging camera 21 of the AOI 20 to capture images that are more suitable for automated visual inspection and visual judgment. [Multiple Inspection Lines] Also, as shown in Figure 31, the visual inspection support system 1 may consist of multiple inspection lines 9 (for example, inspection line 9A, inspection line 9B, inspection line 9C, etc.) each equipped with an AOI 20 and a visual inspection device 2A, and one visual inspection terminal 3A. The single visual inspection terminal 3A may sequentially display the visual inspection images and visual inspection support information captured by the visual inspection devices 2A of each inspection line 9, allowing for centralized visual inspection.

[0155] 1 Visual inspection support system, 2 Visual inspection unit, 2A Visual inspection device, 3 Visual inspection unit, 3A Visual inspection terminal, 5 Display unit, 6 Input unit, 7 Printed circuit board, 8 Bus, 9 Inspection line, 10 Database, 11 Imaging camera, 11a Oblique camera, 11b Planar camera, 12 Computer for automatic inspection, 13 Communication unit, 14 Support unit, 17 Communication unit, 18 Control unit, 20 AOI, 21 Imaging camera, 22 Computer for AOI, 23 Communication unit, 24 Control unit

Claims

1. Visual Inspection Support System comprising: an imaging device that captures images of predetermined inspection areas of an object to be inspected for visual inspection by an inspector, automatically performs a visual inspection using the captured images, quantitatively evaluates the quality of the predetermined inspection areas, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection areas based on the quantitative evaluation; a display unit that displays images captured by the imaging device, and an input unit that inputs the results of the visual inspection by an inspector, wherein the display unit displays visual inspection images captured by the imaging device of the inspection areas of the object to be inspected, which have been inspected by the visual inspection unit and are subject to visual inspection by an inspector, and the visual inspection support information.

2. The visual inspection support system according to claim 1, wherein the quantitative evaluation involves scoring the quality of the inspected area, and the display unit displays visual inspection support information based on the score.

3. The visual inspection support system according to claim 2, wherein the visual inspection support information is divided into multiple categories for each predetermined range based on the score, and the category is identifiable information.

4. The visual inspection support system according to claim 3, wherein the visual inspection support information is displayed in a manner corresponding to the name and / or category of the defect.

5. The visual inspection support system according to claim 3, wherein the threshold scores for classifying the multiple categories can be arbitrarily set and / or changed.

6. The visual inspection support system according to claim 4, wherein the inspection areas classified into high-quality product category and low-quality product category in the visual inspection support information are excluded from the scope of visual inspection by the visual inspection unit.

7. The visual inspection support system according to any one of claims 1 to 6, wherein the visual inspection support information is displayed on the display unit together with the captured image when the inspector makes a visual judgment.

8. The visual inspection support system according to any one of claims 1 to 6, wherein the visual inspection support information is displayed on the display unit after the visual judgment result is entered by the inspector.

9. The visual inspection support system according to claim 8, wherein the visual inspection support information is displayed on the display unit when the visual inspection support information or the automated inspection result does not match the judgment result entered by the inspector.

10. The visual inspection support system according to any one of claims 1 to 6, wherein if the visual inspection support information or automated inspection result does not match the judgment result entered by the inspector, the system does not accept the input of the judgment result by the inspector.

11. The visual inspection support system according to any one of claims 1 to 6, wherein, in the event that the visual inspection support information or automated inspection result does not match the judgment result entered by the inspector, the system also requires the input of judgment results by other inspectors in order to accept the input of the judgment result.

12. The visual inspection support system according to any one of claims 1 to 6, wherein the visual inspection support information and the visual judgment result are displayed on the display unit as reference information together with the visual inspection support information of the other objects to be inspected when visually inspecting other objects to be inspected.

13. A visual inspection support system according to any one of claims 1 to 6, comprising a visual inspection terminal constituting the visual inspection unit and a visual inspection terminal constituting the visual inspection unit.

14. The visual inspection support system according to claim 13, comprising: a plurality of inspection lines, each having the aforementioned visual inspection terminal; and a database storing images captured by the visual inspection terminals, location information of the inspection area, visual inspection support information, and visual judgment results, wherein the visual inspection terminal is connected to each of the plurality of inspection lines' visual inspection terminals and the database, and centrally performs visual inspection of the inspection target objects being transported to each of the inspection lines.

15. A visual inspection support system according to any one of claims 1 to 6, comprising an automatic optical inspection device that automatically performs a visual inspection of the object to be inspected prior to a visual inspection by the visual inspection unit, wherein the visual inspection unit performs the visual inspection using the parts determined to be defective by the automatic optical inspection device as the inspection areas.

16. The visual inspection support system according to claim 15, wherein the imaging conditions of the object to be inspected by the visual inspection unit are different from the imaging conditions of the object to be inspected by the automatic optical inspection device.

17. A visual inspection method comprising: a step of performing an automated visual inspection of an object to be inspected by an inspector using a visual inspection unit, and quantitatively evaluating the quality of the inspected area; a step of generating visual inspection support information that serves as an indicator for determining the quality of the inspected area based on the quantitative evaluation; a step of displaying on the display unit of the visual inspection unit a visual inspection image taken by the imaging device of the visual inspection unit of the inspected area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is to be inspected by an inspector, and the visual inspection support information; and a step of receiving input of the visual judgment result from the inspector.

18. The visual inspection method according to claim 17, comprising: an appearance inspection device constituting the appearance inspection unit; a visual inspection terminal constituting the visual inspection unit; a plurality of inspection lines, each having the appearance inspection device; and a database for storing automated inspection data including images captured by the appearance inspection device and visual inspection support information, wherein the visual inspection terminal is connected to each appearance inspection device of the plurality of inspection lines and the database, and centrally performs visual inspection of the objects to be inspected being transported to each of the inspection lines.

19. The visual inspection method according to claim 18, wherein the visual inspection terminal sequentially performs visual inspections on multiple objects to be inspected after the database has recorded multiple sets of automatic inspection data for objects to be inspected that require visual inspection.

20. An appearance inspection unit having an imaging device that captures a predetermined inspection area of ​​an object to be inspected for visual inspection by an inspector, automatically performs an appearance inspection using the captured image, scores the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the score; and a visual inspection unit having a display unit that displays a visual inspection image of the inspection area of ​​the object to be inspected, which has been inspected by the appearance inspection unit and is to be inspected for visual inspection by an inspector, the visual inspection support information, and an input unit for inputting the visual judgment result by the inspector, wherein the inspection area is divided into a high-quality category based on the score, a low-quality category based on the score, a low-quality category based on the score, and a medium-quality category between the high-quality category and the low-quality category. A visual inspection support system that excludes the inspection areas, which are classified into high-quality product category and low-quality product category, from the scope of visual inspection using the visual inspection unit.

21. The visual inspection unit has the following steps: to photograph a predetermined inspection area of ​​an object to be inspected by an inspector using an imaging device, to automatically perform a visual inspection using the captured image, to score the quality of the predetermined inspection area, and to generate visual inspection support information that serves as an indicator for judging the quality of the inspection area based on the score; to display on the display unit of the visual inspection unit the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is to be inspected by an inspector, and the visual inspection support information; and to receive input of the visual judgment result from the inspector. The inspection area is divided into a high-quality category based on the score, where there is a high probability that the inspection area is a good product, a low-quality category where there is a low probability that the inspection area is a good product, and a medium-quality category between the high-quality category and the low-quality category. A visual inspection method comprising excluding the inspection areas, which are classified into high-quality product category and low-quality product category, from the scope of visual inspection using the visual inspection unit.

Images