Visual inspection support system

Abstract

This invention provides a visual inspection support device that displays visual inspection support information during visual inspections by inspectors, thereby maintaining the quality of the inspection. [Solution] The visual inspection support system 1 includes an imaging device 11 which captures a predetermined inspection area of ​​an object to be inspected with the imaging device 11, automatically performs a visual inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information based on the quantitative evaluation; a visual inspection unit 3 which has a display unit 5 which displays the image captured by the imaging device 11, and an input unit 6 which inputs the visual judgment result by the inspector, and the display unit 5 displays the visual inspection support information.

Description

Technical Field

[0001] The present technology relates to an inspection apparatus for inspecting various inspection objects such as printed circuit boards, mounting boards, liquid crystal panels, etc., and relates to a visual inspection support system having a support function for efficiently performing visual inspection by an inspector.

Background Art

[0002] Conventionally, automatic inspection apparatuses have been provided for inspecting pads, wiring patterns, mounting states of electronic components, etc. in various inspection objects such as printed circuit boards, mounting boards, liquid crystal panels, etc. For inspection objects determined to be defective by the automatic inspection apparatus, subsequent visual inspection is then performed by an inspector.

[0003] In visual inspection, generally, an inspector visually reinspects an inspection image taken by an automatic inspection apparatus to determine whether it is a good product or a defective product.

[0004] Also, a plurality of inspection lines each equipped with an automatic inspection apparatus are provided, and inspection images related to inspection objects determined to be defective in the automatic inspection apparatuses of each inspection line are sequentially displayed on one inspection terminal, and centralized visual inspection is also performed on one inspection terminal.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The inspection process is provided to prevent defective products from flowing out to subsequent processes. Therefore, in the inspection process, it is necessary to prevent, as much as possible, determining defective products as good products. On the other hand, when a good product is determined to be defective, at least the situation where defective products flow out to subsequent processes does not occur.

[0007] Therefore, the automated inspection system determines that all items except those that are definitively good, including those whose quality is uncertain, are defective and subjected to visual inspection by inspectors. In other words, many good items may be included in the items subjected to visual inspection. As a result, many of the items subjected to visual inspection by inspectors are judged to be good.

[0008] Thus, if a single inspector consistently assigns good marks to items during visual inspection, the quality of the inspection may decline, creating a risk that items that should be incorrectly classified as defective may be mistakenly identified as good due to inertia. Reasons for an inspector incorrectly classifying a defective item as good include errors in the inspector's judgment or erroneous input where an item judged as defective is mistakenly recorded as good during the input process.

[0009] Furthermore, while visual inspections can yield different results depending on the inspector's experience and ability, it is necessary to minimize the variability in pass / fail judgments based on differences in experience and ability among inspectors.

[0010] One aspect of the present invention aims to provide a visual inspection support device that can display visual inspection support information during visual inspection by an inspector, thereby maintaining or improving the quality of the inspection.

[0011] Furthermore, the description of this problem does not preclude the existence of other problems. Moreover, one aspect of the present invention does not need to solve all of these problems. It is possible to extract other problems from the description in the specification, drawings, and claims. [Means for solving the problem]

[0012] To solve one of the above-mentioned problems, a visual inspection support system according to one aspect of the present invention includes: an appearance inspection unit having an imaging device that captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation; a display unit that displays the image captured by the imaging device; and an input unit that inputs the visual inspection result by the inspector, wherein the display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the appearance inspection unit and is subject to visual inspection by an inspector, and the visual inspection support information. The quantitative evaluation involves scoring the quality of the inspected area, and the display unit displays visual inspection support information based on the score. The visual inspection support information is divided into multiple categories based on the score, within predetermined ranges, and is information that allows for the identification of the category. The visual inspection support information is displayed in a manner corresponding to the defect name and / or category. In the visual inspection support information, the inspected areas that are divided into high-quality category and low-quality category are excluded from the visual inspection by the visual inspection unit. It is.

[0013] Furthermore, in order to solve one of the above-mentioned problems, a visual inspection method according to one aspect of the present invention comprises the steps of: performing an automatic visual inspection of an object to be inspected by an inspector using a visual inspection unit and quantitatively evaluating the quality of the inspected area; generating visual inspection support information that serves as an indicator for determining the quality of the inspected area based on the quantitative evaluation; displaying on the display unit of the visual inspection unit a visual inspection image taken by the imaging device of the visual inspection unit of the inspected area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, and the visual inspection support information; and receiving input of the visual judgment result from the inspector. The system comprises an appearance inspection unit comprising an appearance inspection device and a visual inspection terminal comprising a visual inspection unit comprising a plurality of inspection lines, each having the appearance inspection device, and a database that stores automated inspection data including images captured by the appearance inspection device and visual inspection support information. The visual inspection terminal is connected to each appearance inspection device in the plurality of inspection lines and to the database, and centrally performs visual inspection of the objects to be inspected being transported to each of the inspection lines. It is. [Effects of the Invention]

[0014] According to one aspect of the present invention, the display unit of the visual inspection unit displays an image of the inspection area and visual inspection support information. This prevents errors in judgment where an inspector mistakenly identifies an object that should be judged as defective as a good product, and prevents errors in input where an inspector mistakenly enters a good product as the result of a judgment that the inspector has judged as defective. Therefore, according to one aspect of the present invention, it is possible to prevent a situation where a defective product is judged as good and flows to a later process. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a diagram showing an example of the overall configuration of the inspection system. [Figure 2] Figure 2 is a diagram showing an example of the overall configuration of the inspection system. [Figure 3] Figure 3 is a diagram showing an example of the connection configuration of the inspection system. [Figure 4] Figure 4 is a diagram showing the main configuration of the appearance inspection device (appearance inspection unit). [Figure 5] Figure 5 is a perspective view of the appearance of the appearance inspection device. [Figure 6] Figure 6 is a diagram showing the configuration of the imaging camera. [Figure 7] Figure 7 is a diagram showing a state in which the substrate is flat, warped upward, or warped downward. [Figure 8] Figure 8 is a diagram showing a configuration in which the matching score is set with a score (numerical value) of 100 levels. [Figure 9] Figure 9 is a diagram showing a configuration in which the matching score is divided into three categories: a high good product rate category, a low good product rate category, and a medium good product rate category. [Figure 10] Figure 10 is a diagram showing a configuration in which the inspection object or visual inspection part belonging to the high good product rate category and the low good product rate category is excluded from the object of visual inspection, and only the inspection object or visual inspection part belonging to the medium good product rate category is used as the object of visual inspection. <D000086><D000087>Figure 11 is a flowchart showing the operation flow of the appearance inspection device. [Figure 12] Figure 12 is a diagram showing the main configuration of the visual inspection terminal (visual inspection unit). [Figure 13] Figure 13 is a diagram showing an example of a display on the display unit during visual inspection. [Figure 14] Figure 14 is a diagram showing another example of a display on the display unit during visual inspection. [Figure 15] Figure 15 is a diagram showing another example of a display on the display unit during visual inspection. [Figure 16]Figure 16 is a flowchart showing the operation flow of the visual inspection terminal. [Figure 17] Figure 17 shows the data stored in the database. [Figure 18] Figure 18 is a flowchart showing an example of the inspection process for a visual inspection support system. [Figure 19] Figure 19 is a flowchart showing other examples of inspection processes in the visual inspection support system. [Figure 20] Figure 20 is a flowchart showing other operation flows of the visual inspection terminal. [Figure 21] Figure 21 shows an example of the display shown on the display unit during the inspection process when the inspector makes a judgment, in the process of displaying visual inspection support information after the judgment. [Figure 22] Figure 22 shows an example of the display unit when the judgment input is restricted. [Figure 23] Figure 23 shows an example display where the "Administrator Only" icon is shown in the input field. [Figure 24] Figure 24 shows an example display where the input section shows an "NG" icon, an "OK" icon, and an "Acceptable Product" icon. [Figure 25] Figure 25 is a flowchart showing the operation flow of the visual inspection terminal that performs additional imaging. [Figure 26] Figure 26 shows an example of the overall configuration of a visual inspection support system using AOI (Automated Optical Inspection). [Figure 27] Figure 27 shows an example of a connection configuration for a visual inspection support system using AOI (Automated Optical Inspection). [Figure 28] Figure 28 shows an example of the overall configuration of AOI. [Figure 29] Figure 29 is a flowchart showing an example of the inspection process of a visual inspection support system using AOI. [Figure 30] Figure 30 is a flowchart showing other examples of inspection processes in a visual inspection support system using AOI. [Figure 31] Figure 31 shows an example of the overall configuration of an inspection system using AOI. [Modes for carrying out the invention]

[0016] The following describes in detail a visual inspection support system to which the present invention is applied, with reference to the drawings. It should be noted that the present invention is not limited to the following embodiments, and various modifications are possible without departing from the spirit of the invention. Furthermore, the drawings are schematic, and the proportions of dimensions may differ from those of reality. Specific dimensions should be determined by referring to the following description. It should also be noted that there are parts where the relationships and proportions of dimensions differ between drawings.

[0017] A visual inspection support system 1 according to one aspect of the present invention, as shown in Figure 1, comprises a visual inspection unit 2 having an imaging camera 11 that captures a predetermined inspection area of ​​an object to be inspected with the imaging camera 11, and automatically performs a visual inspection, i.e., quantitative evaluation of the quality of the visually inspected area, and generates visual inspection support information based on the quantitative evaluation, using the captured image; a visual inspection unit 3 having a display unit 5 that displays the image of the visually inspected area captured by the imaging camera 11, and an input unit 6 that inputs the visual judgment result by an inspector, and the display unit 5 displays visual inspection support information based on the quantitative evaluation.

[0018] The visual inspection unit 2 is a unit that photographs the inspection area of ​​the object to be inspected and automatically quantitatively evaluates the quality of the inspection area based on the image data, and is composed of, for example, an automatic inspection device 2A. The automatic inspection device 2A is installed, for example, on the inspection line through which the object to be inspected is transported.

[0019] The visual inspection unit 3 is equipped with a display monitor that constitutes the display unit 5 and various input means that constitute the input unit 6. Based on the visual inspection image displayed on the display unit 5, the quality of the visual inspection part is visually judged by the inspector, and the judgment result is input via the input means. For example, it is composed of a visual inspection terminal 3A.

[0020] The objects to be inspected are not particularly limited as long as they are components subjected to visual inspection. For example, they include printed circuit boards and mounted boards on which various electronic components such as ICs, LSIs, resistors, and capacitors are mounted, or on which circuit patterns and resist patterns are formed, as well as liquid crystal panels and other various objects to be inspected.

[0021] The designated inspection areas refer to parts of the object to be inspected that require inspection, such as the mounting state of electronic components or the formation state of circuit patterns, or parts that have been judged as defective or not judged as "OK" by the automated optical inspection device (AOI) described later. The parts that are automatically visually inspected by this visual inspection unit 2 are called inspection-required areas.

[0022] Visual inspection refers to the quantitative evaluation of the quality of a designated inspection area, performed by the visual inspection unit 2 based on images of that area. In this specification, it is also referred to as automated visual inspection. The images of the inspection area captured by the visual inspection unit 2 are called automated inspection images.

[0023] A visual inspection area is a part that is visually inspected by an inspector, and refers to the part that is subject to visual inspection among the parts requiring inspection that have been automatically visually inspected by the visual inspection unit 2. In addition, among the automatically inspected images, the images used for visual inspection are called visual inspection images.

[0024] Visual inspection support information refers to information that serves as an indicator for inspectors to determine whether an item is good or bad. It is based on a quantitative evaluation of the degree of goodness or badness obtained through automated visual inspection, and displays the degree of goodness or badness using numerical values, probabilities, images, characters, sentences, colors, etc.

[0025] During visual inspection, the display unit 5 of the visual inspection unit 2 displays an image of the area being inspected, captured by the imaging camera 11, along with visual inspection support information. This prevents inspectors from misjudging defective items as good products, and prevents erroneous inputs where an inspector mistakenly enters a good product as a defective product. Therefore, the visual inspection support system 1 according to the present invention prevents defective products from being mistakenly identified as good products and sent to subsequent processes.

[0026] The following describes the various components of the visual inspection support system 1. As mentioned above, the visual inspection support system 1 may be configured with the visual inspection unit 2 and the visual inspection unit 3 as a single device, or with the visual inspection device 2A and the visual inspection terminal 3A as separate terminals. Furthermore, as shown in Figure 2, the visual inspection support system 1 may be configured with multiple inspection lines 9 (for example, inspection line 9A, inspection line 9B, inspection line 9C, etc.) each equipped with a visual inspection device 2A, and one visual inspection terminal 3A, and the single visual inspection terminal 3A may sequentially display the visual inspection images and visual inspection support information captured by the visual inspection devices 2A of each inspection line 9, thereby enabling centralized visual inspection. In the following description, the visual inspection support system 1 equipped with multiple inspection lines 9 and one visual inspection terminal 3A will be used as an example.

[0027] Furthermore, the visual inspection support system 1 is used to inspect various objects, including printed circuit boards, mounted circuit boards, and liquid crystal panels, on which various electronic components such as ICs, LSIs, resistors, and capacitors are mounted, or on which circuit patterns and resist patterns are formed. However, the objects to be inspected are not limited to these. In the following, a printed circuit board 7 on which electronic components are surface-mounted will be used as an example of an object to be inspected.

[0028] Each of the multiple inspection lines 9 has a transport means such as a belt conveyor to transport the objects to be inspected, and the visual inspection device 2A is positioned on the transport line for the objects to be inspected. As a result, the inspection line 9 can transport the objects to be inspected one after another and perform inspections continuously.

[0029] Furthermore, as shown in Figure 2, the visual inspection devices 2A and visual inspection terminals 3A installed on each inspection line 9 are connected to the database 10 via the bus 8, enabling mutual data communication. The database 10 stores, in an interconnected manner, the following: automated inspection images (visual inspection images) from the visual inspection device 2A, location information of areas requiring inspection (visual inspection areas), visual inspection support information, visual judgment results entered into the visual inspection terminal 3A by the inspector, IDs and part IDs of the object being inspected, and automated inspection results (score, defect name, etc.) from the visual inspection device 2A. Note that the connection configuration of each device 2A, 3A and the database 10 via the bus 8 in the visual inspection support system 1 is illustrative, and of course, other network connection configurations besides the bus type may be used (see Figure 3). Also, the database 10 may consist of one or more databases.

[0030] [Visual Inspection Device 2A] The visual inspection device 2A is a device that photographs the objects to be inspected on the inspection line 9 and quantitatively evaluates the quality of the objects based on the image data. The visual inspection device 2 can employ either a manual or automatic method for attaching and detaching the objects to be inspected, but an automatic method is preferable in order to shorten the inspection cycle time.

[0031] As shown in Figures 4 and 5, the visual inspection device 2A includes an imaging camera 11 such as a CCD camera for photographing the object to be inspected, an automated inspection computer 12 that uses the imaging camera 11 to photograph the object to be inspected by referring to predetermined inspection area data (XYθ coordinates), and performs quantitative evaluation of the quality of the inspection area based on the captured image data, and generates visual inspection support information based on the quantitative evaluation, a communication unit 13 connected to the bus 8 and communicating with a database 10, etc., and a support unit 14 for supporting the object to be inspected.

[0032] [Imaging camera 11] The imaging camera 11 is positioned on the upper side of the device housing and includes an oblique camera 11a that photographs a predetermined inspection area of ​​the object to be inspected from an oblique angle above. The imaging camera 11 may also include a planar camera 11b that photographs the inspection area of ​​the object to be inspected from above. In the visual inspection device 2A, the imaging camera 11 is positioned on the upper side of the device housing, and a movement mechanism in the XYθ direction is provided on the support part 14 that supports the object to be inspected or on the imaging camera 11, so that the imaging camera 11 can move relative to the object to be inspected supported by the support part 14, and imaging can be taken from any position on the object to be inspected. In addition, in the visual inspection device 2A, a rotation mechanism is provided on the support part 14 that supports the object to be inspected or on the imaging camera 11, so that the imaging camera 11 can rotate relative to the object to be inspected supported by the support part 14, and imaging can be taken from any direction on the object to be inspected.

[0033] The oblique camera 11a is supported with its camera optical axis α1 tilted at an angle to the object being inspected, and it photographs the object being inspected from an oblique angle above. As shown in Figure 6, the oblique camera 11a is positioned so that its optical axis is at an angle of 45° to, for example, the printed circuit board 7 which is the object being inspected. The planar camera 11b is supported with its camera optical axis α2 aligned with the rotation axis of the rotation mechanism, and it photographs the object being inspected from directly above.

[0034] The imaging camera 11 has predetermined imaging lenses and image sensors incorporated into the lens barrels of the oblique camera 11a and the planar camera 11b. The captured image data is stored in the database 10 and reproduced and displayed on the display unit 5 of the visual inspection terminal 3A during visual inspection. Alternatively, during visual inspection at the visual inspection terminal 3A, the images captured by the oblique camera 11a and the planar camera 11b of the imaging camera 11 may be displayed as real-time images on the display unit 5 of the visual inspection terminal 3A.

[0035] The imaging camera 11 may be equipped with focus and zoom functions for the oblique camera 11a and the planar camera 11b, or it may not be equipped with these functions, but may use a camera that has a focal length and aperture to achieve a deep depth of field and is capable of pan-focus shooting, and the image may be displayed at an appropriate size by processing on the automatic inspection computer 12 or the visual inspection terminal 3A.

[0036] Furthermore, the imaging camera 11 may be equipped with multiple oblique cameras 11a. For example, the imaging camera 11 can be equipped with a pair of oblique cameras 11a, consisting of a high-magnification camera and a low-magnification camera, at opposite positions via the rotation axis of the rotation mechanism, allowing for selection of use depending on the size of the area to be inspected. Alternatively, the imaging camera 11 may be equipped with multiple oblique cameras 11a concentrically around the rotation axis of the rotation mechanism, or the multiple oblique cameras 11a may be arranged at equal intervals or at unequal intervals.

[0037] When an object to be inspected is brought in, the imaging camera 11 refers to the inspection area data (XYθ coordinate data) stored in the database 10 and takes a photograph of the area to be inspected at a predetermined angle and magnification. The captured image is stored in the database 10 and read out according to the inspector's operation, etc., and displayed on the display unit 5 of the visual inspection terminal 3A. In addition, when additional images are taken as described later, the captured image is displayed as a real-time image. The inspector operating the visual inspection terminal 3A makes a pass / fail judgment by looking at the visual inspection image and real-time image displayed on the display unit 5. The visual inspection image of the area to be inspected displayed on the display unit 5 during visual inspection is not limited to one image; multiple visual inspection images with different angles and magnifications may be displayed.

[0038] [Support part 14] The support portion 14 can be any mechanism that detachably fixes and supports the object to be inspected. For example, the support portion 14 may have a pair of long-side frames with support surfaces that protrude to support the lower surface of the outer edge of the printed circuit board 7, a pair of connecting bars that connect both ends of the long-side frames, and fixing pins that slide along the longitudinal direction of the long-side frames to fix the printed circuit board 7.

[0039] [Z-axis correction value] Furthermore, the visual inspection device 2A may be equipped with a vertical movement mechanism (in the Z-axis direction) and a correction means in the Z-axis direction on the imaging camera 11 or the support part 14 that supports the object to be inspected, so as to vary the relative distance in the Z-axis direction between the imaging camera 11 and the object to be inspected. This ensures that even if a plate-shaped object to be inspected, such as a printed circuit board 7, becomes warped, the imaging position of the imaging camera 11 can be corrected in the Z-axis direction to reliably capture the area that needs to be inspected.

[0040] In other words, as shown in Figure 7(A), if there is no warping in the printed circuit board 7, the inspection area identified based on the position data of the area to be inspected will coincide with the area to be subjected to automated visual inspection or displayed on the display unit 5, and the area to be actually inspected by automated visual inspection or visual inspection. Therefore, by setting the relative position between the printed circuit board 7 and the imaging camera 11 based on the position data (XYθ coordinates) of the area to be inspected, the optical axis of the oblique camera 11a will coincide with the area to be inspected of the electronic component set on the printed circuit board 7, and the area to be inspected by the oblique camera 11a can be captured.

[0041] However, as shown in Figure 7(B), if the printed circuit board 7 warps downward, or as shown in Figure 7(C), if the printed circuit board 7 warps upward, setting the relative position between the printed circuit board 7 and the oblique camera 11a of the imaging camera 11 based on the position data of the area to be inspected set on the printed circuit board 7 may result in a discrepancy between the area to be inspected identified based on the position data and the image from the oblique camera 11a. This may prevent the oblique camera 11a from capturing the area that needs to be inspected, or it may result in an out-of-focus image.

[0042] In Figure 7(c), the printed circuit board 5 is curved upward, causing the optical axis α1 of the oblique camera 11a to capture the area in front of the inspection site. In Figure 7(b), the printed circuit board 5 is curved downward, causing the optical axis α1 of the oblique camera 11a to capture the area beyond the inspection site. In this way, if the relative position of the printed circuit board 7 and the imaging camera 11 is set based on the position data of the area to be inspected, it becomes impossible to capture the necessary areas for automated visual inspection or visual inspection, making it impossible to perform an appropriate quantitative evaluation of the quality. Furthermore, inspectors will not know which position of which surface-mount component to visually inspect. Especially in recent years, extremely small chip components with sides of less than 1 mm, such as 0603 chips, are used as surface-mount components, and these extremely small chips are mounted at high density, so it is necessary to accurately capture the inspection site.

[0043] Therefore, it is preferable that the visual inspection device 2A is equipped with a vertical movement mechanism (in the Z-axis direction) and a correction means in the Z-axis direction on the imaging camera 11 or the support part 14 that supports the object to be inspected, so as to move the relative distance in the Z-axis direction between the imaging camera 11 and the object to be inspected.

[0044] A known movement mechanism can be used as the vertical movement mechanism for the imaging camera 11 or the support unit 14.

[0045] As a means of correcting in the Z-axis direction, for example, a method can be adopted in which the amount of deflection of the object to be inspected (printed circuit board 7) is measured using a laser displacement meter, and the relative distance (height direction: Z-axis direction) of the imaging camera 11 or support part 14 is corrected based on that value. In order to determine the amount of deflection, the height of a reference point on the printed circuit board 7 is measured in advance. For example, the height is measured at points on the outer perimeter of the board, such as near the origin (X:Y=0:0) in the lower left corner of a rectangular board in a plan view, near the upper left corner, near the upper right corner, near the lower right corner, and other points where the amount of deflection can be assumed to be approximately 0 structurally. One measurement point is sufficient, but multiple points are preferable. If there is one measurement point, the height of that measurement point is used as the reference value, and if there are multiple measurement points, the average of the heights of each measurement point is used as the reference value.

[0046] Next, the height of the substrate at the inspection-required area of ​​the object to be inspected (printed circuit board 7) is measured using a laser displacement meter or the like, and the difference from the reference value is calculated. This difference value becomes the correction amount in the Z-axis direction. This correction amount in the Z-axis direction is measured in advance prior to the inspection process and stored in the database 10 in association with the substrate ID, etc. Note that the correction means in the Z-axis direction is not limited to the method described above, but any known method can be used.

[0047] When the imaging camera 11 of the visual inspection device 2A captures an area requiring inspection, the relative height between the imaging camera 11 and the support part 14 is corrected by referring to the correction amount in the Z-axis direction of the area requiring inspection on the substrate. As a result, in the visual inspection process, a visual inspection image that has been corrected in the Z-axis direction in advance is displayed, eliminating the need for additional shooting such as manually correcting the imaging camera 11 in the Z-axis direction, and thus shortening the cycle time of the inspection process.

[0048] The Z-axis correction amount is referenced not only during imaging in automated visual inspection but also during additional imaging. Alternatively, the Z-axis correction amount may be measured during the inspection of the first object to be inspected (printed circuit board 7) in the inspection process, stored in the database 10 in association with the board type ID, etc., and referenced in subsequent automated visual inspections and additional imaging of objects of the same board type. Alternatively, the Z-axis correction amount may be measured each time an object to be inspected is brought into the visual inspection device 2A and used for imaging in automated visual inspection and additional imaging.

[0049] The device housing may also be provided with lighting means (not shown) to illuminate the inside of the housing.

[0050] [Automated inspection computer 12] The automated inspection computer 12, based on the automated inspection processing program, uses the imaging camera 11 to photograph predetermined areas of the object to be inspected, and uses these captured images (also called "automatic inspection images") to perform a quantitative evaluation of the quality of the predetermined areas to be inspected. Specifically, the visual inspection device 2A has inspection area data indicating the location of the areas to be inspected on the printed circuit board 7, or obtains it from the database 10, and sequentially, for example in order of proximity, captures inspection images such as planar images and oblique images of the locations indicated by the inspection area data according to the program.

[0051] The position data of the area to be inspected is determined in an XYθ coordinate system specific to the imaging camera 11. Of these, XY are the two orthogonal coordinates within the inspection surface of the object to be inspected, and define the imaging position of the imaging camera 11. θ is the angle obtained when the object to be inspected is rotated counterclockwise around the X-axis, with the center point of the XY coordinate as the rotation center, and defines the imaging direction (θ direction) by the oblique camera 11a.

[0052] Furthermore, the automated inspection computer 12 may classify the defect at the inspection site against a "defect name database" which contains pre-categorized types of defects (e.g., bridging, misalignment, electrode detachment, missing electrode, foreign matter, dirt, insufficient soldering, etc.). Defect names such as bridging may be displayed on the display unit 5 along with visual inspection support information.

[0053] The inspection results obtained by the automated inspection computer 12 are called automated inspection results. The automated inspection results include quantitative evaluation values ​​(scores) generated based on the quantitative evaluation described below, and information on the defect name (e.g., bridge, misalignment, electrode detachment, missing electrode, foreign matter, dirt, insufficient soldering, etc.).

[0054] Furthermore, the automated inspection computer 12 quantitatively evaluates the quality of the areas requiring inspection based on the automated inspection images and generates visual inspection support information based on the quantitative evaluation.

[0055] Furthermore, areas suspected of having mounting defects, identified by the visual inspection device 2A, that require visual inspection by an inspector are referred to as "visual inspection areas." If the entire product is subject to visual inspection, all areas requiring inspection will be considered visual inspection areas. Also, as described later, if the inspection target is limited according to quantitative evaluation values ​​(scores), etc., the areas among the areas requiring inspection that are subjected to visual inspection will be considered visual inspection areas.

[0056] The quantitative evaluation of the quality of the areas requiring inspection, the generation of quantitative evaluation values ​​(scores), and the generation of visual inspection support information are performed by an automated inspection processing program using a known image processing program.

[0057] [Quantitative evaluation] The quantitative evaluation of the quality is performed by calculating a matching score, which is an evaluation of the similarity representing the matching state using a predetermined formula, utilizing known processes such as pattern matching and artificial intelligence (AI). Known pattern matching methods such as template matching and feature-based matching can be employed.

[0058] As shown in Figure 8, the matching score is set as a numerical score on a scale of, for example, 100, with 100 representing the highest value for the good product rate and 0 representing the lowest value. In other words, the closer the score is to 100, the higher the probability that the product is good, and the lower the score, the higher the probability that the product is defective.

[0059] [Visual Inspection Support Information] Visual inspection support information is information that serves as an indicator for inspectors to determine whether an item is good or bad. It is generated based on a quantitative evaluation of the quality by the automated inspection computer 12, and displays the quality using numerical values, probabilities, text, colors, etc. In other words, visual inspection support information is a display generated based on a score calculated by quantitative evaluation. Various displays can be used, such as the score itself, text such as "high good product rate" or "low good product rate" corresponding to the score, warning messages such as "caution," the probability of a good product corresponding to the score, a blue mark indicating a high good product rate or a red mark indicating a low good product rate according to the score, or combinations thereof.

[0060] During visual inspection, visual inspection support information is displayed on the display unit 5 along with the visual inspection image. This helps to prevent errors in judgment, such as the inspector mistakenly identifying an item that should be deemed defective as a good product, or the inspector mistakenly entering a good product as the result of a defect.

[0061] Furthermore, the visual inspection support information may be divided into multiple categories based on the score, within predetermined ranges, and may be information that allows for the identification of each category. For example, as shown in Figure 9, if the score is set on a scale of 100, with 100 being the highest value for the high good product rate and 0 being the lowest value, two thresholds are set to divide the products into three categories: a high good product rate category with a score of 80 or higher, a low good product rate category with a score of 40 or lower, and a medium good product rate category between the high and low good product rate categories. The visual inspection support information may then display text such as "high good product rate," "medium good product rate," or "low good product rate," a blue mark indicating the high good product rate category, a gray mark indicating the medium good product rate category, a red mark indicating the low good product rate category, or a combination of these.

[0062] Alternatively, the display format of the visual inspection support information may be changed according to each category. For example, for items classified in the medium good product rate category, a message such as "Please check carefully" may be displayed, while for items classified in the high good product rate category, the visual inspection support information may not be displayed. By hiding the visual inspection support information, the time required for visual inspection will be reduced.

[0063] The score threshold may be one or three or more.

[0064] Furthermore, the threshold scores used to distinguish each category may be fixed, arbitrarily set, or changeable. By changing the thresholds according to the type of substrate or component, for example, the lower limit of the threshold indicating a high good product rate can be lowered for components with a relatively high good product rate, or the upper limit of the threshold indicating a low good product rate category can be raised for components that require particularly careful visual inspection, thereby allowing for the emphasis placed on visual inspection. Also, for example, setting the high good product rate category to a score of 90 or higher broadens the categories that inspectors should focus on. Also, for example, setting the low good product rate category to a score of 50 or lower reduces the possibility of defective products being judged as good products. In other words, inspectors can relatively easily determine whether an object or visual inspection area belonging to the high good product rate category or the low good product rate category is good or bad, and by focusing on objects or visual inspection areas belonging to the intermediate good product rate category between these two, they can perform visual inspections efficiently and prevent misjudgments and incorrect inputs.

[0065] Furthermore, the threshold scores used to categorize each item may be set for each defect name. For example, in the case of a defect name "missing parts," it can be divided into four categories: score less than 10, score between 10 and 70, score between 70 and 90, and score of 90 or more. In the case of a defect name "incorrect polarity," it can be divided into four categories: score less than 10, score between 10 and 50, score between 50 and 90, and score of 90 or more. Since the reliability of automated inspection results and visual inspection results by inspectors differs depending on the defect name, i.e., the nature of the defect, setting a threshold for each defect name can increase the reliability of the visual inspection support system according to the present invention.

[0066] Furthermore, the visual inspection support system 1 may be configured to limit the visual inspection to only those objects belonging to a predetermined category, thereby reducing the number of items to be inspected and easing the burden on inspectors, thus preventing misjudgments and incorrect inputs. For example, as shown in Figure 10, by excluding objects or parts for visual inspection belonging to the high-quality product rate category and the low-quality product rate category from the visual inspection, only the medium-quality product rate category between the high-quality product rate category and the low-quality product rate category is visually inspected. This reduces the number of visual inspections performed by inspectors, allowing them to concentrate their visual inspection efforts. By setting a high threshold for the score of the high-quality product rate category (for example, 95 or higher), it is possible to prevent defective products from being released without undergoing visual inspection by inspectors. Also, by setting a low threshold for the score of the low-quality product rate category (for example, 40 or lower), defective products can be removed from the line without undergoing visual inspection by inspectors, preventing both defective products from being released and good products from being removed from the line.

[0067] [Automated inspection data] The automated inspection results from the automated inspection computer 12 and the visual inspection support information are stored in the database 10 along with the automated inspection images, which are the imaging data used for automated judgment. The database 10 manages automated inspection data such as automated inspection results, visual inspection support information, automated inspection images, automated shooting condition data (angle, magnification, etc.), substrate ID, inspection area data (XYθ coordinate data), defect name, and quantitative evaluation value (score). Note that the automated inspection data is not limited to those described above.

[0068] [Operation Flow] Next, the operation of the visual inspection device 2A will be described. As shown in Figure 11, first, in step S1, the object to be inspected, such as the printed circuit board 7, is brought into the visual inspection device 2A. Then, in step S2, the imaging camera 11 takes a photograph of a predetermined inspection area from diagonally above and / or from above.

[0069] Then, in step S3, an automated visual inspection is performed. Specifically, the automated inspection processing program uses the automated inspection image to quantitatively evaluate the quality of the product at a predetermined inspection location and generate visual inspection support information. In step S4, the visual inspection device 2A transmits the automated inspection results, visual inspection support information, automated inspection image used for the automated visual inspection, inspection location data (XYθ coordinate data), and various automated inspection data such as defect names to the database 10. Each transmitted data is stored in the database 10.

[0070] Of this automated inspection data, the data obtained by adding reference image data indicating the correct implementation state of the parts to be inspected by an inspector to the automated inspection data relating to the objects to be inspected by an inspector is called visual inspection data. For example, if objects or parts to be inspected that belong to the high good product rate category or the low good product rate category are excluded from the visual inspection, the data obtained by adding reference image data to the automated inspection data of objects or parts to be inspected that are classified as the medium good product rate category will be called visual inspection data. Also, if all objects to be inspected that have undergone automated visual inspection are to be subjected to visual inspection, the data obtained by adding reference image data to the fully automated inspection data will be called visual inspection data. Note that not all of the visual inspection data is necessarily displayed on the display unit 5 of the visual inspection terminal 3A. For example, the reference image may or may not be displayed.

[0071] Subsequently, in step S5, the object to be inspected is removed, and the next object to be inspected is brought in (step S1). In this way, the visual inspection device 2A receives the objects to be inspected as they are transported along the inspection line 9 and performs inspections continuously.

[0072] [Visual Inspection Terminal 3A] The visual inspection terminal 3A is a device that sequentially displays visual inspection images taken by the visual inspection devices 2A on multiple inspection lines 9, allowing for centralized visual inspection of each object being transported on multiple inspection lines 9 in one location. It is connected to the visual inspection devices 2A and the database 10 via the bus 8.

[0073] As shown in Figure 12, the visual inspection terminal 3A includes an input unit 6 into which the inspector inputs visual judgment results, a display unit 5 that displays visual inspection images (automatically captured images) taken by the visual inspection device 2A and visual inspection support information for the visual inspection images, a communication unit 17 that communicates with the database 10 and the visual inspection device 2A, and a control unit 18 that controls the operation of the entire visual inspection support system 1, including image switching of the display unit 5 and communication with the database 10 and the visual inspection device 2A.

[0074] The input unit 6 receives instructions from the inspector to execute the inspection process of the visual inspection support system 1, such as inputting the results of visual inspection, instructing the capture of additional visual inspection images, and instructing the display unit 5 to switch between visual inspection areas and objects being inspected. Known input methods can be used, such as an operation panel with input buttons and levers, or a mouse or keyboard used to input instructions on the operation screen displayed on the display unit 5.

[0075] The inspector operating the visual inspection terminal 3A can perform any operation on the visual inspection device 2A and the database 10 via the input unit 6, such as inputting the visual judgment results, switching the visual inspection image of the object being inspected to a visual inspection image of another object, or issuing a screen display switching instruction to switch to a different visual inspection image of the same object.

[0076] The visual inspection results entered via the input unit 6 may include not only the pass / fail judgment result but also the name of the defect. The name of the defect may be displayed on the display unit 5 from a pre-categorized list of defect names, allowing the user to select one or more of them, or a free-form field may be provided where the name of the defect can be entered.

[0077] The screen display switching instruction displays a list of inspection targets that have been automatically inspected by the visual inspection device 2A and are subject to visual inspection by an inspector on the display unit 5, and switches the visual inspection images in the order they are displayed according to the switching instruction. Alternatively, the inspector may select any one of the displayed inspection targets from the list, thereby switching the visual inspection image displayed on the display unit 5.

[0078] The display unit 5 has one or more monitors and displays visual inspection images (automatically captured images) taken by the visual inspection device 2A, reference images showing the correct mounting state to be compared with the visual inspection images, a list of visual inspection devices 2A on the inspection line 9 that are performing visual inspection or are in standby mode for visual inspection, operation screens, etc.

[0079] [Visual inspection data] As described above, the display unit 5 of the visual inspection terminal 3A displays visual inspection data. The visual inspection data displayed on the display unit 5 includes data necessary for visual inspection by an inspector, and includes at least a visual inspection image (automatic inspection image), visual inspection support information, and a reference image showing the correct implementation state of the parts to be inspected. The inspector compares the visual inspection image with the reference image to determine whether it is good or bad. The inspector inputs the visual inspection result from the input unit 6. The input visual inspection result is associated with other automatic inspection data of the object to be inspected or the parts to be inspected, and is stored in the database 10 via the communication unit 17.

[0080] The control unit 18 enables centralized inspection across multiple inspection lines 9 by controlling each part of the visual inspection terminal 3A in response to the inspector's operation, and is configured, for example, by a computer. The control unit 18 can remotely operate the visual inspection device 2A and the database 10 via the bus 8 or other network and have them perform the necessary processing.

[0081] Figure 13 shows an example of the display on the display unit 5 during visual inspection. As a visual inspection image, a front view image 30 of the area being inspected is displayed in the center, and oblique images 31a and 31b are displayed on the left and right. A reference image 32 is displayed in the upper right. In the lower right, "NG (defective)" and "OK (good)" icons, which are input units 6 for entering the visual inspection result, are displayed. The inspector can input the visual inspection result by clicking the "NG" or "OK" icon with a cursor displayed on the display unit 5 using an input means such as a mouse.

[0082] Visual inspection support information 33 is displayed in the upper left of the display unit 5. In the display example shown in Figure 13, the defect names "Foreign matter / dirt," "Insufficient soldering," and "Solder bridging between pins" are displayed, and next to each defect name, the defect probability (%) calculated based on the quantitative evaluation value is displayed. In addition, a graph showing the categories and thresholds is displayed. In the display example shown in Figure 13, it is divided into four categories, and the thresholds are 30%, 70%, and 90%. By referring to the visual inspection support information 33 in addition to the visual inspection images 30, 31a, and 31b, inspectors can prevent judgment errors such as misjudging an inspected item that should be judged as defective as a good product, or misinputting an item that the inspector judged as defective as a good product when inputting the judgment result.

[0083] Visual inspection support information is displayed in a manner appropriate to the category. Figure 14 shows another example of display on the display unit 5 during visual inspection. The difference from Figure 13 is that the text "Please check carefully. There is a possibility of solder bridging between pins." is displayed in the upper center as visual inspection support information 33. Among the visual inspection support information 33 displayed in the upper left, "solder bridging between pins" has an 86% failure rate. In this way, when categorized as a category with a high failure rate, additional messages or other displays may be added as visual inspection support information to draw more attention.

[0084] Figure 15 shows another example of the display unit 5 during visual inspection. The difference from Figure 14 is , positive The key feature is the display of a frame surrounding the suspected defective area in surface image 30. When quantitatively evaluating the quality based on automated inspection images (visual inspection images), the suspected defective area is also identified. Ruko This can be done. The frame may be colored, flashed, or otherwise made to draw attention.

[0085] [Operation Flow] The operation of the visual inspection terminal 3A will now be explained. As shown in Figure 16, in step S11, the visual inspection terminal 3A receives a screen display switching instruction from the inspector by operating the input unit 6. Next, in step S12, the visual inspection terminal 3A retrieves the visual inspection data of the object to be inspected (printed circuit board 7) to be displayed on the display unit 5 from the database 10.

[0086] Then, in step S13, the visual inspection terminal 3A displays the visual inspection image (automatic inspection image), reference image, and visual inspection support information of the object to be inspected that has been switched on the display unit 5, and the inspector performs a visual inspection. The visual inspection terminal 3A may also display other visual inspection data. In addition, the visual inspection image of the part to be inspected displayed on the display unit 5 during the visual inspection is not limited to one image, but may display multiple visual inspection images with different angles and magnifications. In step S14, the inspector performs a visual inspection based on the visual inspection data and inputs the visual judgment result via the input unit 6. In step S15, the visual inspection support terminal 3A saves the visual judgment result in the database 10 in association with the visual inspection data.

[0087] Then, when the visual inspection terminal 3A receives an input (step S11) for switching the screen display to switch to the visual inspection image of another inspection line 9, it reads the visual inspection image (automatic inspection image) and other visual inspection data of the specified inspection line 9 from the database 10 and displays it on the display unit 5. In this way, the visual inspection terminal 3A can display visual inspection images taken by the visual inspection devices 2A of multiple inspection lines 9, and one inspector can perform visual inspections of multiple inspection lines 9.

[0088] [Database 10] As shown in Figure 17, the database 10 contains automated inspection data, visual inspection data, and inspection line data. The automated inspection data stores data related to the object to be inspected (printed circuit board 7), data related to automated imaging, and data related to quantitative evaluation and visual inspection support information. Data related to the object to be inspected includes, for example, board type ID, board ID, component ID, inspection area data (XYθ coordinates), and automated inspection images. Data related to automated imaging includes, for example, imaging condition data (angle, magnification, etc.) and Z-axis correction value. Data related to quantitative evaluation and visual inspection support information includes automated inspection results (defect name, quantitative evaluation value (score)) and visual inspection support information.

[0089] Furthermore, data related to the inspected object is recorded after the visual inspection, along with the visual judgment results made by the inspector. Other data recorded after the visual inspection of the inspected object may include additional shooting condition data (position, angle, magnification, etc.), defect name (after visual inspection), and inspection time (time taken from displaying the visual inspection image to inputting the visual judgment results).

[0090] The visual inspection data consists of automated inspection data related to the object being inspected, plus reference images. During the visual inspection, the visual inspection data is read by the visual inspection terminal 3A and displayed on the display unit 5 as appropriate.

[0091] The inspection line data manages information such as the inspection line ID, which is an identification ID assigned to each inspection line; the visual inspection device ID, which is an identification ID assigned to each visual inspection device 2A; the visual inspection terminal ID, which is an identification ID assigned to each inspector operating the visual inspection terminal 3A; the date and time of inspection; and the quantity inspected.

[0092] The data managed in database 10 is not limited to these.

[0093] Each data point is managed in an interconnected manner and referenced according to the operation of the visual inspection terminal 3A. Furthermore, each data point is managed in an interconnected manner and used for verifying judgment accuracy, improving automatic judgment accuracy, and evaluating and improving the inspection capabilities of inspectors.

[0094] The database 10 may be integrated with the visual inspection device 2A or the visual inspection terminal 3A, or it may be connected to the visual inspection device 2A and the visual inspection terminal 3A via the bus 8 or the internet.

[0095] Furthermore, while database 10 manages automated inspection data, visual inspection data, and inspection line data in a single database, it may also be composed of multiple databases depending on the type and purpose of the data, such as data on the object being inspected, data on automated photography, quantitative evaluation, and visual inspection support information.

[0096] [Inspection Flow] Next, the inspection process of the entire visual inspection support system 1 will be explained with reference to Figure 18. In the following inspection flow steps, the visual inspection support system 1 using the visual inspection device 2A as the visual inspection unit 2 and the visual inspection terminal 3A as the visual inspection unit 3 will be explained as an example.

[0097] First, an automated visual inspection is performed by the visual inspection device 2A (step S31). The automated inspection flow is carried out according to the flow shown in Figure 11, as described above. That is, when the object to be inspected is brought in, the visual inspection device 2A takes a photograph of the designated areas to be inspected and performs an automated visual inspection based on the obtained automated inspection image. This generates automated inspection data, which includes visual inspection support information. The automated inspection data is stored in the database 10.

[0098] Next, a visual inspection is performed by an inspector using the visual inspection terminal 3A (step S32). The visual inspection flow is carried out according to the flow shown in Figure 16, as described above. That is, when the inspector inputs a screen display switching instruction, the visual inspection terminal 3A retrieves visual inspection data from the database 10 and displays it on the display unit 5. The display unit 5 displays the visual inspection image (automatic inspection image) of the object to be inspected, a reference image, visual inspection support information, and other visual inspection data. Based on this visual inspection data, the inspector makes a judgment on the quality of the visually inspected parts.

[0099] Here, the display unit 5 of the visual inspection terminal 3A displays visual inspection support information along with the visual inspection image. This allows inspectors to perform visual inspections while referring to the visual inspection support information, thereby reducing the variability in pass / fail judgments based on differences in experience and ability among inspectors. In other words, by referring to the visual inspection support information as an indicator for pass / fail judgment, it is possible to reduce the variability in judgments among inspectors, such as when one inspector judges an item as defective but another inspector judges it as good.

[0100] Furthermore, if a single inspector consistently assigns good results to inspections during visual inspection, the quality of the inspection may decline due to inertia or fatigue, increasing the risk of inconsistent judgments, such as incorrectly classifying defective items as good. Besides errors in judgment, inspectors may also mistakenly classify defective items as good due to input errors. However, by referencing visual inspection support information as an indicator for quality judgments, the risk of such misjudgments and input errors can be reduced.

[0101] Furthermore, visual inspection support information is divided into multiple categories based on the score, and by making the category identifiable information, inspectors can allocate inspection time and concentration according to the category. For example, if the inspected items are classified into a high-quality category, a low-quality category, and an intermediate-quality category according to the score, the quality of the inspected items or visual inspection parts belonging to the high-quality or low-quality category can be judged relatively easily. By focusing on the inspected items or visual inspection parts belonging to the intermediate-quality category, visual judgment can be performed efficiently, and this can prevent misjudgments and incorrect input.

[0102] [Selection of objects to be inspected] In Figure 18, all objects that underwent automated visual inspection by the visual inspection device 2A were subject to visual inspection. However, as shown in Figure 19, the number of objects to be visually inspected may be reduced by selecting which objects to inspect visually. As described above, by setting thresholds for quantitative evaluation values ​​(scores), objects to be inspected can be classified into multiple categories, and objects belonging to a predetermined category can be excluded from visual inspection, or only objects belonging to a predetermined category can be subject to visual inspection.

[0103] Specifically, after an automated visual inspection by the visual inspection device 2A (step S33), the control unit 18 of the visual inspection terminal 3A selects only the visual inspection data of the inspected objects belonging to a predetermined category (step S34). For example, inspected objects or visual inspection parts belonging to the high-quality-rate category and the low-quality-rate category are excluded from the visual inspection. Then, only the visual inspection data of inspected objects belonging to the medium-quality-rate category, which is between the high-quality-rate category and the low-quality-rate category, is displayed on the display unit 5 (step S35).

[0104] When inspectors perform an excessive number of judgment tasks, fatigue and complacency can lead to misjudgments and incorrect data entry. By focusing on inspection items where pass / fail judgment is ambiguous, the number of visual inspections performed by inspectors can be reduced, allowing them to concentrate on the visual inspection.

[0105] [Display after judgment] In the inspection flow described above, visual inspection support information was displayed during the visual inspection by the inspector. However, the visual inspection support system 1 may also be configured to display the visual inspection support information after the inspector has entered their judgment result. This allows the inspector to compare their own judgment result with the judgment made by the automated visual inspection system to confirm and verify the accuracy of their own judgment.

[0106] Figure 20 shows the process of displaying visual inspection support information after this determination. Similar to the flow shown in Figure 16, when a screen display switching instruction is input (step S11), the visual inspection terminal 3A retrieves the visual inspection data of the object to be inspected (printed circuit board 7) to be displayed on the display unit 5 from the database 10 (step S12).

[0107] In step S16, the visual inspection terminal 3A displays the visual inspection image (automatic inspection image) and reference image of the object to be inspected that was switched on the display unit 5, and the inspector performs the visual inspection. The visual inspection terminal 3A may also display other visual inspection data in addition to the visual inspection support information.

[0108] Figure 21 shows an example of the display shown on the display unit 5 during the process of displaying visual inspection support information after the judgment. As a visual inspection image, a frontal image 30 of the visual inspection area is displayed in the center, and oblique images 31a and 31b are displayed on the left and right. A reference image 32 is displayed in the upper right. In the lower right, "NG" and "OK" icons, which are input units 6 for entering the visual judgment result, are displayed. However, the visual inspection support information 33 is not displayed in the upper left of the display unit 5. The inspector can input the visual judgment result by clicking the "NG" and "OK" icons with a cursor displayed on the display unit 5 using an input means such as a mouse.

[0109] In step S17, the inspector performs a visual inspection based on the visual inspection data and inputs the visual judgment result via the input unit 6. In step S18, the visual inspection support terminal 3A displays the visual inspection support information for the visual inspection area of ​​the object being inspected on the display unit 5 (see Figures 14 and 13). The automatic inspection results of the visual inspection device 2A may also be displayed. This allows the inspector to refer to the visual inspection support information and the automatic inspection results of the visual inspection device 2A to confirm and verify their own judgment result. Subsequently, in step S19, the visual judgment result is re-entered as necessary. If there is no need to change the visual judgment result, no input operation is performed, or input indicating that no change is necessary is made. In step S20, the visual inspection support terminal 3A saves the visual judgment result in the database 10 in association with the visual inspection data.

[0110] Furthermore, the step of displaying visual inspection support information after the inspector has entered the judgment result may be performed if certain conditions are met, for example, if there is a discrepancy between the visual inspection support information or the automated inspection result and the judgment result entered by the inspector.

[0111] For example, visual inspection support information may be displayed on the display unit 5 when an inspector determines that an object or part of an object classified into a low good product category in an automated visual inspection is a good product. In this case, if the inspector determines that an object is defective, the visual inspection support information will not be displayed, thus shortening the inspection time.

[0112] Furthermore, there are no particular restrictions on the conditions for displaying visual inspection support information after inputting the judgment result. In addition to the above, it can be set as appropriate, such as when an inspector judges an object or visual inspection area classified in the high good product rate category as defective in automated visual inspection, or when an inspector judges an object or visual inspection area classified in the low good product rate category as good in automated visual inspection.

[0113] Furthermore, the visual inspection support system 1 may display visual inspection support information along with a notation indicating a match when the judgment result of the automated visual inspection matches the visual judgment result. In addition, the visual inspection support system 1 may display the visual inspection support information when the inspector makes a visual judgment and may redisplay it after the visual judgment result has been entered.

[0114] [Input restriction for judgment] Furthermore, the visual inspection support system 1 may, under certain conditions, for example, prevent the inspector from entering a judgment of good or bad product if the visual inspection support information or automatic inspection results do not match the judgment result entered by the inspector. For example, the visual inspection support system 1 may restrict the inspector from entering a judgment of good product for an object or part of a visual inspection that has been classified into a low good product rate category in the automatic visual inspection of the visual inspection device 2A, and may also display a warning. This can suppress misjudgments or incorrect inputs by inexperienced inspectors or inspectors who have accumulated fatigue.

[0115] There are no particular restrictions on the conditions for restricting input for good product judgment. In addition to the above, it is possible to set conditions as appropriate, such as restricting inspectors from judging inspected items or visually inspected areas that are classified into the high good product rate category as defective in automated visual inspection, or restricting inspectors from judging inspected items or visually inspected areas that are classified into the low good product rate category as good products in automated visual inspection.

[0116] Figure 22 shows an example of the display on the display unit 5 when judgment input is restricted. Of the "NG (defective judgment)" and "OK (good product judgment)" icons, which are input units 6, the "OK" icon is grayed out and cannot be clicked.

[0117] [Multiple person detection] Furthermore, if certain conditions are met, for example, if the visual inspection support system 1 does not match the judgment result entered by the inspector with the visual inspection support information or automated inspection result, the system may require the input of judgment results from other inspectors in addition to the inspector's judgment result in order to accept the judgment input of good or defective products from the inspector. In this case, the system will only accept input if the judgment results of the inspector and the other inspectors are the same. Other inspectors may include inspectors with more experience than the inspector, supervisors, or managers. The number of other inspectors may be one or two or more. This allows for accurate visual judgment, and also allows the inspector to compare their own judgment results with those of the automated visual inspection to confirm and verify the accuracy of their own judgments.

[0118] There are no particular restrictions on the conditions requiring judgment by multiple people. For example, they can be set as appropriate, such as when an inspector judges an object or visually inspected area classified in the low good product rate category as a good product in an automated visual inspection, or when an inspector judges an object or visually inspected area classified in the high good product rate category as a defective product in an automated visual inspection.

[0119] [Administrator judgment] Furthermore, the visual inspection support system 1 may, under certain conditions, require the administrator to perform a visual inspection and input the visual inspection result when an inspector is judging an object or part of an object that has been classified into a category that is difficult to judge even within the medium-good product rate category in an automated visual inspection. Figure 23 shows an example of a display where the "Administrator Only" icon is displayed on the input unit 6. As shown in Figure 23, the display unit 5 does not display the "NG" or "OK" icons that the administrator uses as input unit 6, but instead displays the "Administrator Only" icon. Then, by entering a password known only to the administrator, the "NG" or "OK" icons are displayed, and the visual inspection result can be entered. This makes it possible to accurately perform visual inspections of objects or parts of an object that have been classified into a category that is difficult to judge.

[0120] [Acceptable item icon] Furthermore, as shown in Figure 24, the visual inspection support system 1 may display an "acceptable item" icon in addition to the "NG (defective)" and "OK (good)" icons as the input unit 6 for inputting visual judgment results. The "acceptable item" icon is entered for inspection targets or visual inspection parts that are not clearly good products but are implemented at a level that does not pose practical problems even if they are distributed in the market.

[0121] This allows us to collect data on how inspectors visually judged the results of the automated inspection by the visual inspection device 2A, and statistically analyze this data to help, for example, set thresholds for each category.

[0122] Furthermore, this "acceptable item" icon may be displayed in all visual inspections, or it may be displayed only in visual inspections of a specified defect name (defect description). In other words, some defects may require a strict pass / fail judgment, while others may allow for a certain degree of flexibility in pass / fail judgment. By providing inspectors with options for inputting the pass / fail result in the latter type of visual inspection, the results of the visual inspection can be appropriately reflected.

[0123] Table 1 shows an example of setting defect names, categories, and thresholds. The "Defect Name" column lists the defect names that describe the nature of the defect. In the settings shown in Table 1, categories 1-4 are set, and the threshold for each category differs for each defect name. The threshold uses the defect probability (%) calculated based on a quantitative evaluation value of the quality. In the settings shown in Table 1, the higher the defect probability, the higher the category number. That is, Category 1 is a high-quality product category, Category 4 is a low-quality product category, Categories 2 and 3 are medium-quality product categories, and Category 3 has a higher defect probability than Category 2.

[0124] [Table 1]

[0125] Each category has defined control details for visual inspection. Examples of control details are as follows. As shown in Table 1, visual inspection support information is displayed in a manner corresponding to the defect name and / or category. Note that the present invention is not limited to the defect names, categories, and thresholds shown in Table 1. Automatic OK: Automatically determines OK. The inspector does nothing. (If the default setting is "Display after judgment," it will behave the same as "OK only.") OK only: Only the OK button is enabled. The NG button is disabled. The operator has no choice but to press the OK button to determine it as OK. —: Visual inspection support information is displayed, but no warning message is displayed at the top of display unit 5; standard inspection Warning: In addition to visual inspection support information, a warning message is displayed at the top of display unit 5. Administrator Call: Requires an administrator-only password to determine OK / NG status. NG Only: Only the NG button is enabled. The OK button is disabled. The inspector has no choice but to press the NG button to determine a result as NG. Automatic NG: Automatically determines the item as NG. The inspector does nothing. (If the default setting is "Display after judgment," it will behave the same as "NG only.")

[0126] [feedback] Furthermore, the visual inspection support system 1 may display the visual inspection support information and visual judgment results on the display unit as reference information when visually inspecting other objects, along with the visual inspection support information for other objects. That is, in the inspection of the same type of object or the same part requiring inspection on the same type of object, the judgment results of a previously performed visual inspection may be displayed in addition to the visual inspection support information.

[0127] The visual inspection results from previous visual inspections may include, for example, the number of good and defective items. In addition, the visual inspection images used for the judgment, the name of the inspector who made the judgment, or the inspector's ID may also be displayed.

[0128] [Bulk inspection of all items to be inspected] Furthermore, the visual inspection support system 1 may perform sequential visual inspections of multiple objects requiring visual inspection after the database 10 has recorded multiple sets of automated inspection data for those objects. This control is performed by the control unit 18 of the visual inspection terminal 3A. After multiple sets of automated inspection data have been recorded, an inspection order is assigned to these visual inspection data, and the display unit 5 is displayed, the visual inspection is performed, and the visual inspection results are input sequentially (see Figure 16, etc.).

[0129] [Pre-emptive shooting] The visual inspection support system 1 may perform an automated visual inspection of the parts of an object to be inspected being transported on one inspection line 9 using the visual inspection terminal 3A while the visual inspection terminal 3A is performing a visual inspection of the object to be inspected being transported on the other inspection line 9 using the visual inspection device 2A on the other inspection line 9, and save the automated inspection data in the database 10. After performing the automated visual inspection of an object, the other inspection line 9 can perform the automated visual inspection of the next object without waiting for the visual inspection terminal 3A to finish, thereby shortening the cycle time of the automated visual inspection process without any waiting time for visual inspection.

[0130] When the visual inspection terminal 3A completes the visual inspection of an object being transported on one inspection line 9, it switches its display screen to read the visual inspection data from the other inspection line 9 from the database 10 and displays it on the display unit 5, and then performs the visual inspection. In this way, the visual inspection support system 1 can shorten the cycle time of the visual inspection process by having the visual inspection device 2A installed on each inspection line 9 perform the automatic visual inspection in advance, and then having the visual inspection terminal 3A sequentially read the visual inspection data stored in the database 10 and perform the visual inspection.

[0131] In other words, the visual inspection support system 1 is equipped with an appearance inspection device 2A on each inspection line 9, but the visual inspection of the objects to be inspected transported on each inspection line 9 is performed centrally by displaying the visual inspection images on a display unit 5 such as a monitor provided on the visual inspection terminal 3A. Furthermore, the visual inspection by the visual inspection terminal 3A is performed using images captured by the appearance inspection device 2A.

[0132] Therefore, if visual inspection is required simultaneously on multiple inspection lines 9, the other inspection lines will be on standby while the visual inspection of the object being inspected is being performed on one inspection line 9. Then, after the visual inspection on one inspection line 9 is completed, if the visual inspection device 2A on another inspection line moves its imaging camera to the part of the object that needs to be inspected, time is required for the camera to move and focus in order to photograph the part of the object that needs to be inspected at a predetermined angle and depth of field. During this time, the inspector is unable to proceed with the inspection using the visual inspection terminal 3A and is in an inspection waiting state, resulting in a loss of cycle time for the inspection process. If there are multiple parts that need to be inspected on a single object, the time required to move the imaging camera will increase further, extending the cycle time of the inspection process.

[0133] Therefore, the visual inspection support system 1 related to this technology performs an inspection of one inspection line 9 using a visual inspection terminal 3A, while the visual inspection device 2A of the other inspection line 9 takes photographs of the parts of the object to be inspected on that other inspection line 9 under predetermined shooting conditions. In this technology, this is called pre-emptive shooting. The automated inspection images acquired by pre-emptive shooting are stored in the database 10 along with other automated inspection data. When the inspector finishes inspecting one inspection line 9, the display on the display unit 5 of the visual inspection terminal 3A is switched to the pre-emptively captured visual inspection images of the other inspection line 9. The pre-emptively captured visual inspection image data of the other inspection line 9 is read from the database 10 and displayed on the display unit of the visual inspection terminal 3A. This allows the inspector to quickly perform a visual inspection of the object to be inspected on that other inspection line 9.

[0134] In this way, the visual inspection support system 1 can shorten the cycle time of the visual inspection process by displaying the visual inspection images captured in advance by the visual appearance inspection devices 3 installed on each inspection line 9 on the visual inspection terminal 3A.

[0135] [Additional shooting] Furthermore, the visual inspection support system 1 may perform additional imaging with the visual inspection device 2A while a visual inspection is being performed with the visual inspection terminal 3A. When the visual inspection device 2A receives an instruction to take additional images from the visual inspection terminal 3A during a visual inspection using the visual inspection images, the imaging camera 11 is driven in accordance with the operation of the visual inspection terminal 3A and the object to be inspected is photographed. The additionally photographed images are displayed as real-time images on the display unit 5 of the visual inspection terminal 3A. This allows the inspector operating the visual inspection terminal 3A to display the object to be inspected at a desired angle and magnification, and to observe areas requiring further inspection in more detail, such as when it is difficult to make a judgment using the predetermined angle and magnification of the visual inspection images.

[0136] Furthermore, real-time images additionally captured by the visual inspection device 2A may be converted into data in response to the operation of the visual inspection terminal 3A, and this additionally captured video data may be stored in the database 10 as the visual inspection image used for judgment, associated with the judgment result and other information. Alternatively, the visual inspection device 2A may capture one or more still images of the real-time images obtained through additional capture in response to the operation of the visual inspection terminal 3A, and store the additionally captured image data in the database 10.

[0137] Furthermore, additional shooting condition data, such as angle and magnification set by the inspector, can be stored in the database 10 as visual inspection data, associated with the substrate type ID, etc. Subsequent automatic shooting for the same substrate type, etc., can be performed based on this additional shooting condition data, or subsequent automatic shooting can be performed based on the additional shooting condition data in addition to the predetermined shooting conditions. This allows for automatic shooting with desired shooting conditions for inspection targets of the same substrate type, etc., enabling efficient visual inspection by the inspector.

[0138] Furthermore, when the visual inspection image is displayed on the display unit 5 and the inspector performs a visual inspection, the visual inspection device 2A may be configured to wait at the position where the imaging camera 11, which performed the automatic shooting, has captured the first inspection area that the inspector will visually inspect first using the visual inspection image. If there are multiple inspection areas for a single object to be inspected, that is, if there are multiple visual inspection images that are the subject of the visual inspection, the visual inspection images are displayed on the display unit 5 in a predetermined order and made available for the inspector's visual inspection.

[0139] Here, the visual inspection device 2A positions the imaging camera 11 to capture the first inspection area in preparation for the inspector's instruction to take additional images. The first inspection area refers to the area of ​​the object being inspected during visual inspection that is first displayed on the display unit 5 as a visual inspection image. As a result, when the inspector instructs for additional images to be taken, the imaging camera 11 can quickly take additional images, eliminating the time required to move the imaging camera 11 and shortening the cycle time of the inspection process.

[0140] Alternatively, the visual inspection device 2A may move the imaging camera 11 to the area corresponding to the visual inspection image sequentially displayed on the display unit 5. As a result, when the inspector instructs the imaging camera 11 to take additional shots, the imaging camera 11 can quickly take additional shots as the imaging camera 11 moves to the area displayed on the display unit 5, thereby reducing the time required to move the imaging camera 11 and shortening the cycle time of the inspection process.

[0141] Figure 25 is a flowchart showing the operation flow of the visual inspection terminal 3A when an additional imaging instruction is given. Similar to the flow shown in Figure 16, when a screen display switching instruction is input (step S11), the visual inspection terminal 3A retrieves the visual inspection data of the object to be inspected (printed circuit board 7) to be displayed on the display unit 5 from the database 10 (step S12). Then, the visual inspection terminal 3A displays the visual inspection image (automatic inspection image), reference image, and visual inspection support information of the object to be inspected that was switched on the display unit 5, and the inspector performs a visual inspection (step S13).

[0142] Next, the inspector inputs an instruction to take additional images via the input unit 6 (step S21). The visual inspection device 2A drives the imaging camera 11 in response to the operation signal from the input unit 6 and takes images of the object to be inspected at the angle and magnification corresponding to the operation. The captured images are displayed as real-time images on the display unit 5, and the inspector performs a visual inspection. The display unit 5 may also display the visual inspection images along with the additional captured images.

[0143] Next, the inspector inputs the judgment result (step S22). At this time, if any corrections are made to the automated inspection data, such as the defect name, the correction details are also input. Then, the visual inspection terminal 3A stores the judgment result in the database 10 in association with visual inspection data such as the board ID, via the control unit 18 (step S23). In addition, the visual inspection images (additional images) used for the judgment are converted into data and stored in the database 10 in association with the judgment result and visual inspection data.

[0144] The digitization of visual inspection images (additional images) and their storage in database 10 is the operation of storing the visual inspection images used as the basis for the judgment as image data in the first database 10A. The images additionally captured by the visual appearance inspection device 3 based on the inspector's instructions are real-time images and are not replayed images recorded in database 10. Therefore, the visual inspection images used as the basis for the pass / fail judgment are stored in database 10 as video data and / or image data, associated with data such as the pass / fail judgment result, defect name, and board ID.

[0145] [AOI] Furthermore, the visual inspection support system 1 may perform an automated inspection using an automated optical inspection (AOI) 20 or other automated inspection device before the automated visual inspection by the visual inspection unit 2 or the visual inspection device 2A. As shown in Figure 26, the AOI 20 is connected via a bus 8 to the visual inspection devices 2A, visual inspection terminals 3A, and database 10 installed on each inspection line 9, enabling mutual data communication. Of course, the visual inspection support system 1 may also use a network connection configuration other than the bus type (see Figure 27).

[0146] AOI20 automatically performs visual inspection of the parts of an object that need to be inspected. It photographs the object on the inspection line 9 and automatically determines the quality of the object and the parts that need to be inspected based on the image data. AOI20 is installed on the inspection line 9 upstream of the visual inspection device 2A in the direction of transport of the object.

[0147] The AOI 20 has a configuration similar to that of known AOIs, and for example, as shown in Figure 28, it comprises an imaging camera 21 such as a CCD camera that photographs the object to be inspected, an AOI computer 22 that performs pass / fail judgment based on the image data captured by the imaging camera 21, a communication unit 23 connected to the bus 8 and communicating with a database 10, etc., and a control unit 24 that controls the operation of the entire AOI.

[0148] The AOI computer 22, based on an automatic judgment processing program, uses the imaging camera 21 to photograph predetermined inspection points on the object to be inspected and uses the captured image data to determine whether the object is good or bad at those predetermined inspection points. Specifically, the automatic visual inspection device 2 has pre-existing position data (XYθ coordinates) indicating the inspection points of the printed circuit board 7 to be inspected, and, according to the program, sequentially captures inspection images of the positions indicated by the position data, for example, in order of proximity. The automatic judgment result can be arbitrarily set to, for example, "OK," "NG," or "Unable to determine." A known image processing program can be used for the good or bad judgment.

[0149] As a result, the inspection system 1 can use the AOI 20 to automatically determine the quality of the soldering of electronic components mounted on the printed circuit board 7, and save the automatic determination results, along with the location data of the inspection area, to the database 10 via the communication unit 23.

[0150] [Inspection Flow] Here, the visual inspection support system 1 may perform automatic visual inspection by the visual inspection device 2A on all items, similar to the AOI 20, or it may limit the inspection to items excluding those judged as "OK" or parts requiring inspection by the AOI 20. Specifically, as shown in Figure 29, in step S41, an automatic inspection is performed by the AOI 20. This performs a visual inspection on all items. Next, in step S42, an automatic visual inspection is performed by the visual inspection device 2A. The target of this automatic visual inspection is limited to items or parts requiring inspection that have been judged as "NG" or "undecidable," for example, excluding those judged as "OK" or parts requiring inspection by the AOI 20. The position data (XYθ coordinates) of the parts requiring inspection to be performed by the visual inspection device 2A are read from the database 10 along with the automatic judgment result of the AOI 20. After that, in step S43, a visual inspection is performed using the visual inspection terminal 3A.

[0151] This reduces the number of inspections performed by the automated visual inspection device 2A and the visual inspection terminal 3A, thereby shortening the cycle time for visual inspections and reducing the burden on inspectors, thus preventing misjudgments and incorrect inputs.

[0152] In Figure 29, all objects that underwent automated visual inspection by the visual inspection device 2A were subject to visual inspection. However, as shown in Figure 30, the number of objects to be visually inspected may be reduced by selecting which objects to inspect visually. As described above, by setting thresholds for quantitative evaluation values ​​(scores), objects to be inspected can be classified into multiple categories, and objects belonging to a predetermined category can be excluded from visual inspection, or only objects belonging to a predetermined category can be subject to visual inspection.

[0153] Specifically, an automated inspection by AOI20 (step S44) and an automated visual inspection by the visual inspection device 2A (step S45) are performed. The objects to be inspected by the visual inspection device 2A may be all the objects that AOI20 has targeted for inspection, or they may be objects other than those that AOI20 has judged as "OK". Subsequently, the control unit 18 of the visual inspection terminal 3A selects only the visual inspection data of objects belonging to a predetermined category (step S46). For example, objects or visual inspection parts belonging to the high good product rate category and the low good product rate category are excluded from the visual inspection. Then, only the visual inspection data of objects belonging to the medium good product rate category between the high good product rate category and the low good product rate category is displayed on the display unit 5 (step S47). This reduces the number of visual inspections performed by the inspector and allows them to concentrate on the visual inspection.

[0154] [Change in shooting conditions] The imaging conditions for the object to be inspected by the visual inspection device 2A may differ from those for the object to be inspected by the AOI 20. For example, when the visual inspection device 2A is used to photograph areas requiring inspection, the angles and magnifications of the imaging cameras 11 (oblique camera 11a and planar camera 11b) may be set differently from those of the imaging camera 21 of the AOI 20 to capture images more suitable for automated visual inspection or visual judgment.

[0155] [Multiple testing lines] Furthermore, as shown in Figure 31, the visual inspection support system 1 may consist of a plurality of inspection lines 9 (for example, inspection line 9A, inspection line 9B, inspection line 9C, etc.) each equipped with an AOI 20 and an appearance inspection device 2A, and a single visual inspection terminal 3A. The single visual inspection terminal 3A may sequentially display the visual inspection images and visual inspection support information captured by the appearance inspection devices 2A of each inspection line 9, allowing for centralized visual inspection. [Explanation of Symbols]

[0156] 1 Visual inspection support system, 2 Visual inspection unit, 2A Visual inspection device, 3 Visual inspection unit, 3A Visual inspection terminal, 5 Display unit, 6 Input unit, 7 Printed circuit board, 8 Bus, 9 Inspection line, 10 Database, 11 Imaging camera, 11a Oblique camera, 11b Planar camera, 12 Computer for automatic inspection, 13 Communication unit, 14 Support unit, 17 Communication unit, 18 Control unit, 20 AOI, 21 Imaging camera, 22 Computer for AOI, 23 Communication unit, 24 Control unit

Claims

1. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. The aforementioned quantitative evaluation involves scoring the quality of the inspected area. The display unit shows visual inspection support information based on the score. The aforementioned visual inspection support information is divided into multiple categories based on the score, within predetermined ranges, and is information that allows for the identification of the categories. The aforementioned visual inspection support information is displayed in a manner corresponding to the defect name and / or category. A visual inspection support system that excludes the inspection areas, which are categorized into high-quality product category and low-quality product category in the aforementioned visual inspection support information, from the scope of visual inspection by the visual inspection unit.

2. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. The aforementioned visual inspection support information is displayed on the display unit after the visual inspection result is entered by the inspector, as part of the visual inspection support system.

3. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. A visual inspection support system that, if the visual inspection support information or automated inspection results do not match the judgment results entered by the inspector, refuses to accept the judgment results entered by the inspector.

4. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. A visual inspection support system that, in the event of a discrepancy between the visual inspection support information or automated inspection results and the judgment results entered by the inspector, requires the input of judgment results by other inspectors in order to accept the input of the judgment results.

5. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. A visual inspection support system in which the aforementioned visual inspection support information and the aforementioned visual judgment results are displayed on the display unit as reference information along with the aforementioned visual inspection support information for other objects being inspected during a visual inspection of other objects being inspected.

6. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. The appearance inspection terminals that constitute the appearance inspection unit, The visual inspection unit includes a visual inspection terminal. Multiple inspection lines, each having the aforementioned visual inspection terminal, The system includes a database that stores the images captured by the visual inspection terminal, the location information of the inspection area, the visual inspection support information, and the visual judgment results. The aforementioned visual inspection terminal is connected to each of the multiple inspection lines' visual inspection terminals and the database, and is a visual inspection support system that centrally performs visual inspections of the objects to be inspected being transported to each of the inspection lines.

7. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, quantitatively evaluates the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the quantitative evaluation, The visual inspection unit includes a display unit that displays images captured by the aforementioned imaging device, and an input unit that inputs the results of visual inspection by an inspector. The display unit displays the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, as well as the visual inspection support information. The system includes an automatic optical inspection device that automatically performs an external inspection of the object to be inspected prior to the external inspection by the external inspection unit, The aforementioned visual inspection unit is a visual inspection support system that performs the visual inspection on the parts that have been determined to be defective by the automatic optical inspection device, using those parts as the inspection areas.

8. The aforementioned quantitative evaluation involves scoring the quality of the inspected area. The visual inspection support system according to any one of claims 2 to 7, wherein the display unit displays visual inspection support information based on the score.

9. The visual inspection support system according to claim 8, wherein the visual inspection support information is divided into multiple categories for each predetermined range based on the score, and the category is identifiable information.

10. The visual inspection support system according to claim 9, wherein the visual inspection support information is displayed in a manner corresponding to the name and / or category of the defect.

11. The visual inspection support system according to claim 1, wherein the threshold scores for classifying the multiple categories can be arbitrarily set and / or changed.

12. The visual inspection support system according to claim 9, wherein the threshold scores for classifying the multiple categories can be arbitrarily set and / or changed.

13. The visual inspection support system according to claim 1 or 11, wherein the visual inspection support information is displayed on the display unit together with the captured image when the inspector makes a visual judgment.

14. The visual inspection support system according to claim 8, wherein the visual inspection support information is displayed on the display unit together with the captured image when the inspector makes a visual judgment.

15. The visual inspection support system according to claim 2, wherein the visual inspection support information is displayed on the display unit when the visual inspection support information or the automated inspection result does not match the judgment result entered by the inspector.

16. The appearance inspection terminals that constitute the appearance inspection unit, The visual inspection support system according to claim 1 or 11, comprising a visual inspection terminal that constitutes the visual inspection unit.

17. The visual inspection support system according to claim 7, wherein the imaging conditions of the object to be inspected by the visual inspection unit are different from the imaging conditions of the object to be inspected by the automatic optical inspection device.

18. A step of performing an automated visual inspection of an object to be inspected by an inspector using an visual inspection unit, and quantitatively evaluating the quality of the inspected part, A step of generating visual inspection support information that serves as an indicator for determining the quality of the inspected area based on the quantitative evaluation, A step of displaying on the display unit of the visual inspection unit the visual inspection image captured by the imaging device of the visual inspection unit of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, and the visual inspection support information, It has a process for receiving input of visual inspection results from inspectors. The appearance inspection device that constitutes the appearance inspection unit, The visual inspection unit includes a visual inspection terminal, Multiple inspection lines, each having the aforementioned visual inspection device, The system includes a database that stores automated inspection data, including images captured by the visual inspection device and the visual inspection support information. The visual inspection terminal is connected to each of the multiple inspection lines' visual inspection devices and the database, and the visual inspection method centrally performs the visual inspection of the objects to be inspected being transported to each of the inspection lines.

19. The visual inspection method according to claim 18, wherein the visual inspection terminal sequentially performs visual inspections on multiple objects requiring visual inspection after the database has recorded multiple sets of automatic inspection data for those objects.

20. An appearance inspection unit having an imaging device, which captures a predetermined inspection area of ​​an object to be inspected by an inspector using the imaging device, automatically performs an appearance inspection using the captured image, scores the quality of the predetermined inspection area, and generates visual inspection support information that serves as an indicator for determining the quality of the inspection area based on the score, The visual inspection unit includes a display unit that displays visual inspection images of the inspection area of ​​the object to be inspected, which is subjected to visual inspection by the visual inspection unit and subject to visual inspection by the inspector, as captured by the imaging device, as well as visual inspection support information, and an input unit that inputs the visual judgment results by the inspector. Based on the score, the inspection area is divided into three categories: a high-quality category where the inspection area is likely to be a good product, a low-quality category where the inspection area is unlikely to be a good product, and a medium-quality category between the high-quality category and the low-quality category. The inspection areas, which are classified into high-quality product category and low-quality product category, are excluded from the visual inspection using the visual inspection unit. Visual inspection support system.

21. The process involves the visual inspection unit capturing images of designated inspection areas of an object to be inspected by an inspector using an imaging device, automatically performing a visual inspection using the captured images, scoring the quality of the designated inspection areas, and generating visual inspection support information that serves as an indicator for determining the quality of the inspection areas based on the scores. A step of displaying on the display unit of the visual inspection unit the visual inspection image captured by the imaging device of the inspection area of ​​the object to be inspected, which has been inspected by the visual inspection unit and is subject to visual inspection by an inspector, and the visual inspection support information, It has a process for receiving input of visual inspection results from inspectors. Based on the score, the inspection area is divided into three categories: a high-quality category where the inspection area is likely to be a good product, a low-quality category where the inspection area is unlikely to be a good product, and a medium-quality category between the high-quality category and the low-quality category. The inspection areas, which are classified into high-quality product category and low-quality product category, are excluded from the visual inspection using the visual inspection unit. Visual inspection method.

Images