Optimizing reference groups for visual inspection

Abstract

A method and system for visual inspection automatically detects an image of an item, which can help maximize correlation of a reference image in a reference group to an inspection of the item. The detected image can be automatically added to or removed from the reference group. Optionally, recommendations to the user as to how to improve and optimize the reference group can be generated based on the detected images.

Description

Technical Field

[0001] The present invention relates to visual inspection processes, for example, image-based inspection of items on a production / inspection line. Background Art

[0002] Inspection during the production process helps control the quality of products by identifying defects and acting on defect detection (e.g., by fixing the defects or discarding the defective parts), and thus plays a role in improving productivity, reducing defect rates, and reducing rework and waste.

[0003] Traditionally, inspection tasks were performed by human workers. With the growing success of machine learning techniques, more and more inspection tasks are now automated. Some automated visual inspection solutions compare an image of the inspected item with one or more reference images in order to detect defects in the inspected item (or perform other inspection tasks).

[0004] Reference images can be acquired during a setup phase prior to inspecting an item by sequentially imaging a sample of the item. Typically, sequentially captured images do not vary significantly from one another. Therefore, acquiring a set of reference images that represent all possible variations of the inspected item may require the acquisition of a large number of images and may be time consuming.

[0005] Today's automated vision inspection solutions fail to meet the needs of industrial plants for flexibility and improvement. Summary of the invention

[0006] Embodiments of the present invention provide a method for optimizing a set of reference images (also referred to as a "reference set") used in a visual inspection process. Some embodiments of the present invention provide a tool for a user (such as a technician, inspector, and / or inspection line operator) to assist the user in optimizing the set of reference images.

[0007] Methods and systems according to embodiments of the present invention automatically detect images of an article that can help maximize the relevance of reference images in a reference group to the inspection of the article. The detected images can then be automatically added to the reference group or removed from the reference group. In some embodiments, recommendations to users on how to improve and optimize the reference group can be generated based on the detected images. The recommendations generated based on the automatic detection typically include suggesting to the user actions related to the reference group (e.g., adding images to the reference group or removing images from the reference group) that can optimize the reference group. The recommendation can then be communicated to the user, thereby providing the user with tools for optimizing the reference group without the user having to change or complicate his inspection routine.

[0008] Thus, embodiments of the present invention simplify the inspection process for the user while maximizing the accuracy and efficiency of the visual inspection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present invention will now be described in conjunction with the following examples and embodiments with reference to the following schematic drawings, so that the present invention can be more fully understood. In the drawings:

[0010] Figure 1 Schematically illustrates a system according to an embodiment of the present invention;

[0011] Figure 2A and Figure 2B A method for optimizing a reference group according to an embodiment of the present invention is schematically shown;

[0012] Figure 3A , Figure 3B and Figure 3C schematically illustrates a method for optimizing a reference group according to another embodiment of the present invention; and

[0013] Figure 4A and Figure 4B A method for generating a recommendation about a reference group based on detected differences between an inspection image and a previously captured image according to an embodiment of the present invention is schematically illustrated. DETAILED DESCRIPTION

[0014] Embodiments of the present invention provide methods and systems for optimizing a reference group for visual inspection of items by analyzing images of the same type of items (including inspection images and previously captured images), detecting differences between the inspection images and the previously captured images, and generating a signal about the reference group based on the detected differences (e.g., a signal to add an image to the reference group or to remove an image from the reference group). Images can be automatically added or deleted based on the generated signals. Alternatively or additionally, a recommendation about the reference group can be generated based on the signal, and the recommendation can be communicated to a user. The recommendation communicated to the user can include, for example, a suggestion to add an inspection image to the reference group, or, for example, to delete or remove a reference image from the reference group.

[0015] Automatically adding or removing images from a reference group and / or recommendations for adding or removing images from a reference group (generated by a processor based on analysis of images of the same type of items) effortlessly create a group of related images and / or provide a user with tools to create a group of related images that can be used as a reference for inspecting these items.

[0016] The following terms will be used in this manual:

[0017] The "setup phase" is a phase in the visual inspection process where images of examples of the same type of artifact (which are typically provided by the user) are provided to the visual inspection machine for processing. The setup phase is typically the initial phase followed by the inspection phase, but the setup can also be performed after the inspection phase has already begun.

[0018] The "inspection phase" is the phase in the visual inspection process after the initial setup phase, in which images of inspected items of the same type as the samples used in the initial setup phase are analyzed for the inspection task.

[0019] An "inspection task" relates to a function of the inspection process (typically the inspection phase), such as defect detection, defect location, quality assurance, classification and / or counting, gating, etc.

[0020] A "reference image" refers to an image of an item whose status has been confirmed by a user. For example, an image obtained during a setup phase may be considered "confirmed" because the item is provided by a user. In addition, an image approved by a user (e.g., via a user interface device) as being free of defects or having defects may be considered "confirmed."

[0021] A "check image" refers to an image of an item of the same type as the examples used in the setup phase, however, the item in the check image has an unknown status.

[0022] The "status" of an item or image relates to the inspection information results or predictions associated with the imaged item. For example, the status of an item may be "defective" or "non-defective" or "unknown".

[0023] "Articles of the same type" means articles or objects that have the same physical composition and are similar to each other in shape and size and possibly color and other physical characteristics. Generally, articles of a single production series, a batch of articles of the same type, or a batch of articles at the same stage of their production line can be "articles of the same type". For example, if the inspected articles are sanitary products, different sink bowls from the same batch are articles of the same type.

[0024] "Defects" may include, for example, visible flaws on the surface of the item, an undesirable size of the item or part of the item, an undesirable shape or color of the item or part of the item, an undesirable number of parts of the item, incorrect or missing assembly of the interface of the item, damaged or burned parts, and incorrect alignment of the item or part of the item, incorrect or defective barcodes, serial numbers, text, icons, etc., and any differences between non-defective samples and the inspected item (in general), which defects will be obvious to the user (i.e., inspector) from the image. In some embodiments, defects may include defects that are only visible in magnified or high-resolution images (such as images obtained through a microscope or other specialized camera).

[0025] In addition, in the following description, for the purpose of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be practiced without the specific details presented herein. In addition, well-known features may be omitted or simplified in order not to obscure the present invention.

[0026] Unless specifically stated otherwise, as will be apparent from the following discussion, it should be understood that in this specification, discussions using terms such as "analyze," "process," "calculate," "compute," "determine," "detect," "recognize," "learn," etc. refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or transform data represented as physical (e.g., electronic) quantities within registers and / or memories of the computing system to other data similarly represented as physical quantities within the memory, registers, or other such information storage, transmission, or display devices of the computing system. Unless otherwise stated, these terms refer to the automatic actions of a processor independent of and without any action by an operator.

[0027] The visual inspection process can include a setup phase and an inspection phase. In the setup phase, examples of manufactured products of the same type and in a known state (e.g., non-defective items or defective items) are successively placed within the field of view (FOV) of the camera(s) to provide a reference image.

[0028] In an inspection phase, typically following an initial setup phase, an inspected item of the same type as the item in the reference image and of unknown status (e.g., may or may not have a defect) is imaged, thereby providing an inspection image. The inspection image is analyzed using computer vision techniques (e.g., a machine learning process) and compared to the reference image to detect defects in the inspected item and / or for other inspection tasks.

[0029] In this setup phase, the processor learns setup parameters, such as spatial characteristics and features or attributes that uniquely represent the item in the image, and optimal parameters for the image of the item, such as optimal imaging or optical parameters (e.g., exposure time, focus, and lighting). These parameters can be learned, for example, by analyzing images of non-defective items using different imaging parameters and by analyzing the relationship between different images of the same type of non-defective items. This analysis during the setup phase enables the detection of the same type of items (non-defective or defective) in new images, regardless of the imaging environment of the new images, and enables the continuous optimization of imaging parameters during the following inspection phase with minimal processing time.

[0030] In one embodiment, analysis of the setup images is used to determine a spatial range in which items in the setup images do not exhibit stereo distortion. For example, the level of stereo distortion between items in different images may be analyzed by detecting areas in the items that do not have corresponding features between the setup images, by analyzing intersection locations and angles between regions of interest of item boundaries or markings on the items, etc. The boundaries of the spatial range may be calculated by comparing two (or more) setup images (in which the items may be positioned and / or oriented differently) and determining which image exhibits stereo distortion and which image does not exhibit stereo distortion.

[0031] The calculated range can then be used to determine the boundaries of where and / or what orientations, scales, or other arrangements of the inspected item can be placed on the inspection line in order to avoid distortion. In addition, by using a set of setup images as a reference to each other, the processor can detect images with similar spatial resolution, and the set of images can then be analyzed to see if there are enough similar setup images to allow registration, defect detection, and other analysis for every possible position / location on the inspection line.

[0032] "Sufficient set images" are collected when a substantially complete representation of one type of item is achieved. For example, as described above, when enough images are collected to be able to determine the spatial range within which each reference image can be used as an undistorted reference, or when the set images are compared to each other, no new tolerances or surface changes are discovered as new set images are added.

[0033] Analysis of the setup images may be performed to gather information about the item's likely 2D shape and 3D properties (eg, rotation on a check line) or to find unique distinguishing features of the item and the spatial relationships between these unique features as preserved between setup images.

[0034] Based on the information gathered from the setup images, the processor can detect a second item of the same type and perform the inspection task, even if the second item was not previously learned by the processor. This allows the processor to detect when a new item (of the same type) is imaged, and then analyze the new item based on the analysis of the setup images, for example searching for defects on the inspected item.

[0035] Therefore, a set of reference images of the inspected object should provide a more or less complete representation of the object type in order to maximize the inspection accuracy of the object.

[0036] exist Figure 1 In one embodiment of the invention shown, a system for visual inspection includes a processor 102 in communication with a camera 103 (one or more cameras) and a user interface (UI) device 106. The camera 103 captures images (e.g., reference images and inspection images) of the same type of items 104, for example, on an inspection line 105. The inspection line 105 may include, for example, a conveyor belt on which the items 104 are placed, such that movement of the conveyor belt continuously brings the items into the FOV 103' of the camera 103. The images of the items may be displayed to a user, such as a technician, an inspector, and / or an inspection line operator, for example, via the UI device 106.

[0037] The captured images may be stored in storage device 108. Storage device 108 may store a reference set of items 104 (e.g., in a database of reference images) and / or may store other images of items 104, for example, inspection images may be stored in an inspection image database.

[0038] The components of the system may be in wired or wireless communication and may include suitable ports and / or network hubs. In some embodiments, the processor 102 may communicate with devices such as the storage device 108 and / or the user interface device 106 via a controller such as a programmable logic controller (PLC) that is typically used in manufacturing processes, such as for data processing, storage, processing power, and communication capabilities. The controller may communicate with the processor 102, the storage device 108, the user interface device 106, and / or other components of the system via USB, Ethernet, appropriate cables, etc.

[0039] Processor 102 may include, for example, one or more processors and may be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a microprocessor, a controller, a chip, a microchip, an integrated circuit (IC), or any other suitable multi-purpose or specific processor or controller. Processor 102 may be locally embedded or remote.

[0040] The user interface device 106 may include a display, such as a monitor or screen, having a graphical user interface (GUI) for displaying images, instructions, recommendations, and / or notifications to the user (e.g., via text or other content displayed on the monitor). In other embodiments, the user interface device 106 may include other or additional components, such as sound or light-emitting components, by which instructions, recommendations, and / or notifications are transmitted to the user. The user interface device 106 can also be designed to receive input from the user. For example, the user interface device 106 may include a monitor and a keyboard and / or a mouse and / or a touch screen to enable the user to input feedback, instructions, etc.

[0041] The storage device 108 may be a server including, for example, volatile and / or non-volatile storage media such as a hard disk drive (HDD) or a solid state drive (SSD). The storage device 108 may be connected locally or remotely, such as in the cloud. In some embodiments, the storage device 108 may include software for receiving and managing image data (e.g., reference images).

[0042] The camera 103 may include a CCD or CMOS or other suitable image sensor. The camera 103 may be a 2D or 3D camera. In some embodiments, the camera 103 may include a standard camera provided by a mobile device such as a smart phone or a tablet computer. In other embodiments, the camera 103 is a dedicated camera, such as a camera for obtaining high-resolution images or a UV, near-infrared or infrared dedicated camera. In other embodiments, the camera 103 includes a non-optical camera, such as a neutron camera, a RADAR camera, etc.

[0043] The system may also include a light source, such as an LED or other suitable light source, to illuminate the FOV 103 ′ of the camera 103 , for example to illuminate the items 104 on the inspection line 105 .

[0044] Processor 102 receives image data (which may include, for example, data representing the intensity of reflected light and pixel values ​​of part or all of the image or video) of items 104 on inspection line 105 from one or more cameras 103 and runs processes according to embodiments of the present invention.

[0045] The processor 102 is typically in communication with a memory unit 112. The memory unit 112 may store at least a portion of the image data received from the camera 103. The memory unit 112 may include, for example, random access memory (RAM), dynamic RAM (DRAM), flash memory, volatile memory, non-volatile memory, cache memory, a buffer, a short-term memory unit, a long-term memory unit, or other suitable memory unit or storage unit.

[0046] In some embodiments, memory unit 112 stores executable instructions that, when executed by processor 102 , facilitate performance of operations of processor 102 , as described herein.

[0047] In one embodiment, processor 102 detects differences between an inspected image of item 104 and a previously captured image of item 104. Based on the detected differences, processor 102 may generate a signal regarding a reference set of images of item 104 (e.g., a signal to automatically add or remove an image from the reference set). In some embodiments, processor 102 generates a recommendation to a user based on the detected differences and causes the recommendation to be communicated to the user, e.g., via user interface device 106.

[0048] During the setup phase, the user may provide a small number of confirmed items 104 to be imaged by the camera 103 and / or the user may confirm a small number of images of the items 104 taken by the camera 103. The images of the confirmed items or the confirmed images may include images of defect-free and / or defective items. The user may then continue to provide items 104 of unknown state to be imaged by the camera 103. The confirmed images will be grouped into a reference group by the processor 102 (e.g., in the storage device 108), and the images of the items 104 of unknown state (i.e., the inspection images of the items 104) will be looked up for differences between them and the images previously taken by the camera 103 (in this case, the images in the reference group). If a difference is detected, the processor 102 generates a signal about the reference group. Based on the signal, an automatic action may be performed on the reference group (e.g., an image may be added to the reference group or removed from the reference group) or a recommendation may be generated for the user, which can help the user optimize the reference group. The automatic actions or recommendations generated by the processor streamline the user's inspection process without requiring the user to change or complicate his inspection routine.

[0049] In one example, Figure 2A and Figure 2B Schematically shown in , optimizing the reference group may include providing an inspection image 201 and a previously captured image 203 for analysis by the processor 102. The previously captured image 203 may include a reference image and / or a previously inspected image. Typically, as described above, the reference image includes an image confirmed by a user. In one embodiment, the previously captured image includes an image of an item of the same type without defects. In another embodiment, the previously captured image includes an image of an item of the same type with defects. In other embodiments, an image of an unknown state may be included in the previously captured image.

[0050] The processor 102 looks for differences or changes between the inspection image 201 and the previously captured image 203 (step 204). Based on the detected differences, the processor 102 generates a signal to optimize the reference set (step 206). For example, the processor 102 determines whether to add or remove an image from the reference set and generates a signal accordingly. The processor 102 can then cause an automatic action based on the signal.

[0051] In another embodiment, Figure 2B As schematically shown in , based on the signal generated in step 206, the processor 102 may cause a recommendation regarding the reference group to be communicated to the user (step 208). For example, if a predefined difference is detected, a recommendation is generated and communicated to the user, typically via the UI device 106. For example, the recommendation may be displayed to the user on a monitor. If no predefined difference is detected in step 204 and / or no signal regarding the reference group is generated in step 206, no recommendation is generated or displayed, and the inspection image 201 is added to the previously captured image 203, and the difference between the new inspection image and the previously captured image 203 is found. This process may continue until a complete representation of the item in the reference group is achieved. In some embodiments, achieving the complete representation is a time-based process (e.g., the complete representation is achieved after a predetermined or calculated time has passed) and / or based on the number of images acquired (e.g., the complete representation is achieved after a predetermined or calculated number of images have been acquired).

[0052] The differences or changes looked for by the processor may be based on statistical values ​​of parameters or features compared between the inspection image 201 and the previously captured image 203 .

[0053] The difference(s) found in step 204 may be differences above a threshold. In other embodiments, the difference(s) found in step 204 may be differences below a threshold. Figure 3A-3C In some embodiments, the threshold value may depend on the level of variability between items 104, such as Figure 4A and Figure 4B Further examples are given in .

[0054] exist Figure 3AIn one embodiment schematically shown in , the processor 102 looks for differences or changes between the inspection image 301 and the previously captured image 303 (step 304), and if a difference (e.g., a difference above a threshold) is detected, the processor 102 generates a signal to add the inspection image 301 to the reference set (and potentially communicates a recommendation to the user) (step 308). This embodiment ensures that significantly different images will be included in the reference set, thereby providing a better representation of all possible variations of an item (e.g., item 104). Variations in the image of item 104 may exist for a variety of reasons, such as different users performing the inspection, different incoming materials, different environmental conditions (e.g., day / night). These variations may not be fully represented in the small number of initially confirmed images that initially constitute the reference set, and therefore, adding images to the reference set that are different (e.g., above a threshold) from those already in the reference set helps to optimize the reference set.

[0055] exist Figure 3B In another embodiment schematically shown in FIG. 3 , in step 310, the processor 102 looks for differences or changes between the inspection image 301 or multiple inspection images 301, 301', 301", etc. and a previously captured image 303 (in this example, a reference image). If a difference is detected (e.g., a difference above a threshold), the processor 102 generates a signal to remove the reference image (previously captured image 303) from the reference group (and possibly communicates a recommendation to the user) (step 312). This embodiment ensures that reference images that are no longer relevant to the inspected item are not continued to be used as a reference. For example, the inspected item may change significantly over time (e.g., due to changes in production materials, changes in environmental conditions, etc.), so the item imaged in the initial reference image may no longer be used as a reference image and should be removed from the reference group.

[0056] Figure 3CAnother embodiment schematically illustrated in is an example in which a particular area or feature of an imaged article is erroneously detected as a defect or causes a defect to be erroneously determined in an image of a reference group. In step 314, the processor 102 looks for differences or changes between the inspection image 301 and a previously captured image 303 (in this example, the reference image). In this example, the difference looked for in step 314 is a difference in a particular area or feature that causes a defect to be erroneously detected. If there is no difference or a small difference between the area or feature in the inspection image 301 and the reference image 303, adding the inspection image to the reference group will increase the number of different images with areas or features that cause a defect to be erroneously detected, thereby providing a better representation of different changes in the particular area or feature. Therefore, if no difference (e.g., a difference below a threshold) is detected in step 314, the processor 102 generates a signal to add the inspection image 301 to the reference group (and, possibly, communicates a recommendation to the user) (step 316).

[0057] exist Figure 4A In one embodiment schematically shown in , the difference between the inspection image and the previously captured image comprises a change in a characteristic between the inspection image and the previously captured image. Thus, the processor 102 looks for a change in a characteristic between the inspection image 401 and the previously captured image 403 (step 404), and generates a signal about the reference group (and possibly communicates a recommendation to the user) (step 406), such as to add the inspection image to the reference group or remove the reference image from the reference group.

[0058] Changes in characteristics may include, for example, differences between parameters of the item in the inspection image and parameters of the same type of item in a previously captured image. Parameters may include, for example, relevance scores, visual similarity scores, number of abnormal regions, predictions of defects in abnormal regions, features used by machine learning algorithms to describe items and abnormal locations on items.

[0059] Thus, a method according to one embodiment may include comparing previously captured images 403 to each other to obtain a relevance score of the previously captured image relative to itself (e.g., a score of visual similarity or other similarities that may not be visible, such as correlation of statistical features), and if a difference is detected between the relevance score of the inspection image 401 relative to the previously captured image 403 and the relevance score of the previously captured image 403 relative to itself, a signal is generated (e.g., as described above) and a recommendation may be communicated to the user (step 406).

[0060] In yet another embodiment, the method includes comparing previously captured images 403 with each other to obtain the number of abnormal areas in items of the same type, and if there is a difference between the number of abnormal areas in the inspection image 401 and the number of abnormal areas in items of the same type in the previously captured images, a signal is generated (e.g., as described above) and a recommendation may be communicated to the user (step 406).

[0061] In another embodiment, the method includes obtaining a prediction of a defect in an abnormal region in a previously captured image 403 (e.g., by using a machine learning algorithm as described below), and if there is a difference between the prediction of a defect in an abnormal region in the inspection image 401 and the prediction of a defect in an abnormal region in the previously captured image 403, generating a signal (e.g., as described above), and a recommendation may be communicated to a user (step 406).

[0062] According to an embodiment of the present invention, a machine learning algorithm for obtaining a prediction of a defect may include, for example, detecting spots of suspected defects in an inspection image and spots of suspected defects in one or more previously captured images (e.g., a pixel group at a certain location in the inspection image is different from a pixel group at the same location in a reference image). If the distance between the spots is above a threshold, a signal may be generated to add the inspection image to the reference group. For example, if the distance between a spot in the inspection image and an area (spot) where a defect was erroneously detected in a previously captured image is below a threshold, a signal may be generated to add the inspection image to the reference group.

[0063] In some embodiments, the change in characteristics includes a difference between features used by the machine learning algorithm to describe the item in inspection image 401 (and possibly the location of anomalies on the item) and features used by the machine learning algorithm to describe the same type of item (and possibly the location of anomalies on the same type of item) in previously captured image 403. Thus, if there is a difference between features used by the machine learning algorithm to describe the item in the inspection image and features used by the machine learning algorithm to describe the same type of item in previously captured images, a signal is generated (e.g., as described above), and a recommendation may be communicated to the user (step 406).

[0064] Features used by the machine learning algorithm to describe the location of an item and / or anomaly on the item may include, for example, outputs of an intermediate layer of a deep learning network (e.g., outputs of an auxiliary loss in a downscaled version of an image). Thus, if such outputs for an inspection image differ from outputs for one or more previously captured images, a signal may be generated accordingly.

[0065] exist Figure 4BIn one embodiment schematically shown in , the difference between the inspection image and the previously captured image is related to the setting parameters. In this embodiment, the processor 102 finds the difference in one or more setting parameters between the inspection image 401 and the previously captured image 403 (step 414), and generates a signal about the reference group (and possibly communicates a recommendation to the user) (step 416) based on the difference, such as adding the inspection image to the reference group or removing the reference image from the reference group.

[0066] The setting parameters may include, for example, the 3D location of the item in the image or optical parameters of the image of the item (e.g., as described above). Thus, in one embodiment, the difference determined in step 414 is the change between the 3D location of the item in the inspection image and the 3D location of the same type of item in a previously captured image.

[0067] In another embodiment, the difference determined in step 414 is a change between the optical parameters of the inspection image and the optical parameters of a previously captured image.

[0068] Some differences or variations may have a wide range of tolerances (high levels of variability between items of the same type), and therefore, the threshold used to determine whether an image has changed or is different may be high or low, depending on the tolerance of the difference or variation being looked for and based on the type of item.

Claims

1. A method for optimizing a reference set, the reference set comprising reference images of objects of the same type, the reference images being compared with inspection images for visual inspection of the objects in the inspection images, the method comprising: detecting differences between the inspection image and a previously captured image; as well as Based on the detected differences, a signal is generated for adding an image to or removing an image from the reference set.

2. The method according to claim 1, wherein: The previously captured image comprises a reference image.

3. The method according to claim 1, wherein: The previously captured image includes an inspection image.

4. The method according to claim 1, wherein: The reference set includes images of items of the same type that are free of defects.

5. The method according to claim 1, wherein: The reference group includes images confirmed by a user.

6. The method according to claim 1, comprising: Differences above a threshold are detected, where the threshold depends on the level of variability between items of the same type.

7. The method according to claim 1, wherein: The difference comprises a change in a characteristic between the inspection image and the previously captured image.

8. The method according to claim 7, wherein: The change in characteristic comprises a difference between a parameter of the object in the inspection image and a parameter of the same type of object in the previously captured image.

9. The method according to claim 8, comprising: comparing the previously captured images to each other to obtain a relevance score of the previously captured images relative to themselves; and The change in the characteristic includes a difference between a correlation score of the inspection image relative to the previously captured image and a correlation score of the previously captured image relative to itself.

10. The method according to claim 8, comprising: comparing the previously captured images with each other to obtain the number of abnormal areas in the same type of objects; and The change in the characteristic includes a difference between the number of abnormal regions in the inspection image and the number of abnormal regions in the same type of article.

11. The method according to claim 8, comprising: obtaining a prediction of a defect in an abnormal region in the previously captured image; and The change in the characteristic includes a difference between a prediction of a defect in the abnormal region in the inspection image and a prediction of a defect in the abnormal region in the previously captured image.

12. The method according to claim 8, wherein: The change in characteristics includes a difference between features used by the machine learning algorithm to describe the object in the inspection image and features used by the machine learning algorithm to describe the same type of object in the previously captured image.

13. The method according to claim 1, wherein: The differences are related to the setup parameters.

14. The method according to claim 13, wherein: The difference comprises a change between the 3D location of the object in the inspection image and the 3D location of the same type of object in the previously captured image.

15. The method according to claim 13, wherein: The difference comprises a change between optical parameters of the inspection image and optical parameters of the previously captured image.

16. The method according to claim 1, wherein: The signal is used to add the inspection image to the reference set.

17. The method according to claim 1, wherein: The previously captured image comprises a reference image, and the method comprises: detecting a difference between the inspection image and a reference image; and Based on the detected difference, a signal is generated for removing the reference image from the reference set.

18. The method of claim 17, comprising detecting differences between a plurality of inspection images and a reference image.

19. The method of claim 1, comprising communicating a recommendation about the reference group to a user based on the generated signal.

20. A system for visual inspection, the system comprising: a processor in communication with a camera for capturing a reference image and an inspection image of an object of the same type; as well as a user interface in communication with the processor, The processor detects differences between previously captured images of the same type of item and the inspection image, and generates a signal for adding or removing an image to or from a reference set based on the detected differences.

21. The system of claim 20, wherein: The processor causes communication to a user via the user interface of a recommendation to add an image to the reference set or to remove an image from the reference set.

Images