Appearance inspection system, control device, and control method

Abstract

It is possible to provide an appearance inspection system, a control device, and a control method in which the time required for determining the quality of an inspection target is less likely to increase even when the number of inspection items of defects to be detected increases.SOLUTION: An appearance inspection system 1 includes a photographing unit 10 that photographs an inspection target while changing a photographing condition, an illumination unit 20 that irradiates the inspection target with irradiation light, a diagnosis unit 80 that diagnoses an appearance of the inspection target based on an image photographed by the photographing unit 10 using a learning model 70 learned for each photographing condition based on an inspection target that is a non-defective product, and an inspection order determination unit 100 that determines an inspection order in which the photographing condition is changed based on an occurrence frequency of a defect with respect to the photographing condition.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an appearance inspection system, a control device, and a control method, and more particularly to an appearance inspection system, a control device, and a control method that diagnose the appearance of an inspection object using a learning model that is trained based on inspection objects that are non-defective. [Background technology]

[0002] In recent years, the development of visual inspection equipment that applies AI (Artificial Intelligence) has progressed, and it is expected that labor-saving measures will be implemented. One method for this visual inspection equipment is to create a learning model by learning using training data, including defective products, and then use the created learning model to inspect the inspection object.

[0003] Patent Document 1 describes an appearance inspection apparatus that includes an imaging device, a feature extraction unit, and a judgment unit. The imaging device captures appearance images of an object using inspection light irradiated onto the object at multiple wavelengths and irradiation angles. The feature extraction unit extracts appearance features of the object using a bidirectional reflectance distribution function based on the appearance images captured by the imaging device at multiple wavelengths and irradiation angles. The judgment unit judges whether the appearance of the object being inspected is good or bad based on the features extracted by the feature extraction unit and a learning model generated by machine learning using teacher images generated by capturing images of a reference object, which is a learning object, at multiple wavelengths and irradiation angles. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-51076 Summary of the Invention [Problem to be solved by the invention]

[0005] On the other hand, there is a method of creating a learning model using learning without teacher data (good product learning AI) based only on the inspection target of good products, and then performing the inspection. However, in conventional visual inspections using good-quality learning AI, it is necessary to make inferences using multiple learning models trained on images taken under optimized conditions for each defect to be detected. As a result, as the number of defect inspection items increases, it takes a long time to determine whether the inspection items are good or bad. The present invention aims to provide an appearance inspection system, a control device, and a control method that do not increase the time required to determine whether an inspection object is good or bad, even if the number of inspection items for defects to be detected increases. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides an appearance inspection system that includes an imaging unit that images an inspection object under different imaging conditions, an illumination unit that irradiates the inspection object with irradiation light, a diagnosis unit that diagnoses the appearance of the inspection object based on the image captured by the imaging unit using a learning model that has been trained for each imaging condition based on inspection objects that are good, and an inspection order determination unit that determines the inspection order under different imaging conditions based on the frequency of defects occurring for the imaging conditions. In this case, it is possible to provide an appearance inspection system in which the time required to determine the pass / fail of an inspection object does not increase even if the number of inspection items for defects to be detected increases.

[0007] Here, for example, the photographing condition is the color of the irradiated light, and the inspection order determination unit determines the inspection order so that the irradiated light is irradiated in order from the color with the highest frequency of occurrence of defects to the color with the lowest frequency of occurrence of defects. In this case, the color of the irradiated light that makes it easier to diagnose defects can be selected, and defects can be found more quickly. Furthermore, for example, the diagnosing unit diagnoses the color of the irradiated light in association with the type of defect detected, which makes it easier to grasp the relationship between the imaging conditions and the type of defect. Furthermore, for example, a probability calculation unit is further provided for calculating the frequency of occurrence of a defect for a given imaging condition. In this case, even if the frequency of occurrence of a defect is unknown, the frequency of occurrence of a defect can be calculated. Furthermore, the apparatus may further include a display unit that displays, for example, the conventional examination order and the examination order determined by the examination order determination unit based on the occurrence frequencies calculated by the probability calculation unit, together with the imaging conditions and the occurrence frequencies. In this case, it is possible to suggest to the user a change in the examination order to shorten the examination time. The system further includes a pre-processing unit that pre-processes the captured images before the diagnosis unit makes a diagnosis. The pre-processing unit creates a composite image by combining images captured under multiple imaging conditions, and the diagnosis unit diagnoses the test object using the composite image using a learning model trained based on the composite image. In this case, the diagnosis unit can use fewer processing devices such as AI accelerators. The diagnosis unit also visualizes under which imaging conditions the defect occurred in the composite image, allowing the user to understand under which imaging conditions the defect occurred. Furthermore, the diagnostic unit visualizes the composite image as a heat map showing the frequency of defects, allowing the user to understand under which imaging conditions the defects occurred. Furthermore, the diagnostic unit performs diagnosis using a composite image, thereby simultaneously diagnosing multiple defect types, which reduces inspection time and costs. The preprocessing unit also creates a composite image that is maximized relative to the angle of view at which the inspection object is photographed, thereby improving the accuracy of defect detection. Furthermore, there are multiple imaging units and lighting units, and imaging conditions are changed by selecting one of the imaging units and lighting units, respectively. In this case, the imaging conditions of the imaging units and lighting units are not switched, or only switched a little.

[0008] The present invention also provides a control device for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions, the control device including: a diagnosis unit that diagnoses the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good, and an inspection order determination unit that determines the inspection order for which the shooting conditions have been changed based on the frequency of occurrence of defects for the shooting conditions. In this case, it is possible to provide a control device that does not increase the time required to determine the pass / fail of the inspection object even if the number of inspection items for defects to be detected increases.

[0009] Furthermore, the present invention provides a control method for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions by having a processor execute a program stored in a memory, the control method diagnosing the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good, and determining the inspection order for the changed shooting conditions based on the frequency of occurrence of defects for the shooting conditions. In this case, it is possible to provide a control method that does not increase the time required to determine the pass / fail of the inspection object even if the number of inspection items for defects to be detected increases. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an appearance inspection system, a control device, and a control method in which the time required to determine whether an inspection object is good or bad does not increase even if the number of inspection items for defects to be detected increases. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing the overall configuration of a visual inspection system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a workflow of the visual inspection system according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing the overall configuration of a visual inspection system according to a second embodiment. [Figure 4]FIG. 10 is a diagram illustrating a workflow of the visual inspection system according to the second embodiment. [Figure 5] (a) is a conceptual diagram of the imaging conditions when the lighting unit is changed and each image is captured as a separate image. (b) is a conceptual diagram of a composite image obtained by combining the images in (a) into a single image by the pre-processing unit. (c) is a diagram showing an example of the composite image visualized as a heat map. [Figure 6] FIG. 10 is a diagram showing the overall configuration of a visual inspection system according to a fourth embodiment. [Figure 7] FIG. 10 is a diagram showing a case where information about an examination order is displayed on a display unit. [Figure 8] 10(a) to 10(c) are diagrams illustrating the effect of changing the inspection order depending on the frequency of defects. DETAILED DESCRIPTION OF THE INVENTION

[0012] The embodiments will be described in detail with reference to the drawings. However, the present invention should not be interpreted as being limited to the description of the embodiments shown below. Those skilled in the art will easily understand that the specific configuration can be changed without departing from the concept or spirit of the present invention. In the configuration of the invention described below, the same parts or parts having similar functions are denoted by the same reference numerals in different drawings, and redundant explanations may be omitted. When there are multiple elements having the same or similar functions, they may be described using the same reference numeral with different subscripts. However, when there is no need to distinguish between multiple elements, the subscripts may be omitted.

[0013] The designations "first," "second," "third," etc. in this specification are used to identify components and do not necessarily limit the number, order, or content thereof. Furthermore, numbers used to identify components are used in different contexts, and numbers used in one context do not necessarily indicate the same configuration in another context. Furthermore, this does not prevent a component identified by a certain number from also serving the function of a component identified by another number.

[0014] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings etc. may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings etc. All publications, patents, and patent applications cited herein are incorporated by reference in their entirety. As used herein, elements referred to in the singular are intended to include the plural unless the context clearly indicates otherwise.

[0015] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)), and performs processing defined by the program using storage resources (e.g., a memory), interface devices (e.g., a communication port), and the like. Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0016] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0017] The present invention will be described below with reference to first to fourth embodiments. Here, a first embodiment will be described first. In the first embodiment, a case will be described in which, when inspecting a workpiece W as an inspection target, the probability of occurrence of defects for each inspection item for inspecting defects is known.

[0018] [First embodiment] <Overall explanation of visual inspection system 1> FIG. 1 is a diagram showing the overall configuration of a visual inspection system 1 according to the first embodiment. The visual inspection system 1 shown in the figure diagnoses the workpiece W to be inspected for each predetermined inspection item and determines whether it is a good product or a defective product. The workpiece W is not particularly limited, and may be, for example, a part, a container such as a bottle, can, or bag, or food such as bread. The inspection items for the workpiece W relate to defects in the appearance of the workpiece W to be inspected, and may be, for example, dents, scratches, chips, cracks, dirt, uneven paint, scorch marks, etc. The visual inspection system 1 includes a photographing unit 10, an illumination unit 20 (20a, 20b), a delivery unit 30, a control unit 40, a storage 50, a pre-processing unit 60, a learning model 70, a diagnosis unit 80, a display unit 90, and an inspection order determination unit 100.

[0019] The photographing unit 10 photographs the workpiece W. The photographing unit 10 can change photographing conditions such as the photographing angle, filter, color setting, exposure, and aperture. That is, the photographing unit 10 photographs the workpiece W, which is the inspection target, by changing the photographing conditions. The photographing unit 10 is, for example, a camera that photographs the workpiece W. In this case, the photographing unit 10 includes, for example, an optical system that focuses an image of the workpiece W and an image sensor that detects the image focused by the optical system. The image sensor is configured by arranging imaging elements such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera can be selected depending on the purpose of the inspection, and an RGB camera, an infrared camera, a hyperspectral camera, etc. can be used. In addition, the camera may not only photograph images but also acquire frequency information.

[0020] The illumination unit 20 irradiates the workpiece W with irradiation light. In this embodiment, two illumination units 20 are provided, illumination unit 20a and illumination unit 20b. However, the number of illumination units 20 may be any number. In this embodiment, the illumination units 20 are capable of changing the imaging conditions, such as the color, intensity, and illumination angle of the irradiated light, and the background color. To change the imaging conditions, the same number of illumination units 20 as the imaging conditions may be prepared, or the imaging conditions may be changed by switching one or more illumination units 20. The illumination unit 20 is, for example, an LED (Light Emitting Diode) light, a fluorescent lamp, an incandescent lamp, or the like, and is selected appropriately depending on the purpose of the inspection.

[0021] The delivery unit 30 transports the workpiece W while it is placed thereon. The delivery unit 30 has at least the functions of sending, stopping, and rejecting the workpiece W. The delivery unit 30 is, for example, a belt conveyor, and transports the workpiece W to a position where the photographing unit 10 and the lighting unit 20 are installed. Then, when the workpiece W is transported to this position, the photographing unit 10 photographs the workpiece W illuminated with light by the lighting unit 20.

[0022] The control unit 40 controls the entire visual inspection system 1. The control unit 40 is, for example, a computer device such as a control unit for control. The control unit 40 includes a processor such as a CPU, which is a calculation means, a main memory, which is a storage means, and a storage. The processor executes various software such as an OS (operating system) and an application program (application software). The main memory is a storage area that stores various software and data used for executing the software. The storage is a storage area that stores input data for the various software and output data from the various software.

[0023] The storage 50 stores image data captured by the imaging unit 10. At this time, the data is provided with imaging conditions transmitted from the control unit 40. The storage 50 is, for example, a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a memory card.

[0024] The pre-processing unit 60 pre-processes the captured image before diagnosis is performed by the diagnosing unit 80. The pre-processing is, for example, a process of trimming the image data input to the storage 50 as needed. The pre-processing is also a process of blacking out the expiration date if it is written on a food package, or deleting unnecessary background information. The pre-processing unit 60 is, for example, a CPU or a GPU.

[0025] The learning model 70 is an unsupervised learning model that learns only from non-defective products and is created by learning based on a large amount of input image data. The learning model 70 is trained for each imaging condition based on the workpiece W, which is a non-defective product.

[0026] The diagnosis unit 80 diagnoses the appearance of the workpiece W using the learning model 70 based on the image captured by the imaging unit 10. That is, the diagnosis unit 80 uses the learning model 70 to determine whether the workpiece W is a good or defective product based on the input image data. If the imaging condition is the color of the irradiated light, the diagnosis unit 80 associates the color of the irradiated light with the type of defect detected and diagnoses it. This makes it easier to understand the relationship between the imaging conditions and the type of defect. The diagnosis unit 80 sends the diagnosis result to the display unit 90 along with the sample number and imaging conditions as the diagnosis result, and if a defect is determined, sends a rejection instruction to the control unit 40. The diagnosis unit 80 is, for example, an AI accelerator.

[0027] The display unit 90 displays the diagnosis result output by the diagnosis unit 80. The display unit 90 is, for example, a display device such as a liquid crystal display.

[0028] The inspection order determination unit 100 determines the inspection order with modified imaging conditions based on the frequency of defect occurrence for the imaging conditions. In practice, the inspection order determination unit 100 determines the inspection order based on the probability of defect occurrence for each inspection item and sends it to the control unit 40. In this embodiment, the probability of defect occurrence for each inspection is known in advance. The control unit 40 controls the imaging unit 10 and the illumination unit 20 according to this inspection order and the imaging conditions determined for each inspection. If the imaging condition is the color of the irradiation light, the inspection order determination unit 100 determines the inspection order so that the irradiation light is irradiated in order from the color with the highest frequency of defect occurrence to the color with the lowest frequency of defect occurrence. This allows the selection of an irradiation light color that makes it easier to diagnose defects, enabling defects to be found more quickly.

[0029] In the first embodiment, the control unit 40, storage 50, pre-processing unit 60, learning model 70, diagnosis unit 80 and inspection order determination unit 100 can be considered to constitute a control device 2 that controls the visual inspection system 1 when inspecting the work W.

[0030] <Explanation of the operation of the visual inspection system 1> FIG. 2 is a diagram illustrating the workflow of the visual inspection system 1 in the first embodiment. The illustrated workflow consists of an imaging condition determination phase, an inspection order determination phase, a trial run phase, and an actual operation phase. In the photographing condition determination phase (S201), the control unit 40 determines the photographing conditions for the photographing unit 10 and the lighting unit 20. For example, the lighting unit 20 searches for lighting conditions that make it easier to detect defective products while reducing the variation in non-defective products (for example, maximizing the contrast with non-defective parts). The photographing unit 10 optimizes, for example, the angle of view, focus, white balance, color, and filter. The control unit 40 may also select the type of camera and sensor. The photographing conditions may be determined manually by an examiner, or photographs may be taken automatically from multiple candidates, and optimal settings may be determined based on the examiner's criteria. For example, a list of previously inspected objects, their characteristics, and photographing conditions may be stored in the storage 50. Then, when the workpiece W to be inspected is photographed, the control unit 40 lists the optimal photographing condition candidates that are closest to the characteristics of the photographed workpiece and automatically evaluates the workpiece under these multiple conditions. For example, the control unit 40 selects the conditions that maximize the contrast of defective parts.

[0031] In the inspection order determination phase (S202), the probability of defect occurrence for each inspection item is input to the inspection order determination unit 100, and the imaging conditions are determined in descending order of defect occurrence probability, and the inspection process is determined.

[0032] In the trial run phase (S203), a test run is performed to check whether any problems will occur with the imaging conditions and inspection sequence determined in the imaging condition determination phase. The test run may be performed, for example, using an actual line in conjunction with an inspector to prevent defective products from being shipped, or even if an actual line is not used, the test run may be performed with the number of products that would flow on the actual line for a day or half a day.

[0033] The purpose of this phase is to test inspect good products and known defective products that will be inspected during actual operation, and to determine whether or not the system can be moved to actual operation. If a problem occurs (Yes in S204), the system returns to the imaging condition determination phase (S201) and reassess the imaging conditions. If there are no problems (No in S204), the system moves to the actual operation phase (S205). The judgment threshold may be determined during trial operation.

[0034] In the actual operation phase (S205), the actual inspection target workpiece W is transported by the delivery unit 30 using an actual line, while being irradiated with light by the lighting unit 20 under predetermined imaging conditions, photographed by the imaging unit 10, and diagnosed by the diagnosis unit 80. If the diagnosis result is bad, a rejection instruction is sent to the control unit 40, and the defective product is rejected. At this time, the diagnosis unit 80 displays on the display unit 90 the number of the inspection target workpiece W, the pass / fail judgment result, and, in the case of a bad judgment, which inspection item was found to be defective. The diagnosis unit 80 may also simultaneously display other related information, such as the inspection time, inspection location, and inspector name, on the display unit 90.

[0035] When actual operation begins, defective types that were not initially anticipated and were not included in the inspection items may be found. Also, the surrounding environment may have changed and the judgment threshold may no longer be optimal. If a problem like this occurs (Yes in S206), return to the imaging condition determination phase (S201) and review the imaging conditions. If there are no problems (No in S206), operation continues as is.

[0036] [Second embodiment] Next, a second embodiment will be described. In the second embodiment, a case will be described in which, when inspecting a workpiece W as an inspection target, the probability of defect occurrence for each inspection item for inspecting defects is unknown.

[0037] <Overall explanation of visual inspection system 1> FIG. 3 is a diagram showing the overall configuration of a visual inspection system 1 according to the second embodiment. The appearance inspection system 1 shown in Fig. 3 differs from the appearance inspection system 1 described in Fig. 1 in that it adds a probability calculation unit 110, but is otherwise similar except for the fact that there is only one illumination unit 20. Note that the illumination unit 20 may have the same configuration as in the first embodiment.

[0038] The probability calculation unit 110 calculates the probability of defect occurrence for each inspection item. This can also be said to mean that the probability calculation unit 110 determines the frequency of defect occurrence for the imaging conditions. In practice, the probability calculation unit 110 actually carries out inspection by feeding the workpieces to be inspected down the line during the trial run phase. The probability calculation unit 110 then saves the results output by the diagnosis unit 80 and calculates the probability of defect occurrence for each inspection item. Furthermore, the results are input to the inspection order determination unit 100 to determine the inspection order. In this way, the frequency of defect occurrence can be determined even if the frequency of defect occurrence is unknown.

[0039] In the second embodiment, the control unit 40, storage 50, pre-processing unit 60, learning model 70, diagnosis unit 80, inspection order determination unit 100 and probability calculation unit 110 can be considered to constitute a control device 2 that controls the visual inspection system 1 when inspecting the work W.

[0040] <Explanation of the operation of the visual inspection system 1> FIG. 4 is a diagram illustrating the workflow of the visual inspection system 1 according to the second embodiment. The illustrated workflow consists of an imaging condition determination phase, a trial run phase, an inspection order determination phase, and an actual operation phase. In other words, compared to the workflow in Figure 2, the trial run phase and the inspection order determination phase are in the reverse order. S401, and S404 to S406 in FIG. 4 are the same as S201, and S204 to S206 in FIG. 2, and therefore the description thereof will be omitted.

[0041] In the trial run phase (S402) of Fig. 4, a test run is carried out to check whether any problems will occur with the imaging conditions and inspection order determined in the imaging condition determination phase. Here, a certain number of works W are inspected in full, and the probability of defects occurring is calculated. In the inspection order determination phase (S403), the inspection order is determined using the calculation results from the trial operation phase. As a result, the imaging conditions are determined in descending order of the probability of defect occurrence, and the inspection process is determined.

[0042] [Third embodiment] Next, a third embodiment will be described. In the third embodiment, a preprocessing unit 60 creates a composite image by combining images captured under a plurality of imaging conditions, and a diagnosing unit 80 uses a learning model 70 that has been trained based on the composite image to diagnose a workpiece W to be inspected using the composite image.

[0043] 5(a) is a conceptual diagram showing the results when the lighting unit 20 is changed to lighting 1 and 2 as the shooting conditions, and different images are shot for each lighting. In this case, it is shown that there is one defective part in each lighting.

[0044] FIG. 5(b) is a conceptual diagram of a composite image created by the preprocessing unit 60 by combining the images shown in FIG. 5(a) into a single image. The learning model 70 is then created based on this composite image. Furthermore, the diagnosis unit 80 uses the learning model 70, which has been trained based on the composite image, to diagnose the workpiece W to be inspected using the composite image. By combining images captured under different imaging conditions into a single image for learning and diagnosis, the system requires only one learning model and one AI accelerator, whereas the system requires two learning models and one AI accelerator in FIG. 5(a). In other words, the diagnosis unit 80 requires fewer processing devices, such as AI accelerators. Furthermore, the diagnosis unit 80 uses the composite image to diagnose multiple defect types. This reduces inspection time and costs. However, in non-defective product learning, if a composite image is used for diagnosis and a defect is determined, it is not possible to determine under which imaging conditions the defect occurred. Therefore, the diagnosis unit 80 visualizes under which imaging conditions the defects occurred for the composite image. As a specific example, the diagnosis unit 80 visualizes as a heat map showing the frequency of defects for the composite image. This allows the user to understand under which imaging conditions the defects occurred.

[0045] FIG. 5(c) is a diagram showing an example of visualizing the composite image as a heat map. In the illustrated example, the frequency of defects is expressed by the density of gray. The system is also provided with a function to identify under which imaging conditions the defects occurred. This allows the user to understand under which imaging conditions the defects occurred. It is preferable that the pre-processing unit 60 creates a composite image that is as large as possible relative to the angle of view at which the workpiece W is photographed. This allows a larger workpiece W to be accommodated in a limited space, improving the accuracy of defect detection.

[0046] [Fourth embodiment] Next, a fourth embodiment will be described. In the fourth embodiment, there are a plurality of imaging units 10 and a plurality of lighting units 20, and imaging conditions are changed by selecting one of the plurality of imaging units 10 and one of the plurality of lighting units 20.

[0047] FIG. 6 is a diagram showing the overall configuration of a visual inspection system 1 according to the fourth embodiment. The appearance inspection system 1 illustrated in Fig. 6 is similar to the appearance inspection system 1 described in Fig. 3 except that there are two photographing units 10 and two lighting units 20. In Fig. 6, the two photographing units 10 are illustrated as photographing units 10a and 10b, and the two lighting units 20 are illustrated as lighting units 20a and 20b. In this case, the imaging units 10a and 10b and the lighting units 20a and 20b can be used to capture images under different imaging conditions, which eliminates or reduces the need to switch imaging conditions for a single imaging unit 10 or lighting unit 20.

[0048] FIG. 7 is a diagram showing a case where information about the examination order is displayed on the display unit 90. In FIG. Here, the conventional inspection order and the inspection order determined by the inspection order determination unit 100 based on the occurrence frequency calculated by the probability calculation unit 110 are displayed. These inspection orders are also displayed together with the imaging conditions and occurrence frequencies. Here, the conventional inspection order is illustrated as the current setting, and the inspection order determined by the inspection order determination unit 100 is illustrated as the recommended setting. The respective imaging conditions and occurrence frequencies indicated by the defect occurrence probability are then displayed. More specifically, in the second embodiment, when the defect occurrence probability is unknown, the current setting (imaging conditions 1, 2, 3) and the recommended setting (imaging conditions 2, 3, 1, which inspects in descending order of defect occurrence probability) are displayed, with the order changed based on the calculated defect occurrence probability. This clarifies the difference between the two and allows the user to suggest changes to the inspection order to shorten the inspection time. Note that scratches, dents, and chips are defect details, and users can input them themselves to easily understand the purpose of the test. In addition, in FIG. 7, the occurrence frequencies are also displayed as bar graphs. However, this is not limited to this, and line graphs and tables may also be used.

[0049] In general, labor shortages are becoming more serious worldwide, and the manufacturing industry is expected to face an increasing shortage of skilled workers. For example, mechanization of inspection processes, particularly in the manufacturing industry, has been lagging behind because visual evaluation by skilled personnel is the norm. In recent years, the development of AI-based visual inspection equipment has progressed, with the hope of reducing labor. Among these methods, a method that creates a learning model using unsupervised learning (good product learning AI) based solely on good products to be inspected and then performs the inspection is particularly user-friendly, as it does not require the creation of training data (annotation) or defective product data, compared to supervised learning that includes defective products. However, this method suffers from low recognition accuracy and often requires optimization of imaging conditions and preprocessing, such as lighting, background, and camera settings, depending on the inspection target and the defects to be detected. In this embodiment, as described above in detail, the inspection order is changed from inspection items with a high frequency of occurrence of unwanted products to inspection items with a low frequency of occurrence of unwanted products, depending on the frequency of occurrence of defective products. This produces the following effects.

[0050] 8(a) to 8(c) are diagrams illustrating the effect of changing the inspection order depending on the frequency of occurrence of defects. Of these, Figure 8(a) is a conceptual diagram showing the case where inspection is performed according to the conventional inspection order. In this case, the inspection target is workpiece W with No. (N), and inspections 1 to 4 are performed in this order. GPUs 1 to 4 are used as AI accelerators for the inspection.

[0051] FIG. 8(b) is a conceptual diagram showing a case where the inspection order is changed depending on the frequency of defects as described in the second embodiment, and where inspection is performed using a composite image as described in the third embodiment. In this case, the inspection target is workpiece W No. (N), and inspection 3, inspection 4, and inspection 1 & inspection 2 are performed in this order. Here, inspection 1 & inspection 2 indicates that the inspection is performed using a composite image obtained by combining the images taken in inspection 1 and inspection 2. In this case, because a composite image is used in inspection 1 & inspection 2, GPUs 1 to 3 are used as the AI ​​accelerators used for inspection. In other words, the number of AI accelerators required is small.

[0052] In this case, inspection items with a higher frequency of defects will be inspected first. FIG. 8(c) is a diagram showing the effect in this case. In this case, a defect was found for workpiece W No. (N) in inspection 3, which has the highest defect occurrence frequency, so at this point, workpiece W No. (N) was rejected as a defective product. Also, inspection began for the next workpiece W No. (N+1), and inspection 3, inspection 4, inspection 1, and inspection 2 were carried out in that order. In other words, by inspecting the inspection items with a higher defect occurrence frequency first, defects in workpiece W are often discovered more quickly. This reduces the inspection time. As a result, even if the number of inspection items for defects to be detected increases, the time required to determine whether the inspection items are good or bad is unlikely to increase. Furthermore, the inspection method of this embodiment follows a sequence similar to that of a manual inspection, and as a result, there is no significant change in the inspection process before and after the introduction of the visual inspection system 1, making it possible to partially automate the process. Furthermore, by knowing the frequency of occurrence, it becomes easier to grasp problems in the manufacturing process of the work W, which leads to an improvement in the yield of the work W.

[0053] <Control method explanation> The above-described processes performed by the control device 2 are realized by the cooperation of software and hardware resources. That is, a processor such as a CPU provided in the control device 2 loads into a main memory and executes a program that realizes each function of the control device 2, thereby realizing each of the above-described functions.

[0054] Therefore, the processing performed by the control device 2 described above can be considered as a control method for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions by having a processor execute a program stored in memory, and that diagnoses the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good, and determines the inspection order with changed shooting conditions based on the frequency of defects occurring for the shooting conditions. This makes it possible to provide a control method that does not increase the time required to determine the pass / fail of an inspection object even if the number of inspection items for defects to be detected increases.

[0055] The program for realizing this embodiment can be provided not only by communication means but also by being stored on a recording medium such as a CD-ROM. [Explanation of symbols]

[0056] 1...visual inspection system, 10, 10a, 10b...imaging unit, 20, 20a, 20b...illumination unit, 30...transport unit, 40...control unit, 50...storage, 60...preprocessing unit, 70...learning model, 80...diagnosis unit, 90...display unit, 100...inspection order determination unit, 110...probability calculation unit, W...work

Claims

1. an imaging unit that changes imaging conditions to image the inspection object; an illumination unit that irradiates the inspection object with irradiation light; a diagnosis unit that diagnoses the appearance of the inspection object based on the image captured by the imaging unit using a learning model that has been trained for each imaging condition based on inspection objects that are non-defective; an inspection order determination unit that determines an inspection order in which the imaging conditions are changed based on the frequency of occurrence of defects for the imaging conditions; A visual inspection system comprising:

2. the photographing condition is the color of the irradiated light, 2. The visual inspection system according to claim 1, wherein the inspection order determination unit determines the inspection order so that the irradiation light is applied in order from a color with a high frequency of occurrence of defects to a color with a low frequency of occurrence of defects.

3. 3. The visual inspection system according to claim 2, wherein the diagnosing unit diagnoses the color of the irradiated light in association with the type of defect detected.

4. The visual inspection system according to claim 1 , further comprising a probability calculation unit for calculating the frequency of occurrence of defects for the imaging conditions.

5. 5. The appearance inspection system according to claim 4, further comprising a display unit that displays a conventional inspection order and the inspection order determined by the inspection order determination unit based on the occurrence frequency calculated by the probability calculation unit, together with the shooting conditions and the occurrence frequency.

6. a pre-processing unit that performs pre-processing of the captured image before the diagnosis is made by the diagnosis unit; the preprocessing unit creates a composite image by combining images captured under a plurality of imaging conditions; The visual inspection system according to claim 1 , wherein the diagnosing unit diagnoses the inspection object using the composite image by using a learning model that has been trained based on the composite image.

7. The visual inspection system according to claim 6 , wherein the diagnosis unit visualizes under which of the photographing conditions a defect occurred in the composite image.

8. The visual inspection system according to claim 7 , wherein the diagnosing unit visualizes the composite image as a heat map showing the frequency of defects.

9. The visual inspection system according to claim 6 , wherein the diagnosing unit performs diagnosis using the composite image, thereby simultaneously diagnosing a plurality of defect types.

10. 7. The visual inspection system according to claim 6, wherein the preprocessing unit creates a composite image so as to maximize the image angle for capturing the inspection object.

11. 2. The appearance inspection system according to claim 1, wherein the photographing unit and the lighting unit are each plural, and the photographing conditions are changed by selecting one of the plural photographing units and one of the plural lighting units.

12. A control device for controlling an appearance inspection system that irradiates an inspection object with irradiation light, changes imaging conditions, and diagnoses the appearance of the inspection object based on an image of the inspection object, a diagnosis unit that diagnoses the appearance of the inspection object based on the captured image using a learning model that has been trained for each of the imaging conditions based on inspection objects that are non-defective; an inspection order determination unit that determines an inspection order in which the imaging conditions are changed based on the frequency of occurrence of defects for the imaging conditions; A control device comprising:

13. The processor executes the program stored in the memory. A control method for controlling an appearance inspection system that irradiates an inspection object with irradiation light, changes imaging conditions, and diagnoses the appearance of the inspection object based on images of the inspection object, comprising: Based on the captured image, the appearance of the inspection object is diagnosed using a learning model that has been trained for each of the imaging conditions based on inspection objects that are non-defective; determining an inspection sequence in which the imaging conditions are changed based on the frequency of occurrence of defects for the imaging conditions; Control method.

Images