Appearance inspection system, control device, and control method

Abstract

An appearance inspection system 1 comprises: a photographing part 10 for photographing an inspection object while changing photographing conditions; illumination parts 20 for irradiating the inspection object with irradiation light; a diagnosis part 80 for diagnosing appearance of the inspection object on the basis of the image photographed by the photographing part 10, using a trained model 70 trained for each photographing condition on the basis of a non-defective inspection object; and an inspection order determination part 100 for determining an inspection order in which the photographing conditions are changed, on the basis of occurrence frequency of defects for each photographing condition. As a result, it is possible to provide an appearance inspection system, a control device, and a control method that suppress an increase in time required for a pass / fail determination of an inspection object, even if the number of inspection items for defects to be detected increases.

Description

Visual inspection system, control device and control method

[0001] The present invention relates to an appearance inspection system, a control device, and a control method, and more particularly to an appearance inspection system, a control device, and a control method that diagnose the appearance of an inspection object using a learning model that is trained based on inspection objects that are non-defective.

[0002] In recent years, the development of visual inspection equipment that uses AI (Artificial Intelligence) has progressed, and it is expected that it will reduce the number of people required. One method for this visual inspection equipment is to create a learning model by learning using training data, including defective products, and then inspect the inspection object using the created learning model.

[0003] Patent Document 1 describes an appearance inspection apparatus that includes an imaging device, a feature extraction unit, and a determination unit. The imaging device captures appearance images of an object using inspection light irradiated onto the object at multiple wavelengths and irradiation angles. The feature extraction unit extracts appearance features of the object using a bidirectional reflectance distribution function based on the appearance images captured by the imaging device at multiple wavelengths and irradiation angles. The determination unit determines whether the appearance of the object being inspected is good or bad based on the features extracted by the feature extraction unit and a learning model generated by machine learning using teacher images generated by capturing images of a reference object, which is a learning object, at multiple wavelengths and irradiation angles.

[0004] Japanese Patent Application Laid-Open No. 2022-51076

[0005] On the other hand, there is a method of creating a learning model using learning without teacher data (good product learning AI) based only on good product inspection objects, and then performing inspection. However, in conventional visual inspection using good product learning AI, it is necessary to make inferences using multiple learning models trained on images captured under optimized conditions for each defect item to be detected. Therefore, as the number of defect inspection items increases, there is a problem in that it takes a lot of time to determine the pass / fail of the inspection objects. The present invention aims to provide a visual inspection system, control device, and control method that do not increase the time required to determine the pass / fail of inspection objects even when the number of defect inspection items to be detected increases.

[0006] In order to solve the above problems, the present invention provides an appearance inspection system that includes an imaging unit that images an inspection object under different imaging conditions, an illumination unit that irradiates the inspection object with irradiation light, a diagnosis unit that diagnoses the appearance of the inspection object based on the image captured by the imaging unit using a learning model that has been trained for each imaging condition based on inspection objects that are good, and an inspection order determination unit that determines the inspection order under different imaging conditions based on the frequency of defects occurring for the imaging conditions. In this case, it is possible to provide an appearance inspection system in which the time required to determine the pass / fail of an inspection object does not increase even if the number of inspection items for defects to be detected increases.

[0007] Here, for example, the imaging condition is the color of the irradiation light, and the inspection order determination unit determines the inspection order so that the irradiation light is applied in order from the color with the highest defect occurrence frequency to the color with the lowest defect occurrence frequency. In this case, the color of irradiation light that is easier to diagnose defects is selected, allowing defects to be found more quickly. Furthermore, for example, the diagnosis unit performs diagnosis by associating the color of the irradiation light with the type of defect detected. In this case, it is easier to understand the relationship between the imaging condition and the defect type. Furthermore, for example, the device may further include a probability calculation unit that calculates the defect occurrence frequency for the imaging condition. In this case, the defect occurrence frequency can be calculated even if the defect occurrence frequency is unknown. Furthermore, for example, the device may further include a display unit that displays the conventional inspection order and the inspection order determined by the inspection order determination unit based on the occurrence frequency calculated by the probability calculation unit, along with the imaging condition and the occurrence frequency. In this case, it is possible to suggest to the user a change in the inspection order to shorten the inspection time. Furthermore, the device may further include a preprocessing unit that preprocesses the captured images before the diagnosis unit performs diagnosis. The preprocessing unit creates a composite image by combining images captured under multiple imaging conditions, and the diagnosis unit diagnoses the inspection object using the composite image using a learning model trained based on the composite image. In this case, fewer processing devices such as AI accelerators are required by the diagnosis unit. The diagnosis unit visualizes under which imaging conditions defects occurred for the composite image. In this case, the user can understand under which imaging conditions defects occurred. The diagnosis unit visualizes the composite image as a heat map showing the frequency of defects. In this case, the user can understand under which imaging conditions defects occurred from the heat map. Furthermore, the diagnosis unit diagnoses multiple defect types by using the composite image for diagnosis. In this case, inspection time is shortened and costs are reduced. The preprocessing unit creates a composite image that is maximized for the angle of view used to capture the inspection target. In this case, the accuracy of defect detection is improved. Furthermore, there are multiple imaging units and lighting units, and imaging conditions are changed by selecting one of the multiple imaging units and lighting units. In this case, switching of imaging conditions for the imaging units and lighting units is unnecessary or required only occasionally.

[0008] The present invention also provides a control device for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions, the control device including: a diagnosis unit that diagnoses the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good, and an inspection order determination unit that determines the inspection order for which the shooting conditions have been changed based on the frequency of occurrence of defects for the shooting conditions. In this case, it is possible to provide a control device that does not increase the time required to determine the pass / fail of the inspection object even if the number of inspection items for defects to be detected increases.

[0009] Furthermore, the present invention provides a control method for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions by having a processor execute a program stored in a memory, the control method diagnosing the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good, and determining the inspection order for the changed shooting conditions based on the frequency of occurrence of defects for the shooting conditions. In this case, it is possible to provide a control method that does not increase the time required to determine the pass / fail of the inspection object even if the number of inspection items for defects to be detected increases.

[0010] According to the present invention, it is possible to provide an appearance inspection system, a control device, and a control method in which the time required to determine whether an inspection object is good or bad does not increase even if the number of inspection items for defects to be detected increases.

[0011] 1 is a diagram showing the overall configuration of a visual inspection system according to a first embodiment; FIG. 2 is a diagram explaining the workflow of the visual inspection system according to the first embodiment; FIG. 3 is a diagram explaining the overall configuration of a visual inspection system according to a second embodiment; FIG. 4 is a diagram explaining the workflow of the visual inspection system according to the second embodiment; (a) is a conceptual diagram when the lighting unit is changed as a shooting condition, and each image is shot as a separate image; (b) is a conceptual diagram of a composite image obtained by combining the images of (a) into a single image by a pre-processing unit; (c) is a diagram showing an example of visualizing the composite image as a heat map; FIG. 4 is a diagram showing the overall configuration of a visual inspection system according to a fourth embodiment; (a) is a diagram showing a case where information about the inspection order is displayed on the display unit; (a) to (c) are diagrams explaining the effect of changing the inspection order according to the frequency of defect occurrence.

[0012] The embodiments will be described in detail with reference to the drawings. However, the present invention should not be construed as being limited to the description of the embodiments shown below. Those skilled in the art will readily understand that the specific configuration can be modified without departing from the concept or spirit of the present invention. In the configuration of the invention described below, the same reference numerals are used in common between different drawings for the same parts or parts having similar functions, and duplicated explanations may be omitted. When there are multiple elements having the same or similar functions, they may be described using the same reference numerals with different subscripts. However, when there is no need to distinguish between multiple elements, the subscripts may be omitted in the description.

[0013] The terms "first," "second," "third," and the like used in this specification are used to identify components and do not necessarily limit the number, order, or content of the components. Furthermore, numbers used to identify components are used in different contexts, and a number used in one context does not necessarily indicate the same configuration in another context. Furthermore, this does not prevent a component identified by a certain number from also serving the function of a component identified by another number.

[0014] The position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings, etc. Publications, patents, and patent applications cited in this specification constitute part of the description of this specification in their entirety. In this specification, components expressed in the singular form include the plural form unless otherwise clearly indicated in the context.

[0015] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)), and performs processing defined by the program using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0016] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in the embodiments, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0017] The present invention will be described below with reference to first to fourth embodiments. First, the first embodiment will be described. In the first embodiment, a case will be described in which, when inspecting a workpiece W to be inspected, the probability of defect occurrence for each inspection item for inspecting defects is known.

[0018] First Embodiment <Explanation of the Overall Visual Inspection System 1> FIG. 1 is a diagram illustrating the overall configuration of a visual inspection system 1 according to a first embodiment. The visual inspection system 1 illustrated in the figure diagnoses a workpiece W to be inspected for each predetermined inspection item and determines whether the workpiece W is a pass or fail product. The workpiece W is not particularly limited and may be, for example, a part, a container such as a bottle, can, or bag, or food such as bread. The inspection items for the workpiece W relate to defects in the appearance of the workpiece W to be inspected, such as dents, scratches, chips, cracks, stains, uneven paint, and burns. The visual inspection system 1 includes a photographing unit 10, an illumination unit 20 (20a, 20b), a delivery unit 30, a control unit 40, a storage unit 50, a preprocessing unit 60, a learning model 70, a diagnosis unit 80, a display unit 90, and an inspection order determination unit 100.

[0019] The photographing unit 10 photographs the workpiece W. The photographing unit 10 can change photographing conditions such as the photographing angle, filter, color setting, exposure, and aperture. That is, the photographing unit 10 photographs the workpiece W, which is the inspection target, by changing the photographing conditions. The photographing unit 10 is, for example, a camera that photographs the workpiece W. In this case, the photographing unit 10 includes, for example, an optical system that focuses an image of the workpiece W and an image sensor that detects the image focused by the optical system. The image sensor is configured by an array of imaging elements such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera can be selected depending on the purpose of the inspection, and an RGB camera, an infrared camera, a hyperspectral camera, etc. can be used. In addition to cameras that photograph images, cameras that acquire frequency information may also be used.

[0020] The illumination unit 20 irradiates the workpiece W with irradiation light. In this embodiment, two illumination units 20 are provided, illumination unit 20a and illumination unit 20b. However, the number of illumination units 20 may be any number. In this embodiment, the illumination units 20 are capable of changing the imaging conditions, such as the color, intensity, and illumination angle of the irradiation light, and the background color. To change the imaging conditions, the same number of illumination units 20 as the imaging conditions may be prepared, or the imaging conditions may be changed by switching between one or more illumination units 20. The illumination unit 20 is, for example, an LED (Light Emitting Diode) light, a fluorescent lamp, an incandescent lamp, or the like, and is selected appropriately depending on the purpose of the inspection.

[0021] The delivery unit 30 transports the workpiece W while it is placed thereon. The delivery unit 30 has at least the functions of sending, stopping, and rejecting the workpiece W. The delivery unit 30 is, for example, a belt conveyor, and transports the workpiece W to a position where the photography unit 10 and the lighting unit 20 are installed. Then, when the workpiece W is transported to this position, the photography unit 10 photographs the workpiece W illuminated with light by the lighting unit 20.

[0022] The control unit 40 controls the entire visual inspection system 1. The control unit 40 is, for example, a computer device such as a control unit for control. The control unit 40 includes a processor such as a CPU, which is a calculation means, a main memory, which is a storage means, and a storage. The processor executes various software such as an OS (operating system) and application programs (application software). The main memory is a storage area that stores various software and data used to execute the software. The storage is a storage area that stores input data for the various software and output data from the various software.

[0023] The storage 50 stores image data captured by the imaging unit 10. At this time, the data is provided with imaging conditions transmitted from the control unit 40. The storage 50 is, for example, a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a memory card.

[0024] The pre-processing unit 60 pre-processes the captured image before diagnosis by the diagnosing unit 80. The pre-processing is, for example, a process of trimming the image data input to the storage 50 as needed. The pre-processing is also a process of blacking out the expiration date if it is written on a food package, or deleting unnecessary background information. The pre-processing unit 60 is, for example, a CPU or a GPU.

[0025] The learning model 70 is an unsupervised learning model that learns only from non-defective products and is created by learning based on a large amount of input image data. The learning model 70 is trained for each imaging condition based on non-defective workpieces W.

[0026] The diagnosis unit 80 diagnoses the appearance of the workpiece W using the learning model 70 based on the image captured by the imaging unit 10. That is, the diagnosis unit 80 uses the learning model 70 to determine whether the workpiece W is a good or bad product based on the input image data. If the imaging condition is the color of the irradiated light, the diagnosis unit 80 associates the color of the irradiated light with the detected defect type and makes a diagnosis. This makes it easier to understand the relationship between the imaging condition and the defect type. The diagnosis unit 80 sends the diagnosis result to the display unit 90 along with the sample number and imaging condition as the diagnosis result, or sends a rejection instruction to the control unit 40 if a defect is determined. The diagnosis unit 80 is, for example, an AI accelerator.

[0027] The display unit 90 displays the diagnosis results output by the diagnosis unit 80. The display unit 90 is, for example, a display device such as a liquid crystal display.

[0028] The inspection order determination unit 100 determines an inspection order with modified imaging conditions based on the frequency of defect occurrence for the imaging conditions. In practice, the inspection order determination unit 100 determines the inspection order based on the probability of defect occurrence for each inspection item and sends the determined inspection order to the control unit 40. In this embodiment, the probability of defect occurrence for each inspection is determined in advance and is known. The control unit 40 controls the imaging unit 10 and the illumination unit 20 according to this inspection order and the imaging conditions determined for each inspection. If the imaging condition is the color of the irradiation light, the inspection order determination unit 100 determines the inspection order so that the irradiation light is applied in order from the color with the highest frequency of defect occurrence to the color with the lowest frequency of defect occurrence. This allows the selection of an irradiation light color that makes it easier to diagnose defects, enabling defects to be found more quickly.

[0029] In the first embodiment, the control unit 40, storage 50, pre-processing unit 60, learning model 70, diagnosis unit 80 and inspection order determination unit 100 can be considered to constitute a control device 2 that controls the visual inspection system 1 when inspecting the workpiece W.

[0030] <Explanation of Operation of Visual Inspection System 1> FIG. 2 is a diagram illustrating the workflow of the visual inspection system 1 according to the first embodiment. The illustrated workflow comprises a photographing condition determination phase, an inspection order determination phase, a trial run phase, and an actual operation phase. In the photographing condition determination phase (S201), the control unit 40 determines the photographing conditions for the photographing unit 10 and the lighting unit 20. For example, the lighting unit 20 searches for lighting conditions that facilitate the detection of defective products while reducing the variability of non-defective products (e.g., maximizing the contrast with non-defective parts). The photographing unit 10 optimizes, for example, the angle of view, focus, white balance, color, and filter. The control unit 40 may also select the type of camera or sensor. The photographing conditions may be determined manually by an examiner, or photographs may be automatically taken from multiple candidates, and optimal settings may be determined based on the examiner's criteria. For example, a list of previously inspected objects, their characteristics, and photographing conditions may be stored in the storage 50. When the workpiece W to be inspected is photographed, the control unit 40 lists candidates for the optimum photographing conditions that are closest to the characteristics of the workpiece W, and automatically performs evaluation under these multiple conditions. For example, the control unit 40 selects the conditions that maximize the contrast of defects.

[0031] In the inspection order determination phase (S202), the probability of defect occurrence for each inspection item is input to the inspection order determination unit 100, and the imaging conditions are determined in descending order of defect occurrence probability, and the inspection process is determined.

[0032] In the trial run phase (S203), a test run is performed to check whether any problems will occur with the imaging conditions and inspection sequence determined in the imaging condition determination phase. The test run may be performed, for example, using an actual line in conjunction with an inspector to prevent defective products from being shipped, or even if an actual line is not used, the test run may be performed with the number of products that would flow on the actual line for about a day or half a day.

[0033] The purpose of this phase is to test inspect good products and known defective products to be inspected during actual operation and determine whether or not the system can be moved to actual operation. If a problem occurs (Yes in S204), the system returns to the imaging condition determination phase (S201) and reconsiders the imaging conditions. If there is no problem (No in S204), the system moves to the actual operation phase (S205). The judgment threshold may be determined during the trial run.

[0034] In the actual operation phase (S205), the workpiece W to be inspected is transported by the delivery unit 30 using an actual line, while being irradiated with light by the illumination unit 20 under predetermined photographing conditions, photographed by the photographing unit 10, and diagnosed by the diagnosis unit 80. If the diagnosis result is bad, a rejection instruction is sent to the control unit 40, and the defective product is rejected. At this time, the diagnosis unit 80 displays on the display unit 90 the number of the workpiece W to be inspected, the pass / fail judgment result, and, in the case of a bad judgment, which inspection item was found to be defective. The diagnosis unit 80 may also simultaneously display other related information, such as the inspection time, inspection location, and inspector name, on the display unit 90.

[0035] When actual operation begins, defective types that were not initially anticipated and were not included in the inspection items may be found. Also, the surrounding environment may have changed and the judgment threshold may no longer be optimal. If a problem like this occurs (Yes in S206), return to the imaging condition determination phase (S201) and review the imaging conditions. If there is no problem (No in S206), operation continues as is.

[0036] Second Embodiment Next, a second embodiment will be described. In the second embodiment, a case will be described in which, when inspecting a workpiece W as an inspection target, the probability of defect occurrence for each inspection item for inspecting defects is unknown.

[0037] <Explanation of the Overall Visual Inspection System 1> Figure 3 is a diagram showing the overall configuration of the visual inspection system 1 in the second embodiment. The visual inspection system 1 shown in Figure 3 differs from the visual inspection system 1 described in Figure 1 in that it adds a probability calculation unit 110, and is otherwise similar except that it has a single illumination unit 20. Note that the illumination unit 20 may have the same configuration as in the first embodiment.

[0038] The probability calculation unit 110 calculates the probability of defect occurrence for each inspection item. This can also be said to mean that the probability calculation unit 110 determines the frequency of defect occurrence for the imaging conditions. In practice, the probability calculation unit 110 actually carries out inspection by feeding the workpieces to be inspected down the line during the trial run phase. The probability calculation unit 110 then saves the results output by the diagnosis unit 80 and calculates the probability of defect occurrence for each inspection item. Furthermore, the results are input to the inspection order determination unit 100 to determine the inspection order. In this way, the frequency of defect occurrence can be determined even if the frequency of defect occurrence is unknown.

[0039] In the second embodiment, the control unit 40, storage 50, pre-processing unit 60, learning model 70, diagnosis unit 80, inspection order determination unit 100 and probability calculation unit 110 can be considered to constitute a control device 2 that controls the visual inspection system 1 when inspecting the work W.

[0040] <Explanation of Operation of Visual Inspection System 1> Figure 4 is a diagram illustrating the workflow of the visual inspection system 1 in the second embodiment. The illustrated workflow is composed of an imaging condition determination phase, a trial run phase, an inspection order determination phase, and an actual operation phase. In other words, compared to the workflow in Figure 2, the trial run phase and the inspection order determination phase are in the reverse order. S401 and S404 to S406 in Figure 4 are the same as S201 and S204 to S206 in Figure 2, and therefore their explanation will be omitted.

[0041] In the trial run phase (S402) of Fig. 4, a trial run is performed to confirm that no problems will occur with the imaging conditions and inspection order determined in the imaging condition determination phase. Here, a certain number of works W are inspected in full, and the probability of defects occurring is calculated. In the inspection order determination phase (S403), the inspection order is determined using the calculation results from the trial run phase. As a result, the imaging conditions are determined in descending order of the probability of defects occurring, and the inspection process is determined.

[0042] [Third Embodiment] Next, a third embodiment will be described. In the third embodiment, a preprocessing unit 60 creates a composite image by combining images captured under a plurality of imaging conditions, and a diagnosing unit 80 uses a learning model 70 that has been trained based on the composite image to diagnose a workpiece W to be inspected using the composite image.

[0043] 5A is a conceptual diagram showing different images taken under different imaging conditions, with the illumination unit 20 being set to illuminations 1 and 2. In this case, one defective part is shown in each image.

[0044] FIG. 5(b) is a conceptual diagram of a composite image created by the preprocessing unit 60 by combining the images in FIG. 5(a) into a single image. The learning model 70 is then created based on this composite image. Furthermore, the diagnosis unit 80 uses the learning model 70, which has been trained based on the composite image, to diagnose the workpiece W to be inspected using the composite image. By combining images captured under different shooting conditions into a single image for learning and diagnosis, while two learning models and one AI accelerator were required in FIG. 5(a), only one learning model and one AI accelerator are required in FIG. 5(b). In other words, fewer processing devices, such as AI accelerators, are required by the diagnosis unit 80. Furthermore, by using the composite image for diagnosis, the diagnosis unit 80 can simultaneously diagnose multiple defect types. This reduces inspection time and costs. However, in non-defective product learning, if a composite photograph is used for diagnosis and a defect is determined, it is not possible to determine under which shooting conditions the defect occurred. Therefore, the diagnosis unit 80 visualizes under which imaging conditions the defects occurred for the composite image. As a specific example, the diagnosis unit 80 visualizes as a heat map the occurrence frequency of defects for the composite image. This allows the user to understand under which imaging conditions the defects occurred.

[0045] FIG. 5(c) is a diagram showing an example of a composite image visualized as a heat map. In the illustrated example, the frequency of defects is represented by gray levels. The system is also provided with a function to identify under which imaging conditions the defects occurred. This allows the user to understand under which imaging conditions the defects occurred. It is preferable that the pre-processing unit 60 creates a composite image that is as large as possible relative to the angle of view used to capture the workpiece W. This allows a larger workpiece W to be accommodated in a limited space, improving the accuracy of defect detection.

[0046] [Fourth Embodiment] Next, a fourth embodiment will be described. In the fourth embodiment, there are a plurality of image capturing units 10 and a plurality of illumination units 20, and the image capturing conditions are changed by selecting one of the image capturing units 10 and one of the illumination units 20.

[0047] FIG. 6 is a diagram showing the overall configuration of an appearance inspection system 1 according to a fourth embodiment. The appearance inspection system 1 shown in FIG. 6 differs from the appearance inspection system 1 described in FIG. 3 in that it has two photographing units 10 and two lighting units 20, but is otherwise similar. In FIG. 6, the two photographing units 10 are shown as photographing units 10a and 10b, and the two lighting units 20 are shown as lighting units 20a and 20b. In this case, photographing can be performed under different photographing conditions using the photographing units 10a and 10b and the lighting units 20a and 20b. This eliminates or reduces the need to switch photographing conditions for a single photographing unit 10 or lighting unit 20.

[0048] FIG. 7 shows the display unit 90 displaying information about the inspection order. Here, the display unit 90 displays the conventional inspection order and the inspection order determined by the inspection order determination unit 100 based on the occurrence frequency calculated by the probability calculation unit 110. These inspection orders are also displayed along with the imaging conditions and occurrence frequencies. The conventional inspection order is shown as the current setting, and the inspection order determined by the inspection order determination unit 100 is shown as the recommended setting. The imaging conditions and occurrence frequencies indicated by the defect occurrence probability are also displayed. More specifically, in the second embodiment, when the defect occurrence probability is unknown, the display unit 90 displays the current setting (imaging conditions 1, 2, 3) and the recommended setting (imaging conditions 2, 3, 1, which inspects in descending order of defect occurrence probability) in a reversed order based on the calculated defect occurrence probability. This clarifies the differences between the two settings and allows the user to suggest changes to the inspection order to shorten the inspection time. Note that scratches, dents, and chips are defect details, and users can input them themselves to easily understand the purpose of the test. FIG. 7 also displays the occurrence frequencies in a bar graph. However, the present invention is not limited to this, and may be a line graph or a table.

[0049] In general, labor shortages are becoming more serious worldwide, and the manufacturing industry is expected to face an increasing shortage of skilled workers. For example, in the manufacturing industry, mechanization is lagging behind, particularly in the inspection process, where visual evaluation by skilled personnel is the norm. In recent years, the development of visual inspection equipment using AI has progressed, and labor savings are expected. Among these, a method of creating a learning model using training without teacher data (good product training AI) based only on inspection targets of good products and then performing inspections is particularly user-friendly compared to supervised learning that includes defective products, as it does not require the creation of teacher data (annotation) and does not require defective product data. However, this method has low recognition accuracy and often requires optimization of imaging conditions and preprocessing, such as lighting, background, and camera settings, depending on the inspection target and the type of defect to be detected. In this embodiment, as described above, the inspection order is rearranged based on the frequency of defect occurrence, from inspection items with a high frequency of defective to inspection items with a low frequency of defective. This produces the effects described below.

[0050] 8(a) to 8(c) are diagrams explaining the effect of changing the inspection order depending on the frequency of defects. Of these, FIG. 8(a) is a conceptual diagram showing the case where inspection is performed using the conventional inspection order. In this case, the inspection target is workpiece W No. (N), and inspections 1 to 4 are performed in this order. It also shows that GPUs 1 to 4 are used as AI accelerators for the inspection.

[0051] 8B is a conceptual diagram showing a case where the inspection order is changed depending on the frequency of defects as described in the second embodiment, and where inspection is performed using a composite image as described in the third embodiment. In this case, the inspection target is workpiece W No. (N), and inspection 3, inspection 4, and inspection 1 & inspection 2 are performed in this order. Here, inspection 1 & inspection 2 indicates that the inspection is performed using a composite image obtained by combining the images taken in inspection 1 and inspection 2. In this case, because a composite image is used in inspection 1 & inspection 2, GPUs 1 to 3 are used as the AI ​​accelerators used for inspection. In other words, the number of AI accelerators required is small.

[0052] In this case, inspection items with a higher defect frequency are inspected first. FIG. 8( c) illustrates the effect of this case. In this case, a defect was found for workpiece W No. (N) in inspection 3, which had the highest defect frequency, and therefore workpiece W No. (N) was rejected as a defective product at this point. Also, inspection of the next workpiece W No. (N+1) was started, and inspection 3, inspection 4, inspection 1, and inspection 2 were performed in that order. In other words, by inspecting inspection items with a higher defect frequency first, defects in workpieces W are often discovered more quickly. This shortens the inspection time. As a result, even if the number of inspection items for defects to be detected increases, the time required to determine the quality of the inspection target does not increase. Furthermore, the inspection method of this embodiment has a sequence similar to that of manual inspection. As a result, there are no significant changes in the inspection process before and after the introduction of the visual inspection system 1, allowing for partial automation. Furthermore, knowing the occurrence frequency makes it easier to identify problems in the workpiece W manufacturing process, leading to improved workpiece W yield.

[0053] <Explanation of Control Method> The above-described processing performed by the control device 2 is realized by the cooperation of software and hardware resources. That is, a processor such as a CPU provided in the control device 2 loads a program that realizes each function of the control device 2 into a main memory and executes the program, thereby realizing each of the above-described functions.

[0054] Therefore, the processing performed by the control device 2 described above can be considered as a control method for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions by having a processor execute a program stored in memory, and that diagnoses the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good, and determines the inspection order with changed shooting conditions based on the frequency of defects occurring for the shooting conditions. This makes it possible to provide a control method that does not increase the time required to determine the pass / fail of an inspection object even if the number of inspection items for defects to be detected increases.

[0055] The program for realizing this embodiment can be provided not only by communication means but also by being stored on a recording medium such as a CD-ROM.

[0056] 1...visual inspection system, 10, 10a, 10b...imaging unit, 20, 20a, 20b...illumination unit, 30...transport unit, 40...control unit, 50...storage, 60...preprocessing unit, 70...learning model, 80...diagnosis unit, 90...display unit, 100...inspection order determination unit, 110...probability calculation unit, W...work

Claims

1. An appearance inspection system comprising: an imaging unit that images an object to be inspected by changing the imaging conditions; an illumination unit that irradiates the object to be inspected with light; a diagnosis unit that diagnoses the appearance of the object to be inspected based on the image taken by the imaging unit using a learning model that has been trained for each imaging condition based on inspection objects that are good products; and an inspection order determination unit that determines the inspection order for the imaging conditions that have been changed based on the frequency of defects occurring for the imaging conditions.

2. The appearance inspection system of claim 1, wherein the photographing condition is the color of the irradiated light, and the inspection order determination unit determines the inspection order so as to irradiate the irradiated light in order of the color with the highest frequency of defects to the color with the lowest frequency of defects.

3. The visual inspection system according to claim 2, wherein the diagnosing unit diagnoses by associating the color of the irradiated light with the type of defect detected.

4. The visual inspection system according to claim 1, further comprising a probability calculation unit for calculating the frequency of occurrence of defects for said photographing conditions.

5. An appearance inspection system as described in claim 4, further comprising a display unit that displays the conventional inspection order and the inspection order determined by the inspection order determination unit based on the occurrence frequency calculated by the probability calculation unit, together with the shooting conditions and the occurrence frequency.

6. The visual inspection system of claim 1, further comprising a pre-processing unit that pre-processes the captured image before diagnosis by the diagnosis unit, wherein the pre-processing unit creates a composite image by combining images captured under a plurality of shooting conditions, and the diagnosis unit uses a learning model trained based on the composite image to diagnose the inspection object using the composite image.

7. The visual inspection system according to claim 6, wherein the diagnosis unit visualizes under which of the photographing conditions a defect occurred in the composite image.

8. The visual inspection system according to claim 7, wherein the diagnostic unit visualizes the composite image as a heat map showing the frequency of defects.

9. The visual inspection system according to claim 6, wherein the diagnosing unit performs diagnosis using the composite image, thereby simultaneously diagnosing a plurality of defect types.

10. The visual inspection system according to claim 6, wherein the preprocessing unit creates a composite image that is largest relative to the angle of view at which the inspection object is photographed.

11. The appearance inspection system of claim 1, wherein there are multiple photographing units and multiple lighting units, and the photographing conditions are changed by selecting one of the multiple photographing units and one of the multiple lighting units.

12. A control device for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions, the control device comprising: a diagnosis unit that diagnoses the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good products; and an inspection order determination unit that determines the inspection order for which the shooting conditions have been changed based on the frequency of defects occurring under the shooting conditions.

13. A control method for controlling an appearance inspection system that irradiates an inspection object with irradiation light and diagnoses the appearance of the inspection object based on images of the inspection object taken under different shooting conditions by a processor executing a program recorded in memory, the control method diagnosing the appearance of the inspection object based on the taken images using a learning model that has been trained for each shooting condition based on inspection objects that are good products, and determining the inspection order with the changed shooting conditions based on the frequency of defects occurring under the shooting conditions.

Images