Visual inspection system, visual inspection method, and visual inspection program

Abstract

This invention provides a visual sorting system, visual sorting method, and visual sorting program that can more accurately select good products. [Solution] The appearance sorting system 100 comprises an imaging device 10 that images an object to be inspected W and generates an inspection image Wi, a local computer 1 having a first discrimination unit 11 that locally performs a first pass / fail judgment of the inspection image Wi based on a first inspection recipe 11R, a server computer 2 having a second discrimination unit 21 that performs a second pass / fail judgment of the inspection image Wi in the cloud based on a second inspection recipe 21R, and a parallel processing unit 24 that performs the first pass / fail judgment and the second pass / fail judgment in parallel.

Description

Technical Field

[0001] The present invention relates to an appearance sorting system, an appearance sorting method, and an appearance sorting program.

Background Art

[0002] Conventionally, in an appearance sorting machine that determines (judges good or bad) OK products and NG products based on an image of a product and sorts out good products, an image determined as an NG product is further classified into a more detailed NG mode and labeled (multi-label classification), and by using the labeling result, the performance of the inspection recipe may be improved (for example, Patent Document 1).

[0003] Conventional appearance sorting machines need to set a threshold value so that the inspection result is on the safe side in order to meet the requirements of low-cost and high-speed inspection processing and to prevent undetected defective products. In addition to the constraints on the specifications of the image processing PC and the image processing time constraints due to the equipment tact of the product production equipment, there are also constraints due to excessive image detection (misclassification). For this reason, it is difficult to accurately perform multi-label classification during the production of products, and the sorting results that can be obtained by users from conventional appearance sorting machines may be inaccurate.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of the present invention is to provide an appearance sorting system, an appearance sorting method, and an appearance sorting program that can more accurately sort out good products.

Means for Solving the Problems

[0006] (1) An appearance sorting system according to one aspect of the present invention comprises: an imaging device that images an object to be inspected and generates an inspection image; a local computer having a first discrimination unit that locally performs a first good / bad judgment of the inspection image based on a first inspection recipe; a server computer having a second discrimination unit that performs a second good / bad judgment of the inspection image in the cloud based on a second inspection recipe; and a parallel processing unit that performs the first good / bad judgment and the second good / bad judgment in parallel. (2) In the above (1), the first discrimination unit may perform a first multi-label classification in which, if the result of the first good / bad discrimination is a defective product, the inspection image is classified into a defective product mode and labeled; the second discrimination unit may perform a second multi-label classification in which, if the result of the second good / bad discrimination is a defective product, the inspection image is classified into a defective product mode and labeled; and the comparison extraction unit may include a first classification result of the first multi-label classification and a second classification result of the second multi-label classification, and extract inspection images in which the two do not match. (3) In (1) or (2) above, the first inspection recipe may be composed of a combination of a rule-based discrimination logic based on predefined rules and a local AI logic using a trained discrimination AI. (4) In the case of (3) above, if the result of the first pass / fail judgment and the result of the second pass / fail judgment are different, the result of the second pass / fail judgment may be output. (5) An appearance sorting method according to one aspect of the present invention includes an imaging step of imaging an object to be inspected to generate an inspection image, a first determination step of performing a first good or bad determination of the inspection image locally by a local computer based on a first inspection recipe, and a second determination step of performing a second good or bad determination of the inspection image in the cloud by a server computer based on a second inspection recipe, wherein the first determination step and the second determination step are performed in parallel. (6) An appearance sorting program according to one aspect of the present invention causes a computer to perform an imaging function that images an object to be inspected and generates an inspection image; a first judgment function that performs a first pass / fail judgment of the inspection image locally based on a first inspection recipe; a second judgment function that performs a second pass / fail judgment of the inspection image in the cloud based on a second inspection recipe; and a parallel processing function that performs the first pass / fail judgment and the second pass / fail judgment in parallel. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an appearance sorting system, an appearance sorting method, and an appearance sorting program that can sort good products more accurately. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram illustrating the overview of the visual sorting system according to the first embodiment. [Figure 2] This diagram illustrates the flow of the visual inspection method. [Figure 3] This is an explanatory diagram comparing the first multi-label classification and the second multi-label classification. [Figure 4] This diagram illustrates the overview of the visual sorting system according to the second embodiment. [Modes for carrying out the invention]

[0009] (First Embodiment) A first embodiment of the present invention will be described in detail below with reference to the drawings. Figure 1 is a diagram illustrating the overview of the visual sorting system 100 according to the first embodiment. Figure 2 is a diagram illustrating the flow of the visual sorting method. Figure 3 is an explanatory diagram comparing the first multi-label classification and the second multi-label classification. In the following, parts having common functions may be denoted by the same reference numeral or symbol.

[0010] (Visual inspection system) The appearance sorting system 100 according to the first embodiment can be used, for example, to inspect the appearance of products such as electronic components such as capacitors. The appearance sorting system 100 can be used to sort products into good products (also called OK products) or defective products (also called NG products) based on their appearance.

[0011] As shown in Figure 1, the visual inspection system 100 includes an imaging device 10 that images an object W to be inspected and generates an inspection image Wi, a local computer 1 having a first discrimination unit 11 that locally performs a first pass / fail judgment of the inspection image Wi based on a first inspection recipe 11R, and a server computer 2 having a second discrimination unit 21 that performs a second pass / fail judgment of the inspection image Wi in the cloud based on a second inspection recipe.

[0012] Local computer 1 is, for example, built into an inspection device installed along the production line of the object to be inspected W. Local computer 1 is connected to server computer 2 via network N from its communication unit 12.

[0013] The imaging device 10 is an optical machine that takes photographs. The imaging device 10 may be a so-called digital camera that receives light that has passed through a lens with an image sensor, which is a semiconductor sensor such as a CCD or CMOS, and converts it into image data. The imaging device 10 is attached to an inspection device installed along the manufacturing line of the object to be inspected W. The imaging device 10 images the product (part) that is the object to be inspected W. The imaging device 10 acquires multiple images of the same product by taking images from multiple directions (for example, 6 directions) as appropriate. The captured inspection images Wi are sent to the first discrimination unit 11 of the local computer 1 or to a memory not shown.

[0014] The first discrimination unit 11 may perform the first pass / fail discrimination by determining whether it is a good product or a defective product according to a rule-based discrimination logic R1 based on rules defined in advance according to the specifications or types of the inspection object W. The rule-based discrimination logic R1 may be, for example, an integration of algorithms for detecting defective locations provided for each defective item (such as stains, chips, foreign objects, pinholes, etc.). As shown in FIG. 1, the first inspection recipe 11R may include the rule-based discrimination logic R1. Thereby, the burden of the inspection process (image processing, pass / fail discrimination process, multi-label classification process) by the first discrimination unit 11 can be kept low.

[0015] The first discrimination unit 11 may perform the first pass / fail discrimination on the inspection image Wi by means of a local AI logic R2. The local AI logic R2 may be, for example, logistic regression, support vector machine, k-nearest neighbor method, decision tree, neural network deep learning. Thereby, the inspection process by the first discrimination unit 11, especially the multi-label classification with a large processing burden, can be made more accurate.

[0016] The first pass / fail discrimination by the first discrimination unit 11 may include multi-label classification for classifying and labeling an image determined to be a defective product into finer defective items (also referred to as defective product modes). That is, in the first pass / fail discrimination by the first discrimination unit 11, as shown in FIG. 3, in addition to determining whether it is a good product or a defective product, the first pass / fail discrimination may further classify and label the defective product into finer defective items.

[0017] The first inspection recipe 11R may be composed by synthesizing a rule-based discrimination logic R1 based on predefined rules and an AI logic R2 by a learned discrimination AI. For example, for an image that is originally determined to be a good product by the rule-based discrimination logic R1 but is determined to be a defective product, if it is also determined to be a good product by the local AI logic R2, the first inspection recipe 11R determines that image as a good product. Thereby, over-detection of defective products by the first pass / fail discrimination by the first discrimination unit 11 can be suppressed.

[0018] The server computer 2 responds to the resource requests from the local computer 1. The server computer 2 is connected to the local computer 1 via the network N from the communication unit 22 of the server computer 2.

[0019] The second discrimination unit 21 may perform pass / fail discrimination based on the inspection image Wi by the cloud AI logic G. The cloud AI logic G may be, for example, logistic regression, support vector machine, k-nearest neighbor method, decision tree, neural network / deep learning. Thereby, multi-label classification can be made more accurate.

[0020] The second pass / fail discrimination by the second discrimination unit 21 may be performed by determining whether it is a good product or a defective product according to a second inspection recipe 21R set in advance according to the specification or type of the inspection object W. The second pass / fail discrimination by the second discrimination unit 21 may include multi-label classification for classifying and labeling the images determined to be defective products into finer defective product modes. That is, as shown in FIG. 3, in addition to determining whether it is a good product or a defective product, the second pass / fail discrimination may further classify and label the defective products into finer defective items (defective product modes).

[0021] Here, the visual inspection system 100 includes a parallel processing unit 24 that performs a first pass / fail judgment and a second pass / fail judgment in parallel. As shown in Figure 1, the parallel processing unit 24 may be provided in the server computer 2, the local computer 1, or another computer (not shown) on the network N. Based on the detection of imaging by the imaging device 10, the parallel processing unit 24 may perform the first pass / fail judgment and the second pass / fail judgment in parallel using the first judgment unit 11 and the second determination unit 21. Performing pass / fail judgment in parallel means, in other words, executing the two judgment processes simultaneously. This allows time-consuming processing to be divided among the two processors of the local computer 1 and the server computer 2, respectively, and proceed efficiently, thereby improving the overall processing speed and enabling selection by combining the two judgment results. Furthermore, this allows the discrimination process to be relied upon by the server computer 2, which has ample performance headroom, rather than relying on the local computer 1, which has performance (specifications) limitations. By maintaining the equipment cycle time and performing discrimination processing on the local computer 1 while the server computer 2 performs complementary discrimination processing, the two discrimination results can be combined to select products with high accuracy. Thus, the appearance-based selection system 100 allows for more accurate selection of good products even when there are limitations in the specifications of the image processing PC and limitations in image processing time due to the equipment cycle time of the product production equipment.

[0022] Local computer 1 may have relatively lower specifications in terms of image processing performance compared to server computer 2. For example, the GPU in local computer 1 may have lower specifications than the GPU in server computer 2. This reduces the distribution ratio of the processing load of inspection images Wi on local computer 1, which has limited performance scalability because it is attached to the manufacturing equipment of the product to be inspected W, and increases the distribution ratio of the processing load of inspection images Wi on server computer 2, which has ample performance scalability. Thus, inspection images Wi can be processed efficiently.

[0023] By the way, in order to further improve the performance of multi-label classification using Local AI Logic R2 or Cloud AI Logic G, it is effective to train Local AI Logic R2 or Cloud AI Logic G with many labeled images. However, users (mainly clients using the visual inspection service) are required to accurately annotate (label and color-code good / bad or bad mode for each image) by visually checking a large number of collected images one by one in order to feed them to Local AI Logic R2 or Cloud AI Logic G for machine learning. And usually, annotation requires a great deal of effort.

[0024] Therefore, recognizing that the large number of collected images includes both those that contribute to improving the performance of multi-label classification and those that do not, we have made it possible to extract and provide users with only a portion of the images that contribute relatively significantly to machine learning, without forcing them to annotate all of the images. As a result, users only need to annotate the extracted portion of images, reducing their workload and allowing them to efficiently train the local AI logic R2 or cloud AI logic G to improve the performance of multi-label classification performed by the local AI logic R2 or cloud AI logic G.

[0025] In detail, the first discrimination unit 11 performs a first multi-label classification, classifying and labeling the inspection image Wi as a defective product if the result of the first good / bad judgment is a defective product. The second discrimination unit 21 performs a second multi-label classification, classifying and labeling the inspection image Wi as a defective product if the result of the second good / bad judgment is a defective product. The appearance sorting system 100 includes a comparison extraction unit 23 that compares the first classification result of the first multi-label classification with the second classification result of the second multi-label classification and extracts inspection images Wi where the two do not match. The comparison extraction unit 23 may be provided on the local computer 1, on the server computer 2 as shown in Figure 1, or on other computers on the network N (not shown). As a result, by comparing the first classification result with the second classification result and extracting inspection images Wi where the two do not match, it is possible to select images that have been over-detected (misclassified), such as good products being mistakenly identified as defective products, which is likely to contribute relatively significantly to machine learning. Then, annotation can be preferentially performed on these selected extracted images Wp (as appropriate, by the user), and the annotated images Wa, to which the correct labels have been assigned through annotation, can be used to train the local AI logic R2 or the cloud AI logic G. This improves the accuracy of quality judgment and multi-label classification by the first discrimination unit 11 or the second discrimination unit 21.

[0026] As shown by the dashed line in Figure 1, the annotation image Wa, which is obtained by adding labels to the extracted image Wp extracted by the comparison extraction unit 23, may be used to train the local AI logic R2 or the cloud AI logic G. This allows the first discrimination unit 11 including the local AI logic R2 or the second discrimination unit 21 including the cloud AI logic G to train without placing an excessive burden on the annotation. This effectively improves the performance of quality judgment or multi-label classification. Furthermore, the rule-based discrimination logic R1 may be improved by modifying the model parameters or hyperparameters based on the annotation image Wa, either through parameter search using an optimization algorithm or manually.

[0027] The visual inspection system 100 may output the result of the second pass / fail judgment if the result of the first pass / fail judgment for the inspection image Wi differs from the result of the second pass / fail judgment. The result of the pass / fail judgment for the inspection image Wi is output to an output unit provided by the visual inspection system 100. The output unit may be provided on the local computer 1, on the server computer 2, or on another computer (not shown) on the network N. Output to the output unit may be, for example, displayed on a display or printed on a printer. This allows for the adoption of a more reliable result, based on the premise that the result of the second pass / fail judgment processed on the relatively high-spec server computer 2 is set to be more reliable than the result of the first pass / fail judgment processed on the relatively low-spec local computer 1.

[0028] (Visual sorting method) Next, the flow of the visual sorting method will be explained. The visual sorting method can be carried out using the visual sorting system 100. Each step in the visual sorting method can be performed by the corresponding part of the visual sorting system 100. Figure 2 is a diagram illustrating the flow of the visual sorting method. Figure 3 is an explanatory diagram comparing the first multi-label classification and the second multi-label classification.

[0029] (1) As shown in Figure 2, first, the object to be inspected W is imaged by the imaging device 10 to generate an inspection image Wi (imaging step S1).

[0030] (2) Next, the captured inspection image Wi is sent to the first discrimination unit 11 of the local computer 1. At the same time, the communication unit 12 of the local computer 1 sends the inspection image Wi to the second discrimination unit 21 of the server computer 2 via the communication unit 22 of the server computer 2.

[0031] (3) The first pass / fail determination of the inspection image Wi is performed locally by the first determination unit 11 of the local computer 1 based on the first inspection recipe 11R (first determination step S2).

[0032] (3-1) In detail, if the result of the first pass / fail judgment is a defective product, the first discrimination unit 11 classifies the inspection image Wi into defective product mode and labels it (first multi-label classification step S2-1).

[0033] (4) The second pass / fail determination of the inspection image Wi is performed in the cloud by the second determination unit 21 of the server computer 2 based on the second inspection recipe 21R (second determination step S3).

[0034] (4-1) In detail, if the result of the second quality determination is a defective product, the second discrimination unit 21 classifies the inspection image Wi into defective product mode and labels it (second multi-label classification step S3-1).

[0035] (5) Here, the first discrimination step S2 and the second discrimination step S3 are performed in parallel. For example, as shown in Figure 3, the first discrimination unit 11 performs a first pass / fail discrimination on the inspection image Wi, and obtains a first classification result showing the detection result for each defective item, as shown in the table in Figure 3. At the same time, the second discrimination unit 21 performs a second pass / fail discrimination on the inspection image Wi, and obtains a second classification result showing the detection result for each defective item, as shown in the table in Figure 3. This allows time-consuming processing to be divided between the two processors of the local computer 1 and the server computer 2, respectively, and can be carried out efficiently, thereby improving the overall processing speed and enabling selection by combining the two discrimination results. Furthermore, this allows the discrimination processing to be relied upon by the server computer 2, which has a performance margin, rather than the local computer 1, which has performance (specifications) constraints. This maintains the equipment cycle time, allowing the local computer 1 to perform discrimination processing while the server computer 2 performs complementary discrimination processing, enabling accurate selection by combining the two discrimination results. Thus, this visual inspection method allows for more accurate selection of good products, even when faced with limitations in the specifications of the image processing PC and limitations in image processing time due to the equipment cycle time of the product's production equipment.

[0036] (6) Next, the first classification result of the first multi-label classification and the second classification result of the second multi-label classification are compared (comparison step S4).

[0037] (7) In the comparison step S4, if the results of the comparison show that the defective items in the first classification result and the second classification result match, then either classification result, for example, the first classification result, is output (match result output step S5).

[0038] (7-1) In the comparison step S4, if the results of the comparison show that the defective items in the first classification result and the second classification result do not match, the inspection image Wi that was the subject of inspection is extracted as the extracted image Wp (extraction step S6). For example, as shown in Figure 3, the first classification result and the second classification result obtained from the good / bad judgment of the inspection image Wi are compared, and a match or mismatch is determined for each defective item, as shown in the table in Figure 3. If the inspection image Wi contains defective items that were determined to be mismatched, that inspection image Wi is extracted as the extracted image Wp.

[0039] (8) The extracted inspection images Wi are annotated by the user as appropriate. In this way, only inspection images Wi with mismatched defect items are extracted, so annotation for machine learning can be performed efficiently on the extracted inspection images Wi with minimal effort.

[0040] (8-1) Here, the local AI logic R2 or cloud AI logic G may be trained with the annotated image Wa, which has been annotated with the extracted image Wp, as appropriate. This can improve the accuracy of quality judgment and multi-label classification.

[0041] (9) In the comparison step S4, if the results of the comparison show that the defective items in the first classification result and the second classification result do not match, the second classification result is output preferentially (priority output step S7). This makes it possible to output a highly reliable inspection result in which the overdetection of defective items by the first pass / fail judgment is corrected by the second pass / fail judgment.

[0042] (Appearance-based sorting program) The visual inspection program has the function of causing a computer to execute each process performed at each step of the visual inspection method described above. Specifically, the visual inspection program causes the computer to execute, according to each step of the visual inspection method described above, an imaging function that images the object to be inspected W and generates an inspection image Wi; a first judgment function that performs a first pass / fail judgment of the inspection image Wi locally based on the first inspection recipe 11R; a second judgment function that performs a second pass / fail judgment of the inspection image Wi in the cloud based on the second inspection recipe 21R; and a parallel processing function that performs the first pass / fail judgment and the second pass / fail judgment in parallel. The visual inspection program may be installed on the server computer 2, on the local computer 1, or on other computers (not shown) on the network N. This allows time-consuming processing to be divided between the two processors of the local computer 1 and the server computer 2, respectively, enabling efficient calculation, thereby improving the overall processing speed and allowing for selection based on a combination of the two judgment results. Furthermore, this allows the discrimination process to be relied upon by server computer 2, which has ample performance headroom, rather than relying on local computer 1, which has performance (specifications) limitations. By maintaining the equipment cycle time and performing discrimination processing on local computer 1 while server computer 2 performs complementary discrimination processing, the two discrimination results can be combined to select products with high accuracy. In this way, the appearance-based selection program can more accurately select good products even with the limitations of the image processing PC's specifications and the image processing time constraints resulting from the equipment cycle time of the product production equipment.

[0043] (Second Embodiment) Next, the visual sorting system 200 according to the second embodiment will be described. Figure 4 is a diagram illustrating the overview of the visual sorting system 200 according to the second embodiment. Hereinafter, in the visual sorting system 200 according to the second embodiment, elements having the same function as the visual sorting system 100 according to the first embodiment are denoted by the same reference numerals or symbols.

[0044] In the first embodiment of the appearance sorting system 100, the case in which the local AI logic R2 in the first inspection recipe 11R is trained using an annotation image Wa was described. However, it can be assumed that training on the local computer 1 requires stopping the inspection device related to the local computer 1, which is time-consuming. Furthermore, it can be assumed that training on the local computer 1 will cause the content of the local AI logic R2 currently running on the local computer 1 to be automatically changed, making it unmanageable for the user. Therefore, in order to solve these problems, the appearance sorting system 200 according to the second embodiment differs from the appearance sorting system 100 according to the first embodiment, mainly as described below.

[0045] As shown in Figure 4, the appearance sorting system 200 according to the second embodiment includes a server computer 2 equipped with a third discrimination unit 25 having the same structure as the first discrimination unit 11.

[0046] The third discrimination unit 25 has a third inspection recipe 25R which has the same structure as the first inspection recipe 11R. In other words, the third inspection recipe 25R includes an alternative rule-based discrimination logic R3 with the same structure as the rule-based discrimination logic R1, and an alternative local AI logic R4 with the same structure as the local AI logic R2, and is constructed by combining the alternative rule-based discrimination logic R3 and the alternative local AI logic R4. For example, if an image that should have been judged as a good product by the alternative rule-based discrimination logic R3 is judged as a defective product, but the alternative local AI logic R4 also judges it as a good product, then the third inspection recipe 25R will judge that image as a good product.

[0047] Then, the user can, at their discretion, pass the annotation image Wa to an alternative local AI logic R4 instead of the local AI logic R2, and train the alternative local AI logic R4.

[0048] Whenever the user determines that the performance of the third test recipe 25R is sufficiently higher than that of the first test recipe 11R, the user can overwrite the first test recipe 11R on local computer 1 with the third test recipe 25R at any time of their choosing.

[0049] In this case, the comparison extraction unit 23 may compare the first discrimination unit 11 and the third discrimination unit 25 instead of the second discrimination unit 21. That is, in the appearance sorting system 200, the comparison extraction unit 23 compares the first classification result of the first multi-label classification by the first discrimination unit 11 with the third classification result of the third multi-label classification by the third discrimination unit 25 and extracts inspection images Wp for which the two do not match.

[0050] Thus, according to the appearance sorting system 200 of the second embodiment, the server computer 2 is equipped with a third discrimination unit 25 having the same structure as the first discrimination unit 11. As a result, the annotation image Wa is not passed to the local AI logic R2 on the local computer 1, but is passed to the alternative local AI logic R4 included in the third discrimination unit 25 on the server computer 2, allowing the alternative local AI logic R4 to be trained. Therefore, the first determination unit 11 on the local computer 1 can be updated at a timing appropriate to the user without stopping the inspection device on the local computer 1.

[0051] It should be noted that the technical scope of the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the invention. Furthermore, it is possible to replace the components in the embodiments described above with well-known components as appropriate, without departing from the spirit of the invention. In addition, the above-described modifications can be combined as appropriate, without departing from the spirit of the invention.

[0052] As described above, the appearance sorting system 100,200 according to the embodiment includes an imaging device 10 that images an object to be inspected W and generates an inspection image Wi, a local computer 1 having a first discrimination unit 11 that performs a first pass / fail judgment of the inspection image Wi locally based on a first inspection recipe 11R, a server computer 2 having a second discrimination unit 21 that performs a second pass / fail judgment of the inspection image Wi in the cloud based on a second inspection recipe 21R, and a parallel processing unit 24 that performs the first pass / fail judgment and the second pass / fail judgment in parallel. This allows time-consuming processing to be divided between the two processors of the local computer 1 and the server computer 2, respectively, and proceeds efficiently, thereby improving the overall processing speed and enabling sorting by combining the two discrimination results. Furthermore, this allows the discrimination processing to be relied upon by the server computer 2, which has ample performance headroom, rather than relying on the local computer 1, which has performance (specifications) constraints. This maintains the equipment cycle time, allowing the local computer 1 to perform discrimination processing while the server computer 2 performs complementary discrimination processing, enabling accurate sorting by combining the two discrimination results. Thus, the visual inspection system 100 allows for more accurate selection of good products, even when there are limitations in the specifications of the image processing PC and limitations in image processing time due to the equipment cycle time of the product production equipment.

[0053] The visual inspection method according to this embodiment includes an imaging step S1 in which an object to be inspected W is imaged to generate an inspection image Wi; a first discrimination step S2 in which a first good / bad discrimination of the inspection image Wi is performed locally by a local computer 1 based on a first inspection recipe 11R; and a second discrimination step S3 in which a second good / bad discrimination of the inspection image Wi is performed in the cloud by a server computer 2 based on a second inspection recipe 21R. In this visual inspection method, the first discrimination step S2 and the second discrimination step S3 are performed in parallel. This makes it possible to select good products more accurately even if there are limitations in the specifications of the image processing PC and limitations in image processing time due to the equipment cycle time of the product production equipment.

[0054] The visual sorting program according to this embodiment causes the computer to perform an imaging function that captures an object W to be inspected and generates an inspection image Wi; a first judgment function that performs a first pass / fail judgment of the inspection image Wi locally based on a first inspection recipe 11R; a second judgment function that performs a second pass / fail judgment of the inspection image Wi in the cloud based on a second inspection recipe 21R; and a parallel processing function that performs the first pass / fail judgment and the second pass / fail judgment in parallel. This makes it possible to sort good products more accurately even if there are limitations in the specifications of the image processing PC and limitations in image processing time due to the equipment cycle time of the product production equipment. [Explanation of symbols]

[0055] 100,200 Visual Inspection System 1 Local computer 10 Imaging device 11 1st discrimination part 11R First Inspection Recipe 12 Communications Department 2 Server Computers 21 Second discrimination part 21R Second Inspection Recipe 22 Communications Department 23 Comparison Extraction Unit 24 Parallel Processing Unit 25 3rd discrimination part 25R Third Test Recipe N Network R1 Rule-based discrimination logic R2 Local AI Logic R3 Alternative Rule-Based Discrimination Logic R4 Alternative Local AI Logic G Cloud AI Logic W: Items to be inspected Wi-Fi examination images WP extracted images Wa Annotation Images

Claims

1. An imaging device that captures images of the object to be inspected and generates inspection images, A local computer having a first discrimination unit that locally performs a first pass / fail judgment of the aforementioned inspection image based on a first inspection recipe, A server computer having a second discrimination unit that performs a second pass / fail determination of the aforementioned inspection image in the cloud based on a second inspection recipe, The system comprises a parallel processing unit that performs the first quality determination and the second quality determination in parallel, The first test recipe includes a rule-based discrimination logic based on predefined rules, The aforementioned second inspection recipe includes cloud AI logic, The local computer has a lower allocation ratio of processing load related to the image processing performance of the inspection images compared to the server computer. Appearance sorting system.

2. If the result of the first quality determination is that the product is defective, the first discrimination unit performs a first multi-label classification on the inspection image, classifying it into defective product mode and labeling it. If the result of the second quality determination is that the product is defective, the second discrimination unit performs a second multi-label classification on the inspection image, classifying it into the defective product mode and labeling it. The system includes a comparison and extraction unit that compares the first classification result of the first multi-label classification with the second classification result of the second multi-label classification and extracts inspection images where the two results do not match. The appearance sorting system according to claim 1.

3. The first inspection recipe is constructed by combining the rule-based discrimination logic and the local AI logic based on the trained discrimination AI. The appearance sorting system according to claim 1 or claim 2.

4. If the result of the first pass / fail judgment differs from the result of the second pass / fail judgment, the result of the second pass / fail judgment is output. The appearance sorting system according to claim 1 or claim 2.

5. An imaging device that captures images of the object to be inspected and generates inspection images, A local computer having a first discrimination unit that locally performs a first pass / fail judgment of the aforementioned inspection image based on a first inspection recipe, A server computer having a second discrimination unit that performs a second pass / fail determination of the aforementioned inspection image in the cloud based on a second inspection recipe, The system comprises a parallel processing unit that performs the first quality determination and the second quality determination in parallel, If the result of the first quality determination is that the product is defective, the first discrimination unit performs a first multi-label classification on the inspection image, classifying it into defective product mode and labeling it. If the result of the second quality determination is that the product is defective, the second discrimination unit performs a second multi-label classification on the inspection image, classifying it into the defective product mode and labeling it. The system includes a comparison and extraction unit that compares the first classification result of the first multi-label classification with the second classification result of the second multi-label classification and extracts inspection images where the two results do not match. Appearance sorting system.

6. An imaging device that captures images of the object to be inspected and generates inspection images, A local computer having a first discrimination unit that locally performs a first pass / fail judgment of the aforementioned inspection image based on a first inspection recipe, A server computer having a second discrimination unit that performs a second pass / fail determination of the aforementioned inspection image in the cloud based on a second inspection recipe, The system comprises a parallel processing unit that performs the first quality determination and the second quality determination in parallel, If the result of the first pass / fail judgment differs from the result of the second pass / fail judgment, the result of the second pass / fail judgment is output. Appearance sorting system.

7. The imaging step involves capturing an image of the object to be inspected and generating an inspection image, A first determination step in which the first pass / fail determination of the aforementioned inspection image is performed locally by a local computer based on a first inspection recipe, This includes a second determination step in which a second pass / fail determination of the aforementioned inspection image is performed in the cloud by a server computer based on a second inspection recipe, An appearance sorting method that performs the first discrimination step and the second discrimination step in parallel, The first test recipe includes a rule-based discrimination logic based on predefined rules, The aforementioned second inspection recipe includes cloud AI logic, The local computer has a lower allocation ratio of processing load related to the image processing performance of the inspection images compared to the server computer. Method for sorting by appearance.

8. The imaging step involves capturing an image of the object to be inspected and generating an inspection image, A first determination step in which the first pass / fail determination of the aforementioned inspection image is performed locally by a local computer based on a first inspection recipe, This includes a second determination step in which a second pass / fail determination of the aforementioned inspection image is performed in the cloud by a server computer based on a second inspection recipe, An appearance sorting method that performs the first discrimination step and the second discrimination step in parallel, In the first determination step, if the result of the first good / bad determination is a defective product, a first multi-label classification is performed on the inspection image to classify and label it as a defective product. In the second discrimination step, if the result of the second good / bad discrimination is a defective product, a second multi-label classification is performed on the inspection image to classify and label it as a defective product. The method includes a comparison extraction step of comparing the first classification result of the first multi-label classification with the second classification result of the second multi-label classification and extracting examination images in which the two results do not match. Method for sorting by appearance.

9. The imaging step involves capturing an image of the object to be inspected and generating an inspection image, A first determination step in which the first pass / fail determination of the aforementioned inspection image is performed locally by a local computer based on a first inspection recipe, This includes a second determination step in which a second pass / fail determination of the aforementioned inspection image is performed in the cloud by a server computer based on a second inspection recipe, An appearance sorting method that performs the first discrimination step and the second discrimination step in parallel, If the result of the first pass / fail judgment differs from the result of the second pass / fail judgment, the result of the second pass / fail judgment is output. Method for sorting by appearance.

10. On the computer, The imaging function captures images of the object to be inspected and generates inspection images, A first judgment function that performs a first pass / fail determination of the inspection image locally based on a first inspection recipe, A second judgment function that performs a second pass / fail determination of the aforementioned inspection image in the cloud based on a second inspection recipe, A parallel processing function that performs the first pass / fail determination and the second pass / fail determination in parallel, An appearance-based selection program that executes the following: The first test recipe includes a rule-based discrimination logic based on predefined rules, The aforementioned second inspection recipe includes cloud AI logic, The first judgment function, performed locally, has a lower allocation ratio of processing load related to the image processing performance of the inspection images compared to the second judgment function, which is performed in the cloud. Appearance sorting program.

11. On the computer, The imaging function captures images of the object to be inspected and generates inspection images, A first judgment function that performs a first pass / fail determination of the inspection image locally based on a first inspection recipe, A second judgment function that performs a second pass / fail determination of the aforementioned inspection image in the cloud based on a second inspection recipe, A parallel processing function that performs the first pass / fail determination and the second pass / fail determination in parallel, An appearance-based selection program that executes the following: The first judgment function, if the result of the first pass / fail judgment is a defective product, performs a first multi-label classification on the inspection image, classifying it as a defective product and labeling it. The second judgment function, if the result of the second good / bad judgment is a defective product, performs a second multi-label classification on the inspection image, classifying and labeling it as a defective product. The system performs a comparison and extraction function that compares the first classification result of the first multi-label classification with the second classification result of the second multi-label classification and extracts inspection images where the two results do not match. Appearance sorting program.

12. On the computer, The imaging function captures images of the object to be inspected and generates inspection images, A first judgment function that performs a first pass / fail determination of the inspection image locally based on a first inspection recipe, A second judgment function that performs a second pass / fail determination of the aforementioned inspection image in the cloud based on a second inspection recipe, A parallel processing function that performs the first pass / fail determination and the second pass / fail determination in parallel, An appearance-based selection program that executes the following: If the result of the first pass / fail judgment differs from the result of the second pass / fail judgment, the result of the second pass / fail judgment is output. Appearance sorting program.

Images