Article inspection device

Abstract

To provide an article inspection device with improved accuracy.SOLUTION: An inspection processing unit 221 outputs determination data D1 indicating a good-product determination result for an article W by using a good-product learning model 241 trained with good-product images alone obtained by imaging the article W. An inspection processing unit 222 outputs determination data D2 by using a defective-product learning model 242 trained with captured images alone of defective products of the article W. A comprehensive determination unit 23 comprehensively determines the quality state of the article W imaged in a captured image on the basis of determination data D1 to D3 output from a plurality of inspection processing units 221 to 223.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an article inspection device. [Background technology]

[0002] For example, the inspection device disclosed in Patent Document 1 has been proposed as an article inspection device for determining the quality (quality state) of an article. The article inspection device disclosed in Patent Document 1 performs inspection using a learning model that inputs an inspection image of an object to be inspected (article) and outputs the degree of quality defect for each pixel. The learning model is trained using a learning composite image of defective quality, which is a combination of images of good products and images of only defective products.

[0003] However, while the inspection device of Patent Document 1 described above can inspect learned quality defects with high accuracy, there is a risk that unlearned quality defects may be erroneously determined as non-defective. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-114828 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an article inspection device with improved inspection accuracy. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the article inspection device according to the present invention is characterized by the following [1] to [8]. [1] an image storage unit (21) that stores an image of the article (W) captured by the imaging unit; a plurality of inspection processing units (221 to 223, 221B, 222B, 221C, 222C) that perform inspection processing on the captured images and output judgment data indicating the quality state of the items captured in the captured images; and a comprehensive judgment unit (23, 23B) that comprehensively judges the quality state of the item captured in the captured image based on the judgment data output from the plurality of inspection processing units. An article inspection device (1, 1B, 1C), One of the plurality of inspection processing units outputs the judgment data using a non-defective product learning model (241, 2411 to 241n) that has been trained using only non-defective product images obtained by capturing images of non-defective products of the article; One of the plurality of inspection processing units outputs the judgment data using a defective product learning model (242, 2421 to 242m) that has learned only defective product images obtained by capturing images of defective products of the article. It is an item inspection device. [2] In the article inspection device according to [1], a first selection unit (R61) that selects one or more of the plurality of inspection processing units; the comprehensive judgment unit comprehensively judges the quality state of the item captured in the captured image based on the judgment data output from the inspection processing unit selected by the first selection unit. It is an item inspection device. [3] In the article inspection device according to [1], The good product learning model or the defective product learning model is provided with a plurality of learning models with different learning contents, a second selection unit (R62) that selects one or more of the plurality of good product learning models or the plurality of defective product learning models; the comprehensive judgment unit comprehensively judges the quality state of the product captured in the captured image based on the judgment data output from the good product learning model or the defective product learning model selected by the second selection unit. It is an item inspection device. [4] In the article inspection device according to [1], One of the plurality of inspection processing units performs image processing on the input captured image using a predetermined image processing algorithm (243), and outputs the determination data. It is an item inspection device. [5] In the article inspection device according to [1], a judgment rule setting unit that sets rules for judgment by the comprehensive judgment unit; the comprehensive judgment unit makes the comprehensive judgment on the quality state in accordance with the set rule. It is an item inspection device. [6] In the article inspection device according to [1], An image extraction unit (261, 262, 263) that extracts a portion in which the article is captured from the captured image, The inspection processing unit receives an extracted image of the portion extracted by the image extraction unit. It is an item inspection device. [7] [6] The article inspection device according to [6], The image extraction unit inputs the captured image and extracts the captured image of the article using a learning model (271, 272, 273) that has been trained. It is an item inspection device. [8] In the article inspection device according to [1], a plurality of the defective product learning models are provided, each of which is trained separately for each type of heterogeneous object; a third selection unit (R63) that selects the type of the foreign substance; the inspection processing unit outputs the determination data based on the defective product learning model in which the heterogeneous object selected by the third selection unit has been learned. It is an item inspection device.

[0007] According to the configuration of [1] above, by learning about defective products that can be predicted in advance, the inspection processing unit using the defective product learning model can accurately determine that they are defective, and about defective products that are unexpected and have not been learned, the inspection processing unit using the non-defective product learning model can determine that they are defective, thereby improving inspection accuracy. According to the configuration [2] above, if there is an inspection processing unit with poor accuracy, the judgment data from that inspection processing unit can be excluded from the overall judgment, allowing for more accurate inspection. Furthermore, inspection processing units that are not selected do not need to perform inspection processing, reducing the processing load and allowing for faster inspection. According to the configuration [3] above, by selecting a good product learning model or a defective product learning model that is suited to the photographing conditions and the type of article W, it is possible to perform inspection with even greater accuracy. According to the configuration [4] above, unexpected defective products that could not be determined as defective even by the inspection processing unit using the good product learning model can be determined as defective by the inspection processing unit that performs image processing using a predetermined image processing algorithm, thereby further improving inspection accuracy. According to the configuration [5] above, it is possible to set rules for overall judgment that are suited to the accuracy of each of the plurality of inspection processing units, thereby further improving the inspection accuracy. According to the configuration [6] above, the part of the captured image in which the article is not captured is not input to the inspection processing unit, so that the inspection can be performed with even higher accuracy. According to the configuration [7] above, it is possible to extract the part where the article is imaged with high precision. According to the configuration of [8] above, if there is a defective product learning model with poor accuracy, that defective product learning model can be removed, allowing for more accurate inspection. In addition, there is no need to process the defective product learning model corresponding to the unselected heterogeneous object, reducing the processing load and enabling faster inspection. [Effects of the Invention]

[0008] The article inspection device according to the present invention has the effect of improving inspection accuracy.

[0009] The present invention has been briefly described above. The details of the present invention will become clearer by reading the following detailed description of the invention (hereinafter referred to as "embodiments") with reference to the accompanying drawings. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an article inspection device according to the first embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing a learning image to be learned by the non-defective product learning model shown in FIG. [Figure 3] FIG. 3 is a diagram showing learning images and learning labels used for learning by the defective product learning model shown in FIG. [Figure 4] FIG. 4 is a diagram showing a driving screen displayed on the display operation unit shown in FIG. [Figure 5] FIG. 5 is an explanatory diagram for explaining the effect of the article inspection device shown in FIG. [Figure 6] FIG. 6 is a block diagram of an article inspection device according to the second embodiment of the present invention. [Figure 7] FIG. 7 is a block diagram showing the configuration of each defective product learning model shown in FIG. [Figure 8] FIG. 8 is a diagram showing a stop screen displayed on the display operation unit shown in FIG. [Figure 9] FIG. 9 is a block diagram of an article inspection device according to the third embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing learning images, learning labels, and extracted images to be learned by a learning model used in the image extraction unit shown in FIG. [Figure 11] FIG. 11 is a diagram showing learning images used for learning by the non-defective product learning model shown in FIG. [Figure 12]FIG. 12 is a diagram showing learning images and learning labels used for learning by the defective product learning model shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0012] (First embodiment) First, an article inspection device 1 according to a first embodiment will be described. The article inspection device 1 shown in Fig. 1 is a device that inspects the quality (condition) of articles W transported by a conveyor belt 10. In this embodiment, the articles W are "agricultural products of similar shape, such as beans or potatoes," which are transported loose and not packaged. In this embodiment, the article inspection device 1 determines whether a plurality of articles W contains any foreign matter F (such as stones of the same size as the transported agricultural products) as a good product, and whether a plurality of articles W contains any foreign matter F as a defective product.

[0013] As shown in FIG. 1, the article inspection device 1 includes a control unit 2 and a display operation unit 3. The control unit 2 is composed of a computer that controls each unit to inspect the article W. The control unit 2 includes an image storage unit 21, a plurality of inspection processing units 221 to 223, and an overall judgment unit 23. The image storage unit 21 stores images of the article W captured by the imaging unit 11. The imaging unit 11 sequentially captures images of the article W that are sequentially transported by the conveyor belt 10. The images captured by the imaging unit 11 are color images.

[0014] When a captured image is input, each of the plurality of inspection processing units 221 to 223 performs an inspection process on the input captured image, and outputs determination data D1 to D3 of the item captured in the captured image. The inspection processing units 221 to 223 perform inspection processes different from one another.

[0015] In this embodiment, the inspection processing unit 221 outputs the judgment data D1 using a good product learning model 241 that has been trained using only images of good products captured from good products of the article W. In this embodiment, the good product learning model 241 is a learning model generated by machine learning using a well-known neural network structure consisting of an input layer, a hidden layer, and an output layer.

[0016] Next, the learning phase of the good product learning model 241 will be described with reference to Fig. 2. In the learning phase, only good product images (see Fig. 2) that are not contaminated with foreign matter F are used as learning images to train the good product learning model 241. In this embodiment, the good product learning model 241 inputs, for example, about 1000 of the learning images into the good product learning model 241 and adjusts the weights between layers. As a result of this learning, in the inference phase, when a captured image is input to the input layer of the good product learning model 241, the degree to which each pixel of the captured image differs from a good product is output from the output layer.

[0017] When the output of the result of applying the good product learning model 241 to the captured image shows that the degree of difference from a good product is equal to or greater than the judgment value and there are pixels that are not the conveyor belt 10 in the background of the item W, the inspection processing unit 221 outputs an NG judgment indicating the presence of a foreign object F as the judgment data D1. In contrast, when the degree of difference from the good product is equal to or greater than the judgment value and there are no pixels that are not the conveyor belt 10 in the background, the inspection processing unit 221 outputs an OK judgment indicating a good product as the judgment data D1.

[0018] In this embodiment, the inspection processing unit 222 outputs the judgment data D2 using a defective product learning model 242 that has been trained using only defective product images obtained by photographing defective products that have been contaminated with a foreign substance F. In this embodiment, the defective product learning model 242 is a learning model generated by machine learning using a well-known neural network configured to include an input layer, a hidden layer, and an output layer.

[0019] Next, the learning phase of the defective product learning model 242 will be described with reference to FIG. 3. In the learning phase, the defective product learning model 242 is trained using only defective product images containing foreign matter F (see FIG. 3(A)) as learning images and the coordinates of a box enclosing the foreign matter F in the defective product image (see FIG. 3(B)) as learning labels. In this embodiment, only stones of the same size as agricultural crops are trained as the foreign matter F. These training images and training labels, for example, about 1,000 images, are input into the defective product learning model 242, and the weights between layers are adjusted. Through this training, in the inference phase, when a captured image is input to the input layer of the defective product learning model 242, the output layer outputs the coordinates of a box enclosing an area that appears to be foreign matter F in the captured image and its reliability, as shown in FIG. 3(B).

[0020] When the inspection processing unit 222 detects a foreign substance F with a reliability equal to or greater than the judgment value in the output of the result of applying the defective product learning model 242 to the captured image, it outputs an NG judgment indicating the presence of a foreign substance F as judgment data D2. On the other hand, when the inspection processing unit 222 detects no foreign substance F with a reliability equal to or greater than the judgment value, it outputs an OK judgment indicating a non-defective product as judgment data D2.

[0021] The inspection processing unit 223 does not use a learning model, but rather uses a predetermined image processing algorithm 243 on the input captured image, and outputs judgment data D3 based on a comparison of the results of image processing with preset reference conditions. In this embodiment, the image processing by the image processing algorithm 243 is configured, for example, with an image processing filter that emphasizes features that are different from those of a non-defective product. For example, if the results of the image processing match the reference conditions, the inspection processing unit 223 outputs an OK judgment indicating a non-defective product as the judgment data D3. On the other hand, if the results do not match the reference conditions, the inspection processing unit 223 outputs an NG judgment indicating the presence of a foreign substance F as the judgment data D3.

[0022] The overall judgment unit 23 makes an overall judgment as to whether the article W captured in the captured image is good or bad based on the judgment data D1 to D3 output from the plurality of inspection processing units 221 to 223. In this embodiment, the overall judgment unit 23 outputs an NG judgment as the overall judgment when even one of the inspection processing units 221 to 223 outputs an NG judgment, and outputs an OK judgment as the overall judgment when all of the inspection processing units 221 to 223 output an OK judgment.

[0023] In this embodiment, the display operation unit 3 is composed of a touch panel and functions as a display unit and an operation unit. The display operation unit 3 displays the in-operation screen shown in Fig. 4. The in-operation screen is a screen that is displayed while the item inspection device 1 is operating and inspecting an item W. As shown in Fig. 4, the in-operation screen includes an inspection status display area R1, a common information display area R2, a captured image display area R3, an inspection information display area R4, and an operation button area R5.

[0024] The inspection status display area R1 displays the inspection status (whether the machine is in operation, where the transport and inspection of the item W is being carried out, or is stopped, where the transport and inspection of the item W is stopped). The common information display area R2 displays common information such as the date and time. The captured image display area R3 displays the captured image captured by the imaging unit 11. A frame surrounding the pixel (position) determined to be a foreign object F by the overall determination unit 23 is superimposed on the captured image.

[0025] In the inspection information display area R4, the judgment result by the overall judgment unit 23 and the judgment result for each of the inspection processing units 221 to 223 are displayed. In this embodiment, the number of items judged to be foreign matter F by the overall judgment unit 23 (NG number) and the number of items judged to be non-defective (OK number) are displayed as the judgment result by the overall judgment unit 23. In addition, the number of items judged to be foreign matter F for each of the inspection processing units 221 to 223 (NG number) is displayed as the judgment result for each of the inspection processing units 221 to 223. In the example shown in FIG. 4, "Inspection A" indicates the inspection process by the inspection processing unit 221 using the non-defective product learning model 241, "Inspection B" indicates the inspection process by the inspection processing unit 222 using the defective product learning model 242, and "Inspection C" indicates the inspection process by the inspection processing unit 223 using the image processing algorithm 243.

[0026] Next, the effect of the above-described product inspection device 1 will be described with reference to Fig. 5. Consider a case where the captured images P1 to P3 shown in Fig. 5 are inspected by the inspection processing units 221 to 223. The captured image P1 captures foreign objects F that have not been trained by the defective product learning model 242, such as a plastic bottle cap, a leaf, and individually wrapped candy. The captured image P2 captures a stone that is a foreign object F that has been trained by the defective product learning model 242. The captured image P3 captures a foreign object F that has not been trained by the defective product learning model 242, such as a fragment F5 from a packaging bag.

[0027] The inspection processing unit 222 (inspection B) using the defective product learning model 242 can accurately determine (NG judgment) that the captured image P2, which is a photograph of a stone that has been learned, contains a foreign substance F. However, there is a high possibility that inspection B will erroneously determine (OK judgment) that the captured images P1 and P3, which are photographs of a foreign substance F that has not been learned, contain a non-defective product that does not contain the foreign substance F.

[0028] The inspection processing unit 221 (inspection A) using the good product learning model 241 does not have a very high accuracy in detecting foreign objects F, but it may be able to judge an image P1 containing a foreign object F that has not been learned by the defective product learning model 242 as NG.

[0029] The inspection processing unit 223 (C inspection) using a predetermined image processing algorithm 243 can make an NG judgment for an image P3 in which a foreign object F that does not match the above-mentioned reference conditions is captured among foreign objects F that have not been learned by the defective product learning model 242.

[0030] In this embodiment, the comprehensive judgment unit 23 makes a comprehensive judgment on the quality of the item W based on judgment data D1 and D2 from the inspection processing unit 221 (inspection A) using the good product learning model 241 and the inspection processing unit 222 (inspection B) using the defective product learning model 242. As a result, by learning, foreign objects F that can be expected in advance, such as stones, can be accurately judged as NG in inspection B, while foreign objects F that are unexpected and have not been learned may be judged as NG in inspection A. This makes it possible to improve inspection accuracy.

[0031] Furthermore, in this embodiment, the comprehensive judgment unit 23 makes a comprehensive judgment on the quality of the article W based on the judgment data D1, D2 as well as the judgment data D3 from the inspection processing unit 223 using a predetermined image processing algorithm 243. As a result, an unexpected foreign object F that could not be judged as NG in the A inspection may be judged as NG in the C inspection, thereby further improving the inspection accuracy.

[0032] In the first embodiment described above, the overall judgment unit 23 outputs an NG judgment as the overall judgment when at least one of the inspection processing units 221 to 223 outputs an NG judgment, and outputs an OK judgment as the overall judgment in other cases, but this is not limited to this. For example, the overall judgment unit 23 may output an NG judgment as the overall judgment only when the inspection processing unit 222 outputs an NG judgment or when both the inspection processing units 221 and 223 output NG judgments, and may output an OK judgment in other cases. Furthermore, the overall judgment unit 23 may output an NG judgment only when all of the inspection processing units 221 to 223 output NG judgments. In this way, various judgment rules for the overall judgment unit 23 can be considered.

[0033] It is also conceivable that a judgment rule setting area (judgment rule setting area) for setting judgment rules for the overall judgment unit 23 is displayed on the display operation unit 3, and the overall judgment unit 23 judges pass / fail according to the rules set by operating the judgment rule setting area. This makes it possible to set rules for overall judgment that match the accuracy of the inspection processing units 221 to 223, thereby further improving the inspection accuracy.

[0034] (Second embodiment) Next, an article inspection device 1B of a second embodiment will be described with reference to Fig. 6. In Fig. 6, parts equivalent to those of the article inspection device 1 of the first embodiment described above are given the same reference numerals, and detailed description thereof will be omitted. As with the first embodiment, the article inspection device 1B shown in Fig. 6 assumes that the article W is an "agricultural product," and determines that a state in which no foreign matter F is mixed in is a good product, and a state in which a foreign matter F is mixed in is a defective product.

[0035] 6, the article inspection device 1B includes a control unit 2B and a display operation unit 3B. The control unit 2B is composed of a computer that controls each unit to inspect the article W. The control unit 2B includes an image storage unit 21, a plurality of inspection processing units 221B, 222B, 223, and an overall judgment unit 23B.

[0036] The difference between the inspection processing unit 221 of the first embodiment and the inspection processing unit 221B of the second embodiment is that a plurality of good product learning models 2411 to 241n (n is any integer) is provided, and the inspection processing unit 221B outputs judgment data D1 using one or more good product learning models selected by the display operation unit 3B from the plurality of good product learning models 2411 to 241n.

[0037] The non-defective product learning models 2411-241n are separate learning models trained on different images. For example, non-defective product learning models 2411-241n may be provided for each shooting condition (such as the type of imaging unit 11 or the type of lighting) or each type of non-defective product. If the item W is a "crop" as in this embodiment, there are three types of non-defective products: crop C1, crop C2, which is a different variety from crop C1, and a mixture of crops C1 and C2. In this case, the non-defective product learning models 2411-241n may include a learning model trained on captured images of crop C1 as learning images, a learning model trained on captured images of crop C2 as learning images, and a learning model trained on captured images of a mixture of crops C1 and C2 as learning images.

[0038] The inspection processing unit 221B applies one or more selected from the plurality of non-defective product learning models 2411 to 241n to the captured image, and outputs the judgment data D1 based on the output of the selected non-defective product learning models 2411 to 241n. The non-selected non-defective product learning models 2411 to 241n are not applied to the captured image.

[0039] Furthermore, the difference between the inspection processing unit 222 of the first embodiment and the inspection processing unit 222B of the second embodiment is that multiple defective product learning models 2421 to 242m (m is any integer) are provided, and the inspection processing unit 222B outputs judgment data D2 using one or more defective product learning models selected from the multiple defective product learning models 2421 to 242m.

[0040] The defective product learning models 2421 to 242m are different learning models that have learned different images. For example, it is possible to provide the defective product learning models 2421 to 242m for each shooting condition (such as the type of imaging unit 11 or the type of lighting) or for each different type of object.

[0041] The inspection processing unit 222B applies one or more selected defective product learning models from the plurality of defective product learning models 2421-242m to the captured image, and outputs judgment data D2 based on the selected defective product learning models 2421-242m. The unselected defective product learning models 2421-242m are not applied to the captured image.

[0042] In this embodiment, as shown in FIG. 7 , the defective product learning models 2421-242m include multiple (13 in this embodiment) heterogeneous object learning models 251-2539 trained separately for each type of foreign object F for each type of inspected object W (crop C1, crop C2, a mixture of crops C1 and C2). In this embodiment, 13 types of foreign objects F1-F13, such as pieces of plastic, pieces of rubber, stones and pebbles of the same size as crops, are trained separately. For example, heterogeneous object learning model 251 is trained using captured images of only foreign object F1 (e.g., pieces of plastic), and heterogeneous object learning model 252 is trained using captured images of only foreign object F2 (e.g., pieces of rubber). Note that multiple types of heterogeneous objects F may be trained together to create a single defective product learning model for the multiple types of heterogeneous objects F.

[0043] The inspection processing unit 222B applies the heterogeneous object learning model 251-2539 selected by the display operation unit 3B from among the multiple heterogeneous object learning models 251-2539 included in the selected defective product learning models 2421-242m to the captured image, and outputs judgment data D2 based on the output from the selected heterogeneous object learning model 251-2539. In this embodiment, the inspection processing unit 222B outputs an NG judgment when the results output from the heterogeneous object learning models 251-2539 include a heterogeneous object F with a reliability equal to or greater than the judgment value.

[0044] The inspection processing unit 223 is the same as that in the first embodiment, and therefore a detailed description thereof will be omitted here.

[0045] Additionally, the overall judgment unit 23B of the second embodiment differs from the overall judgment unit 23 of the first embodiment in that the overall judgment unit 23B of the present embodiment makes an overall judgment from judgment data D1 to D3 output from one or more of the inspection processing units 221B, 222B, 223 selected by operation of the display operation unit 3B. The unselected inspection processing units 221B, 222B, 223 do not receive captured images and do not perform inspection processing.

[0046] The display operation unit 3B displays an operating screen shown in Fig. 4 and a stopped screen shown in Fig. 8. The operating screen has already been explained in the first embodiment, so details will be omitted. The stopped screen is a screen that is displayed while the item inspection device 1B is stopped and not inspecting an item W. As shown in Fig. 8, the stopped screen includes an inspection status display area R1, a common information display area R2, an inspection information display area R4, an operation button area R5, and a setting display area R6.

[0047] The examination status display area R1, common information display area R2, examination information display area R4, and operation button area R5 are the same as the areas displayed on the operating screen shown in FIG. 4, and therefore detailed description thereof will be omitted.

[0048] On the screen during stoppage, a setting display area R6 is displayed in place of the captured image display area R3 on the screen during operation. The setting display area R6 has a foreign substance inspection setting area R61 as a first selection area for selecting one or more of the plurality of inspection processing units 221B, 222B, 223. The foreign substance inspection setting area R61 displays check boxes corresponding to inspection A (inspection processing unit 221B), inspection B (inspection processing unit 222B), and inspection C (inspection processing unit 223). Touching each check box to check it selects the inspection processing unit 221B, 222B, 223 corresponding to the checked check box.

[0049] The setting display area R6 has an AI model setting area R62 as a second selection section for selecting one or more of the multiple good product learning models 2411-241n and one or more of the multiple defective product learning models 2421-242m. The AI ​​model setting area R62 displays drop-down lists corresponding to model A (inspection processing unit 221B) and model B (inspection processing unit 222B). The drop-down list for model A allows selection of the good product learning models 2411-241n. The drop-down list for model B allows selection of the defective product learning models 2421-242m. The selection items in the drop-down lists are individual models or combinations of models, and selecting one selection item allows selection of one or more learning models. Examples of model combinations include combinations of defective product learning models that can be created by dividing the heterogeneous object learning models 251-2539 by type (good product learning models) of the object W to be inspected, or combinations of any defective product learning models from the defective product learning models 2421-242m. Note that the learning model selection in the AI ​​model setting area R62 may be limited to either the good product learning model or the defective product learning model.

[0050] The setting display area R6 has a mask setting area R63 as a third selection section for selecting the foreign objects F1 to F13. Check boxes corresponding to the foreign objects F1 to F13 are displayed in the mask setting area R63. Touching each check box to check it selects the foreign object F1 to F13 corresponding to the checked check box. Note that because of the mask setting here, the foreign objects F1 to F13 that are unchecked correspond to the selection in the third selection section, and the foreign object learning models 251 to 2539 that have learned the unchecked foreign objects F1 to F13 are used in the inspection processing unit 222B.

[0051] According to the above-described embodiment, one or more of the inspection processing units 221B, 222B, and 223 can be selected by operating the foreign object inspection setting area R61, and the overall judgment unit 23B makes an overall judgment on the quality of the object W captured in the captured image based on the judgment data D1-D3 output from the selected inspection processing units 221B, 222B, and 223. Depending on the learning content and the object W, some of the inspection processing units 221B, 222B, and 223 may have poor accuracy. In this case, the judgment data D1-D3 of the poorly accurate inspection processing units 221B, 222B, and 223 can be excluded from the overall judgment, allowing for more accurate inspection. Furthermore, the inspection processing units 221B, 222B, and 223 that are not selected do not need to perform inspection processing, reducing the processing load and allowing for faster inspection.

[0052] According to the above-described embodiment, the AI ​​model setting area R62 allows one or more learning models to be selected from the plurality of non-defective product learning models 2411-241n or the plurality of defective product learning models 2421-242m, and the inspection processing units 221B and 222B output the judgment data D1 and D2 using the selected non-defective product learning models 2411-241n and defective product learning models 2421-242m. By selecting the non-defective product learning models 2411-241n or defective product learning models 2421-242m that are suitable for the imaging conditions, the type of article W, etc., inspection can be performed with even greater accuracy.

[0053] According to the above-described embodiment, multiple types of heterogeneous objects F1 to F13 can be selected using the mask setting area R63, and the inspection processing unit 222B outputs judgment data D2 using the heterogeneous object learning models 251 to 2539 corresponding to the heterogeneous objects F1 to F13 selected using the mask setting area R63 for the heterogeneous object learning models 251 to 2539 to be inspected (for example, when a defective product learning model or no defective product learning model is selected using the AI ​​model setting area R62). Depending on the learning content, some heterogeneous object learning models 251 to 2539 may have low accuracy. In this case, the heterogeneous object learning models 251 to 2539 with low accuracy can be removed, allowing for more accurate inspection. Furthermore, there is no need to process the heterogeneous object learning models 251 to 2539 that are not selected, reducing the processing load and enabling faster inspection.

[0054] (Third embodiment) Next, an article inspection apparatus 1C of a third embodiment will be described with reference to Fig. 9. In Fig. 9, parts equivalent to those of the article inspection apparatus 1 of the first embodiment described above are given the same reference numerals, and detailed description thereof will be omitted. In this embodiment, as shown in Fig. 10, articles W are packaged in transparent packaging bags 12, two at a time. In this embodiment, the article inspection apparatus 1C determines an article W to be a good product if it has the correct size and shape and is free of defects such as stains, cracks, discoloration, or the presence of foreign matter inside, and determines an article W to be a defective product if it has a defect.

[0055] 9, the article inspection device 1C includes a control unit 2C and a display operation unit 3. The control unit 2C is composed of a computer that controls each unit to inspect the article W. The control unit 2C includes an image storage unit 21, image extraction units 261, 262, 263, a plurality of inspection processing units 221C, 222C, 223, and a comprehensive judgment unit 23.

[0056] The image extraction units 261, 262, and 263 extract the portion in which the article W is captured from the captured image stored in the image storage unit 21, and output extracted images P4 and P5 (see FIG. 10(C)). In this embodiment, the image extraction units 261, 262, and 263 extract the captured image in which the article W is captured using learning models 271, 272, and 273 that have been trained. The learning models 271, 272, and 273 are learning models generated by machine learning using a well-known neural network structure consisting of an input layer, a hidden layer, and an output layer.

[0057] Next, the learning phase of the learning models 271, 272, and 273 will be described with reference to FIG. 10. In the learning phase, the learning models 271, 272, and 273 are trained using captured images (see FIG. 10(A)) of the item W in a packaging bag 12 as learning images and the coordinates of a box surrounding the item W in the learning images (see FIG. 10(B)) as learning labels. These learning images and learning labels, for example, about 1,000, are input into the learning models 271, 272, and 273, and the weights between layers are adjusted. The learning images include not only images of the item W that are good, but also images of the item W that are defective. Through this learning, when a captured image is input to the input layer of the learning models 271, 272, and 273 in the inference phase, the output layer outputs the coordinates of a box surrounding an area that appears to be the item W in the captured image and its reliability, as shown in FIG. 10(B).

[0058] When an item W having a reliability equal to or higher than a judgment value is found in the output of the result of applying the learning models 271, 272, 273 to the captured image, the image extraction units 261, 262, 263 cut out the coordinates of the box and output them as extracted images P4, P5. In this embodiment, two items W are packed in one packaging bag 12, so two extracted images P4, P5 are output.

[0059] 9, the image extraction unit 261 inputs the two extracted extraction images P4 and P5 to the inspection processing unit 221C. The image extraction unit 262 inputs the two extracted extraction images P4 and P5 to the inspection processing unit 222C. The image extraction unit 263 inputs the two extracted extraction images P4 and P5 to the inspection processing unit 223.

[0060] The inspection processing unit 221C outputs the judgment data D1 using a good product learning model 241C that has been trained using only images of good products captured from good products of the article W. In this embodiment, the good product learning model 241C is a learning model generated by machine learning using a well-known neural network structure consisting of an input layer, a hidden layer, and an output layer.

[0061] Next, the learning phase of the good product learning model 241C will be described with reference to Figure 11. In the learning phase, the good product learning model 241C is trained using only the good product images (see Figure 11) of the extracted images as learning images. For example, about 1000 of these learning images are input to the good product learning model 241C, and the weights between layers are adjusted. Through this learning, when the extracted images P4 and P5 are input to the input layer of the good product learning model 241C in the inference phase, the degree to which each pixel of the extracted images differs from a good product is output from the output layer.

[0062] When the output of the result of applying the good product learning model 241C to the extracted image contains pixels whose degree of difference from a good product is equal to or greater than the judgment value, the inspection processing unit 221C outputs an NG judgment indicating a defective product as the judgment data D1. On the other hand, when there are no pixels whose degree of difference from a good product is equal to or greater than the judgment value, the inspection processing unit 221C outputs an OK judgment indicating a good product as the judgment data D1.

[0063] In this embodiment, the inspection processing unit 222C outputs the judgment data D2 using a defective product learning model 242C that has been trained using only images of defective products. In this embodiment, the defective product learning model 242C is a learning model generated by machine learning using a well-known neural network configured with an input layer, a hidden layer, and an output layer.

[0064] Next, the learning phase of the defective product learning model 242C will be described with reference to Figure 12. In the learning phase, only defective product images containing defective parts (blackened parts in the figure) are used as learning images as shown in Figure 12(A), and the coordinates of boxes surrounding the defective parts in the learning images are used as learning labels as shown in Figure 12(B) to train the defective product learning model 242C. These learning images and learning labels, for example, about 1,000 images, are input into the defective product learning model 242, and the weights between layers are adjusted. Through this learning, when a captured image is input to the input layer of the defective product learning model 242C in the inference phase, the output layer outputs the coordinates of a box surrounding an area that appears to be a defective part in the extracted image as shown in Figure 12(B) and its reliability.

[0065] When the output of the result of applying the defective product learning model 242C to the extracted image contains a defective part with a reliability equal to or greater than the judgment value, the inspection processing unit 222C outputs an NG judgment indicating a defective product as judgment data D2. On the other hand, when there is no defective part with a reliability equal to or greater than the judgment value, the inspection processing unit 222C outputs an OK judgment indicating a non-defective product as judgment data D2.

[0066] The inspection processing unit 223 is the same as that in the first embodiment except that the input image is an extracted image, and therefore a detailed description thereof will be omitted here.

[0067] According to the third embodiment described above, the image extraction units 261, 262 output extracted images P4, P5 obtained by extracting portions of the captured image in which the article W is captured, and the extracted images P4, P5 are input to the inspection processing units 221C, 222C. As a result, portions of the captured image in which the article W is not captured are not input to the inspection processing units 221C, 222C, allowing for more accurate inspection.

[0068] According to the above-described third embodiment, the image extraction units 261, 262, and 263 perform extraction using the learning models 271, 272, and 273. This makes it possible to extract with high accuracy the portion in which the image of the article W is captured.

[0069] The present invention is not limited to the above-described embodiments, and can be appropriately modified, improved, etc. Furthermore, the material, shape, size, number, location, etc. of each component in the above-described embodiments are arbitrary and not limited as long as they can achieve the present invention.

[0070] In the above-described embodiment, the captured image is a color image, but the captured image is not limited to this. The captured image may be a black and white image, an X-ray image, an infrared image, or the like.

[0071] According to the above-described embodiment, the article inspection apparatuses 1, 1B, and 1C each have three inspection processing units 221-223, 221B, 222B, and 223, and 221C, 222C, and 223, respectively, but this is not limited to this. The article inspection apparatuses 1, 1B, and 1C may have only two inspection processing units: inspection processing units 221, 221B, and 221C that use non-defective product learning models 241, 2411-241n, and inspection processing units 222, 222B, and 222C that use defective product learning models 242, 2421-242m, and do not necessarily have the inspection processing unit 223. Furthermore, the article inspection apparatuses 1, 1B, and 1C may have four or more inspection processing units.

[0072] According to the third embodiment described above, the article inspection apparatus 1C has three image extraction units 261, 262, and 263 and three learning models 271, 272, and 273, but is not limited to this. The article inspection apparatus 1C may have one image extraction unit 261 and one learning model 271, and may input extracted images P4 and P5 extracted by the image extraction unit 261 to all of the inspection processing units 221C, 222C, and 223.

[0073] Here, the features of the above-described embodiment of the article inspection device according to the present invention will be briefly summarized and listed below in [1] to [8]. [1] an image storage unit (21) that stores an image of the article (W) captured by the imaging unit; a plurality of inspection processing units (221 to 223, 221B, 222B, 221C, 222C) that perform inspection processing on the captured images and output judgment data indicating the quality state of the items captured in the captured images; and a comprehensive judgment unit (23, 23B) that comprehensively judges the quality state of the item captured in the captured image based on the judgment data output from the plurality of inspection processing units. An article inspection device (1, 1B, 1C), One of the plurality of inspection processing units outputs the judgment data using a non-defective product learning model (241, 2411 to 241n) that has learned only non-defective product images obtained by capturing images of non-defective products of the article; One of the plurality of inspection processing units outputs the judgment data using a defective product learning model (242, 2421 to 242m) that has learned only defective product images obtained by capturing images of defective products of the article. Article inspection equipment. [2] In the article inspection device according to [1], a first selection unit (R61) that selects one or more of the plurality of inspection processing units; the comprehensive judgment unit comprehensively judges the quality state of the item captured in the captured image based on the judgment data output from the inspection processing unit selected by the first selection unit. Article inspection equipment. [3] In the article inspection device according to [1], The good product learning model or the defective product learning model is provided with a plurality of learning models with different learning contents, a second selection unit (R62) that selects one or more of the plurality of good product learning models or the plurality of defective product learning models; the comprehensive judgment unit comprehensively judges the quality state of the product captured in the captured image based on the judgment data output from the good product learning model or the defective product learning model selected by the second selection unit. Article inspection equipment. [4] In the article inspection device according to [1], One of the plurality of inspection processing units performs image processing on the input captured image using a predetermined image processing algorithm (243), and outputs the determination data. Article inspection equipment. [5] In the article inspection device according to [1], a judgment rule setting unit that sets rules for judgment by the comprehensive judgment unit; the comprehensive judgment unit makes the comprehensive judgment on the quality state in accordance with the set rule. Article inspection equipment. [6] In the article inspection device according to [1], An image extraction unit (261, 262, 263) that extracts a portion in which the article is captured from the captured image, The inspection processing unit receives an extracted image of the portion extracted by the image extraction unit. Article inspection equipment. [7] [6] The article inspection device according to [6], The image extraction unit inputs the captured image and extracts the captured image of the article using a learning model (271, 272, 273) that has been trained. Article inspection equipment. [8] In the article inspection device according to [1], a plurality of the defective product learning models are provided, each of which is trained separately for each type of heterogeneous object; a third selection unit (R63) that selects the type of the foreign substance; the inspection processing unit outputs the determination data based on the defective product learning model in which the heterogeneous object selected by the third selection unit has been learned. Article inspection equipment. [Explanation of symbols]

[0074] 1, 1B, 1C Item inspection equipment 21 Image storage unit 23,23B General Judging Section 221~223, 221B, 222B, 221C, 222C Inspection processing section 221B~223B Inspection processing section 221C~223C Inspection Processing Section 241,2411~241n Good product learning model 242,2421~242m Defective product learning model 243 Image Processing Algorithms 261,262,263 Image extraction section 271,272,273 Learning Model W Goods R61 Foreign object inspection setting area (first selection section) R62 AI model setting area (second selection section) R63 Mask setting area (third selection section)

Claims

1. an image storage unit (21) that stores an image of the item (W) captured by the imaging unit; a plurality of inspection processing units (221 to 223, 221B, 222B, 221C, 222C) that perform inspection processing on the captured images and output judgment data indicating the quality state of the items captured in the captured images; and a comprehensive judgment unit (23, 23B) that comprehensively judges the quality state of the item captured in the captured image based on the judgment data output from the plurality of inspection processing units. An article inspection device (1, 1B, 1C), One of the plurality of inspection processing units outputs the judgment data using a non-defective product learning model (241, 2411 to 241n) that has learned only non-defective product images obtained by capturing images of non-defective products of the article; One of the plurality of inspection processing units outputs the judgment data using a defective product learning model (242, 2421 to 242m) that has learned only defective product images obtained by capturing images of defective products of the article. Article inspection equipment.

2. 2. The article inspection device according to claim 1, a first selection unit (R61) that selects one or more of the plurality of inspection processing units; the comprehensive judgment unit comprehensively judges the quality state of the item captured in the captured image based on the judgment data output from the inspection processing unit selected by the first selection unit. Article inspection equipment.

3. 2. The article inspection device according to claim 1, The good product learning model or the defective product learning model is provided with a plurality of learning models with different learning contents, a second selection unit (R62) that selects one or more of the plurality of good product learning models or the plurality of defective product learning models; the comprehensive judgment unit comprehensively judges the quality state of the product captured in the captured image based on the judgment data output from the good product learning model or the defective product learning model selected by the second selection unit. Article inspection equipment.

4. 2. The article inspection device according to claim 1, One of the plurality of inspection processing units performs image processing on the input captured image using a predetermined image processing algorithm (243), and outputs the determination data. Article inspection equipment.

5. 2. The article inspection device according to claim 1, a judgment rule setting unit that sets rules for judgment by the comprehensive judgment unit; the comprehensive judgment unit makes the comprehensive judgment on the quality state in accordance with the set rule. Article inspection equipment.

6. 2. The article inspection device according to claim 1, an image extraction unit (261, 262, 263) that extracts a portion in which the article is captured from the captured image; The inspection processing unit receives an extracted image of the portion extracted by the image extraction unit. Article inspection equipment.

7. 7. The article inspection apparatus according to claim 6, The image extraction unit inputs the captured image and extracts the captured image of the article using a learning model (271, 272, 273) that has been trained. Article inspection equipment.

8. 2. The article inspection device according to claim 1, a plurality of the defective product learning models are provided, each of which is trained separately for each type of heterogeneous object; a third selection unit (R63) that selects the type of the foreign substance; the inspection processing unit outputs the determination data based on the defective product learning model in which the heterogeneous object selected by the third selection unit has been learned. Article inspection equipment.

Images