Article inspection device and model update system

Abstract

To provide an article inspection device and a model update system which can surely prevent deterioration of yield due to erroneous detection when updating a trained model, by allowing for quantitatively grasping deterioration of inspection performance accompanying update of the trained model.SOLUTION: The article inspection device comprises: a sensor 23 which outputs a detection signal for inspecting a quality state of an article W; and an inspection unit 20 which inspects the quality state of the article by applying a trained model generated in advance by training to a prescribed detection signal outputted by the sensor during movement of the article. Further, the article inspection device comprises update means 41 which updates the trained model via a prescribed medium, evaluation means 47 which uses a data set for verification which is not used for training of the trained model, to evaluate the trained model, and output means 49 which outputs an evaluation result obtained from the evaluation means as statistical data related to an inspection result obtained from the inspection unit by the data set for verification, for example, a detection rate indicating that the article is a non-defective or defective product.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an article inspection apparatus and a model update system, and more particularly to an article inspection apparatus that inspects the quality state of an article by applying a learned model to inspection images and sensor signals according to the quality state of the article, and a model update system that supports the update of the learned model.

Background Art

[0002] Recently, an article inspection apparatus has been known that inspects the quality state of an article by applying a machine-learned model, so-called AI (artificial intelligence) model, to inspection images and sensor signals including feature amounts according to the quality state of the article.

[0003] For example, in order to improve the article inspection accuracy, a plurality of images with different input systems are captured under predetermined imaging conditions according to each input system, and image data including a plurality of images of the object to be inspected as a set is acquired and stored in an image storage unit. On the other hand, prior to that, a learned model for inspection determination obtained by machine learning using learning image data acquired under the same imaging conditions as the image data of the object to be inspected stored in the image storage unit is created. Then, using the learned model, the image data of the object to be inspected acquired during the actual inspection is processed pixel by pixel to obtain the degree of quality defect, and the quality state of the object to be inspected is determined by comparing the degree of quality defect with a preset threshold value (see, for example, Patent Document 1).

[0004] Also, based on the learning results of X-ray image data in a plurality of energy bands related to the learning target variety, a pseudo-image generation model capable of pseudo-generating X-ray image data in other energy bands corresponding to the X-ray image data in a predetermined energy band is created. Then, based on the X-ray image data of the object to be inspected acquired during the actual inspection, a pseudo-transmission image in another energy band is created by the pseudo-image generation model, and based on the X-ray image data in the predetermined energy band acquired during the actual inspection and the pseudo-transmission image in another energy band created by the pseudo-image generation model, the quality state of the article is determined (see, for example, Patent Document 2).

Prior Art Documents

Patent Document

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the above - described article inspection apparatus, it may be necessary to update the trained model (hereinafter also referred to as an AI model), for example, by increasing the teacher image data of good and defective articles to be inspected and performing additional learning, or to re - train the trained model due to an increase or decrease in the network layers constituting the trained model or a change in hyperparameters.

[0007] However, due to additional learning or re - training, the inspection performance may deteriorate compared to the currently used AI model. Or, although the additional - learned defective products can be detected, the detection accuracy of defective products that could be detected with high accuracy by the AI model before learning may decrease.

[0008] Therefore, in an actual production line, it is necessary to flow a large number of articles of the inspection target variety through test production using the additional - learned model to evaluate the performance of the additional - learned model, which causes the problem of loss of food and packaging materials due to test production.

[0009] Therefore, an object of the present invention is to provide an article inspection apparatus that can quantitatively grasp the deterioration of inspection performance accompanying the update of the trained model and can surely prevent the deterioration of the yield due to false detection when updating the trained model of the inspection unit, and also to provide its model update system.

Means for Solving the Problems

[0010] [[ID=4I]] In order to achieve the above object, the article inspection apparatus according to the present invention includes: (1) a sensor that outputs a detection signal for inspecting the quality state of an article; and a learning model that has been created in advance by learning, for a predetermined detection signal output by the sensor during the movement of the article, an inspection unit that inspects the quality state of the article by applying the learned model. The article inspection apparatus is further provided with: an update means for updating the learned model via a predetermined medium; an evaluation means for evaluating the learned model using a verification data set that has not been used for learning the learned model; and an output means for outputting the evaluation result by the evaluation means as statistical data related to the inspection result by the inspection unit using the verification data set.

[0011] With such a configuration, in the present invention, when the update means updates the learned model applied to the inspection unit via a predetermined medium, such as a transmission medium such as a network or a storage medium such as a USB, the learned models before and after the update can be respectively evaluated by the evaluation means using unknown verification data sets that have not been used for their respective learning. Also, for the learned models (AI (artificial intelligence) models) before and after the update, the respective evaluation results are output by the output means as statistical data (for example, the detection rate (= number of detections / number of inspected articles) indicating that the article is a non-defective or defective product) related to the inspection result by the inspection unit using the same verification data set. Therefore, based on each evaluation result, the old and new learned models before and after the update can be accurately compared and evaluated using the same verification data set. When partially changing the learning algorithm to perform relearning of the entire model or increasing the learning image data of non-defective or defective products for additional learning, the influence of such relearning or additional learning on the inspection performance can be accurately grasped, and a decrease in inspection performance due to the update of the learned model can be reliably prevented in advance.

[0012] Note that the verification dataset mentioned here is a collection of data for specific purposes processed by a program or model with a learning algorithm, but it is not the training data used for weight updates etc. during machine learning for model construction. Instead, it is an unknown dataset that can evaluate the classification and regression prediction performance of a trained model, and it includes at least test data among the test data for evaluating the trained model after learning and the verification data that can be used for calculating accuracy for adjusting hyperparameters in stages such as re - learning.

[0013] In a preferred embodiment of the present invention, it can be configured to further include (2) a storage unit that stores a plurality of verification datasets not used for training the trained model. In this case, a plurality of verification datasets corresponding to a plurality of varieties to be inspected can be stored.

[0014] In a preferred embodiment of the present invention, it can be configured that (3) the output means outputs the evaluation result by the evaluation means as screen display information equivalent to the detection rate of whether the article is a good product or a defective product. In this case, the influence on the inspection performance of re - learning and additional learning can be easily and accurately grasped.

[0015] In a preferred embodiment of the present invention, when the trained model is updated by the (4) update means, the evaluation means can be configured to automatically verify the trained model using the verification dataset. In this way, when the trained model of the inspection unit is updated, the old and new trained models before and after the update can be automatically verified using the verification dataset, and the old and new trained models can be compared and evaluated in a timely manner.

[0016] In a preferred embodiment of the present invention, (5) it is possible to adopt a configuration including a recording unit that records the evaluation result by the automatic verification of the evaluation unit on the condition that the evaluation result is outside a preset evaluation criterion. In this case, when the influence on the inspection performance of relearning or additional learning is not preferable, even if the relearning or additional learning at that stage is terminated early, the evaluation result at that time is recorded, so that the influence on the inspection performance can be more accurately evaluated in subsequent model update operations.

[0017] In a preferred embodiment of the present invention, (6) the storage unit can be configured to store the verification data set in an updatable manner. By doing so, verification data with high user requirements in the actual inspection results, such as defective product image data to be detected near the lower limit accuracy required for inspection performance and good product image data that should not be misjudged as defective, can be added and updated to the verification data set.

[0018] The model update system according to the present invention is, for achieving the above object, (7) a model update system communicably connected to an article inspection apparatus provided with a sensor that outputs a detection signal for inspecting the quality state of an article, an inspection unit that inspects the quality state of the article by applying a learned model created in advance by learning to a predetermined detection signal output by the sensor during the movement of the article, and an update unit that updates the learned model via a predetermined medium, the model update system including an evaluation unit that evaluates the learned model before and after the update by the update unit using a verification data set not used in the learning of the learned model, and an output unit that outputs the evaluation result by the evaluation unit as statistical data related to the inspection result by the inspection unit using the verification data set.

[0019] With such a configuration, in the model update system of the present invention, when the update means in the article inspection apparatus updates the learned model for inspection unit application via a predetermined medium, the learned models before and after the update can be evaluated respectively by the evaluation means using unknown verification data sets that have not been used for their respective learning. Further, regarding the learned models before and after the update, the respective evaluation results will be output by the output means as statistical data related to the inspection results in the inspection unit by the same verification data set. Therefore, based on the respective evaluation results, the old and new learned models before and after the update can be accurately compared and evaluated using the same verification data set. When partially changing the learning algorithm to perform relearning of the entire model or increasing the learning image data of good or defective products for additional learning, it is possible to accurately grasp the impact of such relearning or additional learning on the inspection performance, and reliably prevent a decrease in inspection performance due to the update of the learned model in advance.

[0020] Note that the article inspection apparatus referred to in the present invention may irradiate an article with X-rays to obtain an X-ray inspection image, and apply a predetermined image processing algorithm to the X-ray inspection image to perform X-ray inspection on the quality state of the article. However, it may also use an inspection image of a camera image using any electromagnetic wave other than X-rays, for example, NIR (near-infrared). Further, the article inspection apparatus referred to in the present invention is not limited to a camera image, and is also applicable when a machine-learned model is used for the inspection determination of an article inspection apparatus such as a metal detector that detects the influence of an article passing through the article inspection area on the magnetic field in the article inspection area as a signal waveform.

Advantages of the Invention

[0021] According to the present invention, it is possible to provide an article inspection apparatus and a model update system that can quantitatively grasp the deterioration of inspection performance accompanying the update of a learned model, and reliably prevent the deterioration of yield due to false detection when updating the learned model of the inspection unit in advance.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0023] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

[0024] (One Embodiment) FIGS. 1 to 9 show an article inspection apparatus according to one embodiment of the present invention.

[0025] First, the configuration will be described.

[0026] As shown in FIG. 1, the article inspection apparatus 1 of the present embodiment includes a conveyance unit 10 that conveys an article W to be inspected, an inspection unit 20 that inspects the article W during conveyance, a control unit 30 for main control including control of these conveyance unit 10 and inspection unit 20, and a learning model update control unit 40 that executes a predetermined update process on a learned model (details will be described later) that constitutes a part of the control unit 30. And this article inspection apparatus 1 irradiates the article W conveyed by the conveyor of the conveyance unit 10 with X-rays by the inspection unit 20 and detects image data corresponding to the transmitted X-ray dose distribution, and inspects the quality state of the article W based on the image data. Here, the quality state referred to here is the suitability of the quality and physical quantities required for the article W as a product, such as the presence or absence of foreign matter mixed in, the presence or absence of defective products, the pass or fail of the shape, size, storage state, etc. of the contents, the distribution of density, thickness, volume or mass, etc.

[0027] The conveyance unit 10 is a conveyor that winds a loop-shaped conveyance belt 11 around a plurality of conveyance rollers 12 and 13 and can sequentially convey the article W in the right direction in FIG. 1 by the upper running section 11a of the conveyance belt 11, and is supported by a housing (not shown).

[0028] The inspection unit 20 is an X-ray inspection unit that includes an X-ray generator 21 (X-ray source) that generates X-rays in a predetermined energy band that penetrates the article W conveyed by the conveyance unit 10, and an X-ray detector 23 disposed directly below the upper running section 11a of the conveyor belt 11.

[0029] The X-ray generator 21 generates X-rays with a wavelength and intensity corresponding to the tube current and tube voltage in a known X-ray tube 22, and through the X-ray window portion of an outer casing (not shown in detail), irradiates the article W within a predetermined inspection section Zx on the conveyor belt 11 with fan beam-shaped X-rays directed in the main observation direction orthogonal to the article conveyance direction of the conveyance unit 10.

[0030] This X-ray detector 23 is, although not shown in detail, for example, an X-ray line sensor camera configured by arranging detection elements each composed of a scintillator which is a phosphor and a photodiode or a charge coupled device in an array at a predetermined pitch in the width direction of the conveyance path of the conveyance unit 10 to perform X-ray detection at a predetermined resolution, and is disposed at a predetermined position in the conveyance direction corresponding to the X-ray irradiation position from the X-ray generator 21.

[0031] That is, the X-ray detector 23 detects the X-rays irradiated from the X-ray generator 21 and transmitted through the article W for each predetermined transmission region corresponding to the detection element, converts them into electrical signals corresponding to the transmission amount of the X-rays, and can output X-ray detection signals for generating an X-ray transmission image with the direction in which the X-rays are transmitted as the observation direction. Here, it is assumed that the X-rays irradiated from the X-ray generator 21 or the X-rays detected by the X-ray detector 23 have a certain radiation quality (energy, wavelength) specified according to the quality of the article W. However, by using a plurality of types of X-rays with different radiation qualities, so-called dual energy or multi-energy X-ray images may be generated.

[0032] The control unit 30 includes a conveying control means for controlling the conveying speed and conveying interval of the item W by the conveying belt 11 in the conveying unit 10, and an inspection control means for controlling the X-ray irradiation intensity and irradiation period in the inspection unit 20, and for controlling the X-ray detection period of the X-ray line sensor of the X-ray detector 23 and the detection period for each item W according to the conveying speed of the item W, but detailed illustrations are omitted.

[0033] The control unit 30 also has an inspection image memory unit 31 that sequentially takes in X-ray detection signals from the X-ray detector 23 at predetermined intervals, acquires X-ray transmission images of each item W, and outputs the image data; an image processing unit 32 that takes in the image data output from the inspection image memory unit 31 and performs image analysis processing such as predetermined filter processing (including preprocessing) that can extract image features and feature measurement to determine the feature amounts of the extracted image features; an inspection and judgment unit 33 that performs judgment processing to determine whether or not the item W is in a predetermined quality state based on the feature amount data extracted and measured by the image processing unit 32, such as judgment processing for the presence or absence of foreign matter, the presence or absence of missing parts, and the pass / fail of the shape, size, or storage condition of the contents; a trained model 34 that combines multiple inspection algorithms to perform the image processing used in the image processing unit 32 and the judgment processing used in the inspection and judgment unit 33, and has been machine-learned to optimize their parameters; and a display and operation unit 35 (display) such as a touch panel that can display and output the judgment results from the inspection and judgment unit 33 and input required operations such as variety registration.

[0034] This control unit 30 is composed of, for example, a microcomputer having a CPU, ROM, RAM and I / O interface (not shown), an auxiliary storage device that stores control programs for performing the functions of the image processing unit 32, the inspection and judgment unit 33 and the trained model 34, network structure information and variables, etc. in a readable manner in cooperation with the ROM, and a timer circuit, etc., and is configured so that the CPU executes predetermined arithmetic processing while exchanging data with the RAM, etc., and also executes the control program according to the control program stored in the ROM, etc.

[0035] The inspection image storage unit 31 is an image input unit. For example, it performs A / D conversion on the X-ray detection signals from a plurality of detection elements of the X-ray detector 23, and for each predetermined unit transfer time corresponding to the detection element size in the X-ray detector 23, for all n detection element regions (n is an integer greater than 1, for example, 640), it writes the data of the cumulative transmission amount within that unit time into the image memory as digital data of density levels representing gradations from 0 to 1023 (hereinafter referred to as line scanning).

[0036] Also, when the line scanning by the X-ray detector 23 is repeated a predetermined number of times according to the inspection period of the article W and sequentially written into the image memory, the inspection image storage unit 31 generates imaging data Dpx (X-ray imaging image data) corresponding to the dose distribution of the X-rays transmitted through the article W based on the detection data Lx of the line scanning images, and has a data processing program and a working memory (not shown) that exhibit the function of outputting the imaging data of the article W to the image processing unit 32.

[0037] In the image processing unit 32, a predetermined inspection algorithm combining an image processing filter and the like is preset and stored in order to perform a predetermined article inspection based on the imaging data Dpx of the article W taken in from the inspection image storage unit 31.

[0038] The image processing filter included in the inspection algorithm of the image processing unit 32 is a processing program for extracting predetermined image features (such as edges, lines, corners, regions, shades, textures) necessary for a predetermined article inspection based on the imaging data Dpx of the article W. When the inspection algorithm includes a filter for foreign object detection, for example, a feature extraction filter that performs edge detection processing to emphasize the contour of a foreign object in the article W, and has a differential filter such as a Sobel filter, and performs differential processing based on a predetermined arithmetic formula in the vicinity of the pixel of interest to emphasize the edge of the foreign object. Note that the term "image processing filter and the like" means that it includes preprocessing such as gradation correction and noise removal for the imaging data Dpx from the inspection image storage unit 31 in order to improve the detection processing accuracy of image features.

[0039] In addition, the feature measurement of the image features executed by the image processing unit 32 is to perform necessary pre-processing and image processing on the imaging data Dpx of the article W captured from the inspection image storage unit 31, and for the image obtained, calculate attributes related to grayscale features, color features, shape features, etc. (feature quantities characterizing edges, regions, distances, positions, shapes, etc.), calculate feature quantities representing the spatial relationship between such features, or calculate texture feature quantities related to the spatial frequency distribution and direction components, so as to calculate the feature quantities necessary for the determination process in the inspection determination unit 33.

[0040] The inspection determination unit 33 executes a determination process such as detecting the feature shape, foreign matter, etc. detected in the article W based on the feature quantities extracted and feature-measured by the image processing unit 32, or comparing feature quantities such as the area, contour length, and sum of densities of the detection target with a predetermined determination reference value, to determine whether a local feature shape corresponding to a foreign matter or defective part that satisfies the determination conditions is included in the article W.

[0041] The learned model 34 is an inference program that combines a plurality of algorithms including at least a part of the image processing algorithm used in the image processing unit 32 and at least a part of the determination processing algorithm used in the inspection determination unit 33. By incorporating the learned parameters obtained by optimizing a plurality of parameters (coefficients) in the algorithms with a learning data set, and by accurately adjusting the hyperparameters incorporated before learning, the required image processing for the input image data and the inspection determination based on the image can be respectively executed so as to output results.

[0042] In this way, the control unit 30 applies a predetermined image processing algorithm and a determination processing algorithm that combine a plurality of filter processes, etc. to the imaging data Dpx obtained by imaging the article W of a predetermined variety in the inspection unit 20 to inspect the predetermined quality state of the article W.

[0043] On the one hand, the learning model update control unit 40 includes an update processing unit 41, an evaluation unit 42, and a communication I / F (interface) 43, and is capable of data communication with an external management support server via the communication I / F 43.

[0044] This learning model update control unit 40, for example, like the control unit 30, includes a microcomputer having a CPU, a ROM, a RAM, and an I / O interface (not shown), and an auxiliary storage device that stores, in cooperation with the ROM, a control program for implementing each function such as the update processing unit 41 and the evaluation unit 42 described later in a readable manner. It also includes a timer circuit and the like, and according to the control program stored in the ROM and the like, the CPU exchanges data with the RAM and the like while executing predetermined arithmetic processing and executes the control program.

[0045] Specifically, the update processing unit 41 is configured to be able to execute a predetermined update process to rewrite the learned model 34 into an additional learned model or a relearned model for update (hereinafter referred to as an update model), and includes an update model storage unit 45 that stores at least the additional learned model or the relearned model in a readable and rewritable manner, and a verification data set storage unit 46 that stores and holds the verification data set not used in the learning of the learned model 34 in an updatable manner.

[0046] Note that the verification data set referred to here is a collection of specific-purpose data processed by a program or model having a learning algorithm, but is not the learning data used for weight update or the like during machine learning for model construction. Instead, it is an unknown data set capable of evaluating the classification and regression prediction performance of the learned model 34, and includes at least test data among the test data for evaluating the learned model 34 after learning and the verification data that can be used for accuracy calculation for adjusting hyperparameters at the stage of relearning and the like.

[0047] The evaluation unit 42 has a function of verifying whether the current article inspection accuracy using the old model is impaired by the update process performed by the update processing unit 41, that is, the process of rewriting and updating the learned model 34 (old model), which is the current AI model, to the update model stored in the update model storage unit 45. It is configured to include an evaluation processing unit 47, a verification result recording unit 48, and a result output unit 49.

[0048] The evaluation processing unit 47 is based on the condition that there is an update request input from the display operation unit 35 to the update processing unit 41 or an update request input from an external management support server via the communication I / F 43. First, using the verification dataset stored in the verification dataset storage unit 46, it performs an old model evaluation that evaluates the learned model 34, which is the AI model before the update, using a predetermined evaluation method (details will be described later), and a new model evaluation that evaluates the update model stored in the update model storage unit 45 using the same evaluation method as the old model evaluation using the verification dataset. It serves as an evaluation means.

[0049] The verification result recording unit 48 is a recording means for recording the evaluation result together with the version information of the update model to be evaluated, etc., on the condition that at least the evaluation result by the automatic verification of the evaluation processing unit 47 is outside the preset evaluation criteria. For example, when the update process to the update model by the update processing unit 41 is executed, the evaluation result shows that the current article inspection accuracy using the old model is impaired.

[0050] The result output unit 49 is an output means for outputting the evaluation results of the old model evaluation and the new model evaluation by the evaluation processing unit 47 as statistical data (for example, the detection rate of whether the article W is a good product or a defective product) related to the inspection results in the inspection unit 20 using the verification dataset.

[0051] More specifically, the update processing unit 41 performs the process of updating the learned model 34 to an additionally learned or relearned one step by step. For example, in the step of storing the update model in the update model storage unit 45 so that it can be read out, it is the stage of creating a trial updated learned model. At this stage, first, after executing the old model evaluation using the verification dataset stored in the verification dataset storage unit 46 for the currently learned model 34 which is the AI model, the new model evaluation is executed using the evaluation dataset assuming that the update model is tentatively applied. Then, by comparing the respective evaluation results, on the condition that it is presumed that the current article inspection accuracy using the old model will be impaired when the update processing unit 41 executes the update process for the update model based on the result of the comparative evaluation, it is evaluated that it is outside the preset evaluation criteria, and in other cases, it is evaluated that the current article inspection accuracy will not be impaired by the execution of the update process.

[0052] As shown in FIG. 2, the verification dataset stored in the verification dataset storage unit 46 in a replenishable and updatable manner is a dataset including the image data of each sample (exemplified by the image file name in the figure) for training the learned model 34, the label indicating the inspection algorithm preferably adapted to and used for the image data (exemplified by the image identification label (good product / NG product) and foreign object type in the figure), and the annotation data (exemplified as the annotation data in the figure). Each row of image and the corresponding label and / or annotation data in the figure form one sample record.

[0053] The records of the verification dataset are, for example, as shown in Fig. 3, the image data Drxg of the lump meat W1 that becomes the article W, the foreign object position filling Drxa corresponding to the foreign object sample C1 (the Sus sphere, glass sphere, and bone in the figure) that is assumed to be mixed in the lump meat W1, and the foreign object circumscribing rectangle Drxb that circumscribes the foreign object of the size of the foreign object sample C1. Here, the foreign object position filling Drxa and the foreign object circumscribing rectangle Drxb are annotation data given as information related to learning labeling for the image data Drxg of the lump meat W1.

[0054] When using the learned model 34 trained with labeled learning data for the data of the input inspection image based on the imaging data Dpx, it is image data with unknown labels.

[0055] When the learning model update control unit 40 configured as described above executes the old model evaluation and the new model evaluation using the verification dataset by the evaluation processing unit 47, the model evaluation in the predetermined evaluation method described above is executed in the following procedure.

[0056] First, in order to verify the inference accuracy of the learned model 34 before update or the updated model, the image data included in the verification dataset is sequentially taken in record by record instead of the input inspection image based on the imaging data Dpx. For example, the image data Drxg of the lump meat W1 illustrated in Fig. 3 is taken in as the data of the input inspection image with unknown labels.

[0057] Then, for the learned model 34 (old model) initially stored in the update model storage unit 45 and the updated model (new model) that has undergone additional learning or relearning, the inference results of the old and new models when the image data Drxg of the verification dataset is taken in as the data of the input inspection image are evaluated based on annotation data such as the foreign object position filling Drxa and the foreign object circumscribing rectangle Drxb and identification label information, and the evaluation such as calculating the score of the inference accuracy is executed by the evaluation processing unit 47 for each record of the verification data.

[0058] Furthermore, the evaluation results for the plurality of records are calculated as statistical data related to the inspection results in the inspection unit 20 by the verification dataset, such as the detection rate (= number of detections / number of inspected articles) of whether the article W is a good product or a defective product, and recorded in the verification result recording unit 48 when necessary. On the other hand, immediately after the calculation or when an input of a display request is made, it can be output as information that can be displayed by the result output unit 49.

[0059] Note that the learned model 34 is assumed to perform machine learning processing here, but it may be a deep learning classification method using a convolutional neural network, or a support vector machine (SVM) that performs two-class classification, for example, a non-linear SVM. Also, the input for learning may not be the X-ray image itself, but numerical data such as the average value, variance value, maximum value, and product size of pixels. Furthermore, when using an image for learning input, not only the transmission image by X-ray, but also a difference image using transmission images in different energy bands, an image after filtering the captured image, in addition to images obtained by imaging methods using different optical systems such as visible light NIR, can be used. Also, an image created by combining a plurality of types of images (each of the above-mentioned images), such as an image obtained by assigning an arbitrary image to each channel of a color image such as an RGB image (see, for example, JP-A-2023-114827 and JP-A-2023-114828), may be used.

[0060] As described above, the article inspection apparatus 1 of the present embodiment irradiates the conveyed article W with X-rays in the inspection unit 20 to acquire data of an input inspection image that is an X-ray inspection image, and applies a predetermined image processing algorithm in the image processing unit 32 to the data of the input inspection image to perform X-ray inspection of the quality state of the article W.

[0061] In addition, when an update request input from the display operation unit 35 or an update request input from an external management support server is received via the communication I / F 43 by the update processing unit 41, the evaluation processing unit 47 executes old model evaluation and new model evaluation regarding the learned model 34, and outputs the results in a displayable manner by the result output unit 49 or / and records them in the verification result recording unit 48. Then, whether or not the current article inspection accuracy of the article inspection apparatus 1 is impaired by the update processing by the update processing unit 41 is verified by the evaluation unit 42 of the learning model update control unit 40.

[0062] Next, the operation will be described.

[0063] In the learning stage of the learned model 34 in the present embodiment configured as described above, annotation data is given to each of a plurality of captured images of articles W obtained by imaging a product group of inspection target varieties, and a learning data set including labels and annotation data suitable for each image is prepared. Then, by using a large number of learning data sets and executing learning processing or machine learning processing by, for example, a statistical pattern recognition method, a learned model 34 capable of associating the data of the input inspection image of the article W with any inspection algorithm that fits is created.

[0064] The learned model 34 created in this way is stored in the update model storage unit 45 of the update processing unit 41 via the communication I / F 43 of the learning model update control unit 40. At this time, although it is created as a data set including image data of learning samples similar to the learning data set and labels and annotation data suitable for the image data, an unknown data set that was not used in the learning stage of the learned model 34 is stored in the verification data set storage unit 46 in an updatable manner as a verification data set.

[0065] Next, when a variety registration setting input is made to the control unit 30 by the display operation unit 35 and an article W for variety registration is test-transported to the article inspection apparatus 1, the article W being transported is irradiated with X-rays by the inspection unit 20, the X-rays transmitted through the article W are detected by the X-ray detector 23, and the data of the input inspection image acquired by the inspection image storage unit 31 is subjected to image processing and determination processing by the image processing unit 32 and the inspection determination unit 33 that apply the learned model 34.

[0066] At this time, the learned model 34 extracts image processing and / or determination processing that conforms to the data of the input inspection image, and those processes are executed.

[0067] On the other hand, when adding or re-learning the learned model 34 so as to be suitable for a new variety of inspection object W, on the condition that there is an update request input from the display operation unit 35 to the update processing unit 41 or an update request input from an external management support server via the communication I / F 43, the evaluation processing unit 47 performs an old model evaluation on a plurality of records of verification data, respectively, as shown in the schematic procedure of FIG. 4. After that, as shown in FIG. 5, a new model evaluation is performed, and a determination is made as to whether to update the learned model 34 to an update model according to the respective evaluation results of the plurality of old and new models.

[0068] That is, as shown in FIG. 4, first, after a verification data set including a plurality of records is created and stored using the verification data set stored in the verification data set storage unit 46 or the verification data set to be replenished and updated (step S11), using the image data of each record of this verification data set and its label and annotation data, the learned model 34, which is the old model before the update, is performance-verified by a predetermined evaluation method (a method of verifying the model inference accuracy for the image data of the verification data set with the annotation data) (step S12). The result of the performance verification is output from the result output unit 49, recorded and stored in the verification result recording unit 48 under predetermined recording conditions (step S13), and displayed on the display operation unit 35 such as a touch panel (step S14).

[0069] Next, when the updated model obtained by additional training or retraining of the learned model 34 is taken into the update processing unit 41 and the updated model is stored in the updated model storage unit 45 (step S21), the updated model (new model) is evaluated for performance using a predetermined evaluation method with the verification dataset stored in the verification dataset storage unit 46 or the replenished and updated verification dataset (step S22).

[0070] Then, the results of the respective performance evaluations (evaluation values of inference accuracy) for the old model and the new model are compared (step S23), and the comparison result is displayed on the display screen of the display operation unit 35 which is a display (step S24).

[0071] As shown in FIG. 6, for this comparative evaluation, for each record of the verification data, for the image types labeled with the type of being a good product or not and the foreign object types of NG products, the evaluation data by each of the old model and the new model shows whether it is OK or NG or what the evaluation value is, and the evaluation results of the old model and the new model for each record can be referred to.

[0072] When the evaluation of the old model and the new model for each record of the verification data is completed, then, as shown in FIG. 7, summary data (statistical data) that can be compared and displayed in the form of, for example, the good product determination rate (the ratio of OK for data with the image type being a good product), the false detection rate of NG products (the ratio of OK for data with the image type being NG), the detection rate of foreign object samples, etc., is output, so that the superiority or inferiority of the evaluation of the new model with respect to the old model can be easily grasped. However, the final comparison result is quantitatively compared and displayed as an evaluation value indicating, for example, the probability of being an excellent model, as shown in FIG. 9.

[0073] At this time, simultaneously, it is determined whether the evaluation value of the performance (inference accuracy) of the new model among the evaluation values of the performance of the old model and the new model compared and displayed on the display operation unit 35 is higher than the evaluation of the performance of the old model (step S25). If the determination result is YES, the new model is adopted and the update process of the learned model 34 by the update processing unit 41 is executed (step S26), and this process ends. On the other hand, if the determination result is NO, the old model is adopted and held (step S27), and this process ends without executing the update process of the learned model 34 by the update processing unit 41.

[0074] As shown in FIG. 8, by setting the range of the detection limit for determination by foreign object detection determination, for example, near the limit value L2, it is possible to sufficiently reduce the foreign object undetected rate while allowing a certain degree of false detection rate of good products. When it is desired to suppress the false detection rate of good products to a value below a desired value, if the range of the detection limit is set to the limit value L1 corresponding to the detection rate Ra, for example, the false detection rate of good products can be suppressed to the set detection rate Ra. Thus, since there is a correlation between the false detection rate of good products and the foreign object undetected rate, in the comparative evaluation, for example, the limits of the old and new models are set so that the false detection rates of good products are about the same, and the foreign object undetected rates at that time are compared. Or, the limits of each model may be set so that the foreign object undetected rates are about the same, and the false detection rates of good products at that time may be compared.

[0075] Thus, in this embodiment, when the update processing unit 41 of the learning model update control unit 40 attempts to update the learned model 34 applied in the control unit 30 for inspection via a predetermined medium, for example, a transmission medium such as a network or a storage medium such as a USB, the old and new learned models before and after the update can be evaluated for their performance such as inference accuracy by the evaluation processing unit 47 of the evaluation unit 42 using unknown verification data sets that have not been used in their respective learning.

[0076] Also, for the old and new models before and after the update, statistical data related to the inspection results in the inspection unit 20 by the same verification dataset for which the respective evaluation results are the same, for example, the detection rate (= number of detections / number of inspected items) indicating whether the article W is a non-defective or defective product, is output by the result output unit 49. Therefore, based on each evaluation result, the old and new learned models before and after the update can be accurately compared and evaluated using the same verification dataset. When re-learning the entire model of the learned model 34 by partially changing the learning algorithm or adding learning to the learned model 34 by increasing the learning image data of non-defective and defective products, the influence of such re-learning and additional learning on the inspection performance can be accurately grasped, and a decrease in inspection performance due to the update of the learned model 34 can be reliably prevented in advance.

[0077] In this embodiment, also, the verification dataset storage unit 46 can store a plurality of verification datasets corresponding to a plurality of product types to be inspected, and can easily and accurately cope with multi-product production. Further, the result output unit 49 can output the evaluation result by the evaluation processing unit 47 as screen display information equivalent to the detection rate indicating whether the article W is a non-defective or defective product, and the influence on the inspection performance of re-learning and additional learning can be easily and accurately grasped.

[0078] In this embodiment, further, when the learned model is updated by the update processing unit 41, the evaluation processing unit 47 of the evaluation unit 42 automatically verifies the learned model 34 using the verification dataset. Therefore, when the learned model 34 is updated, the old and new learned models before and after the update can be accurately verified using the verification dataset, and the old and new learned models can be compared and evaluated in a timely manner.

[0079] In addition, in the present embodiment, on the condition that the evaluation result by the automatic verification of the evaluation processing unit 47 is outside the preset evaluation criteria, the evaluation result is recorded in the verification result recording unit 48. Therefore, when it is determined that the influence on the inspection performance of re-learning or additional learning is not preferable, even if the re-learning or additional learning at that stage is terminated early, the evaluation result at that time is recorded, so that the influence on the inspection performance can be more accurately evaluated in the subsequent model update work.

[0080] Also, in the present embodiment, since the verification data set storage unit 46 stores the verification data set in an updatable manner, verification data with high user requirements in the actual inspection results, for example, defective product image data to be detected near the lower limit accuracy required for inspection performance, and good product image data that should not be misjudged as defective can be added and updated to the verification data set.

[0081] As described above, according to the article inspection apparatus 1 of the present embodiment, it is possible to quantitatively grasp the deterioration of the inspection performance accompanying the update of the learned model 34, and it is possible to provide the article inspection apparatus 1 that can surely prevent the deterioration of the yield due to false detection (false rejection) when updating the learned model 34 used for the inspection determination in the inspection unit 20.

[0082] (Other Embodiments) FIG. 10 shows an article inspection apparatus and its model update system according to another embodiment of the present invention.

[0083] The model update system 50 of the present embodiment constitutes an external work support server owned by a user disposed in a production factory or the like that is communicably connected to an article inspection apparatus 1A having a main configuration similar to that of the article inspection apparatus 1 of the foregoing embodiment. Regarding the configuration similar to that of the article inspection apparatus 1 in the article inspection apparatus 1A, the same reference numerals as those in the embodiment shown in FIG. 1 are used to avoid redundant description, and only the differences will be described.

[0084] The article inspection apparatus 1A has a conveyance unit 10, an inspection unit 20, and a control unit 30A.

[0085] In addition to the inspection image storage unit 31, the image processing unit 32, the inspection determination unit 33, the learned model 34, and the display operation unit 35, the control unit 30A of the article inspection device 1A is provided with an update processing unit 36 (update means) that updates the learned model 34 in cooperation with the model update system 50. This update processing unit 36 has a storage unit 37 that can store the learned model 34 and its updated model after additional learning or relearning and the verification data.

[0086] Also, the control unit 30A of the article inspection device 1A is provided with a communication I / F 43 that can be data communication-connected to the model update system 50, and the model update system 50 is also provided with a communication I / F 53 that can be data communication-connected (e.g., VPN connection) to the control unit 30A via the communication I / F 43.

[0087] The model update system 50 has an update processing unit 51, an update control unit 52, and a display operation unit 54. The update control unit 52 is communicatively connected to the article inspection device 1A via the communication I / F 53 so as to enable data communication, and is responsible for passing data between the update processing unit 51 and the display operation unit 54.

[0088] The update processing unit 51 has an old model storage unit 55a that temporarily stores the learned model 34 before the update for which update is recommended, a new model storage unit 55b that temporarily stores the updated model which is the new model after the update for which update is recommended, and a verification data set storage unit 56 that temporarily stores the verification data set not used in the learning of the learned model 34 before the update. The learned model 34 before the update is stored in the old model storage unit 55a at a predetermined timing such as when comparing with the new model or when mounting the new model on the article inspection device 1A, for example, from another device that manages the learned model held by the article inspection device 1A or from the article inspection device 1A via the communication I / F 53.

[0089] The update control unit 52 is equipped with an additional / retraining processing unit 59 that can update the learned model 34, for example, by increasing the teacher image data of good and defective products of the article W to be inspected and performing additional learning, or by retraining the learned model when the number of network layers constituting the learned model 34 is increased or decreased or hyperparameters are changed. Note that the update control unit 52 may also have functions such as annotations necessary for learning data and verification data.

[0090] Further, the update control unit 52 includes an evaluation processing unit 57 (evaluation means) that evaluates the learned model 34 before and after the update by the update processing unit 51 using a verification data set not used for the learning of the learned model 34, and a result output unit 58 (output means) that outputs the evaluation result by the evaluation processing unit 57 as statistical data related to the inspection result in the inspection unit 20 by the verification data set, for example, the detection rate of whether the article W is a good product or a defective product.

[0091] In the model update system 50 of this embodiment configured as described above, when the learned model 34 provided in the control unit 30A of the article inspection apparatus 1A can be updated by additional learning or retraining, the update processing unit 36 of the control unit 30A is notified that it is in an updatable state, and a display indicating that it is updatable is made on an inspection screen or the like displayed on the display operation unit 35, etc., to notify the user.

[0092] Then, when the user desires to update the learned model 34 by operating the display operation unit 35 upon receiving the notification, a data communication path is established between the control unit 30A and the model update system 50, and the necessary update model, data, etc. are taken into the storage unit 37 of the update processing unit 36, and a model update process in a predetermined update procedure is executed.

[0093] Also, prior to the model update process, in the model update system 50, on the condition that a request for model update is received from the control unit 30A, the addition / retraining processing unit 59 increases the teacher image data of good and defective products of the article W to be inspected and performs additional learning, or retrains the learned model in accordance with an increase or decrease in the network layer constituting the learned model 34 or a change in hyperparameters, etc.

[0094] Then, in the article inspection apparatus 1A, when the update processing unit 36 updates the learned model 34 for inspection unit application via a communication interface which is a transmission medium as a predetermined medium, the learned model 34 before update and its updated model are respectively evaluated by the evaluation processing unit 57 using an unknown verification data set not used in their respective learning.

[0095] Also, for the learned model 34 before update and the updated model (old and new models), their respective evaluation results are displayed and output by the result output unit 58 as statistical data related to the inspection results in the inspection unit 20 using the same verification data set.

[0096] Therefore, based on each evaluation result, the old and new learned models before and after update can be accurately compared and evaluated using the same verification data set. When partially changing the learning algorithm of the learned model 34 to perform retraining of the entire model, or increasing the learning image data of good and defective products for additional learning of the learned model 34, the influence of the retraining and additional learning on the inspection performance can be accurately grasped, and a decrease in inspection performance due to the update of the learned model 34 can be surely prevented in advance.

[0097] Note that the article inspection device referred to in the present invention irradiates an article with X-rays to obtain an X-ray inspection image, and applies a predetermined image processing algorithm to the X-ray inspection image to inspect the quality state of the article by X-ray. However, a camera image using any electromagnetic wave other than X-rays, for example, NIR (near-infrared ray), may be used as the inspection image. Further, the article inspection device referred to in the present invention is applicable not only to camera images but also to cases where a machine learning model is used for the inspection determination of an article inspection device such as a metal detector that detects, as a signal waveform, the influence exerted by an article passing through the article inspection area on the magnetic field in the article inspection area.

[0098] Further, in FIG. 10, the model update system 50 is connected to one article inspection device, but it goes without saying that the model update system 50 can be connected to a plurality of article inspection devices, and the model update system 50 can update the learning model corresponding to the inspected articles for a plurality of article inspection devices in the production factory.

[0099] Furthermore, the learnable model 34 can be updated when the model update system 50 receives a new model from the manufacturer via a predetermined medium, for example, a transmission medium such as a network or a storage medium such as a USB, and it is also possible that the user exchanges the learning model between the model update system 50 and the article inspection device 1A at a predetermined timing via a storage medium such as a transmission medium or a USB.

[0100] As described above, the present invention can quantitatively grasp the deterioration of the inspection performance due to the update of the learnable model, and can surely prevent the deterioration of the yield due to false detection when updating the learnable model of the inspection unit. The present invention can provide an article inspection device and a model update system. Such a present invention is useful for an article inspection device that applies a learnable model to an inspection image or a sensor signal according to the quality state of an article to inspect the quality state of the article, and a model update system that supports the update of the learnable model thereof.

Description of Reference Numerals

[0101] 1, 1A Article inspection device 10 Conveyor Section 11 Conveyor Belt 11a Upper Travel Section 12, 13 Conveyor Rollers 20 Inspection Section (X-ray Inspection Section) 21 X-ray Generator 22 X-ray Tube 23 X-ray Detector 23 Sensor 30, 30A Control Section (Inspection Control Section) 31 Inspection Image Memory Section (Image Input Unit) 32 Image Processing Section 33 Inspection Judgment Section 34 Trained Model (Model) 35, 54 Display Operation Section 36 Update Processing Section 37 Memory Section 40 Learning Model Update Control Section 41, 51 Update Processing Section (Update Means) 42 Evaluation Section 43, 53 Communication I / F (Communication Interface) 45 Update Model Memory Section 46, 56 Verification Data Set Memory Section (Memory Section) 47, 57 Evaluation Processing Section (Evaluation Means) 48 Verification Result Recording Section (Recording Means) 49, 58 Result Output Section (Output Means) 50 Model Update System 52 Update Control Section 55a Old Model Memory Section 55b New Model Memory Section 59 Additional / Relearning Processing Section C1 Foreign Object Sample Dpx Imaging Data Drx Verification Data Set (One Record) Drxg Image Data Drxa Foreign Object Position Filling (Annotation Data) Drxb Foreign Object Bounding Rectangle (Annotation Data) L1, L2 Limit Values Lx Detection Data W article (article to be inspected, product) W1 lump of meat (article, article to be inspected, product) Zx inspection section

Claims

1. A sensor (23) that outputs a detection signal for inspecting the quality state of an article (W), An article inspection apparatus comprising: an inspection unit (20) that inspects the quality state of an article by applying a learned model created in advance by learning to a predetermined detection signal output by the sensor while the article is moving, An update means (41) for updating the learned model via a predetermined medium, An evaluation means (47) for evaluating the learned model using a verification dataset not used for learning the learned model, The article inspection apparatus further comprising: an output means (49) for outputting the evaluation result by the evaluation means as statistical data related to the inspection result by the inspection unit using the verification dataset.

2. The article inspection apparatus according to claim 1, further comprising a storage unit (46) that stores a plurality of verification datasets not used for learning the learned model.

3. The article inspection apparatus according to claim 1 or 2, wherein the output means outputs the evaluation result by the evaluation means as screen display information corresponding to the detection rate of whether the article is a good product or a defective product.

4. The article inspection apparatus according to claim 1 or 2, wherein when the learned model is updated by the update means, the evaluation means automatically verifies the learned model using the verification dataset.

5. The article inspection apparatus according to claim 4, further comprising a recording means (48) for recording the evaluation result on the condition that the evaluation result by the automatic verification of the evaluation means is outside a preset evaluation criterion.

6. The article inspection apparatus according to claim 2, wherein the storage unit stores the verification dataset in an updatable manner.

7. A model update system communicably connected to an article inspection apparatus (1A) comprising: a sensor (23) that outputs a detection signal for inspecting the quality state of an article; an inspection unit (20) that inspects the quality state of an article by applying a learned model created in advance by learning to a predetermined detection signal output by the sensor while the article is moving; and an update means (36) for updating the learned model via a predetermined medium, An evaluation means (57) for evaluating the learned model before and after the update by the update means using a verification dataset not used for the learning of the learned model, An output means (59) for outputting the evaluation result by the evaluation means as statistical data related to the inspection result in the inspection unit by the verification dataset, characterized in that the model update system comprises the output means.

Images