Inspection apparatus and inspection method

Abstract

According to the invention, the appearance inspection precision can be improved. An inspection device (1) is provided with: an image processing means (3) that performs image processing on an image obtained by capturing an object to be inspected; an AI processing unit (4) that marks the foreign matter captured in the image of the object to be inspected on the basis of a learning parameter obtained by learning training data in which the image obtained by capturing the object to be inspected and an index indicating whether the object to be inspected is a good product or a defective product are combined; and a determination unit (7) that determines whether the object to be inspected is a good product or a defective product on the basis of the image processed by the image processing unit (3) and the image labeled by the AI processing unit (4).

Description

Technical Field

[0001] The invention relates to an inspection device and an inspection method. Background Art

[0002] Conventionally, when inspecting foreign matter in a liquid filled in a transparent container, an automatic foreign matter inspection device is used as disclosed in Patent Document 1. This automatic foreign matter inspection device performs foreign matter inspection by rotating the transparent container and then stopping it, and then detecting foreign matter suspended in the inertially rotating liquid in the transparent container using a camera or a sensor.

[0003] In addition, a visual inspection is also performed to inspect the appearance of the liquid-filled container.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2014-109526 Summary of the invention

[0007] Problems to be solved by the invention

[0008] Existing inspection devices detect whether there are foreign objects by rotating a liquid-filled container and then stopping it, and then photographing the internal liquid while only the internal liquid is moving, thereby checking whether there are moving objects. However, if there are bubbles, it is difficult to distinguish between bubbles and foreign objects, resulting in frequent false detection problems.

[0009] In addition, in the appearance inspection of liquid-filled containers, the rubber stopper of a vial with a complex shape, the water droplet attachment part near the shoulder of the vial, and the liquid surface area are difficult to inspect due to the influence of the product and the environment. These parts are also inspected manually by visual inspection. Therefore, it is required to perform appearance inspection with sufficient accuracy by machine in such cases.

[0010] Therefore, an object of the present invention is to improve the accuracy of foreign matter inspection and appearance inspection in liquid.

[0011] Technical solutions to solve problems

[0012] In order to solve the above-mentioned problems, the inspection device of the present invention includes: an image processing unit, which performs image processing on an image obtained by photographing an inspection object; an AI processing unit, which labels foreign matter photographed in the image of the inspection object based on learning parameters obtained by learning training data, wherein the training data is a combination of the image of the inspection object and an indicator indicating whether the inspection object is a good product or a defective product; and a judgment unit, which judges whether the inspection object is a good product or a defective product based on the image processed by the image processing unit and the image labeled by the AI ​​processing unit.

[0013] The inspection method of the present invention is characterized in that it includes: a step of performing image processing on an image obtained by photographing an inspection object by an image processing unit; annotating foreign matter photographed in the image of the inspection object by an AI processing unit based on learning parameters obtained by learning training data, wherein the training data is a combination of the image of the inspection object and an indicator indicating whether the inspection object is a good product or a defective product; and a step of judging whether the inspection object is a good product or a defective product by a judging unit based on the image processed by the image processing unit and the image marked by the AI ​​processing unit.

[0014] Other technical solutions are described in the specific implementation manner.

[0015] Effects of the Invention

[0016] According to the present invention, the accuracy of appearance inspection can be improved. In addition, the blind spots of container appearance inspection can be reduced, and the visual inspection sites can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a structural diagram of the inspection device of this embodiment.

[0018] Figure 2 This is a conceptual diagram showing the base and lighting of the inspection device.

[0019] Figure 3 This is a diagram for explaining the operation of the inspection device during production video recording.

[0020] Figure 4 This is a diagram for explaining the operation of the inspection device during playback adjustment.

[0021] Figure 5 It is a structural diagram of the learning environment device.

[0022] Figure 6 This is a diagram illustrating the learning process of the AI ​​processing unit.

[0023] Figure 7 This is a flowchart of the learning process of the AI ​​processing unit.

[0024] Figure 8 This is a flowchart of the inspection process.

[0025] Fig. 9 It is the input image of the black foreign body.

[0026] Fig.10 It is an image obtained by segmenting the input image of black foreign matter and identifying the defective parts.

[0027] Fig.11 It is the input image of the white foreign body.

[0028] Fig.12 It is an image obtained by segmenting the input image of white foreign matter and identifying the defective parts.

[0029] Fig.13 is the input image of the bubble.

[0030] Fig.14 It is an image obtained by segmenting the input image of bubbles. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0032] The first example of the present invention is the foreign matter inspection on the top surface of freeze-dried cakes, which has errors in the state of good products. Good products have larger errors in shape than bad products (cracks in freeze-dried cakes, etc.), so it is difficult to distinguish between good products and bad products. The current problem is that cracks in cakes are mistakenly detected as bad products.

[0033] The second example is the inspection of foreign matter on the side of a syringe rubber stopper, where the inspection area and detection sensitivity are limited. When there is a defect in the shape change part (boundary part, etc.), the inspection area and detection sensitivity are limited. The current problem is that the defect is missed and the boundary part is mistakenly detected as a defect.

[0034] The third example is the inspection of foreign matter inside the rubber stopper of a vial, where the image changes every time during the inspection. Since the inspection image is taken in rotation, the inspection area needs to be set according to the angle. The current problem is that defects are missed and the boundary part is mistakenly detected as defective.

[0035] The following embodiment relates to a foreign matter inspection device, and the foreign matter inspection in a liquid filled in a transparent container such as an ampoule or vial, and the appearance inspection of a liquid filled container are described as examples. However, the present invention is not limited to this, and can also be applied to the first to third examples described above, and is not limited thereto.

[0036] Figure 1 It is a block diagram of the inspection device 1 according to the present embodiment.

[0037] Inspection device 1 inspection Figure 2 The foreign matter in the inspection object container 8 and the appearance of the inspection object container 8 are shown. The inspection device 1 includes a camera 2, a control unit 13, an image processing unit 3, an AI processing unit 4, and a determination unit 7.

[0038] like Figure 2 As shown in FIG. 1 , the inspection device 1 includes a pedestal 11 for rotating the inspection target container 8 and an illumination 12 for irradiating light to the inspection target container 8 from the side.

[0039] The camera 2 captures an image of the inspection target container 8. The camera 2 only needs to capture an image of the inspection target. The inspection target is not limited to the inspection target container 8 and the liquid therein. The control unit 13 satisfies the conditions for capturing an image of the inspection target by performing brightness correction and inspection position correction on the camera 2.

[0040] like Figure 1 As shown, the image processing unit 3 is an image processing unit having an inspection processing unit 31 for filtering the input image signal and an image processing unit for performing image processing on the image of the inspection object container 8 captured by the camera 2. Thus, the judgment unit 7 can judge whether there is a foreign matter in the liquid. The AI ​​processing unit 4 is an AI processing unit that stores the learning parameters 41 obtained by learning the foreign matter and performs AI (Artificial Intelligence) processing on the captured image obtained by the camera 2 and performs region segmentation. The AI ​​processing unit 4 stores the learning parameters 41 obtained by learning the training data composed of a combination of the image of the inspection object container 8 captured by the camera 2 and an indicator indicating whether the inspection object container 8 is a good product or a defective product. The AI ​​processing unit 4 uses the learning parameters 41 to mark the foreign matter captured in the image of the inspection object container 8. Here, marking the foreign matter means that the AI ​​processing unit 4 adds label information indicating that it is a foreign matter to the image area where the foreign matter is captured.

[0041] The determination unit 7 checks whether there is foreign matter, and further checks whether there is a poor appearance, based on the image processed by the image processing unit 3 and the image annotated by the AI ​​processing unit 4. The determination unit 7 can determine whether there is foreign matter for each pixel based on the image processed by AI.

[0042] The judgment unit 7 extracts the first abnormal area from the image processed by the image processing unit 3, and extracts the second abnormal area from the image annotated by the AI ​​processing unit. The judgment unit 7 judges whether the inspection object container 8 and the liquid therein are good or bad products based on the logical product of the first abnormal area and the second abnormal area.

[0043] The inspection device 1 implements the AI ​​processing by the AI ​​processing unit 4 at the same time as the existing inspection by the image processing unit 3. Then, the judgment unit 7 combines the AI ​​processing result with the existing inspection result and makes a comprehensive judgment using the bad judgment (logical product) of both parties. In this way, the accuracy of the appearance inspection performed by the machine can be improved. In addition, the blind spots of the container appearance inspection can be reduced, and the visual inspection sites can be reduced. In addition, because the comprehensive judgment is made using the bad judgment (logical product) of both parties, it is allowed to mistakenly judge good products as bad products in the AI ​​processing to a certain extent. Therefore, the learning workload can be reduced.

[0044] In addition, the judgment unit 7 may combine the AI ​​processing result with the existing inspection result and make a comprehensive judgment using either defect judgment (logical sum). In this case, since a certain degree of error judgment of defective products as good products is allowed, the learning workload can be reduced.

[0045] In addition, although learning is required to perform AI processing, there are problems in that learning on the inspection device 1 is difficult to reproduce, and learning for adjusting parameters cannot be performed in parallel during production.

[0046] like Figure 3 As shown, the image processing unit 3 uses Figure 1 The video recording function of the inspection device 1 shown in the figure digitally records the image of the inspection target container 8 in production captured by the camera 2 in the storage device 6. Thus, all input images can be recorded while performing image evaluation and production operation, and a large amount of data for AI learning can be obtained.

[0047] like Figure 4 As shown, the image recorded in the storage device 6 is reproduced without mechanical operation, and the image processing unit 3 evaluates and adjusts it, thereby outputting the inspection result. Training data can be obtained by manually correcting the inspection result.

[0048] like Figure 5 As shown, a learning unit 52 performs learning using the captured images and training data using a learning environment device 5 different from the inspection device 1 to generate a learning parameter 51. The performance confirmation unit 53 confirms the performance of the inspection performed using the learning parameter 51.

[0049] The performance confirmation unit 53 uses the captured images recorded in the storage device 6 to perform simulation of the learning result, that is, the learning parameter 51, by both conventional processing and AI processing. This allows the results to be compared and the effects to be confirmed.

[0050] After confirming the effect, the learning environment device 5 registers the learning parameter 51 which is the learning result of the AI ​​process in the inspection device 1 as the learning parameter 41. Thus, the learning result is reflected in the inspection device 1.

[0051] AI processing requires a person to specify background, foreign matter, damage, bubbles, etc. for learning. In order to reduce the workload, in this embodiment, the function of the existing image processing unit 3 is used to extract information such as background, foreign matter, and dirt, and the extraction result is displayed on the screen. Based on the screen display, a person confirms whether there is an error in the judgment of the image processing unit 3. In this way, training data can be generated relatively easily.

[0052] Then, the learning environment device 5 learns the training data. In order to reduce human judgment, the learning environment device 5 can reduce the number of sheets to be judged by adding conditions such as ignoring foreign matter below a certain size and treating thinner sheets as good.

[0053] Figure 6 Yes Description Figure 1 The diagram shows the learning process of the AI ​​processing unit 4.

[0054] The learning flow is roughly divided into a process performed by the inspection device 1 , a process performed by the learning unit 52 , and a process performed by the performance confirmation unit 53 .

[0055] In step S10 , the inspection device 1 executes the existing inspection process. The result of the existing inspection process and the captured image are transmitted to the learning unit 52 .

[0056] In step S11, the learning unit 52 learns the elements in the image as labels based on the results of the existing inspection process and the captured image at this time. The learning result is the learning parameter 51 ( Figure 5 ) is transmitted to the performance confirmation unit 53.

[0057] In step S12, the performance confirmation unit 53 uses the evaluation image to confirm the learning parameter 51 ( Figure 5 ) performance. Repeat the processing of steps S11 and S12 until the required performance is obtained. Here, the required performance refers to the judgment of bubbles, etc., which are mistakenly judged as defective by the image processing unit 3 as good products by the AI ​​processing using the learning parameter 51, and the judgment of foreign matter and damage, which are correctly judged as defective by the image processing unit 3 as defective by the AI ​​processing. It may also be a good product in fact, but mistakenly judged as defective by the AI ​​processing using the learning parameter 51. Such misjudgment can be corrected by the logic and of the judgment unit 7, and it is not a problem. In this way, the learning workload of the learning unit 52 and the amount of training data learned by the learning unit 52 can be reduced.

[0058] When the learning result of the required performance is obtained, the learning environment device 5 registers the learning result, namely, the learning parameter 51 , in the inspection device 1 .

[0059] In step S13, the inspection device 1 executes the existing inspection device and AI processing in parallel. Then, in step S14, the judgment unit 7 evaluates the results of the existing inspection processing and the results of the AI ​​processing and makes a comprehensive judgment. In step S15, when the judgment unit 7 selects a normal inspection object, a series of learning processes are completed.

[0060] Figure 7 is a flowchart of the learning process of the AI ​​processing unit 4. Figure 6 The learning process has been modified.

[0061] First, the inspection device 1 collects data using the recording function of the storage device 6 (step S20). Then, the learning environment device 5 generates a model through learning (step S21). Here, the model is composed of the learning parameters 51.

[0062] Next, the operator reflects the model, i.e., the learning parameters 51, to the inspection device 1 and verifies it (step S22), and then applies it to actual production (step S23). Figure 7 processing.

[0063] Figure 8 This is a flowchart of the inspection process. Figure 1 Provide explanation.

[0064] First, the control unit 13 corrects the brightness of the camera 2 (step S30) and corrects the inspection position (step S31) to meet the conditions for capturing an image of the inspection object. Figure 2 The image is sent to the image processing unit 3 and also to the Figure 1 The AI ​​processing unit 4 is shown. Hereinafter, the processing of steps S32 to S34 and the processing of steps S35 to S38 are executed in parallel.

[0065] Figure 1 The image processing unit 3 shown in the figure performs the processing from step S32 to step S34. The image processing unit 3 filters the image using the inspection processing unit 31 (step S32). Then, the image processing unit 3 generates a binary image based on a threshold value of depth (step S33), and extracts the region by annotation (step S34).

[0066] Figure 1 The AI ​​processing unit 4 shown executes the processing of steps S35 to S38. The AI ​​processing unit 4 classifies the foreign matter for each label based on the learning parameter 51 (step S35). Then, the AI ​​processing unit 4 generates an image with the same color or brightness for each label (step S36).

[0067] Then, the AI ​​processing unit 4 generates a binary image based on a threshold value of lightness and darkness (step S37 ), and performs region extraction by labeling (step S38 ).

[0068] In step S39, the determination unit 7 performs determination using the logical product of the processing result of the image processing unit 3 and the processing result of the AI ​​processing unit 4. Here, the determination unit 7 extracts the first abnormal area from the image processed by the image processing unit 3, and extracts the second abnormal area from the image annotated by the AI ​​processing unit 4. Then, the determination unit 7 determines whether the inspection object is a good product or a defective product based on the logical product of the first abnormal area and the second abnormal area.

[0069] Then, the judging unit 7 screens the inspection objects (step S40), and then the process ends. Figure 8 processing.

[0070] The advantage of this embodiment is that it can be added as an option for feature extraction in existing processing. By adding AI processing to existing processing, both processes are performed in parallel, and comprehensive judgment can be made in combination with the results of existing processing.

[0071] In order to improve inspection accuracy, the operator of the inspection device 1 combines existing image processing with AI processing to repeat comparison and evaluation.

[0072] For example, Figure 8 The flowchart of FIG. 1 determines the false detection of foreign matter etc. based on the detected failure. At this time, the determination unit 7 reflects the AI ​​processing result on the failure determination according to the logical product of the first abnormal area and the second abnormal area.

[0073] In addition, there is a case where a new AI process is added to the uninspected area. In such a case, the determination unit 7 reflects the AI ​​process result on the failure determination according to the logical sum of the first abnormal area and the second abnormal area.

[0074] In addition, the determination unit 7 may determine whether the inspection object is a good product or a defective product based on the result of synthesizing a portion of a first specified size in the first abnormal region and a portion of a second specified size in the second abnormal region without limitation.

[0075] The abnormal area that can be properly detected by the image processing unit 3 is larger than the specified value. In contrast, the AI ​​processing can properly detect even smaller areas. Therefore, the judgment unit 7 selects the detection results of both according to the size of the abnormal area, and then synthesizes the two, thereby properly judging whether there is an abnormality. In this case, the learning of the AI ​​processing can be stopped at the time point when the smaller area can be properly detected, so the learning can be completed in a short time.

[0076] Fig. 9This is the input image of a black foreign body. A black area is captured to the left of the center of the image. This black area is the abnormal area.

[0077] Fig.10 Yes Fig. 9 The image shown is an input image of a black foreign body, in which region segmentation is performed and defective parts are identified.

[0078] To the left of the center of the image, an area with a different color from the others is shown. This corresponds to the defective part.

[0079] Fig.11 This is the input image of a white foreign object. A white area is captured slightly above the center of the image.

[0080] Fig.12 Yes Fig.11 The image shown is obtained by performing region segmentation on the input image of the white foreign matter and identifying the defective parts.

[0081] A color area different from the others is shown slightly above the center of the image. This corresponds to the defective part.

[0082] Fig.13 This is the input image of the bubble. The shadow of the bubble is shown in the lower left of the center of the image. This corresponds to a non-defective area.

[0083] Fig.14 Yes Fig.13 An image obtained by performing region segmentation on the input image of the bubbles shown.

[0084] In the image, there is no area with a different color from the others, so the bubble corresponds to the defective part.

[0085] Application examples of this embodiment will be described below.

[0086] 《First application example》

[0087] The inspection device 1 is applied to distinguish bubbles and foreign matter on the rubber stopper of a syringe to reduce false detection. Specifically, all objects reflected on the rubber stopper are detected and marked by the existing inspection, and objects considered to be bubbles are removed from the detected objects by AI processing.

[0088] 《Second application example》

[0089] The inspection device 1 is used to distinguish between bubbles and foreign matter suspended on the liquid surface to reduce false detection. Specifically, the AI ​​learns the degree of liquid level reduction to determine whether there is a foreign matter. In addition, if the shape of the bubble is different from that of the foreign matter to be detected, it is achieved by learning and removing the detected bubble.

[0090] 《Third application example》

[0091] The inspection device 1 learns to distinguish water droplets in the air layer of the main body (shoulder) from damage or dirt. Specifically, all objects reflected in the air layer are detected and marked by the existing inspection. Then, the objects considered to be water droplets are removed from the detected objects by AI processing.

[0092] 《Fourth application example》

[0093] The inspection device 1 learns to distinguish water droplets in the air layer of the freeze-dried agent from damage or dirt. Specifically, all objects reflected in the air layer are detected and marked using existing inspection. Then, objects that are considered to be water droplets are removed from the detected objects using AI processing, thereby achieving this.

[0094] 《Fifth Application Example》

[0095] The inspection device 1 learns to distinguish water drops attached to the inner side of the rubber stopper of the vial from dirt or foreign matter. Specifically, all objects reflected on the rubber stopper are detected and marked by the existing inspection. Then, the shadow of the rubber stopper and objects that are considered to be water drops are removed from the detected objects by AI processing, thereby distinguishing water drops from dirt or foreign matter.

[0096] (Variation Example)

[0097] The present invention is not limited to the above-mentioned embodiments, and includes various modified examples. For example, the above-mentioned embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, and are not limited to all structures that must be described. In addition, a part of the structure of a certain embodiment can be replaced with the structure of other embodiments, and the structure of other embodiments can be added to the structure of a certain embodiment. In addition, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0098] For each of the above structures, functions, processing parts, processing units, etc., for example, a part or all of them can be realized by hardware such as integrated circuits. In addition, each of the above structures, functions, etc. can also be realized by software by interpreting and executing programs that realize each function through a processor. Information such as programs, tables, files, etc. that realize each function can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as a flash memory card and a DVD (Digital Versatile Disk).

[0099] In each embodiment, the control lines and information lines are shown as necessary for explanation, and not all the control lines and information lines on the product are necessarily shown. In fact, it can be considered that almost all structures are connected to each other.

[0100] As modified examples of the present invention, there are, for example, the following (a) to (d).

[0101] (a) The present invention is not limited to appearance inspection, but can also be applied to inspection in liquid.

[0102] (b) The inspection object of the present invention is not limited to vials and freeze-dried agents, and may be containers such as ampoules or any other objects.

[0103] (c) The present invention is not limited to transparent containers.

[0104] (d) The labeling method of the present invention is not limited as long as it uses machine learning to add label information representing a region of a desired foreign object in an image.

[0105] Description of Reference Numerals

[0106] 1. Inspection device

[0107] 2 Cameras

[0108] 3 Image processing unit (image processing unit)

[0109] 4AI processing unit (AI processing unit)

[0110] 31 Inspection and Processing Department

[0111] 41 Learning Parameters

[0112] 5Learning Environment Devices

[0113] 51 Learning Parameters

[0114] 52 Learning Department

[0115] 53 Performance Confirmation Department

[0116] 6 Storage Devices

[0117] 7. Judgment Department

[0118] 8 Check the object container.

Claims

1. An inspection device, characterized in that: include: An image processing unit that performs image processing on an image obtained by photographing the inspection object; an AI processing unit that labels foreign matter captured in the image of the inspection object based on a learning parameter obtained by learning training data, wherein the training data is a combination of the image of the inspection object and an indicator indicating whether the inspection object is a good product or a defective product; and A determination unit determines whether the inspection object is a good product or a defective product based on the image processed by the image processing unit and the image labeled by the AI ​​processing unit.

2. The inspection device according to claim 1, characterized in that: The determination unit extracts a first abnormal region from the image processed by the image processing unit, and extracts a second abnormal region from the image annotated by the AI ​​processing unit.

3. The inspection device according to claim 2, characterized in that: The determination unit determines whether the inspection target object is a good product or a defective product based on a logical product of the first abnormal region and the second abnormal region.

4. The inspection device according to claim 2, characterized in that: The determination unit determines whether the inspection object is a good product or a defective product based on a logical sum of the first abnormal region and the second abnormal region.

5. The inspection device according to claim 2, characterized in that: The determination unit determines whether the inspection object is a good product or a defective product based on a result of combining a portion of a first predetermined size in the first abnormal region and a portion of a second predetermined size in the second abnormal region.

6. An inspection method, characterized in that: include: The step of performing image processing by an image processing unit on an image obtained by photographing the inspection object; annotating, by the AI ​​processing unit, foreign matter captured in the image of the inspection object based on a learning parameter obtained by learning training data, wherein the training data is a combination of the image of the inspection object and an indicator indicating whether the inspection object is a good product or a defective product; and A step of determining whether the inspection object is a good product or a defective product by a determination unit based on the image processed by the image processing unit and the image labeled by the AI ​​processing unit.

Images