Method, program, device and system relating to abnormality detection

Abstract

To properly detect abnormality of a state of a container.SOLUTION: An abnormality detection method includes: obtaining a determined image which is an image of objects to be determined which should be regularly aligned in a state that cylindrical containers are standing (S11); creating a difference image in which the difference between a normal image of the objects to be determined which are regularly aligned and the determined image is emphasized (S12, S13); calculating a multifraction error of the difference image (S14) to determine a maximum value of the multifraction error (S15); and determining whether the objects to be determined are normal or abnormal, on the basis of the maximum value (S16).SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to anomaly detection. [Background technology]

[0002] Generally, in a container manufacturing process, containers being manufactured or manufactured may be arranged in an orderly manner. For example, when the manufactured containers are shipped, the containers are arranged in an orderly manner and loaded onto pallets. For example, Patent Document 1 discloses a technology related to a palletizer that loads cans onto pallets. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-111319 A Summary of the Invention [Problem to be solved by the invention]

[0004] In processes where containers should be arranged in an orderly manner, abnormalities such as containers being missing, some containers being toppled over when they should be standing, and containers being out of order generally need to be removed. Therefore, such abnormalities need to be detected appropriately.

[0005] An object of the present invention is to appropriately detect abnormalities in the state of a container. [Means for solving the problem]

[0006] According to one aspect of the present invention, an abnormality detection method includes acquiring a test image, which is an image of a test object that should be arranged in a regular pattern while the test object is in an upright position in the form of a cylindrical container; creating a difference image in which the difference between the image of a normal test object arranged in a regular pattern and the test image is emphasized; calculating a multi-division error of the difference image to determine a maximum value of the multi-division error; and judging whether the test object is normal or abnormal based on the maximum value. Effect of the Invention

[0007] According to the present invention, abnormalities in the state of a container can be appropriately detected. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of a palletizer system according to an embodiment. [Diagram 2] FIG. 2 shows examples of input images to an autoencoder, output images from the autoencoder, and their difference images, for each of a normal case, an abnormal case in which a missing can has occurred, and an abnormal case in which alignment has been disturbed. [Diagram 3] FIG. 3 is a flowchart showing an outline of the determination process using the first determination method. [Figure 4] FIG. 4 is a diagram showing an example of evaluation values ​​obtained for a plurality of judged images using the first judging method. [Diagram 5] FIG. 5 is a diagram showing an example of evaluation values ​​obtained for a plurality of judged images using a method according to a comparative example. [Figure 6] FIG. 6 is a flowchart showing an outline of the determination process using the second determination method. [Figure 7] FIG. 7 is a diagram showing an example of evaluation values ​​obtained for a plurality of judged images using the second judging method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment will be described with reference to the drawings. This embodiment relates to a palletizer system. This palletizer system is used in the final stage of a manufacturing process for cans, which are a type of bottomed cylindrical container, and stacks the manufactured cans in an orderly manner on pallets for shipping. The palletizer system of this embodiment is equipped with an anomaly detection system that detects cans falling out, cans falling over, cans being out of order, etc. that may occur during stacking, and prevents cans from falling out, falling over, being out of order, etc. in the shipment.

[0010] [Configuration and operation of palletizer system] <Palletizer> FIG. 1 is a schematic diagram showing an outline of a configuration example of a palletizer system 1 according to the present embodiment. The palletizer system 1 includes a palletizer 10 that stacks manufactured empty cans on a pallet for shipment, and a control device 30 that controls the operation of the palletizer 10. The palletizer 10 regularly aligns cans 90 standing with their bottoms facing down, one layer at a time. The palletizer 10 stacks the aligned cans 90 on the pallet in a predetermined number of layers, with a separation sheet sandwiched between each layer. In FIG. 1, the manufactured cans 90 are transported from right to left. The palletizer 10 includes, in order from the upstream side to the downstream side, a supply unit 11, an alignment unit 12, a transfer unit 13, and a palletizing unit 14.

[0011] The manufactured cans 90 are supplied from the supply unit 11 to the palletizer 10. The supply unit 11 is provided with partition plates 22 corresponding to the number of rows of the cans 90 to be aligned, and the cans 90 are aligned in a predetermined number of rows and supplied to the alignment unit 12. The cans 90 are packed tightly in the alignment unit 12, so that the cans 90 are aligned regularly.

[0012] Palletizer 10 is provided with pusher 25 that reciprocates between alignment section 12 and transfer section 13. When one or more layers of cans 90 to be placed on a pallet are aligned in alignment section 12, pusher 25 pushes one layer of aligned cans 90 to transfer section 13. One layer of cans 90 waits in transfer section 13 until it is stacked.

[0013] The palletizing section 14 is provided with a lifting device. A pallet is placed on the lifting device. The lifting device aligns the height of the separation sheet on which the cans 90 are placed to the height of the transfer section 13. The palletizer 10 is provided with a transfer machine 28 that reciprocates between the transfer section 13 and the palletizing section 14. The transfer machine 28 places one layer of cans 90 waiting in the transfer section 13 on the separation sheet of the palletizing section 14. The lifting device then lowers the pallet by the height of one layer. A separation sheet is placed on top of the cans 90, and its height is aligned to the height of the transfer section 13.

[0014] When the cans 90 are stacked in the palletizing unit 14 and the transfer unit 13 becomes empty, the pusher 25 repeatedly pushes out one layer of aligned cans 90 from the alignment unit 12 to the transfer unit 13. The transfer machine 28 repeatedly places a predetermined number of layers of cans 90 waiting in the transfer unit 13 onto the separation sheet of the palletizing unit 14. In this manner, the cans 90 are stacked on the pallet in a predetermined number of layers, aligned regularly and without gaps. Once the predetermined number of layers of cans 90 have been stacked, they are bound together with bands, wrapped in shrink film, and shipped.

[0015] <Anomaly Detection System> The palletizer 10 of the palletizer system 1 of this embodiment is provided with an anomaly detection system 50. The anomaly detection system 50 detects anomalies in the cans 90 to be stacked on a pallet, such as a missing can in which a gap for one can is created among the aligned cans 90, resulting in a shortage of the number of cans 90, a fallen can in which a can 90 falls and is stacked as is, or a disorder in which the alignment of the cans 90 is disturbed and the cans are stacked as is. The anomaly detection system 50 detects the occurrence of a missing can, a fallen can, a disorder in alignment, or the like before the cans are stacked on the pallet. This prevents the cans 90 from being stacked on the pallet in a state where an anomaly such as a missing can, a fallen can, or a disorder in alignment has occurred.

[0016] The anomaly detection system 50 detects the occurrence of an anomaly based on images of the cans 90 aligned by the alignment unit 12. For this purpose, the anomaly detection system 50 has an imaging device 52 provided in the alignment unit 12. The imaging device 52 is provided at the upper part on the downstream side of the alignment unit 12, and is arranged so as to be able to image all the cans 90 pushed out by the extruder 25 from above. As an example, in this embodiment, three imaging devices 52 are provided side by side in the width direction of the alignment unit 12. Each imaging device 52 continuously or intermittently images the cans 90 passing under the imaging device 52 so that all the cans 90 moved by the extruder 25 are imaged. The imaging device 52 is connected to, for example, a PoE (Power over Ethernet) hub 54, is supplied with power from the PoE hub 54, and transmits the captured images via the PoE hub 54, although this is not limited thereto.

[0017] The anomaly detection system 50 includes an anomaly detection device 56. The anomaly detection device 56 is a computer including an integrated circuit such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a central processing unit (CPU), and may include, for example, a graphics processing unit (GPU). The anomaly detection device 56 acquires an image captured by the imaging device 52 via the PoE hub 54. The anomaly detection device 56 uses the acquired image to detect an anomaly such as a missing can, a fallen can, or a misaligned can when the anomaly occurs.

[0018] Anomaly detection device 56 is connected to control device 30, which controls the operation of palletizer 10. When anomaly detection device 56 detects an abnormality, it transmits a signal indicating the detection of an abnormality to control device 30. Upon receiving the abnormality detection signal, control device 30 stops stacking of cans 90. At this time, control device 30 also causes notification device 62, such as a display device or speaker, to output an alarm or the like indicating that an abnormality has occurred.

[0019] [Anomaly detection method] The method of anomaly detection performed by anomaly detection device 56 will now be described. Anomaly detection device 56 acquires, as an image to be determined, an image of cans 90 that should be regularly arranged in alignment unit 12, from imaging device 52. Anomaly detection device 56 creates a difference image in which the difference between the image of normal cans 90 regularly arranged and the image to be determined acquired from imaging device 52 is emphasized.

[0020] As an example, in this embodiment, the anomaly detection device 56 uses an autoencoder using a deep neural network. The autoencoder according to this embodiment is configured to output an image of a normal can 90 in response to an input of an image of a normal can 90, for example, by unsupervised machine learning using normal images, which are images of cans 90 normally arranged in a regular order. The anomaly detection device 56 inputs a judged image acquired from the imaging device 52 to this autoencoder and acquires a reference image, which is the output of the autoencoder. Then, the anomaly detection device 56 calculates the difference between the judged image acquired from the imaging device 52 and the reference image, which is the output of the autoencoder, to create a difference image.

[0021] FIG. 2 shows examples of input and output images to an autoencoder and their difference images for each of (a) a normal case, (b) an abnormal case where a missing can occurs, and (c) an abnormal case where an alignment is disturbed. In the normal case, the difference between the input image to the autoencoder configured as in this embodiment and the output reference image is small, and the error obtained from the difference image is small. In contrast, in the case where an abnormality exists, the difference between the input image to the autoencoder and the output reference image is large, and the error obtained from the difference image is large. Therefore, the abnormality detection device 56 can determine whether the object to be determined is normal or abnormal, that is, whether an abnormality such as a missing can, a fallen can, or alignment is occurring, based on the error obtained from the obtained difference image.

[0022] In this embodiment, the autoencoder generated by machine learning is introduced into a small computer equipped with an FPGA or the like, so that the anomaly detection device 56 can be placed as an edge device in the factory where the palletizer 10 is installed. Note that the anomaly detection device 56 may be connected to a network, and may be configured so that the autoencoder can be reconfigured, re-learned, re-implemented, and the like by remote control.

[0023] In this embodiment, the manufactured cans 90 are used as the object to be judged. The mouth of the can 90 is slightly narrower than the body, so when the can is photographed from above by the imaging device 52, a portion called the shoulder, which is inclined from the body to the mouth of the can 90, is also captured in the image. In addition, the manufactured cans 90 are printed on the outer periphery, including the body and shoulder, according to the product. This printing differs depending on the position of the outer periphery. The circumferential orientation of the cans 90 aligned in the alignment unit 12 of the palletizer 10 is not particularly controlled, so that the orientation of the printing on the shoulder differs for each can 90 in the image to be judged photographed by the imaging device 52.

[0024] Furthermore, in this embodiment, an image of the can 90 being moved by the extruder 25 is acquired as the determined image by the imaging device 52. For this reason, the position of the can 90 in the determined image may differ from image to image.

[0025] Due to these characteristics of the judged image, in this embodiment, the noise in the difference image is large, as can be seen from the normal case in Fig. 2(a). Therefore, in this embodiment, the error in the difference image, which is large in the case of an abnormality, is easily buried in the noise. Therefore, in this embodiment, a judgment method is used that can appropriately detect an abnormality even when the noise in the difference image is large.

[0026] [First Judgment Method] Determining whether an object is normal or abnormal using the first determination method will be described. Fig. 3 is a flow chart showing an outline of an anomaly detection method using the first determination method. The process related to this anomaly detection method is executed by an anomaly detection device 56 by a series of programs.

[0027] In step S11, the anomaly detection device 56 acquires an image obtained by using the imaging device 52 as a judged image. In step S12, the anomaly detection device 56 inputs the acquired judged image to the autoencoder configured as described above to acquire a reference image that is an output image. In step S13, the anomaly detection device 56 creates a difference image between the judged image acquired in step S11 and the reference image acquired in step S12. In step S14, the anomaly detection device 56 calculates a multi-division square error for this difference image. Here, the multi-division square error is a square error of the difference image for each divided region by dividing the difference image into a plurality of regions. The square error is calculated as the sum of the square values ​​of the difference in luminance value between the judged image and the reference image for each pixel. In step S15, the anomaly detection device 56 determines the maximum value of the square error for each divided region calculated in step S14 as an evaluation value. In step S16, the anomaly detection device 56 judges whether the subject image is normal or abnormal based on the maximum value determined in step S15, which is the evaluation value. For example, the anomaly detection device 56 judges that there is an abnormality when the determined maximum value is greater than a predetermined threshold value, and judges that the subject image is normal when the maximum value is equal to or less than the predetermined threshold value.

[0028] <Example of judgment> The above-mentioned steps S11 to S15 described with reference to Fig. 3 were performed on 30198 images judged as normal and 6 images judged as abnormal. Here, in step S12, an autoencoder configured by unsupervised learning using a deep neural network with 36222 images of normal cases as training data was used. In addition, in step S14, as the multi-division squared error, the squared error was calculated for each region obtained by dividing the image into 2 regions vertically and 3 regions horizontally, totaling 6 regions, and in step S15, the maximum value was determined as the evaluation value.

[0029] The obtained processing results are shown in Figure 4. In Figure 4, the vertical axis shows the maximum value of the 6-part square error, which is the evaluation value, and the circles on the left side show the evaluation values ​​related to normal images, while the squares on the right side show the evaluation values ​​related to images with abnormalities. The (a) to (d) attached to the squares with abnormalities are the values ​​corresponding to the images (a) to (d) shown at the bottom of the figure, respectively.

[0030] As a comparative example, instead of performing the processes in steps S14 and S15, the square error of the entire difference image was calculated as the evaluation value without dividing the region for the same image. The result is shown in FIG.

[0031] As shown in Figure 5, when the squared error for the entire difference image is calculated, it is not possible to distinguish between normal and abnormal cases based on the value. In contrast, by using the maximum value of the 6-division squared error as shown in Figure 4, it is possible to set a threshold value, for example, as shown by the dashed line in the graph in Figure 4, and it was confirmed that it is possible to distinguish between normal and abnormal cases.

[0032] [Second Judgment Method] Determining whether an object is normal or abnormal using the second determination method will be described. Fig. 6 is a flow chart showing an outline of an abnormality detection method using the second determination method. The second determination method is different from the first determination method in that a process using a smoothing filter is performed on the created difference image, and a multi-division square error is calculated for the image after the filter process.

[0033] Steps S21 to S23 of the second determination method are the same as steps S11 to S13 of the first determination method. That is, in step S21, the anomaly detection device 56 acquires a determined image from the imaging device 52, in step S22, inputs the acquired determined image to an autoencoder to obtain a reference image, and in step S13, creates a difference image between the input image and the reference image.

[0034] In step S24, the anomaly detection device 56 performs a smoothing filter process on the difference image. The smoothing filter used may be any of various filters for removing noise, such as an averaging filter, a median filter, or a Gaussian filter.

[0035] Steps S25 to S27 of the second determination method are the same as steps S14 to S16 of the first determination method. That is, in step S25, the anomaly detection device 56 calculates a multi-division square error for the difference image after the smoothing filter process, determines the maximum value thereof in step S26, and determines whether the determined image is normal or abnormal based on the determined maximum value in step S27.

[0036] <Example of judgment> As in the results shown in Figures 4 and 5, 30198 images to be judged as normal and 6 images to be judged as abnormal were subjected to the processes in steps S21 to S26 described above with reference to Figure 6. Here, an averaging filter was used as the smoothing filter in step S24. The rest is the same as the judgment example shown in relation to the first judgment method described above.

[0037] The obtained processing results are shown in Figure 7. As in Figure 4, in Figure 7, the vertical axis indicates the maximum value of the 6-part square error, which is the evaluation value, and the circles on the left side indicate the evaluation values ​​related to normal images, while the squares on the right side indicate the evaluation values ​​related to images with anomalies. The (a) to (d) attached to the squares with anomalies are the values ​​corresponding to the images (a) to (d) shown at the bottom of the figure, respectively.

[0038] As shown in FIG. 7, according to the second determination method including the smoothing filter processing, the maximum value of the 6-division square error obtained differs greatly between normal and abnormal cases, and it was confirmed that the second determination method can determine normality or abnormality more accurately than the first determination method.

[0039] As described above, the palletizer system 1 according to this embodiment, which includes the anomaly detection system 50, can accurately detect anomalies before the cans 90 are stacked on a pallet. As a result, the occurrence of defective products can be prevented. In particular, the site where the palletizer 10 stacks the cans 90 is generally mechanized and human intervention is often not possible. The palletizer system 1 according to this embodiment has the advantage of being able to accurately detect anomalies without relying on human intervention. The stacking of the cans 90 by the palletizer 10 is the final stage of the manufacturing process and is the process immediately before the product is shipped, so that anomaly detection at this stage is particularly important in order to prevent the occurrence of defective products, and the effect of the anomaly detection system 50 according to this embodiment is particularly large.

[0040] In the palletizer system 1 of this embodiment, abnormalities are detected based on images captured by the imaging device 52. Alternatively, it is possible to detect fallen or dislodged cans using various sensors such as ultrasonic sensors, optical sensors, and pressure sensors. In this case, however, a large number of sensors would need to be installed to check the status of all of the cans 90 lined up in the width direction. In contrast, image-based analysis as in this embodiment is efficient because a wide range of information can be obtained with a small number of cameras.

[0041] Furthermore, in detection using the above-mentioned sensors, the sensors must be adjusted each time the conditions of the cans 90 change, for example when switching to the production of a different type of cans 90. In particular, such adjustments are required for all of the sensors, which must be provided in large numbers as described above. In contrast, according to this embodiment, it is relatively easy to generate an autoencoder for detecting anomalies in different types of cans 90, and even if the cans 90 to be produced are changed, it is possible to respond by simply changing the autoencoder introduced into the anomaly detection device 56.

[0042] In addition, an analysis similar to that shown in the above judgment example was also performed on cans 90 with different diameters, and it was confirmed that the anomaly detection system 50 of this embodiment functions similarly for cans 90 with different diameters.

[0043] In addition, the anomaly detection device 56 of this embodiment uses an autoencoder using machine learning, so that highly accurate detection of various conditions that was difficult to achieve by conventional rule-based image analysis can be relatively easily achieved. In this embodiment, the anomaly detection device 56 is configured to perform appropriate judgment even for a judged image in which the shoulder of the can 90 is included in the image or the image is acquired while moving, causing a lot of noise in the difference image, by dividing the image into multiple regions, calculating the squared error, and using the maximum value as the evaluation value used for judgment, or by performing processing using a smoothing filter along the way. The fact that accurate judgment can be performed even for an image that includes the shoulder of the can 90 shows that the anomaly detection device 56 of this embodiment has a very high degree of freedom regarding the object to be judged. In addition, the fact that accurate judgment can be performed even for an image acquired while moving shows that the anomaly detection device 56 of this embodiment has few restrictions on image acquisition and a very high degree of freedom regarding system design, including physical aspects such as the performance of the imaging device, the location where the imaging device is set, and the shooting conditions.

[0044] [others] To prevent any abnormality from being overlooked, in addition to the above-mentioned abnormality detection system 50, ultrasonic sensors, optical sensors, pressure sensors, etc. for detecting fallen cans or fallen cans may be further provided to form a multi-layered abnormality detection system. For example, since fallen cans are likely to occur at the outlet of the supply unit 11, a falling sensor may be provided at this position, and a falling sensor may be provided at the outlet of the alignment unit 12.

[0045] Although the present invention has been described above by showing preferred embodiments, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the present invention.

[0046] In the above embodiment, an example has been shown in which an autoencoder using deep learning is used to obtain a difference image in which the difference between the image of the cans 90, which are normal objects to be determined and regularly arranged, and the determined image obtained by the imaging device 52 is emphasized. However, the present invention is not limited to this, and other methods may be used as long as a difference image in which the difference between the image of the normal object to be determined and the determined image is emphasized can be obtained.

[0047] In addition, although the example has been given in which the sum of the squared values ​​of the differences in pixel values ​​for each divided region is used as the error used for evaluation, this is not limiting. Any other value may be used as long as it is a value indicating the error in the difference image for each divided region, i.e., a multi-division error.

[0048] In the above embodiment, an example has been shown in which the anomaly detection system 50 is applied to the palletizer 10 that stacks manufactured cans on a pallet, but the application is not limited to this. For example, the anomaly detection system 50 may be applied to processes related to other containers, such as resin containers and other containers, in addition to cans. These containers may be empty or filled with contents. Furthermore, the anomaly detection system 50 may be applied to other processes in which containers are regularly lined up, in addition to palletizers. The above-described embodiment of the present invention also includes the following inventions. a palletizer configured to align a plurality of containers and stack the aligned plurality of containers in a plurality of tiers on a pallet; an imaging device provided in the palletizer and configured to capture an image of a plurality of aligned containers before being stacked on the pallet while the plurality of containers are moving, thereby acquiring a plurality of divided images of the entirety of the aligned plurality of containers; an anomaly detection device configured to detect anomalies in the aligned containers based on the images acquired by the imaging device; and A palletizer system comprising: [Explanation of symbols]

[0049] 1 Palletizer System 10 Palletizer 11 Supply section 12 Alignment section 13 Transfer section 14 Palletizing Department 22 Partition 25 Extruder 28 Transfer machine 30 Control device 50 Anomaly Detection System 52 Imaging device 54 PoE Hub 56 Anomaly detection device 62 Alarm device 90 cans

Claims

1. Acquiring a subject image, which is an image of a subject object to be determined, in which cylindrical containers are arranged in an orderly manner in an upright state; creating a difference image in which a difference between an image of a normal object to be determined arranged in an orderly manner and the determined image is emphasized; calculating a multi-division error of the difference image and determining a maximum value of the multi-division error; determining whether the object is normal or abnormal based on the maximum value; The anomaly detection method includes:

2. The method further includes performing a smoothing filter process on the difference image before calculating the multi-division error, calculating the multi-division error for the difference image that has been subjected to the smoothing filter processing; The anomaly detection method according to claim 1 .

3. Creating the difference image includes: Inputting a determined image into an autoencoder configured to output an image of a normal determined object in response to an input of the image of the normal determined object by learning using images of normal determined objects arranged in a regular order, and acquiring a reference image which is an output of the autoencoder; calculating a difference between the determined image and the reference image to generate the difference image; The anomaly detection method according to claim 1 or 2, comprising:

4. the object to be determined is a plurality of containers to be stacked on a pallet, The autoencoder is generated by training using normal images of multiple aligned containers stacked on a pallet. The anomaly detection method according to claim 3 .

5. The anomaly detection method according to claim 4 , wherein the determined image includes an image obtained by photographing the determined object while it is moving.

6. The determining step includes comparing the maximum value with a predetermined threshold value, and determining that an abnormality exists when the maximum value is greater than the predetermined threshold value. The anomaly detection method according to any one of claims 1 to 5.

7. An anomaly detection program that causes a computer to execute the anomaly detection method according to any one of claims 1 to 6.

8. An anomaly detection device comprising a computer that executes the anomaly detection method according to any one of claims 1 to 6.

9. an imaging device configured to image the aligned containers prior to being stacked on a pallet; 6. An anomaly detection device comprising a computer that executes the anomaly detection method according to claim 4 or 5, the anomaly detection device acquiring an image obtained by the imaging device as a subject image and determining whether a plurality of containers loaded on the pallet are normal or abnormal; A palletizer anomaly detection system comprising:

10. a tipping sensor configured to detect a tipping container; a dislodging sensor configured to detect dislodging of the container; The palletizer anomaly detection system according to claim 9 , further comprising:

Images