Abnormality detection device

Abstract

To provide an abnormality detection device that can correctly determine a label pasting state and detect abnormalities.SOLUTION: An abnormality detection device includes: a determination model that is trained using an image of an object to which a label is pasted, with respect to product model number information; a camera group that acquires an image of the label pasted to the object; and a computer that is communicably connected to the camera group, and is configured to receive the image of the label from the camera group, generate numerical sequence data by binarizing the image of the label, acquire product model number information from the image of the label, perform abnormality detection determination on the label by using the determination model based on the image of the label and the acquired product model number information, and output a notification of an abnormality of the label if the abnormality is detected in the label.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an anomaly detection device. [Background technology]

[0002] In managing the manufacturing process of a product, a method of attaching a label with necessary information to the object is sometimes used. In this case, it is important that the correct label is attached to the object, that the label is attached in the correct position and facing the correct direction, and that the label is not soiled or damaged.

[0003] To manually prevent forgetting or mis-attaching labels, training is required when changing workers. Furthermore, it is impossible to completely prevent workers from overlooking abnormalities due to carelessness. Therefore, a known technology for inspecting the state of label attachment is to inspect labels attached to objects using image data obtained by photographing the objects.

[0004] For example, Patent Document 1 discloses a method and device for inspecting labels attached to food packs. The method and device in Patent Document 1 uses a line scan camera to capture an image of the label attached to the food pack. The captured image data is compared with reference data for pattern matching. The resulting pattern matching score is evaluated to inspect the label attached to the food pack.

[0005] However, this type of inspection method has difficulty making a correct judgment when labels overlap. Furthermore, if a label is tilted or attached in a position other than its intended position, the accuracy of the judgment may decrease, resulting in an inaccurate judgment. Furthermore, when multiple labels are attached, pattern matching must be performed on each label image as a master image, which increases the number of judgment processes and takes time. Furthermore, a master image is required for each type of label. Furthermore, there is a possibility that the characters printed on the label may differ from the characters printed on the master image used for pattern matching, for example, in terms of the manufacturing date and time. It is not easy to perform an inspection taking such differences into account. Therefore, label inspection methods have generally been developed under the assumption that labels do not overlap and that there are no differences in the characters on the labels. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2022-505986 Summary of the Invention [Problem to be solved by the invention]

[0007] As described above, label inspection methods using pattern matching require the preparation of master images for each type of label. Furthermore, it is not easy to perform inspections that take into account overlapping labels and differences in printed content.

[0008] An object of the present disclosure is to provide an anomaly detection device, an anomaly detection method, and a program that solve the above-mentioned problems. [Means for solving the problem]

[0009] An anomaly detection device according to one aspect of the present disclosure includes: A classification model trained using images of labeled objects for product model number information; A group of cameras that capture images of labels attached to objects; a computer communicatively connected to the group of cameras, receiving an image of the label from the camera group and generating sequence data by binarizing the image of the label; Obtaining product model number information from the image of the label; performing an abnormality detection determination for the label using the determination model based on the image of the label and the acquired product model number information; When an abnormality in the label is detected, a notification of the abnormality in the label is output. a computer configured to: Equipped with.

[0010] An anomaly detection method according to one aspect of the present disclosure includes the steps of generating a judgment model trained using images of labeled objects for product model number information; acquiring an image of a label attached to an object; generating sequence data by binarizing the image of the label; obtaining product model number information from the image of the label; a step of determining whether an abnormality has been detected in the label using the determination model based on the image of the label and the acquired product model number information; outputting a notification of the label abnormality when the label abnormality is detected; Equipped with.

[0011] A computer program according to one aspect of the present disclosure, when executed by a processor, generating a decision model trained using images of labeled objects for product model number information; acquiring an image of a label attached to an object; generating sequence data by binarizing the image of the label; obtaining product model number information from the image of the label; a step of determining whether an abnormality has been detected in the label using the determination model based on the image of the label and the acquired product model number information; outputting a notification of the label abnormality when the label abnormality is detected; The method includes instructions for causing a computer including the processor to execute the above steps. [Effects of the Invention]

[0012] According to the above aspect, it is possible to provide an anomaly detection device that can correctly determine the state of label attachment and detect anomalies without relying on pattern matching. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic block diagram of a label abnormality detection device according to the present disclosure. FIG. [Figure 2] FIG. 2 is a hardware configuration diagram of a computer in a label abnormality detection device according to the present disclosure. [Figure 3] FIG. 1 is a schematic block diagram of a group of cameras in a label anomaly detection device according to the present disclosure. [Figure 4] FIG. 1 is a flow diagram of a label anomaly detection method according to the present disclosure. [Figure 5] 10 is a flowchart showing a label abnormality determination process in the label abnormality detection method according to the present disclosure. FIG. [Figure 6] 10A and 10B are diagrams illustrating pre-processing of a captured image in the label anomaly detection method according to the present disclosure. [Figure 7] 10A and 10B are diagrams illustrating pre-processing of a captured image in the label anomaly detection method according to the present disclosure. [Figure 8] 10A and 10B are diagrams illustrating binarization and conversion of a captured image into sequence data in the label anomaly detection method according to the present disclosure. [Figure 9] 10A and 10B are diagrams illustrating binarization and conversion of a captured image into sequence data in the label anomaly detection method according to the present disclosure. [Figure 10]10A and 10B are diagrams illustrating a captured image and a number sequence data in the label anomaly detection method according to the present disclosure. [Figure 11] 10A and 10B are diagrams illustrating a captured image and a number sequence data in the label anomaly detection method according to the present disclosure. [Figure 12] 10A and 10B are diagrams illustrating a graphing process of sequence data in the label anomaly detection method according to the present disclosure. [Figure 13] 1 is an example illustrating a configuration of a label abnormality detection device according to the present disclosure. [Figure 14] FIG. 14 is a diagram showing a processing flow by the anomaly detection device shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Each embodiment will be described below with reference to the drawings. In all drawings, the same or corresponding components are designated by the same reference numerals, and common descriptions will be omitted.

[0015] First Embodiment Hereinafter, an embodiment according to the present disclosure will be described with reference to the drawings.

[0016] 1 is an exemplary schematic block diagram showing a label anomaly detection device 100 according to the present disclosure. The anomaly detection device 100 includes, for example, an infrared sensor 1 that outputs a signal detecting whether or not an object with a label attached thereto is present in a detection area, a computer 2 that receives the signal from the infrared sensor 1, determines that the object is present in the detection area, and executes label anomaly detection processing, a group of cameras 3 connected to the computer 2 and photographs the object in response to commands from the computer 2, and a database 4 included in or connected to the computer 2 and stores data necessary for the label anomaly detection processing. The computer 2 may be connected to the Internet 5.

[0017] The object may be, for example, a packaging box containing a manufactured product. It may also be a tray or carrier on which a manufactured product is temporarily placed. A label bearing various information necessary for the manufacture of the product, such as product model number information and process information of the contained product, may be affixed to the object. The various information may be printed on the label as a character string, a one-dimensional barcode, or a two-dimensional barcode. Alternatively, other coded information display methods may be used.

[0018] The infrared sensor 1 can be placed near a detection area where objects are placed during label inspection. The detection area may be a simple platform, or an area on a belt conveyor or roller conveyor where objects are transported. The infrared sensor 1 comprises an infrared emitting unit and an infrared detecting unit. The infrared sensor 1 emits infrared rays toward the detection area and detects the intensity of the reflected infrared rays. If the intensity of the reflected infrared rays is below a threshold, the infrared sensor 1 does not output a detection signal. If the intensity of the reflected infrared rays exceeds the threshold, the infrared sensor 1 outputs a detection signal to the computer 2.

[0019] In one embodiment, other sensors may be used instead of the infrared sensor 1. For example, an ultrasonic sensor including an ultrasonic emitter and an ultrasonic detector may be used. The ultrasonic sensor can output a signal to detect the presence of an object by detecting the distance to the object from the reflection time of the ultrasonic waves. Alternatively, a weight sensor provided on a support base on which the object is placed may be used. The weight sensor can detect that the object has been placed on the support base by a change in weight. The presence of an object may also be detected using image recognition by a visible light camera. When using such a visible light camera, a camera included in the camera group 3 may be used. If the object contains metal, a proximity sensor such as a Hall effect element or a magnetic sensor may also be used.

[0020] Fig. 2 is a hardware configuration diagram showing the computer 2 of the label anomaly detection device 100 according to the present disclosure. As shown in Fig. 2, the computer 2 is a computer including various pieces of hardware such as a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 203, a database 204, and a communication module 205.

[0021] FIG. 3 is an exemplary schematic block diagram showing the camera group 3 of the label anomaly detection device 100 according to the present disclosure. The camera group 3 may include multiple cameras 11, 12, 13, etc. The camera group 3 may be arranged near the detection area so as to capture an image of the object. When the computer 2 determines that an object is present in the detection area, the computer 2 may output an instruction to the camera group 3 to capture an image of the object. The cameras 11, 12, 13, etc. may be arranged in positions so as to capture an image of the label affixed to the object. For example, if the detection area is configured so that the surface of the object to which the label is affixed faces a predetermined direction, one of the cameras in the camera group 3 may be arranged to face this surface. If labels are affixed to multiple surfaces of the object, multiple cameras in the camera group 3 may be arranged to face each surface. If the detection area is not configured so that the surface of the object to which the label is attached faces in a predetermined direction, multiple cameras in the camera group 3 may be arranged to surround the object so that the label can be photographed regardless of the direction in which the surface of the object to which the label is attached faces.

[0022] The database 4 can store, for example, a trained decision model used for label anomaly detection, which will be described later. The database 4 can also store other data used for label anomaly detection and temporary data generated in the process of label anomaly detection.

[0023] Fig. 4 is an exemplary flow diagram showing a label anomaly detection method implemented by the label anomaly detection device 100 shown in Figs. 1 and 3. Note that, below, this embodiment will be described using an example in which the object is a packaging box and whether or not the object is present in the detection area is detected using an infrared sensor. However, this embodiment is not limited to this.

[0024] In the anomaly detection method implemented by the label anomaly detection device according to this embodiment, first, in step S101, the infrared sensor 1 emits infrared rays toward the detection area. The infrared sensor 1 also detects the intensity of the reflected infrared rays. If the intensity of the reflected infrared rays is equal to or less than a threshold, the infrared sensor 1 does not output a detection signal. This indicates that a packaging box is not present in the detection area. If a packaging box is present in the detection area, the infrared rays emitted from the infrared sensor 1 are reflected. If the infrared rays are reflected by the packaging box and the intensity of the reflected infrared rays exceeds a predetermined threshold, the infrared sensor 1 outputs a detection signal to the computer 2 indicating that a packaging box is present in the detection area. Note that the infrared sensor 1 may output a first detection signal to the computer 2 indicating that the presence of a packaging box is not detected if the intensity of the reflected infrared rays is equal to or less than the threshold, and may output a second detection signal indicating that the presence of a packaging box is detected if the intensity of the reflected infrared rays exceeds the threshold.

[0025] Next, in step S102, the computer 2 determines whether an object, for example, a packaging box, is present in the detection area based on the detection signal from the infrared sensor 1. If the output from the infrared sensor 1 indicates that a packaging box is not present, the method returns to step S101. If the output from the infrared sensor 1 indicates that a packaging box is present, the method proceeds to step S103.

[0026] In step S103, the computer 2 causes the camera group 3 to photograph the packing box and acquire image data. When the packing box is placed in the detection area so that the surface to which the label is attached faces a specific direction, the camera placed facing the specific direction acquires an image of the packing box and transmits the image data to the computer 2. When the packing box is not configured in the detection area so that the surface to which the label is attached faces a specific direction, the cameras of the camera group 3 are placed to surround the packing box, photograph the packing box from each direction, and transmit the image data to the computer 2.

[0027] Next, in step S104, a process for determining whether a label has been left affixed or affixed incorrectly is performed. Steps S201 to S209 in Fig. 5 are detailed explanations of step S104 in Fig. 4.

[0028] In the anomaly detection method implemented by the label anomaly detection device according to this embodiment, after step S103 is completed, in step S201, pre-processing is performed on the captured image acquired in step S103. The acquired captured image is cropped as shown in FIG. 6 to remove the background, leaving only one side of the packaging box. The cropping can be performed using an object detection algorithm such as YOLO. If the surface of the packaging box is at a predetermined angle with respect to the direction of the camera, angle correction may be performed before cropping to correct the image so that the surface appears as if it were captured from the front. Various corrections such as tilt correction and exposure correction may also be performed.

[0029] Next, as shown in Figure 7, the cropped image is subjected to binarization processing. In the binarization processing, pixels having a predetermined brightness set as a threshold or brighter than this brightness are converted to white, and pixels darker than this brightness are converted to black. For the binarization processing, global binarization is used, in which the same threshold is used for all pixels of the image.

[0030] Next, the image is converted into sequence data. First, as shown in Figure 8, the total number of white pixels is calculated for each column of the binarized image, and sequence 1 is obtained using the image column as an index. In the example shown in Figure 8, the obtained sequence 1 is graphed. It can be seen that in columns where no label exists in the image, such as column 2000, the value of sequence 1 is close to 0, while in columns where a label exists, sequence 1 has a predetermined value greater than 0. It can also be seen that the value of sequence 1 correlates with the height of the label in the image, and there is a low-frequency rectangular fluctuation. Furthermore, it can be seen that there is a high-frequency fluctuation in sequence 1 in the parts of the label where there is printing.

[0031] Next, as shown in Fig. 9, the total number of white pixels is calculated for each row of the binarized image, and sequence 2 is obtained using the row of the image as an index. In the example shown in Fig. 9, the obtained sequence 2 is shown in a graph. The correlation between the image and sequence 2 is the same as sequence 1 for each column of the image, which was explained with reference to Fig. 8.

[0032] Next, as shown in FIG. 10, sequence 1 and sequence 2 are concatenated to obtain one piece of sequence data. FIG. 10 shows an example in which sequence 2 is added after sequence 1 to obtain sequence data. It is also possible to create a two-dimensional matrix in which sequence 1 is placed in the first row and sequence 2 is placed in the second row. In this case, if the length of sequence 2 is shorter than that of sequence 1, a predetermined number of 0s may be added as dummy data to the end of sequence 2 to make sequence 2 the same length as sequence 1. However, the method of obtaining sequence 1, sequence 2, and sequence data is not limited to the example described above.

[0033] Next, returning to FIG. 5, in step S202, the label position is identified from the sequence data. As mentioned above, there is a correlation between the presence of a label and fluctuations in the sequence values. Therefore, it is possible to determine the area where a label exists as pixel coordinates based on fluctuations in the sequence values. More specifically, it is possible to determine the area where a label exists as an area where the sequence value exceeds a predetermined threshold. Note that if multiple labels exist, the coordinate range may be determined for each label. For example, in the examples shown in FIGS. 6 to 9, there are three labels. Therefore, it is possible to identify the coordinate range for each label.

[0034] Next, in step S203, for each coordinate area determined to contain a label, OCR technology is applied to the image to acquire character string data printed on the label. From the acquired character string data, for example, product model number information is identified. Other information may also be identified. Alternatively, instead of using OCR technology, product model number information can be acquired by reading barcode information or two-dimensional barcode information printed on the packaging box.

[0035] Next, in step S204, based on the sequence data and product model number information, the character string data and sequence data of the product model number information are input to a judgment model, and a graph waveform judgment of the sequence data is performed. The judgment model may be constructed, for example, by machine learning using a multilayer neural network. The judgment model is trained, for example, as follows: Product model number information and sequence data obtained from images of packaging boxes to which a label corresponding to the product model number information has been properly attached are input to the judgment model as training data. The judgment model is trained so as to obtain a correspondence between the waveform of the sequence data and the product model number information. Parameters in each layer of the judgment model are updated, for example, by backpropagation. When the character string data and sequence data of the product model number information are input to the trained judgment model, a binary judgment is made as to whether or not there is an abnormality in the graph waveform based on the correspondence between the waveform of the sequence data and the product model number information, and the binary judgment is output. For example, the output value may be 1 if there is an abnormality in the graph waveform, and 0 if there is no abnormality in the graph waveform.

[0036] The judgment model can also be constructed by machine learning using a convolutional neural network. Even when using a convolutional neural network, for example, product model number information and sequence data obtained from images of packaging boxes to which labels corresponding to the product model number information have been properly attached can be used as training data. The parameters of each layer in the neural network are optimized using backpropagation with errors, allowing the convolutional neural network to learn the correspondence between the waveform of the graph and the normal state of the label.

[0037] Label abnormalities may include, for example, damage, soiling, improper placement, improper placement angle, missing labels, overlapping labels, or different labels. For example, sequence data corresponding to a normal label may be a rectangle of a predetermined width and height. However, if the label is damaged and partially missing, the waveform of the graph of the sequence data may appear as if some of the rectangle is missing. If the label is soiled, the waveform of the graph of the sequence data may appear as if some of the rectangle is missing. If the label is positioned abnormally, the rectangle of the waveform of the graph may deviate from a specific range. If the label placement angle is normal, the rectangle of the waveform of the graph may have sharp edges. However, if the label placement angle is abnormal, the rectangle of the waveform of the graph may have gentle edges, forming a trapezoidal shape overall. If no label is attached, no rectangle exists. If labels are attached overlapping, the rectangles corresponding to the two labels may merge into one, resulting in a stepped waveform. If different labels are applied, the range of the rectangle in the graph waveform may differ from the normal range, or the pattern of high frequency fluctuations corresponding to the printing within the rectangle may be different.

[0038] It is also possible to classify overlapping labels as normal. In this case, it is possible to separate the rectangle of the graph waveform that serves as the label into two and determine the label status for each.

[0039] Figure 10 shows an example of a graph of sequence data when there is an abnormality where one label is attached too many times. The graph waveform of the sequence data shows that the number of white pixels is increasing, corresponding to the large number of attached labels.

[0040] 11 shows an example of a graph of sequence data when there is an abnormality to which no label is attached. In the graph waveform of the sequence data, there is no rectangle corresponding to the label.

[0041] Similarly, it is possible to obtain a graph waveform of the corresponding sequence data for abnormalities such as stained or partially damaged labels.

[0042] As described above, when the labeling state is abnormal, a specific change appears in the graph waveform compared to when the labeling state is normal. Therefore, a judgment model can be trained to associate such specific changes with abnormality modes. In this case, images corresponding to each of the above-mentioned label abnormality modes can be used as training data, in addition to the above-mentioned product model number information and images of packaging boxes with properly labeled boxes. The judgment model trained in this way not only simply determines whether the labeling state is normal or not by binarizing it, but also can output the corresponding abnormality mode.

[0043] Next, returning to FIG. 5, in step S205, the result of the binarization judgment output from the judgment model in step S204 is judged. If the output value is 1, that is, if it is judged that there is an abnormality in the waveform of the graph, the process proceeds to step S206, where it is judged that there is a missing or incorrect label. If the output value is 0, that is, if it is judged that there is no abnormality in the waveform of the graph, the process proceeds to step S207, where it is judged that there is no missing or incorrect label. As described above, if the judgment model is configured to output an abnormality mode, it can output a value corresponding to each abnormality mode.

[0044] Next, in step S208, the determination result of step S206 or S207 is stored in the database 4. Various data such as the acquired image, number sequence data, character string data, etc. may also be stored in the database 4.

[0045] Next, in step S209, it is determined whether or not there are any undetermined images. If there are any undetermined images, the process returns to step S201 to determine the images. If there are no undetermined images, the process proceeds to step S105.

[0046] Returning to FIG. 4, if it is determined in step S105 based on the determination result data stored in the database that a label has been left behind or has been affixed incorrectly, the process proceeds to step S106, where a notification of the left behind or affixed incorrectly is output. If a value indicating an abnormal mode is also output as described above, the corresponding abnormal mode may also be notified. For example, the administrator may be notified using a text message, email, visual message, warning sound, etc. via the Internet 5 or another network, such as an intranet. If it is determined that no label has been left behind or affixed incorrectly, the inspection of the label on the packaging box is completed, and the process returns to step S101, where inspection of the label on the next packaging box is started.

[0047] According to this embodiment, it is possible to prevent forgetting or mis-attaching of labels by determining with high accuracy and in a short processing time without requiring human labor costs. Furthermore, by detecting label attachment patterns and identifying attachment positions using a determination model that utilizes AI such as a neural network, it is possible to improve the issues of image pattern matching, such as the reduction in determination accuracy due to tilted or overlapping labels, and the cumbersome need to register a large number of image patterns in advance.

[0048] Fig. 11 is a diagram showing an example of the configuration of a label abnormality detection device, and Fig. 12 is a diagram showing a processing flow of the label abnormality detection device 100 shown in Fig. 11.

[0049] The label anomaly detection device 100 includes at least a computer 2 , a camera group 3 , and a database 4 .

[0050] The camera group 3 photographs the object and acquires image data of the label affixed to the object (step S103). The computer 2 generates sequence data by binarizing the image of the label photographed by the camera group 3 (step S201), acquires product model number information from the image of the label (step S203), and performs a label abnormality detection determination using a determination model stored in the database 4 based on the image of the label and the product model number information (step S204). If an abnormality in the label is detected, a notification of the abnormality in the label is output (step S106).

[0051] The anomaly detection device includes a computer system. The above-described processes are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes the program to perform the above processes. Here, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program.

[0052] The program may also be a program for realizing some of the functions described above. Furthermore, the program may be a so-called differential file (differential program) that can realize the functions described above in combination with a program already recorded in the computer system.

[0053] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0054] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0055] (Appendix 1) A classification model trained using images of labeled objects for product model number information; A group of cameras that capture images of labels attached to objects; a computer communicatively connected to the group of cameras, receiving an image of the label from the camera group and generating sequence data by binarizing the image of the label; Obtaining product model number information from the image of the label; performing an abnormality detection determination for the label using the determination model based on the image of the label and the acquired product model number information; When an abnormality in the label is detected, a notification of the abnormality in the label is output. a computer configured to: Equipped with Anomaly detection device.

[0056] (Appendix 2) 2. The anomaly detection device according to claim 1, wherein the sequence data is generated based on a sum of the number of white pixels for each pixel column and a sum of the number of white pixels for each pixel row of the binarized label image.

[0057] (Appendix 3) The anomaly detection device according to appendix 1 or 2, wherein the anomaly detection judgment of the label is performed by inputting sequence data obtained by binarizing the image of the label and character string data of the product model number information into the judgment model.

[0058] (Appendix 4) The anomaly detection device according to any one of appendixes 1 to 3, wherein the anomaly detection determination for the label is performed based on the waveform of the graph of the sequence data.

[0059] (Appendix 5) The anomaly detection device according to any one of appendices 1 to 4, wherein the label anomaly detection determination is configured to binary-determine whether the state of the label is normal or abnormal and output the binary determination.

[0060] (Appendix 6) The anomaly detection method according to any one of appendices 1 to 5, wherein the label anomaly detection determination is configured to output a value indicating the mode of the label anomaly if the state of the label is abnormal.

[0061] (Appendix 7) The product model number information is Identifying a coordinate range of the label from sequence data obtained by binarizing the image of the label; 9. The anomaly detection device according to any one of appendices 1 to 8, configured to acquire character string data written on the label for each coordinate range of the label.

[0062] (Appendix 8) An anomaly detection device according to any one of appendices 1 to 7, configured to train the judgment model using at least product model number information and sequence data obtained from images in which the label has been properly attached.

[0063] (Appendix 9) generating a decision model trained using images of labeled objects for product model number information; acquiring an image of a label attached to an object; generating sequence data by binarizing the image of the label; obtaining product model number information from the image of the label; a step of determining whether an abnormality has been detected in the label using the determination model based on the image of the label and the acquired product model number information; outputting a notification of the label abnormality when the label abnormality is detected; Equipped with Anomaly detection methods.

[0064] (Appendix 10) 10. The anomaly detection method according to claim 9, wherein the step of generating the sequence data includes generating the sequence data based on a sum of the number of white pixels for each pixel column and a sum of the number of white pixels for each pixel row of the binarized image of the label.

[0065] (Appendix 11) The anomaly detection method according to claim 9 or 10, wherein the step of detecting and determining an anomaly in the label includes inputting sequence data obtained by binarizing the image of the label and character string data of the product model number information into the determination model.

[0066] (Appendix 12) 12. The anomaly detection method according to any one of appendixes 9 to 11, wherein the step of detecting and determining an anomaly in the label includes detecting and determining an anomaly in the label based on a waveform of a graph of the sequence data.

[0067] (Appendix 13) 13. The anomaly detection method according to any one of appendices 9 to 12, wherein the step of detecting and determining an anomaly in the label includes binarizing and outputting whether the state of the label is normal or abnormal.

[0068] (Appendix 14) 14. The anomaly detection method according to any one of appendices 9 to 13, wherein the step of determining whether an anomaly has been detected in the label includes outputting a value indicating the mode of the anomaly in the label if the state of the label is abnormal.

[0069] (Appendix 15) The step of acquiring product model number information includes: Identifying a coordinate range of the label from sequence data obtained by binarizing the image of the label; and acquiring character string data written on the label for each coordinate range of the label.

[0070] (Appendix 16) 16. The anomaly detection method according to any one of appendixes 9 to 15, wherein the learning of the judgment model includes using at least product model number information and sequence data obtained from an image in which the label has been properly attached.

[0071] (Appendix 17) When executed by a processor, generating a decision model trained using images of labeled objects for product model number information; acquiring an image of a label attached to an object; generating sequence data by binarizing the image of the label; obtaining product model number information from the image of the label; performing an abnormality detection determination for the label using the determination model based on the image of the label and the acquired product model number information; outputting a notification of the label abnormality when the label abnormality is detected; a computer program comprising instructions for causing a computer including the processor to execute the above-mentioned program;

[0072] (Appendix 18) 18. The computer program of claim 17, wherein generating the sequence data includes generating the sequence data based on a sum of white pixels for each pixel column and a sum of white pixels for each pixel row of the binarized label image.

[0073] (Appendix 19) The anomaly detection method according to claim 17 or 18, wherein the performing of the anomaly detection determination for the label includes inputting sequence data obtained by binarizing the image of the label and character string data of the product model number information into the determination model.

[0074] (Appendix 20) 20. The anomaly detection method according to any one of appendixes 17 to 19, wherein performing an anomaly detection determination on the label includes performing an anomaly detection determination on the label based on a waveform of a graph of the sequence data.

[0075] (Appendix 21) 21. The anomaly detection method according to any one of appendices 17 to 20, wherein performing an anomaly detection determination for the label includes binarizing and outputting whether the state of the label is normal or abnormal.

[0076] (Appendix 22) 22. The anomaly detection method according to any one of appendices 17 to 21, wherein performing an anomaly detection determination for the label includes outputting a value indicating an anomaly mode for the label if the state of the label is abnormal.

[0077] (Appendix 23) The acquisition of the product model number information includes: Identifying a coordinate range of the label from sequence data obtained by binarizing the image of the label; and acquiring character string data written on the label for each coordinate range of the label.

[0078] (Appendix 24) 24. The anomaly detection method according to any one of appendices 17 to 23, wherein the learning of the judgment model includes using at least product model number information and sequence data obtained from an image in which the label has been properly attached. [Explanation of symbols]

[0079] 100 Label anomaly detection device 1. Infrared sensor 2. Computer 201 CPU(Central Processing Unit) 202 ROM (Read Only Memory) 203 RAM (Random Access Memory) 204 Database 205 Communication Module 3 Cameras 4 Database 5. Internet

Claims

1. A database that stores a decision model trained using images of labeled objects for product model number information; A group of cameras that capture images of labels attached to objects; a computer communicatively connected to the group of cameras, receiving an image of the label from the camera group and generating sequence data by binarizing the image of the label; Obtaining product model number information from the image of the label; performing an abnormality detection determination for the label using the determination model based on the image of the label and the acquired product model number information; When an abnormality in the label is detected, a notification of the abnormality in the label is output. a computer configured to: Equipped with Anomaly detection device.

2. 2. The anomaly detection device according to claim 1, wherein the sequence data is generated based on a sum of the number of white pixels for each pixel column and a sum of the number of white pixels for each pixel row of the binarized label image.

3. 2. The anomaly detection device according to claim 1, wherein the anomaly detection determination for the label is performed by inputting sequence data obtained by binarizing the image of the label and character string data of the product model number information into the determination model.

4. The anomaly detection device according to claim 1 , wherein the determination of anomaly detection of the label is performed based on a waveform of a graph of the sequence data.

5. 2. The anomaly detection device according to claim 1, wherein the label anomaly detection determination is configured to binary-determine whether the state of the label is normal or abnormal and output the binary-determined result.

6. The anomaly detection device according to claim 1 , wherein the label anomaly detection determination is configured to output a value indicating a mode of the anomaly of the label when the state of the label is abnormal.

7. The product model number information is Identifying a coordinate range of the label from sequence data obtained by binarizing the image of the label; The anomaly detection device according to claim 1 , configured to acquire the anomaly detection information by acquiring character string data written on the label for each coordinate range of the label.

8. 2. The anomaly detection device according to claim 1, wherein the learning of the judgment model is performed using at least product model number information and sequence data obtained from images in which the label has been properly attached.

9. generating a decision model trained using images of labeled objects for product model number information; acquiring an image of a label attached to an object; generating sequence data by binarizing the image of the label; obtaining product model number information from the image of the label; a step of determining whether an abnormality has been detected in the label using the determination model based on the image of the label and the acquired product model number information; outputting a notification of the label abnormality when the label abnormality is detected; Equipped with Anomaly detection methods.

10. When executed by a processor, generating a decision model trained using images of labeled objects for product model number information; acquiring an image of a label attached to an object; generating sequence data by binarizing the image of the label; obtaining product model number information from the image of the label; performing an abnormality detection determination for the label using the determination model based on the image of the label and the acquired product model number information; outputting a notification of the label abnormality when the label abnormality is detected; a computer program comprising instructions for causing a computer including the processor to execute the above-mentioned program;

Images