Anomaly detection system and method for detecting anomalies

Abstract

The present invention relates to an abnormality detection system configured to detect an abnormality on a surface of an object via a contact image sensor line scan camera, in which a capture speed is adjusted to a moving speed of the object, and acquired image data is analyzed using a machine learning model, and a processor to provide information on the detected anomaly on the user interface, information on the necessary action on the user interface, and a command on the necessary action, where the necessary action is based on the detected anomaly. The invention also relates to an anomaly detection method.

Description

Technical Field

[0001] The present invention relates to an anomaly detection system and a method for detecting anomalies. Such a system and / or such a method can preferably be used to detect anomalies on a lamination belt. The anomalies can represent defects on the lamination belt. Background Art

[0002] US Pat. No. 6,748,124 B1 discloses an image processing system. The system is arranged above a conveyor belt and includes an illumination device and an optical system for imaging an image of the surface of an object on a line sensor. The illumination device and optical system are moved relative to the object while controlling the clock rate supplied to the line sensor, thereby adjusting the image signal transmission speed of the line sensor to match the processing speed of the image processing system.

[0003] Detecting anomalies on the object is particularly important during the lamination process. Defects on the laminated object must be detected as anomalies, and necessary actions must be taken to reduce waste. For example, when using a laminating system with a laminating tape, defects on the tape, such as glue marks, scratches, or small paper residues, can be transferred to the laminated object. For example, glue stuck to the laminating tape can create black marks on the laminated object (e.g., a data sheet). This results in waste and must be avoided.

[0004] One solution is to manually inspect the object or laminate, assess any detected anomalies (such as defects), and decide on necessary actions, such as cleaning the object or laminate or changing process parameters. However, this is time-consuming and results in downtime for the machinery being used. Simply recording the surface of the object or laminate and detecting defects as anomalies presents additional challenges, such as space limitations and difficulty recording images of sufficient quality, depending on the method used. When recording moving objects, such as laminate, the object's varying speed and surface characteristics can create additional difficulties. Furthermore, conventional area scan or line scan sensors are not suitable for monitoring in settings with reflective surfaces and the necessary resolution to detect small anomalies. They must be positioned at a considerable distance from the object being recorded and require lighting, which can further generate unwanted reflections.

[0005] CN 2 08 366 870 U relates to an online measuring device for label printing defects comprising a CIS sensor.

[0006] DE 10 201 4 011 268 A1 relates to a device which is configured to optically detect defects on an object, in particular micropores in a material film, using a focused line laser. Summary of the Invention

[0007] Therefore, the object of the present invention is to provide a system and method for detecting anomalies in an object that solves the aforementioned problems. In particular, the present invention aims to allow for the detection of anomalies in an object and the provision of information about the detected anomalies to a user. Another object of the present invention is to provide the user with information about necessary actions based on the detected anomalies, or to provide a signal corresponding to a necessary action.

[0008] These objects are solved by an anomaly identification system and a method for detecting anomalies according to the independent claims. Preferred embodiments of the solution are defined by the dependent claims.

[0009] An abnormality recognition system includes: a contact image sensor line scan camera, arranged above an object and configured to record image data of a surface of the object, wherein the contact image sensor line scan camera includes an image sensor and an illumination device; a height-adjustable mounting member, wherein at least the contact image sensor line scan camera is arranged above the object by the height-adjustable mounting member; an encoder, configured to record a moving speed of the object; and a frame grabber, coupled to the encoder and the contact image sensor line scan camera, wherein the frame grabber is configured to receive information about the moving speed of the object from the encoder and is further configured to control the contact image sensor line scan camera to move the object. A contact image sensor line scan camera is configured to record image data when an object moves, and is configured to adjust the capture speed of the contact image sensor line scan camera according to the movement speed of the object when the object moves; an image processing device is configured to process the image data provided by the contact image sensor line scan camera, and use a trained machine learning model and / or use a rule-based model to detect anomalies in the image data, and is configured to provide information about the detected anomaly on a user interface and / or provide information about necessary actions on the user interface and / or provide commands about necessary actions, wherein the necessary actions are based on the detected anomaly.

[0010] When the object is not moving, the frame grabber may be configured to control the contact image sensor line scan camera to stop recording image data.

[0011] The capture speed of a contact image sensor line scan camera may refer to the speed at which triggers are provided to the contact image sensor line scan camera. A frame grabber may configure the contact image sensor line scan camera to capture a line with the contact image sensor line scan camera whenever a trigger arrives. The trigger may be provided by an encoder and a frame grabber, which are configured based on the speed of movement of the object. Such a configuration ensures that there is no oversampling or undersampling of the object. Oversampling occurs when the contact image sensor line scan camera captures the same line multiple times due to overtriggering. Undersampling occurs when the trigger is slower than the speed of movement of the object. Therefore, increasing the capture speed of the contact image sensor line scan camera may correspond to decreasing the time interval between two consecutive triggers. Decreasing the capture speed of the contact image sensor line scan camera may correspond to increasing the time interval between two consecutive triggers.

[0012] The frame grabber can be configured to provide multiple triggers to the contact image sensor line scan camera at a high speed and / or variable speed synchronized with the speed of movement of the object. The frame grabber can also be configured to transmit the captured image data to the image processing device. The image data can be high-resolution image data.

[0013] The detected anomalies may represent defects on the surface of the object.The surface of the object is considered to be the main surface directed towards a contact image sensor line scan camera (hereinafter also referred to as a CISL camera).

[0014] CISL cameras are designed to be placed very close to the objects they record. To achieve this close arrangement, the distance between the camera and the surface of the recorded object can be less than 50 mm, preferably less than 30 mm, more preferably less than 20 mm, and most preferably 12 mm or less than 10 mm.

[0015] The CISL camera includes a contact image sensor (CIS) configured as a line scan sensor and an illumination device. Because the CISL camera is positioned close to the surface of the object to be recorded and includes its own illumination device, unwanted reflections on the surface of the object can be minimized or prevented, even when scanning reflective objects such as laminated tape.

[0016] Furthermore, reflections from other objects surrounding the CISL camera are prevented, and undesirable light interference can also be reduced or prevented. This is also important because CIS cameras provide image data as an image of light reflectivity and can be susceptible to undesirable lighting. CIS cameras allow for distortion-free recording of image data because they include 1:1 optics and are mounted directly above the object being recorded.

[0017] The picture angle can be constant over its read width, since the sensor is as wide as the object to be recorded, for example the sensor can be as wide as the recorded strip.

[0018] The height-adjustable mount allows adjustment of the CISL camera-to-subject distance, thereby enabling the use of the anomaly recognition system (hereinafter referred to primarily as the system) for varying subjects or varying subject distances. The height-adjustable mount can be controlled by the image processing component to adjust its height. This allows for height changes when the subject distance is insufficient or when the image processing device detects a change in subject distance.

[0019] Using CISL cameras reduces the system's space requirements. Due to their sensor type and integrated lighting, CISL cameras require significantly less space than conventional recording devices or line scan cameras. This allows the system to be used even in space-constrained settings for detecting anomalies, such as when detecting anomalies in laminating systems using laminating tapes.

[0020] In the following, the description refers to exceptions, however, the details described also apply to a single exception.

[0021] Because the recorded image data is processed and one or more anomalies are detected by a trained machine learning model and / or by a rule-based model, the system can be used in different settings with adjusted models depending on the exact requirements of the setting. Detecting anomalies via the model allows the detected anomalies to be interpreted and classified according to their type. Such types could be glue marks, scratches, or other defects on the surface of the recorded object. CISL cameras record image data based on the amount of light reflected from the recorded object. Thus, anomalies can be recorded as changes in reflectivity and / or changes in color, and therefore as changes in the light reflected into the sensor. Each pixel can record this information, and changes in the amount of light detected by the pixel can be related to an anomaly.

[0022] The image processing device is configured to detect anomalies in image data using a trained machine learning model and / or using a rule-based model. The trained machine learning model can be trained to detect anomalies by detecting differences from image data without anomalies and then classifying the anomalies. Alternatively, the trained machine learning model can be trained to directly detect anomalies by detecting similarities to anomalies defined in the training data. This includes detecting anomaly types that are related to defect types. In addition, these two alternatives can also be implemented by rule-based methods. The rule-based method may include rules for detecting anomalies by comparing differences in recorded image data with image data without anomalies and then classifying the detected anomalies. Or the rule-based method may include rules for directly detecting anomalies by comparing similarities in the recorded image data with defined anomaly types. The rule-based model can be based on a conventional rule-based computer algorithm, wherein the trained machine learning model can be based on a data-driven machine learning algorithm. In the following, the term "detection" of anomalies describes either of these two alternatives used with either model.

[0023] The rules used can include a recorded change in a specific number of pixels or a defined area related to the size of the recorded change. For example, a rule could be that if a certain number of pixels or a certain area size shows a defined change in color or reflectivity, then the area is classified as a detected anomaly. Rules can also include pixel intensity, which defines how bright a pixel is. Another aspect that can be part of the rules is the location of the recorded change on the object. Certain areas may be more relevant to the detection of an anomaly or can be ignored.

[0024] By providing information about detected anomalies on the user interface, users can identify anomalies without having to manually inspect the object. This can reduce downtime of machines using the system.

[0025] The object is recorded while it moves, and even at varying speeds, the frame grabber controls the CISL camera to adjust its capture speed and ensure sufficient image quality for model analysis. Since image data is recorded only while the object is moving, a stoppage of the object (e.g., a belt stoppage) does not result in incorrect multiple detections of anomalies located in the area covered by the CIS when the object stops. Furthermore, the amount of data required for processing and analysis is reduced. The frame grabber can be configured to synchronize the capture speed of the CISL camera with the object's speed / movement velocity. Therefore, as the object's speed changes, the capture speed of the CISL camera can vary accordingly. For example, when the object's speed decreases, the frame grabber can be configured to reduce the capture speed by increasing the time interval between successive triggers to capture new images. This reduces the capture frequency, thereby reducing or even preventing overlap between adjacent parts of the object. Similarly, when the object's speed increases, the frame grabber can be configured to increase the capture speed by decreasing the time interval between successive triggers to capture new images, thereby avoiding undersampling of the object and, consequently, an incomplete image of the object. Therefore, adapting the capture speed based on the object's moving speed can ensure that the image quality is sufficient for subsequent analysis by the image processing device regardless of the object's moving speed. In particular, if the object is a rotating belt, adjusting the capture speed based on the object's moving speed can ensure that the image quality is sufficient for subsequent analysis by the image processing device regardless of the rotation speed of the rotating belt.

[0026] If the object to be recorded is a rotating belt, the anomaly detection system may be configured to recognize a complete rotation of the belt and thus be able to stop recording images or start analyzing image data if the entire surface of the belt has been recorded and processed.

[0027] Furthermore, the system can be configured to identify and record the location of the anomalies recorded and / or analyzed. This allows the system to be used to check the presence of identified anomalies, for example, after a cleaning or repair process. Thus, the image processing device and the model used can also be configured to compare anomalies detected in at least two recordings of the same area of ​​an object.

[0028] The image processing device is configured to process image data provided by the CISL camera and provide processed image data adapted to the requirements of the model. For example, the image processing device may be configured to divide a recorded object into sections and further divide the image data of the sections into subsections before analyzing the model. The image processing device may be configured to detect anomalies in each subsection of the section and may be configured to combine the results of each subsection and provide the combined information on the user interface.

[0029] According to an embodiment of the present invention, the object is a belt. The belt may be a laminated belt or a conveyor belt. The belt or laminated belt may be a rotating belt. Therefore, the system detects anomalies on the surface of the belt or laminated belt. The system can also be configured to detect anomalies on the belt or laminated belt and anomalies on the surface of an object placed on the belt or laminated belt. Therefore, the trained machine learning model and / or rule-based model is configured to distinguish between the belt and objects on the belt. Anomalies on the belt may be defects on the belt, such as glue marks, scratches, or small paper residues.

[0030] According to an embodiment of the present invention, the object is a rotating belt; the encoder is configured to pick up a motor signal of a motor driving the belt; or the encoder is coupled to a rotating drum guiding and / or driving the belt. The rotating belt may be a laminate belt.

[0031] Depending on the type of motor used to drive the belt, the motor can provide a motor signal that provides information about the angular position or angular motion, and therefore about the motor's rotational speed, which defines the speed of movement of the belt. The motor signal can be generated by an encoder of the motor. Therefore, the encoder can be composed of this signal coupled to the frame grabber. The encoder can also be coupled to a rotating drum that guides and / or drives the belt. The rotating drum can be rotated by the motor or by the belt itself. The encoder can be a rotary encoder and can be of mechanical, optical, or magnetic type. The encoder and rotating drum are configured to interact so as to allow the encoder to record changes in the rotation angle and provide this information to the frame grabber. For example, it can be configured to detect the angular position or movement of the rotating drum through multiple signals or marks and convert them into analog or digital signals that are provided to the frame grabber. The frame grabber is configured to calculate the speed of movement of the belt based on the configuration of the elements of the moving belt (for example, the diameter of the rotating drum).

[0032] According to an embodiment of the present invention, an encoder is configured to record the speed of movement of an object from the surface of the object, and the surface is provided with a mark to be recorded by the encoder. For example, the encoder is configured to record the speed of movement by means of a dedicated mark provided on the surface of the object or by means of a textured surface of the object. It can be configured to convert the recorded information into an analog or digital signal that is provided to a frame grabber. The encoder can be arranged above the object via a height-adjustable mount. This allows the system to be adapted to a variety of use cases, depending only on the provision of a recordable surface of the object to be recorded. In addition, this allows the system to be configured with fewer components for starting the system. For example, a CISL camera, a height-adjustable mount, an encoder, a frame grabber, and an image processing device can be arranged in one unit, and the user interface can be part of the unit or can represent a second unit. The encoder can also be represented by a CISL camera.

[0033] According to an embodiment of the present invention, the image processing device is further configured to provide information and / or provide commands regarding necessary actions by processing the detected anomalies and deciding necessary actions using a trained machine learning model and / or using a rule-based model.

[0034] The information and / or commands about the necessary actions may include instructions such as stopping the object, cleaning the object or changing parameters of the applied process. In the following, the description also relates to actions, however, the details described also apply to single actions.

[0035] Utilizing a rule-based model, the image processing device can be configured to provide information about the detected anomaly on the user interface and / or provide information about necessary actions on the user interface and / or provide commands about necessary actions based on the detected anomaly by selecting the information to be provided or the commands to be provided from a list according to a rule. For example, if the anomaly is classified or detected as a specific type, the image processing component can select the corresponding entry associated with this type in the list based on the rule-based model. The list can include multiple different options that can be associated with the size and / or type of the detected anomaly. For example, if the detected anomaly is classified as a glue mark, the rule-based model can prompt the image processing device to select an entry associated with the glue mark type. This can be information about the detected glue mark, a recommended necessary action or a command. For example, in the case where the object is a laminate tape, the information about the necessary action can include cleaning the laminate tape, monitoring the detected anomaly or ignoring the detected anomaly. The command can, for example, be a command provided to a cleaning tool to clean the object.

[0036] Alternatively or additionally, a trained machine learning model is trained to evaluate detected anomalies and decide whether action is necessary, and if so, what action to provide as information on a user interface and / or as a command. The machine learning model can be trained for a use case of the system. For example, in the case where the object is a laminating tape, the machine learning model can be trained for anomalies that occur on the laminating tape.

[0037] The image processing apparatus may be configured to apply a machine learning model and / or a rule-based model to the results of anomaly detection of each portion and / or sub-portion of the portion, and may be configured to combine the results of each portion and / or sub-portion and provide combined information on the user interface. The image processing apparatus may be configured to apply the model to the combined results of anomaly detection of each portion and / or sub-portion.

[0038] The use of machine learning and / or rule-based models significantly improves the accuracy and adequacy of the necessary actions determined and helps reduce downtime for machines using the system. For example, the machine typically does not have to stop to clean an object, as this is unnecessary and the object can be cleaned during the machine's next scheduled stop. In the exemplary case of a laminating belt, if a detected anomaly affects the object being laminated, the belt only needs to be stopped and cleaned abnormally.

[0039] According to an embodiment of the present invention, the machine learning model and / or rule-based model for providing information and / or providing commands about necessary actions is a machine learning model and / or rule-based model for detecting anomalies. The machine learning model of such an embodiment can be trained to detect anomalies, evaluate them, and decide on necessary actions. In addition, a machine learning model can be used to detect anomalies, and a rule-based model can be used to provide information and / or provide commands about necessary actions. On the other hand, a rule-based model can be used to detect anomalies, and a machine learning model can be used to provide information and / or provide commands about necessary actions.

[0040] The present invention also relates to a method for detecting anomalies on an object. The method comprises the following steps: recording a speed of movement of the object; recording image data from a surface of the object with a contact image sensor line scan camera, wherein the image data is recorded as the object moves and wherein a capture speed of the image data is adjusted according to the speed of movement of the object; processing the image data; detecting anomalies in the image data using a trained machine learning model and / or a rule-based model; providing information about the detected anomaly to a user and / or providing information about a necessary action to the user and / or providing a command about the necessary action, wherein the necessary action is based on the detected anomaly.

[0041] When image data is recorded by a contact image sensor line scan camera (also referred to as a CISL camera in the following), the image data is recorded as the amount of light detected by each pixel of the CISL camera, which is based on the amount of light reflected by the object. Since the CISL camera records close to the object surface and illuminates its recording area with its own lighting device, unwanted reflections on the surface of the object can be minimized or prevented even when scanning reflective surfaces such as laminating tape. As a result, the recorded image data includes anomaly areas, which are characterized by differences in the amount of reflected light from areas of the object. Recording with a CISL camera also reduces the space requirements of the system. This allows the method to be applied even in spatially restricted settings for detecting anomalies, such as when detecting anomalies in laminating systems using laminating tape.

[0042] Because the recorded image data is processed and anomalies are detected using a trained machine learning model and / or rule-based model, the system can be used in different settings with adjusted models, depending on the exact requirements of the setting. Anomalies can represent defects on the surface of the recorded object. Detecting anomalies using the used model allows for interpretation of detected anomalies and classification according to their type, or for direct detection of defined types of anomalies. Such types could include glue marks, scratches, or other defects on the surface of the object being inspected. By providing information about detected anomalies based on the image data recorded from the moving object, the user can identify anomalies without having to manually inspect the object. This reduces downtime for machines using this method.

[0043] Recording image data while adjusting the capture speed according to the object's movement speed improves the quality of the recorded image data and ensures successful and effective analysis by any model. Adjusting the capture speed also ensures that areas of the object that are not being recorded are not missed or recorded more frequently than necessary. Since image data is recorded only when the object is moving, the required data processing capacity and processing time are reduced. Furthermore, stopping the object, such as stopping the belt, does not result in incorrect multiple recordings of the same area of ​​the object. This could unnecessarily increase the amount of data required to be processed and may result in multiple detections of the same anomaly, depending on the image processing.

[0044] According to an embodiment of the present invention, the step of providing information about necessary actions to the user and / or the step of providing commands about necessary actions includes processing the detected anomalies and utilizing a trained machine learning model and / or a rule-based model to decide the necessary actions.

[0045] The information and / or commands about the necessary actions may include instructions such as stopping the object, cleaning the object or changing parameters of the applied process. In the following, the description also relates to actions, however, the details described also apply to single actions.

[0046] The steps of providing information about the detected anomaly to the user and / or providing information about necessary actions to the user and / or providing commands about necessary actions can be performed by a rule-based model, wherein the necessary actions are based on the detected anomaly. For example, this includes selecting the information to be provided or the commands to be provided from a list. For example, if the anomaly is classified as a specific type, the step includes selecting the corresponding entry associated with that type in the list. The list can include several different options that can be associated with the size and / or type of the detected anomaly. For example, if the detected anomaly is classified as a glue mark, the step will select the entry associated with the glue mark type according to the defined rules. This can be information about the detected glue mark, a recommended necessary action or a command. For example, in the case where the object is a laminate tape, the information about the necessary actions can include cleaning the laminate tape, monitoring the detected anomaly or ignoring the detected anomaly. The command can, for example, be a command provided to a cleaning tool to clean defects from the object.

[0047] The trained machine learning model is trained to evaluate detected anomalies and decide whether action is necessary, and if so, what action to provide to the user as information and / or as a command. The machine learning model can be trained for a use case of the system, for example, in the case where the object is a laminating tape, the machine learning model can be trained for anomalies that occur on the laminating tape. For example, the machine learning model can be trained to evaluate the type, size, and location of a detected anomaly and decide whether action is necessary, and if so, which action is most appropriate.

[0048] The use of machine learning models and / or rule-based models significantly improves the accuracy and adequacy of the necessary actions determined and helps reduce downtime for machines using the method. For example, the machine typically does not have to stop to clean an object, as this is unnecessary and the object can be cleaned during the machine's next scheduled stop. In the exemplary case of a laminating belt, if a detected anomaly affects the object being laminated, the belt only needs to be stopped and abnormally cleaned.

[0049] If the object being recorded is a rotating belt, the anomaly detection method can be configured to identify a complete rotation of the belt, thus enabling the recording or analysis of images to be stopped once the entire belt surface has been recorded and processed. Furthermore, the method can be configured to identify and record the location of recorded and / or analyzed anomalies on the object. This allows the system to be used to verify the presence of identified anomalies, for example, after a cleaning or repair process. Therefore, the method and the model used can also be configured to compare anomalies detected in at least two recordings of the same area of ​​the object.

[0050] According to an embodiment of the present invention, the method further includes the step of adjusting the height of the contact image sensor line scan camera above the object. This step is preferably performed before the step of recording image data. This step includes adjusting the height of the CISL camera to a defined working distance from the object.

[0051] By performing this step, the method can be adapted to the height of the object being recorded. For example, the size of the object being recorded can change, which would move the distance between the object's surface and the CISL camera beyond the optimal or necessary working distance. By performing the height adjustment step, the defined optimal working distance can be adjusted. This step can also be performed during the recording of image data.

[0052] According to embodiments of the present invention, the step of using a trained machine learning model and / or rule-based model to determine necessary actions can be performed using a machine learning model and / or rule-based model for detecting anomalies. The machine learning model of such an embodiment can be trained to detect anomalies, evaluate them, and determine necessary actions. In addition, a machine learning model can be used in the step of determining necessary actions, and a rule-based model can be used to detect anomalies. On the other hand, a rule-based model can be used in the step of determining necessary actions, and a machine learning model can be used to detect anomalies.

[0053] According to an embodiment of the present invention, the step of processing image data includes dividing the image data into parts and / or sub-parts, and wherein abnormalities are detected separately for each part and / or sub-part, and wherein providing information about the detected abnormalities to the user and / or providing information about necessary actions to the user and / or providing commands about necessary actions is based on the combined parts and / or sub-parts.

[0054] This embodiment allows for efficient processing of image data recorded from large objects and accelerates the process of detecting anomalies in the image data. CISL cameras record image data as image lines that are equal to the length of the CIS (CIS scan length). Therefore, the recorded image data is provided in the form of number of lines x CIS scan length. Because the CIS provides very high-resolution image lines, and the recording length can result in a large number of recorded lines, the image data is provided in the form of very high-resolution images. Applying models to detect anomalies and optionally determining necessary actions on this very high-resolution image data requires significant processing power, particularly when using machine learning models. Therefore, the image data can be divided into sections and / or subsections. A section can represent a portion of the recorded object. A subsection can represent a subsection of the recorded image or section in the form of tiles. Such subsections still contain high-resolution image data sufficient to detect small anomalies, but require less processing power to be processed by the corresponding model. Because the machine learning model used to process the image data uses a convolutional neural network (CNN) operating at a defined input resolution, the sections or subsections can have the desired defined input resolution. The image data can be resized before or after division into sections and / or subsections. The method may apply a model for processing detected anomalies and deciding necessary actions to the results of anomaly detection for each sub-part and / or portion individually, and may be configured to combine the results of each sub-part and / or portion and provide the combined information to the user. Alternatively, the method may be configured to apply a model for processing detected anomalies and deciding necessary actions to the combined results of anomaly detection for each sub-part and / or portion.

[0055] According to an embodiment of the present invention, the object is a rotating belt, and the step of recording the movement speed of the belt includes: recording a motor signal of a motor driving the belt, and / or recording the rotation speed of a rotating roller that guides and / or drives the belt, and / or recording the surface of the belt. The rotating belt may be a laminated belt.

[0056] Thus, the method detects anomalies on the belt surface. The method can also be configured to detect anomalies on the belt and anomalies on the surface of an object placed on the belt. Thus, the corresponding model is configured to distinguish between the belt and an object on the belt. The detected anomalies can represent defects on the belt, such as glue marks, scratches, or small paper residues.

[0057] The described anomaly identification system can be configured to perform the described anomaly detection method, and the anomaly detection method can be configured to be executed on the anomaly identification system. Naturally, if technically possible, the described embodiment can only perform the method or be executed on the system. The subject matter of the present invention is not limited to the features of the individual embodiments, but may also include any technically possible combination of the described embodiments that fall within the independent claims or their corresponding dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Hereinafter, preferred embodiments of the present invention are described with reference to the accompanying drawings:

[0059] Figure 1 A first embodiment of an anomaly detection system according to the present invention is shown;

[0060] Figure 2 A second embodiment of an anomaly detection system according to the present invention is shown;

[0061] Figure 3 A third embodiment of an anomaly detection system according to the present invention is shown;

[0062] Figure 4 A belt on which the abnormality detection system according to any one of the first to third embodiments is used is shown;

[0063] Figure 5a shows stylized processing steps performed by the system of any one of the first to third embodiments using a method according to an embodiment of the present invention;

[0064] Figure 5b shows stylized processing steps performed by the system of any one of the first to third embodiments using a method according to an embodiment of the present invention;

[0065] Figure 6 The steps of a method according to an embodiment of the present invention are shown. DETAILED DESCRIPTION

[0066] Figure 1 A first embodiment of an anomaly detection system according to the invention is shown. The anomaly detection system 10 comprises a contact image sensor line scan camera 12 arranged above an object 14. In the present embodiment, the object 14 is a laminating belt of a laminating machine and is therefore referred to as belt 14 hereinafter, without limiting the embodiment to such an object. Such a laminating machine comprises two laminating belts which convey and laminate objects, such as data pages for official documents, such as for passports. Due to the narrow space around the belts and the reflective surface of the belts, conventional anomaly detection systems may not be used in this case. The CISL camera 12 of the inventive system 10 comprises a contact image sensor 16 and its own lighting device 18. It is arranged very close to the belt 14 at a distance d. In the present embodiment, this distance may be, for example, 12 mm above the belt 14. As described above, the use of a CISL camera 12 provides a number of advantages, in particular when used in an environment where the object is a reflective laminating belt. The CISL camera 12 records image data of the surface 20 of the belt 14 and is configured to extend over the entire width of the belt 14 (for details see Figure 4). The CISL camera 12 is arranged above the belt 14 by means of a height-adjustable mount 22. Using the mount 12, the distance d can be varied so that the CISL camera 12 is always arranged at a preferred distance d from the belt 14, as explained, for example, 12 mm. This is particularly advantageous because the belt 14 can change its position depending on the objects to be laminated, which may include objects of varying thicknesses.

[0067] The belt 14 is a rotating belt that rotates around two rotating drums 24. In this embodiment, at least one of the rotating drums 24 is equipped with a motor 26 to drive the belt 14. Here, the left rotating drum 24 is equipped with a motor 26. In this embodiment, the motor 26 provides a motor signal 28. The motor signal 28 provides information about the rotation of the motor 26 by providing information about the angular position or angular motion. Therefore, the motor signal 28 can be an encoder signal of an encoder of the motor. When the belt 14 is driven by the motor 26, the rotation of the motor 26 is related to the moving speed of the belt 14. Therefore, the motor signal 28 constitutes an encoder 30 that records the moving speed of the belt 14.

[0068] Information regarding the speed of movement, here in the form of motor signal 28, is provided to frame grabber 32. Frame grabber 32 is coupled to encoder 30 (here in the form of motor signal 28) and CISL camera 12. It is configured to control CISL camera 12 to adjust its capture speed based on the speed of movement of belt 14. For example, if motor 26 rotates at 10 rpm, belt 14 moves at a defined speed depending on the size of rotating drum 24. Consequently, surface 20 of belt 14 moves a defined distance in a defined amount of time, resulting in a defined speed.

[0069] Motor signal 28 returns a specific number of ticks (N) per motor revolution. Assuming the number of revolutions of motor 26 is the same as the number of revolutions of rotating drum 24, variable L can be defined as the circumference of rotating drum 24 together with belt 14. Therefore, the motion (M) of belt 14 is defined by M = L / N.

[0070] By this movement, an area of ​​the belt 14 passes through the recording area of ​​the CISL camera 12. To ensure clear and unblurred recorded image data, the frame grabber 32 controls the CISL camera 12 and can adjust its capture speed (also defined as the temporal resolution R) accordingly. To achieve this adjustment, the motor signal 28 can be multiplied or divided by the frame grabber 32. The corresponding multiplication factor is defined as R / M. After this operation, the frame grabber 32 can adjust the CISL camera 12 accordingly. The motor 30 can provide a fixed signal for which the number of ticks per revolution cannot be changed, or the motor 30 can provide a programmable signal for which the number of ticks per revolution can be programmed.

[0071] Furthermore, if the encoder 30 provides zero velocity, e.g., no change in angular position, the frame grabber 32 controls the CISL camera 12 to stop recording image data of the surface of the tape 14. This prevents multiple recordings of the same area of ​​the object covered by the CIS 16 during the stop and subsequent anomaly detection, and helps reduce the amount of data necessary to be analyzed. Similarly, the motor signal 28 serves as a trigger to correspondingly start recording image data.

[0072] The recorded image data is provided to an image processing device 34. The image processing device 34 is configured to process the image data and utilize a trained machine learning model and / or a rule-based model to detect anomalies in the image data. Hereinafter, this will be referred to simply as a machine learning model. This is merely for ease of description; a rule-based model may alternatively or additionally be used as described above. The image processing device is also configured to provide information about the detected anomaly and / or information about necessary actions and / or commands (not shown) on the user interface 36, where the necessary actions are based on the detected anomaly.

[0073] Figure 2 A second embodiment of an anomaly detection system 110 according to the present invention is shown. Hereinafter, the anomaly detection system 110 of the second embodiment will be described solely based on its differences from the first embodiment. As previously mentioned, the above also applies to system 110. System 110 according to the second embodiment includes an encoder 130 that reads information about the angular position or angular motion of the motor 26 and / or the shaft of the rotating drum 24 through direct or indirect measurement. For example, encoder 130 is mounted to the rotating drum 24 or the shaft of the rotating drum. Encoder 130 provides a signal 128 that returns a specific number of ticks (N) per revolution of the measured shaft. Signal 128 can then be processed by the frame grabber 32, as previously described. Encoder 130 can be a fixed-signal encoder, where the number of ticks per revolution cannot be changed, or a programmable encoder, where the number of ticks per revolution can be programmed.

[0074] Figure 3A third embodiment of an anomaly detection system according to the present invention is shown. Hereinafter, the anomaly detection system 210 of the third embodiment will be described only in terms of its differences from the first and second embodiments, with the aforementioned also applying to system 210. System 210 according to the third embodiment includes an encoder 230 disposed above the surface 20 of the belt 14. In the present case, it is also disposed by a height-adjustable mount 22. Encoder 230 is configured to record the moving speed of the belt 14 from its surface 20 and provide a signal 228 to a frame grabber 32. Surface 20 is provided with marks 38 (only a few exemplary marks 38 are shown) to be recorded by encoder 230. Marks 38 are configured to correspond to the exact type of encoder 230, which can be an optical, magnetic, or resistive encoder. Marks 38 can be arranged so that each mark represents the number of ticks N as described above. Encoder 230 can also be represented by a CISL camera 12, which can also function as encoder 230 by recording corresponding marks 38 on the belt 14. In this case, the CISL camera 12 may provide a corresponding signal 228 or information to the frame grabber 32, which may be implemented via the image processing device 34. Such a signal or information may also be represented by some type of detected anomaly associated with the marker 38.

[0075] Figure 4 Shown according to Figures 1 to 3 The above views of any system of an embodiment of the present invention are provided on which the anomaly detection method of the present invention is performed. The top surface 20 of the belt 14 is shown, as it is a rotating belt, and half of the recordable surface 20 is shown. The CISL camera 12 is configured to extend across the entire width of the belt 14 so that the CISL camera 12 can record the entire belt 14 as the belt 14 moves accordingly. The belt 14 includes a plurality of anomalies 40 on its surface 20. These may represent defects such as glue marks, scratches, or paper residue. As the belt 14 moves beneath the CISL camera 12 in its normal operating procedure, the defects 40 each pass through the sensor area of ​​the CIS 16. They are then recorded by the CISL camera 12 and provided to the image processing device 34 as part of the image data of the belt surface.

[0076] Figure 5a Shown in accordance with Figure 6 Step S3 is shown processing image data 50 relating to the surface 20 of the belt 14 . Figures 1 to 3 The entire surface 20 of the belt 14 of any embodiment is represented in the image data 50, so the image data 50 is recorded from an entire rotation of the belt 14. The image data 50 need not be recorded continuously, but can be stitched together from different recording steps, which can also include overlapping areas of the belt 14.

[0077] Because the entire image data 50 has a very high resolution resulting from the size of the strip 14 and the resolution of the CIS 16, a machine learning model may not be able to economically analyze it all at once. Therefore, the image data 50 can be divided into equal sections 42, 44, 46, 48. These can also represent four repeating sections of the surface 20 of the strip 14. To further reduce the size of the image data to be analyzed at once, each section is divided into a left section 42a, 44a, 46a, 48a and a right section 42b, 44b, 46b, 48b. In other words, the image data 50 is segmented into eight sections. Each section 42a, 44a, 46a, 48a, 48b, 48a, 48b, 42b, 44b, 46b, 48b can have a resolution of 1024×1024 pixels. For each section, a model of the present method can be applied. Since the machine learning models that process image data can use convolutional neural networks (CNNs) that operate on a defined input resolution, the parts can be further divided into subparts to match the corresponding input resolution of the corresponding machine learning model applied. Figure 5b In the processing, each portion 42a, 44a, 46a, 48a, 42b, 44b, 46b, 48b is further divided into four equal sub-portions I, II, III, and IV. Each sub-portion can have a resolution of, for example, 256×256 pixels. Each sub-portion I, II, III, and IV is analyzed by a trained machine learning model to detect anomalies in the image data 50 of the corresponding sub-portion I, II, III, and IV, as shown in FIG5 . For example, two identical machine learning models are simultaneously applied to each portion 42, 44, 46, 48, where each model analyzes one of the two portions 42a, 44a, 46a, 48a, 42b, 44b, 46b, 48b associated with each portion 42, 44, 46, 48. Each model analyzes sub-portions I through IV individually and provides results based on the entire portion 42a, 44a, 46a, 48a, 42b, 44b, 46b, 48b. Furthermore, the results of the portions 42 , 44 , 46 , 48 may be combined before output. In this case, the output of the detected anomalies may include the detected anomalies for the entire image data 50 .

[0078] Figure 6 The steps of an embodiment of the anomaly detection method of the present invention are shown. In the following, the method is described as follows: Figures 1 to 3 The present invention may be performed on any embodiment of the present invention, but is not limited thereto.

[0079] In step S1, the speed of movement of the object (in this case, the belt 14) is recorded. This can be achieved by the different embodiments of the encoders 30, 130, 230 described. As soon as the belt 14 moves, the frame grabber 32 can instruct the CISL camera 12 via an input signal to record image data (step S2). The recording can also depend on a second input signal 52, here for example by the image processing device 34. Depending on the requirements of the setup using the method, continuous recording may not be required every time a movement is recorded in step S1. For example, recording may only be necessary at regular intervals. In addition, the frame grabber 32 and / or the image processing device 34 can provide a signal that controls the image recording based on the area of ​​the belt 14 in the recording area of ​​the CIS 16. Depending on the movement recorded in step S1, the capture speed can also be adjusted in step S2 to match the movement as described above.

[0080] In step S3, the recorded image data is processed and, for example, can be divided into sections and subsections as shown in FIG5. A trained machine learning model is applied to the processed image data for anomaly detection (step S4). In this embodiment, two identical machine learning models are applied to respective sections of the image data 50, as described with respect to FIG5. The model (hereinafter referred to as one model, as they are identical) is trained to detect anomalies in the image data associated with anomalies 40 on the surface 20 of the belt 14. For example, the model can be trained to distinguish between undesirable anomalies and other anomalies associated with known characteristics of the belt. Such undesirable anomalies may be defects on the surface 20. The model is trained to detect such defects, classify them according to their type and size, and link them to information about such defects from a database.

[0081] After detecting an anomaly (step S4), the method includes various options for continuing. The detected anomaly and corresponding information can be provided to the user on the user interface 36 (step S5). Using this information, the user can identify the detected defect and act accordingly, such as by changing parameters of the lamination process on which the method is being used or by cleaning or replacing the belt 14.

[0082] Furthermore, the detected anomalies can be processed by a machine learning model to evaluate them and determine whether action is necessary, and if so, which action is necessary (step S6). This includes evaluating the detected anomalies based on their classification type, size, and location on the belt 14. The machine learning model is preferably trained to the context of the method. In this embodiment, it is trained to the lamination process settings and to detect anomalies associated with defects 40 and distinguish these defects from other anomalies present on objects conveyed on the belt 14 or on the surface of the belt 14. Such other anomalies can be markings 38.

[0083] The machine learning model can be applied to the image data 50 including the detected anomalies. If applied to the image data 50, it can be applied to the various parts described in FIG. 5 . In this case, the model can also be a model for detecting anomalies, and steps S4 and S6 can be performed by the same model. Such a combined model is trained to detect anomalies and evaluate them accordingly. The machine learning model of step S6 can also be trained to evaluate only the areas identified as anomalies by step S4 and / or to evaluate detected anomalies based on the results of step S4 including size and classification information.

[0084] If the machine learning model of step S6 determines a necessary action, information about the necessary action in the current situation can be presented to the user via user interface 36 (step S7). This information can include instructions on how to perform the necessary action. Based on this information, the user can, for example, stop the machine and clean the belt 14 based on the type of defect detected, or repair the defective area. Alternatively or additionally, the method provides a command regarding the selected necessary action (step S8). This command can be processed by the machine using the method; for example, it can cause the belt 14 to stop and / or a cleaning tool to be applied to the belt 14. Therefore, the method can also be applied to machines that do not require a user to react to detected anomalies and / or necessary actions. This further improves the usability of the method.

[0085] The embodiments described and shown in the figures can of course be combined with one another in any way, as long as this is technically possible. The invention is therefore not limited to the embodiments shown, but comprises any combination of embodiments covered by the claims.

[0086] Reference Signs List

[0087] d is the distance to the object

[0088] L circumference

[0089] M Sport

[0090] Compound 1 ticks per revolution

[0091] R resolution (capture speed)

[0092] 10;110;210 Anomaly Detection System

[0093] 12 Contact Image Sensor Line Scan Camera (CISL Camera)

[0094] 14 objects (with)

[0095] 16-line contact image sensor (CIS)

[0096] 18 lighting fixtures

[0097] 20 Object (belt) surfaces

[0098] 22 height adjustable mounting parts

[0099] 24 rotating drum

[0100] 26 motors

[0101] 28;128;228 signal

[0102] 30;130;230 encoder

[0103] 32-frame grabber

[0104] 34 Image processing device

[0105] 36 User Interface

[0106] 38 Markings on the object (belt)

[0107] 40 Abnormalities on the surface 20

[0108] 42,44,46,48 Part of image data 50

[0109] 42a, 44a, 46a, 48a, left part of parts 42, 44, 46, 48

[0110] 42b, 44b, 46b, 48b, right part of parts 42, 44, 46, 48

[0111] 50 image data

[0112] 52 input signals

[0113] I, II, III; Subpart IV

Claims

1. An abnormality identification system (10; 110; 210), including: a contact image sensor line scan camera (12) arranged above an object (14) and configured to record image data (50) of a surface (20) of the object (14), wherein the contact image sensor line scan camera (12) comprises an image sensor (16) and an illumination device (18); a height-adjustable mounting member (22) by which at least the contact image sensor line scan camera (12) is arranged above the object (14); an encoder (30; 130; 230) configured to record a moving speed of the object (14); a frame grabber (32) coupled to the encoder (30; 130; 230) and the contact image sensor line scan camera (12), wherein the frame grabber (32) is configured to receive information (28; 128; 228) about the movement speed of the object (14) from the encoder (30; 130; 230), is configured to control the contact image sensor line scan camera (12) to record image data (50) when the object (14) moves, and is configured to adjust a capture speed of the contact image sensor line scan camera (12) according to the movement speed of the object (14); An image processing device (34) is configured to process image data (50) provided by the contact image sensor line scan camera (12) and detect anomalies (40) in the image data (50) using a trained machine learning model and / or using a rule-based model, and is configured to provide information about the detected anomaly (40) on a user interface (36) and / or provide information about a necessary action on the user interface (36) and / or provide a command about a necessary action, wherein the necessary action is based on the detected anomaly (40).

2. The abnormality identification system (10; 110; 210) according to claim 1, wherein: The object (14) is a rotating belt.

3. The abnormality identification system (10, 110) according to claim 2, wherein the encoder (30) is configured to pick up a motor signal (28) of a motor (26) driving the belt; or in, The encoder (130) is coupled to a rotating drum (24) that guides and / or drives the belt.

4. The anomaly identification system (210) according to any one of the preceding claims, wherein: The encoder (230) is configured to record a speed of movement of the object (14) from a surface (20) of the object (14), and wherein the surface (20) is provided with markings (38) to be recorded by the encoder (230).

5. The abnormality identification system (210) according to claim 4, wherein: The encoder (230) is arranged above the object (14) via the height-adjustable mount (22).

6. Anomaly identification system (10; 110; 210) according to any one of the preceding claims, wherein The image processing device (34) is configured to provide the information and / or provide a command regarding the necessary action by processing the detected anomaly (40) and deciding the necessary action using a trained machine learning model and / or using a rule-based model.

7. The abnormality identification system (10; 110; 210) according to claim 6, wherein: The machine learning model and / or rule-based model for providing said information and / or providing commands regarding said necessary actions is a machine learning model and / or rule-based model for detecting said anomaly (40).

8. Anomaly identification system (10; 110; 210) according to any one of the preceding claims, wherein The anomaly (40) represents a defect on the surface (20) of the object (14).

9. Anomaly identification system (10; 110; 210) according to any one of the preceding claims, wherein: The system (10; 110; 210) is configured to perform the method of any one of claims 10 to 14.

10. A method for detecting anomalies, comprising the following steps: Recording the moving speed of the object (step S1); recording image data from the surface of the object with a contact image sensor line scan camera, wherein the image data is recorded as the object moves, and wherein a capture speed of the image data is adjusted according to a speed of movement of the object (step S2); processing the image data (step S3); Detecting anomalies in the image data using the trained machine learning model and / or rule-based model (step S4); Providing information to the user about the detected anomaly (step S5) and / or Providing information to the user about necessary actions (step S7) and / or A command regarding a necessary action is provided (step S8), wherein the necessary action is based on the detected anomaly.

11. The abnormality detection method according to claim 10, wherein: The step of providing information about the necessary action to the user (step S7) and / or the step of providing a command about the necessary action (step S8) includes processing the detected anomaly and utilizing a trained machine learning model and / or a rule-based model to decide the necessary action (step S6).

12. The abnormality detection method according to claim 11, wherein: The machine learning model and / or the rule-based model is a machine learning model and / or a rule-based model for detecting anomalies.

13. The abnormality detection method according to any one of claims 10 to 12, wherein: The step of processing the image data (step S4) comprises dividing the image data into parts (42, 44, 46, 48) and / or sub-parts (I, II, III; IV) and wherein the anomaly is detected for each part (42, 44, 46, 48) and / or sub-part (I, II, III; IV) individually, and wherein providing information about the detected anomaly to the user and / or providing information about necessary actions to the user and / or providing commands about necessary actions is based on the combined parts (42, 44, 46, 48) and / or sub-parts (I, II, III; IV).

14. The abnormality detection method according to claims 10 to 13, wherein: The object is a rotating belt, and the step of recording the moving speed of the belt comprises recording a motor signal of a motor driving the belt, and / or recording the rotation speed of a rotating drum guiding and / or driving the belt, and / or recording the surface of the belt.

15. The abnormality detection method according to any one of claims 10 to 14, wherein: The method is configured to be executed on an anomaly identification system (10; 110; 210) according to any one of claims 1 to 8.

Images