System and method for detecting an irregularity of an object

Abstract

The invention relates to observation and inspection, respectively, of an object, e.g. for quality control purposes. The solution for camera-based inspection of an object for detecting a presence of an irregularity of the object assumes that the object and the camera are moving relative to each other. The camera is embodied as a DVS camera which is configured pixel-wise generation of event signals in case a light flux on a pixel changes by more than a given threshold. The presence of an irregularity is assumed in case a constellation of event signal generating pixels deviates from a predefined pattern.

Description

[0001] Description

[0002] System and method for detecting an irregularity of an obj ect

[0003] The invention relates to observation and inspection, respectively, of an obj ect , e . g . for quality control purposes .

[0004] Optical inspection in mass production suf fers the problem of a huge amount of data being created when obj ects shall be inspected at their velocity of transport in the production with high lateral resolution, i . e . in the direction of the transport as well as perpendicular to the transport direction . Typical data rates caused by corresponding camera systems might be in the range above 1 or even 5 Gbit / s . This amount of data needs to be transmitted and pre-evaluated in a smart camera or an edge device in order to make the data stream to an end device like a superordinated production control system manageable . Lower data rates would result in less resolution and, therewith, lower inspection accuracy which would in the end have a negative impact on the quality of the production process .

[0005] State of the art inspection systems capture the image data in high volume with a suitable camera and transmit captured data to an edge device which performs data pre-evaluation or even full evaluation . In alternative solutions , the data is transmitted to a cloud system for analysis . In some cases , smart cameras are used where CPUs or FPGAs are incorporated in order to identi fy defects at an early stage of the process and to crop a relevant area of interest in the images or to aggregate the data e . g . by packaging etc . In many applications line-scan cameras are in use in order to have a more constant data rate than with full field cameras , but the general topic of the high data volume stays exactly the same for the same resolution . Thus, the market does not have a suitable concept in place which reduces the data stream on the one side and, on the other side, still covers a large dynamic range of e.g. 80 to lOOdB or even higher in applications of inspecting objects in scenarios in which the object to be inspected and the inspecting camera system are moving relative to each other. The high dynamic range is necessary to cover different illumination conditions in the field of view of the camera and to maintain good imaging properties to ensure the high inspection accuracy and quality. Typical full field or line cameras only have 40 to 60dB of maximal dynamic range.

[0006] Therefore, a solution is required which enables fast and reliable detection of an irregularity of an object. This is solved by the method suggested in claim 1 and by the system as per claim 13. Dependent claims describe beneficial embodiments of the method and system.

[0007] The method for camera based inspection of an object for detecting a presence of an irregularity of the object, e.g. defects or production errors, assumes that the object and the camera are moving relative to each other during the inspection in a transport direction TD. Therein, the expression "object moving relative to the camera" covers a scenario with a moving object and a motionless camera, a scenario with a motionless object and a moving camera, as well as a scenario with a moving object and a moving camera. The camera is embodied as a DVS camera which comprises a sensor chip with a plurality of sensor elements SE(i) , i.e. pixels, with i=l,...,I and I representing the total number of sensor elements of the sensor chip. The DVS camera is configured to generate for each sensor element SE(i) individually, i.e. pixel wise, an event signal SIG(i) , selected from a predefined group GRP of event types. The group GRP can include at least a positive event and a negative event. An event signal SIG(i) is generated only in case a light intensity or light flux, respectively, impinging onto the respective sensor element SE(i) changes by more than a given threshold THRES, e.g. 15% , while no event signal is generated in case the flux change is less than the threshold THRES. The method comprises an irregularity detection stage, during which the presence of an irregularity of the object is assumed in case one or more event signals SIG(i) generated by the DVS camera during the relative movement of a section of the object through a field of view FOV of the DVS camera fulfill a predefined condition.

[0008] The DVS camera is configured and arranged such that it captures a given field of view and such that the field of view covers a section of the surface of the object.

[0009] The predefined group GRP of event types includes a positive event and a negative event, wherein the event signal SIG(i) , in case generated, of a respective sensor element SE(i) is a positive event in case the light flux changes to a higher light flux by an amount DELTAFX with DELTAFXRTHRES . The event signal SIG(i) , in case generated, is a negative event in case the light flux changes to a lower light flux by an amount DELTAFX with DELTAFXRTHRES . In case the light flux change DELTAFX is lower than or equal to the threshold THRES, i.e.

[0010] DELTAFX<THRES , no event signal SIG(i) at all is generated for the respective sensor element SE(i) . Thus, event signals are only generated in case of significant, i.e. DELTAFXRTHRES , flux changes.

[0011] A presence of an irregularity is assumed only in case one or more event signals SIG(i) are generated for one or more sensor elements SE(i) .

[0012] More sophisticated, the presence of an irregularity is assumed only in case a temporal and / or spatial constellation of those one or more sensor elements SE(i) , for which respective event signals SIG(i) are generated during the irregularity detection stage, forms a pattern in time, i.e. considering series of event signals of the same sensor element over time, and / or in space, i.e. considering event signals of different sensor elements, which either corresponds to or deviates from a predefined reference pattern .

[0013] Therein, the reference pattern can be a pattern in space. The reference pattern might be such a spatial constellation of sensor elements momentarily generating an event signal, which constellation would be caused by a certain section of the object momentarily positioned in the field of view in case no significant irregularities of the object are present in the certain section. In that case, the presence of an irregularity can be assumed in case a constellation of the one or more sensor elements SE(i) , for which respective event signals SIG(i) are momentarily generated, forms a pattern in space, which deviates from the predefined reference pattern.

[0014] In other words, such reference pattern corresponds to and represents, respectively, an expected momentary appearance or spatial constellation of event signal generating sensor elements of the sensor chip, under the precondition that the object has no irregularity in its section which is momentarily located in the DVS camera's field of view. This utilizes that the optical appearance of the object and its sections in the field of view are known so that event signals generated by the sensor elements of the DVS camera during movement of the sections through the field of view, i.e. "constellations of sensor elements", can be easily modeled. As soon as a momentary constellation of event signal generating sensor elements deviates from such a modeled constellation, the presence of an irregularity can be assumed .

[0015] The use of a well-defined reference pattern is especially beneficial in case the object surface is inhomogeneous, e.g. in case the object consists of a plurality of components like in a mass production system or in case a texture on a surface of the object shows a significant roughness or contrast elements. The inhomogeneity results in a plurality of event signals, even in case no irregularity is present. Thus, in contrast to the scenario in which the surface of the object is essentially homogeneous, the use of the threshold THRES alone does not suffice to ensure that irregularities can be detected with high reliability.

[0016] However, the constellation of sensor elements generating event signals depends on the known arrangement or form the inhomogeneity. Thus, the constellation can be assumed to be known and deviation from that known constellation can be interpreted as presence of an irregularity.

[0017] The method may furthermore comprises a stage of adoption which is executed before the irregularity detection stage, wherein in the stage of adoption at least one spatial reference pattern is automatically determined. For that purpose, the DVS camera might capture the object during its relative movement through the field of view FOV to generate at least one spatial constellation of event signal SIG(j) generating sensor elements SE(j) and the so generated at least one spatial constellation of event signals is used as the at least one spatial reference pattern.

[0018] The reference pattern might also be at least a pattern in time with, for the same sensor element SE(i) , a first event signal at a point tl in time and a different, second event signal at a later point t2 in time. A presence of an irregularity can be assumed in case during a given time span dTl>t2-tl, and between two points tl, t2 in time, for the same sensor element SE(i) , the generated event signal SIG(i) changes from a first event type of the group GRP to a second event type of the group GRP, the second event type being different from the first event type, i.e. from a positive event to a negative event or vice versa. The maximum reasonable time span dT depends on the transport velocity, the maximum size of an irregularity to be detected, and the size of the sensor elements. The reference pattern might also be essentially a pattern in space, wherein a presence of an irregularity can be assumed in case, during a time span dT2, event signals SIG(jl) , SIG(j2) , SIG(j3) of the same event type, e.g. positive events, are generated for a plurality of N>2 different sensor elements SE(jl) , SE(j2) , SE(j3) for different points tl, t2, t3 in time, wherein the different sensor elements SE(jl) , SE(j2) , SE(j3) are lying on an essentially straight line. Thus, the reference pattern would comprise a plurality of points which are lying on such straight line.

[0019] With the reference pattern being a pattern in space, a presence of an irregularity can be assumed in case for a first sensor element SE(jl) an event signal SIG(jl) of a first event type of the group GRP is generated and, essentially at the same time tO, for a second sensor element SE(j2) , which is located adjacent to the first sensor element SE(jl) , an event signal SIG(j2) of a different, second event type of the group GRP is generated. I.e. the first event type and the second event type are different from each other.

[0020] The method might be applied with an object, the surface of which is essentially homogeneous, such that, in case no irregularity is present, no event signals, i.e. neither positive events nor negative events, would be generated because fluctuations of the intensities and light fluxes would not exceed a noise level which remains typically below the given threshold. However, "essentially homogeneous" might still be applicable to a certain extent for textures at the surface which are characterized by a certain roughness. The given threshold THREE can be set based on a roughness of a texture at the surface, e.g. instead of 15% as mentioned above, a lower threshold value is set for more fine textures and a higher threshold values is set for more rough textures. The roughness of the texture might depend on the tissue type or patterns included in the object. The given threshold THRES might be automatically set in a stage of adoption of the method which would be executed before the irregularity detection stage . In the stage of adoption, the threshold THRES might be varied until a relationship REL=f (NGEN, REFPARA) depending on the number NGEN of generated event signals related to a reference parameter REFPARA reaches an expectation value X . For example , the reference parameter REFPARA might be a time span REFTIME , e . g . in seconds , or an area REFAREA, e . g . in square meters . Thus , the relationship REL might represent a number NGEN of generated event signals per square meter or per second, i . e . REL=NGEN / REFTIME . As soon as such relationship REL reaches the expectation value X, which might be X=0 in the extreme case or any other value which represents a tolerance for potential false alarms , the momentary threshold THRES , which has a direct impact on N, would be selected to be the set threshold THRES .

[0021] Di f ferent methods of illuminating the obj ect might be applied for di f ferent zones of the camera' s field of view, e . g . a bright field transmissive light illumination in a first zone , a combination of transmissive and reflective light in a second zone , and / or reflective light only in a third zone . For some inspection tasks , even dark field kinds of illumination in the reflective or transmissive regime can be employed as well . The di f ferently illuminated zones of the field of view can be arranged behind each other in the transport direction TD .

[0022] Therein, in case a first event signal of a first event type of the group GRP is generated for a first sensor element in a first one of the di f ferently illuminated zones at a first point tl in time , presence of an irregularity can be assumed in case a second event signal of the corresponding event type as the first event type is generated for a second sensor element at a second point t2 in time and in case the second sensor element is located in a second one of the di f ferently illuminated zones and corresponds to the first sensor element . Therein, sensor elements in di f ferent zones correspond to each other in case their mutual distance DIST from each other in transport direction TD is essentially proportional to the distance , which a point on the surface of the obj ect travels relative to the field of view between the points tl and t2 in time , i . e . DIST=PROP*VEL* ( t2-tl ) . Therein, the proportionality factor PROP depends on the imaging characteristics of the DVS camera and its optics , e . g . scaling of the imaged obj ect . Thus , the distance DIST and the correspondence depend on the velocity VEL of the relative movement of DVS camera and obj ect .

[0023] The sensor chip might be a rectangular chip with its sensor elements arranged in rows and columns perpendicular to each other, wherein the DVS camera and its sensor chip are arranged such that 0 °<ALPHA<90 ° is applicable for the angle ALPHA between the rows or, correspondingly, the columns , and the transport direction TD, preferably ALPHA<10 ° . This nonparallel alignment improves the sampling accuracy of the irregularity to a sub-pixel level . Multiple events from di f ferent rows of the sensor chip can be used to fuse the data with respect to the feed motion to create a higher resolved ef fective irregularity image .

[0024] A corresponding system for camera-based inspection of an obj ect for detecting a presence of an irregularity of the obj ect when the obj ect and the camera are moving relative to each other during the inspection in a transport direction TD comprises a DVS camera and an inspection system configured to execute the method described above .

[0025] The DVS camera might be arranged such that rows of sensor elements of its sensor chip are aligned under an angle 0 °<ALPHA<90 ° with regard to the transportation direction TD .

[0026] The system might comprise an illumination system with a plurality of light sources for illuminating the field of view . The light sources might be arranged such that di f ferent light sources of the illumination system illuminate different zones of the field of view and / or the same zone of the field of view with different angles of incidence for the illumination .

[0027] The invention advantageously implements a camera applying the "Dynamic Vision System" (DVS) principle. Typically, such a DVS camera, also known as "event camera" or "neuromorphic camera", is a full field camera with a chip of aspect ratio 4:3 or 16:9 or the like, currently being available in versions with >106individual sensor elements (e.g. Prophesee's "Metavision" sensor) . The essential properties of a DVS camera applied in the solution described herein include that the DVS camera detects events, i.e. changes of light flux exceeding a pre-defined threshold value THRES of e.g. 15%, and that the light flux is detected pixelwise with an actual value FX, so that in effect the DVS camera has an effective dynamic range of 120db or even higher, as a result of the pixels' full well capacity and adoptions of exposure time according to the minimum and maximum allowable values.

[0028] In summary, major differences of the DVS camera based inspection system over conventional inspection systems are based on the properties of a DVS camera as described below to detect events instead of full frame imaging, on the significantly higher speed for DVS camera data acquisition compared to typical frame rates of full frame cameras, on a close and precise coupling of the timing due to the significant speed advantage of the DVS principle over conventional cameras, while such coupling is not needed in that accuracy for conventional cameras, on the significant reduction in data volume, since only relevant events get reported instead of a full image capture of the total surface of the object under inspection, on the color view mode of inspection with an event based DVS camera plus the potential to differentiate between different imaging modes, and on event based data fusion plus creation of a "normal" area type of image. It is to be understood that the elements and features recited in the appended claims may be combined in di f ferent ways to produce new claims that likewise fall within the scope of the present invention . Thus , whereas the dependent claims appended below depend from speci fic independent or dependent claims , it is to be understood that these dependent claims can, alternatively, be made to depend in the alternative from any preceding or following claim, whether independent or dependent , and that such new combinations are to be understood as forming a part of the present speci fication .

[0029] DESCRIPTION OF THE FIGURES

[0030] In the following, possible embodiments of the di f ferent aspects of the present invention are described in more detail with reference to the enclosed figures . The obj ects as well as further advantages of the present embodiments will become more apparent and readily appreciated from the following description of the preferred embodiments , taken in conj unction with the accompanying figure in which :

[0031] FIG 1 shows a side view on a mass production facility for production of an obj ect ,

[0032] FIG 2 shows a flow chart of a method METH for obj ect inspection,

[0033] FIG 3 shows a top view on a sensor chip,

[0034] FIG 4 shows a top view on a produced obj ect suitable for a first embodiment ,

[0035] FIGs 5 show a top view on a section of a sensor chip to illustrate a first scenario ,

[0036] FIGs 6 show a top view on a section of a sensor chip to illustrate a second scenario ,

[0037] FIG 7 shows a top view on a section of a sensor chip to illustrate a third scenario ,

[0038] FIG 8 shows a top view on a produced obj ect suitable for a second embodiment , FIG 9 shows a top view on a section of a sensor chip to illustrate a constellation of event signal generating sensor elements for a produced component,

[0039] FIG 10 shows a top view on a section of a sensor chip to illustrate a constellation of event signal generating sensor elements for a produced component with an irregularity .

[0040] DETAILED DESCRIPTION

[0041] FIG 1 shows a side view on a mass production facility 100 for production of an object 10. The object 10 might be a batch of a plurality of mass production components 10' as indicated in FIG 8, e.g. any kind of component like solid parts made from metal or plastic, machined, printed, injection molded etc. or even soft materials like sheets of tissue, paper, fabric or food like any product from cheese, meat, sausages, cookies, bread, tofu and also agricultural goods like eggs, plants, fish etc. The produced object 10 is transported, for example with a conveyor belt 130, in a transportation direction TD from a first station 110 of the mass production facility 100 to a second station 120. The stations 110, 120 might be machines or other components of the mass production facility 100 for implementing a mass production process applied in the mass production facility 100 to produce the object 10.

[0042] In between the stations 110, 120 a DVS camera 210 of an inspection system 200 for camera-based inspection of the object 10 is installed. The inspection system 200 is configured to detect irregularities IRR of the object 10, e.g. defects or any production errors, while the object 10 moves along and relative to, respectively, the DVS camera 210. The DVS camera 210 is configured and arranged such that it captures a given field of view FOV and such that the field of view FOV covers a section of the surface of the object 10. Preferably, the field of view FOV covers the whole width W of the object 10 and has an extension H in the transportation direction TD. Thus, at a certain point t(i) in time the DVS camera 210 captures that section of the object 10, which lies in the field of view FOV of the DVS camera 210 at that point t(i) in time. Since the object 10 moves relative to the DVS camera 210, different sections of the object's 10 surface are captured at different points t(i) in time.

[0043] The inspection system 200 comprises an evaluation system 220. The evaluation system 220 is connected to the DVS camera 210 via a wired or non-wired connection 221 and configured to control the DVS camera 210 and to receive imaging data SIG(i) from the DVS camera 210, both via the connection 221. Moreover, the evaluation system 220 is configured to determine whether an irregularity is present at the object 10 based on an evaluation of the imaging data SIG(i) provided by the DVS camera 210. For the purpose of evaluation, the evaluation system 220 applies a method METH including a stage INSPECT of detecting irregularities, e.g. defects or production errors, of the object 10 and, optionally, a stage ADOPT of adoption evaluation parameters applied by the evaluation system 220. The method METH as described below in the context of FIG 2 in more detail might be implemented as a respective software executed by a processor 222 of the evaluation system 220. The evaluation system 220 might furthermore comprise a memory 223 for storing any data to be processed, e.g. the imaging data SIG(i) , as well as the software to be executed by the processor 222, and / or any other suitable or required data. The evaluation system 220 and, therewith, the inspection system 200 is configured to create an alarm signal and / or to interfere with the mass production process executed by the mass production facility 100 in case a presence of an irregularity is assumed based on the evaluation of the imaging data SIG(i) .

[0044] The DVS camera 210 has a sensor chip 211 with a plurality of light sensitive sensor elements SE(j) with j = l,...,J and J representing the total number of sensor elements SE(j) , e.g. J>106. As a simplified example, a top view of a sensor chip 211 with only 16*24=384 sensor elements SE(j) is shown in FIG 3. The sensor chip 211 is a rectangular chip 211 with its sensor elements SE(j) arranged in rows and columns perpendicular to each other. Only a few of the sensor elements SE(j) have been marked with reference signs.

[0045] In contrast to regular cameras, a signal SIG(j) generated by a DVS camera 210 for a given sensor element SE(j) does not directly or, for example, proportionally represent the flux (e.g. in lumen) of light onto the respective sensor element SE(j) , but it depends on a relationship between a light flux onto the sensor element SE(j) at a point t(i) in time and a light flux onto the same sensor element SE(j) at an earlier point t(i-l) in time. More concrete, it shows whether a light flux FX at the sensor element SE(j) for a point t(i) in time differs from a light flux FX at the same sensor element SE(j) at an earlier point t(i-l) in time by more than a given threshold THRES . In case such difference is indeed higher than the threshold THRES, a positive or a negative event is generated for the respective sensor element SE(j) .

[0046] Thus, the DVS camera 210 is configured to generate for each sensor element SE(j) individually a signal SIG(j) , in the following referred to as an "event signal" SIG(j) , in case a light flux onto the respective sensor element SE(j) changes by more than the given threshold THRES, e.g. 15% , while no event signal is generated in case the flux change is less than or equal to the threshold THRES. Therein, the event signals SIG(j) are not assigned free floating values, but each event signal SIG(j) is selected from a predefined group GRP of event types, which group GRP can include at least a positive event and a negative event. Thus, an event signal SIG(j) , in case generated, can only be a positive event in case the flux changes to a higher flux by an amount DELTAFX with DELTAFXRTHRES or it can be a negative event in case the flux changes to a lower flux by an amount DELTAFX with DELTAFXRTHRES . In case the change DELTAFX of flux onto a sensor element SE ( j ) is lower than or equal to the threshold THRES, i.e. DELTAFX<THRES , no event signal SIG(j) is generated for the respective sensor element SE(j) . Thus, event signals SIG(j) are only generated in case of significant, i.e. DELTAFX>THRES , flux changes.

[0047] More concrete, considering a specific sensor element SE(j) of the sensor chip 211 of the DVS camera 210: In case the momentary flux FX(t(i) ) onto the sensor element SE(j) is higher than the earlier flux FX(t(i-l) ) onto the same sensor element SE(j) by more than THREE, i.e. FX ( t ( i ) ) -FX ( t ( i- 1)>THRES, the event signal SIG(j) generated for the specific sensor element SE(j) is a positive event, illustrated in respective figures by a symbol "+". In case the momentary flux FX(t(i) ) onto the sensor element SE(j) is lower than the earlier flux FX(t(i-l) ) onto the same sensor element SE ( j ) by more than THREE, i.e. FX ( t ( i-1 ) ) -FX ( t ( i ) >THRES, the event signal SIG(j) generated for the specific sensor element SE ( j ) is a negative event, illustrated in respective figures by a symbol . In all other cases, in which the amount I FX ( t ( i ) ) -FX ( t ( i-1 ) ) | of the difference between the fluxes FX(t(i) ) , FX(t(i-l) ) is less than THREE, i.e. |FX(t(i) )- FX(t(i-l) |<THRES, no signal at all is generated for the respective sensor element SE(j) . Correspondingly, the sensor elements SE(j) for which no event signal is generated are not assigned any symbol in respective figures.

[0048] The method METH for detecting an irregularity IRR of the object 10 beneficially applies the characteristics of such a DVS camera 210. Very basically, a presence of an irregularity IRR can be assumed in case one or more event signals SIG(i) , either positive or negative, are generated for one or more sensor elements SE(i) . This solution might be prone to false alarms. Therefore, in a more robust approach, the presence of an irregularity IRR can be assumed in case a temporal and / or spatial constellation of those one or more sensor elements SE(i) , for which respective event signals SIG(i) are generated during the irregularity detection stage INSPECT, forms a pattern in time, i.e. considering series of event signals of the same sensor element SE(j) over time, and / or in space, i.e. considering event signals SIG(jl) , SIG(j2) , SIG(j3) , ... of different sensor elements SE(jl) , SE(j2) , SE(j3) , ..., which either corresponds to or deviates from a predefined reference pattern. Thus, the assumption of the presence of an irregularity IRR depends on a comparison of the constellation formed by event signal SIG(j) generating sensor elements SE(j) with the predefined reference pattern. For example, the term "constellation" can be interpreted as the spatial arrangement of the entirety of event signal generating sensor elements SIG(j) .

[0049] In general terms, such a reference pattern might correspond to and represent, respectively, an expected momentary appearance or spatial constellation of event signal SIG(j) generating sensor elements SE(j) of the sensor chip 211, under the precondition that the object 10 has no irregularity in its section which is momentarily located in the DVS camera's 210 field of view FOV. This utilizes that the optical appearance of the object 10 and its sections in the field of view FOV are known so that event signals SIG(j) generated by the sensor elements SE(j) of the DVS camera 210 during movement of the sections through the field of view FOV, i.e. "constellations of sensor elements", can be easily predicted or modeled. As soon as a momentary constellation of event signal SIG(j) generating sensor elements SE ( j ) deviates from such a modeled constellation, the presence of an irregularity IRR can be assumed.

[0050] A top view on an object 10 suitable for a first embodiment of the method METH, executed by the inspection system 200 for inspection of the object 10 moving relative to the DVS camera 210 for detecting irregularities IRR, is shown in FIG 4. In the first embodiment, it is assumed that the surface of the object 10 which is captured section-wise by the DVS camera 210, is essentially homogeneous, such that, in case no irregularity is present, no event signals SIG(j) , i.e. neither positive events nor negative events, would be generated because regular fluctuations of the fluxes onto the sensor elements would not exceed a certain noise level .

[0051] Such noise level would typically remain below the given threshold THRES or, correspondingly, the threshold THRES is set such that it is not exceeded by such noise . Nevertheless , "essentially homogeneous" might still be applicable to a certain extent for textures at the surface of the obj ect 10 which might be characteri zed by a certain roughness which might depend on the tissue type or patterns included in the obj ect 10 . In that case , the threshold THRES can be set in the optional stage ADOPT of adoption based on the roughness of the texture at the surface , e . g . instead of a default value THRES=15% as mentioned above , a lower threshold value might be set for more fine textures and a higher threshold values might be set for more rough textures .

[0052] In an exemplary reali zation of the obj ect 10 to be applied in the first embodiment and to be described in the following, the obj ect 10 is a large , continuous fabric with an essentially continuous surface . As another example , the obj ect 10 might be a large , continuous metal sheet etc .

[0053] The method METH comprises the irregularity detection stage INSPECT of detecting irregularities IRR, e . g . defects or production errors , of the obj ect 10 and, optionally, the stage ADOPT of adoption which would be executed, i f applied, before the stage INSPECT of detecting irregularities IRR to individually set the threshold THRES instead of applying a default value as a threshold .

[0054] In case executed, the stage ADOPT achieves automatic determination and setting of the threshold THRES for the obj ect 10 to be inspected . For example , the threshold THRES can be varied, starting from a starting point like THRES ( 0 ) =0% until a relationship REL=f (NGEN, REFPARA) depending on the number NGEN of generated event signals related to a reference parameter REFPARA reaches an expectation value X.

[0055] For example, the reference parameter REFPARA might be a time span REFTIME, e.g. in seconds. Thus, the relationship REL might represent a number NGEN of generated event signals per second, i.e. REL=NGEN / REFTIME . Such relationship REL of course depends on the actual threshold value THRES so that a variation of THRES leads to a change in REL. As soon as such relationship REL reaches the expectation value X, which might be X=0 in the extreme case or any other value which represents a tolerance for potential false alarms, the momentary threshold THRES would be selected to be the set threshold THRES.

[0056] In another example the reference parameter REFPARA might be an area REFAREA, e.g. in square meters, on the surface of the object 10 which has passed through the field of view FOV during execution of the stage ADOPT, depending in the transportation velocity VEL. In this scenario, the relationship REL represents the number N of event signals per square meter. Again, a variation of THRES has an impact on N and as soon as such relationship REL reaches the expectation value X, which might again be X=0 in the extreme case or any other value which represents a tolerance for potential false alarms, the momentary threshold THRES would be selected to be the set threshold THRES.

[0057] However, the irregularity detection stage INSPECT includes that the DVS camera 210 is used to capture the field of view FOV during movement of the object 10. Imaging data, i.e. event signals SIG(j) , are provided to the evaluation system 220 as soon as they appear for evaluation for determination whether an irregularity IRR is present.

[0058] Basically, the presence of an irregularity IRR is assumed by the evaluation system 220 based on the provided imaging data SIG(j) only in case a generated event signal SIG(j) , i.e. a positive event or a negative event, is present for at least one sensor element SE(j) . As indicated above, a more robust approach considers constellations of event signal SIG(j) generating sensor elements SE(j) of the sensor chip 211 and compares such constellations with a reference pattern.

[0059] In a first approach the reference pattern can be understood as at least a pattern in time with a first event signal SIGl (jl) for a specific sensor element SE(jl) at a point tl in time and, for the same sensor element SE(jl) , a different, second event signal SIG2 (jl) at a later point t2 in time. For example, the first event signal SIGl (jl) might a be positive event and the second event signal SIG2 (jl) might be a negative event or vice versa. A presence of an irregularity IRR might be assumed in case, during a given time span dTl>t2-tl during the irregularity detection stage INSPECT, and between two points tl, t2 in time, respectively, the event signal SIG(jl) generated for a specific sensor element SE(jl) changes from a first event type of the group GRP to a second, different event type of the group GRP, the second event type being different from the first event type, i.e. from a positive event to a negative event or vice versa. This includes that a sequence of plurality of first event signals SIGl (jl) has been generated before the point tl in time and that a sequence of plurality of second event signals SIG2 (jl) has been generated after the point t2 in time. Nevertheless, the event signal changes between tl and t2. In other words, a presence of an irregularity IRR might be assumed in case the DVS camera 210 first provides at least one positive event and shortly later provides at least one negative event, or vice versa, for the same sensor element SE(jl) . This actually means that a change has been detected at the sensor element SE(jl) , e.g. resulting in a positive event, and that this change has disappeared again for that same sensor element SE(jl) at a later point in time due to the movement of the object 10, e.g. resulting in a negative event for SE(jl) . This is indicated in FIGs 5a, 5b which show only a limited number of sensor elements SE of the sensor chip 211 for the sake of clarity: FIG 5a represents a first point tl in time at which a positive event is generated for sensor element SE(jl) . FIG 5b represents a second point t2 in time with t2>tl at which a negative event is generated for the same sensor element SE(jl) . Thus, presence of an irregularity IRR can be assumed.

[0060] The maximum reasonable time span dTl during which the change of event signal type should happen depends on the transportation velocity VEL, the maximum size of an irregularity to be detected, and the size of the sensor elements SE .

[0061] In a second approach, which can be applied in addition or alternatively to the first approach, the reference pattern can be essentially understood as a pattern in space, wherein a presence of an irregularity IRR is assumed in case, during a time span dT2, event signals SIG(jl) , SIG(j2) , SIG(j3) of the same event type, e.g. positive events, are generated for a plurality of N>2 different sensor elements SE(jl) , SE(j2) , SE(j3) for different points tl, t2>tl, t3>t2 in time. Therein it is required that those different sensor elements SE(jl) , SE(j2) , SE(j3) are lying on an essentially straight line. Of course, the consideration can be extended to more than three instances of time and the irregularity IRR is assumed in case the pixels of different images with the same event type lie on a straight line. Thus, the spatial reference pattern would comprise a plurality of points which are lying on such straight line.

[0062] An irregularity IRR causes a positive event or a negative event and it has to be assumed that the same event type will be generated at later points in time at different positions and sensor elements, respectively, due to the relative movement. Those sensor elements have to lie on a straight line, actually in parallel to the transport direction TD. FIGs 6a-6c show the corresponding scenario, wherein FIGs 6a- 6c again only show a limited number of sensor elements SE of the sensor chip 211 for the sake of clarity. FIG 6a represents a first point tl in time at which a positive event is generated for sensor element SE(jl) , FIG 6b represents a first point t2 in time at which a positive event is generated for a different sensor element SE(j2) , and FIG 6c represents a first point t3 in time at which a positive event is generated for a different sensor element SE(j3) . As can be seen, the sensor elements SE(jl) , SE(j2) , SE(j3) are indeed lying on an essentially straight line, e.g. when the illustrations of FIGs 6a-6c are, at least virtually, congruently overlaid or superimposed, respectively.

[0063] In a third approach which can be applied in addition or alternatively to the first and / or second approach the reference pattern can be understood as a pattern in space, wherein a presence of an irregularity IRR is assumed in case, for a first sensor element SE(jl) an event signal SIG(jl) of a first event type of the group GRP is generated, e.g. a positive event, and, essentially at the same time tO, for a second sensor element SE(j2) , which is located in the neighborhood of the first sensor element SE(jl) , preferably adjacent to the first sensor element SE(jl) , an event signal SIG(j2) of a different, second event type of the group GRP is generated, e.g. a negative event. I.e. the first event type and the second event type are different from each other. This third approach again accounts for the movement of the object 10 and of the irregularity IRR relative to the DVS camera 210.

[0064] This scenario is illustrated in FIG 7, which again only shows a limited number of sensor elements SE of the sensor chip 211 for the sake of clarity. A negative event is generated for a first sensor element SE(jl) while, at the same time tO, a positive event is generated for a second sensor element SE(j2) , which is located adjacent to the first sensor element

[0065] SE ( j 1 ) .

[0066] The above approaches work best with objects 10 the surfaces of which are essentially homogeneous. However, this cannot necessarily be guaranteed for each and every object 10. A top view on an object 10 suitable for a second embodiment of the method METH, executed by the inspection system 200 for inspection of the object 10 moving relative to the DVS camera 210 for detecting irregularities IRR, is shown in FIG 8. In the second embodiment, it is assumed that the surface of the object 10 which is captured section-wise by the DVS camera 210, is not of an essentially homogeneous type like in case of the fabric described in the first embodiment or, for example, an extensive sheet metal or the like. Consequently and due to the inhomogeneous surface of the object 10, it has to be assumed that the DVS camera 210 detects a plurality of events even in case the object 10 has no defects: For example and as shown in FIG 8, the object 10 might be a batch of a plurality of mass production components 10' , e.g. food or agricultural goods like any product from cheese, meat, sausages, cookies, bread, tofu, eggs, plant, or fish, but also components from other industrial mass production, e.g. screws, circuit boards, fabric with textures etc. For the sake of clarity, only few components 10' of the object 10 have been marked with reference signs. The overall surface of such an object 10 consisting of a plurality of individual components 10' includes the surfaces of the plurality components 10' which are more or less adjacent to each other, so that such overall surface is not essentially homogeneous as in the example of the first embodiment, but the overall surface will typically be characterized by a high-contrast texture with flux gradients at least in the areas between neighboring components. The same is applicable for objects 10 which do not consist of individual components but the surfaces of which have intrinsic inhomogeneities and, correspondingly, textures on the surfaces show a significant roughness or contrast elements. In both cases, the DVS camera 210 will detect a plurality of event signals SIG ( j ) of positive and negative event types due to the high-contrast texture of the overall surface , even in case no irregularity is present . Thus , in contrast to the scenario in which the surface of the obj ect 10 is essentially homogeneous , the use of the threshold THRES alone does not suf fice to ensure that irregularities IRR can be detected with high reliability . However, it can still be assumed that the imaging data S IG ( j ) captured by the DVS camera 210 over time t comprise certain characteristics , according to the regular properties of the components 10 ' .

[0067] FIG 9 , which depicts only a limited number of sensor elements SE of the sensor chip 211 for the sake of clarity, shows a constellation of event signal generating sensor elements S IG ( j ) for only one of the components 10 ' shown in FIG 8 , which might be essentially round platelets .

[0068] The reference pattern according to such an obj ect 10 and components 10 ' , respectively, might consist of one or more constellations as shown in FIG 9 . In more general terms , the reference pattern might be a pattern in space , wherein the reference pattern is such a spatial constellation of sensor elements momentarily generating an event signal of any type , which would be caused by a certain section of the obj ect 10 momentarily positioned in the field of view FOV in case no signi ficant irregularities of the obj ect 10 are present in that certain section in the field of view FOV . In the first embodiment , such a " spatial constellation of sensor elements momentarily generating an event signal" would not comprise a single sensor element because of the homogeneous surface assumed in the first embodiment . In the second embodiment , event signals would at least be generated when a rim or edge of a component 10 ' moves into and out of the area within the field of view FOV which is captured by a specific sensor element . The constellation of sensor elements SE(j) generating event signals SIG(j) depends on the known inhomogeneity, e.g. on the arrangement and form of components 10' . Thus, the constellation can be assumed to be known for objects 10 without irregularities so that such known constellation can be used as reference pattern. Deviations from that known constellation and reference pattern, respectively, can be interpreted as presence of an irregularity IRR.

[0069] In the second embodiment, the method METH comprises the irregularity detection stage INSPECT, e.g. defects or production errors, of the object 10 and, optionally, the stage ADOPT of adoption which would be executed, if applied, before the stage INSPECT of detecting irregularities IRR.

[0070] In case executed, the stage ADOPT includes determination of at least one spatial reference pattern. For that purpose, the DVS camera 210 might capture the object 10 during its relative movement through the field of view FOV to generate at least one spatial constellation of event signal SIG(j) generating sensor elements SE(j) . That at least one spatial constellation of event signal SIG(j) generating sensor elements SE(j) is used as the at least one spatial reference pattern .

[0071] The irregularity detection stage INSPECT includes that the DVS camera 210 is used to capture the field of view FOV during movement of the object 10. Imaging data, i.e. event signals SIG(j) , are provided to the evaluation system 220 as soon as they appear for evaluation for determination whether an irregularity is present. The presence of an irregularity IRR might be assumed in case a momentary constellation of the one or more sensor elements SE(i) , for which respective event signals SIG(i) are momentarily generated, as provided by the DVS camera 210 during the inspection stage forms a pattern in space, which deviates from the predefined reference pattern. For that purpose, the reference pattern might be stored in memory 223 of the evaluation system 220 so that the processor 222 can compare the momentary constellation as provided by the DVS camera 210 with the reference pattern.

[0072] While FIG 9 shows a constellation which can be used as a reference pattern, FIG 10 shows a momentary constellation of sensor elements SE(i) , for which event signals SIG(i) are momentarily generated in case the component 10' indeed has an irregularity IRR. As can be seen from a comparison with FIG 9, the momentary constellation of FIG 10 includes other sensor elements than the reference pattern of FIG 9 which means that the component 10' has an irregularity IRR.

[0073] The evaluation system 220 which might apply an artificial intelligence (Al) system to compare momentary constellations with a respective reference pattern. Such an Al system might have been trained earlier based on a batch of definitely damage free components 10' . In that scenario, the imaging data SIG(j) captured by the DVS camera 210 over time t as described above are provided to the trained Al system which is able to detect an irregularity IRR despite the inhomogeneous overall surface of the object 10.

[0074] In both embodiments, suitable illumination of the object 10 by an illumination system for better imaging might apply different setups. For example, the illumination system might comprise one or more light sources 231, 232, 233, e.g. LED systems, possibly applying illumination in different colors and / or being arranged such that the object is illuminated in a transmissive and in a reflective light setup.

[0075] In a transmissive light setup, which is suitable in case the object 10 is at least semi-transparent and in which at least one 233 of the light sources of the illumination system and the DVS camera 210 might be arranged on different sides of the object 10, i.e. the object 10 is positioned between the lights source 233 and the DVS camera 210, an irregularity IRR embodied as a hole in the object 10 would allow a higher light flux to the DVS camera 210, thus a positive event or even a series of positive events is created at those pixels PX and respective sensor elements SE the hole is imaged onto. When the hole passes by and leaves the field of view FOV, a negative event or a series of negative events is created. Thus, each sensor element SE directly captures its portion of the irregularity IRR and a group of neighboring sensor elements SE would image the irregularity IRR as a structure of positive and negative events. The same holds true and only reverses the types of events, when the irregularity IRR is embodied as an obstructive element in the object 10 like a knot or some inclusion of a particle, being less transmissive than the object's 10 regular texture. The respective threshold THRES for signal changes to trigger an event can be set according to the texture of the object 10, e.g. instead of the typical 15% to lower values for very fine textures and higher values for rougher textures, depending on the object's 10 material or tissue type or patterns included.

[0076] A reflective light setup, in which a light source 231, 232 of the illumination system and DVS camera 210 are on the same side of the object 10, is applicable independently from transparency of the object 10.

[0077] Preferably, the object 10 can be illuminated by light sources

[0078] 231, 232, and / or 233 of the illumination system of the inspection system 200, especially in the field of view FOV, as indicated in FIG 1. Preferably but not necessarily, the illumination system and its one or more light sources 231,

[0079] 232, 233 are configured and arranged such that different methods of illuminating the object 10 are applied for different zones Zl, Z2, Z3,... of the camera's field of view, e.g. a bright field transmissive light illumination in a first zone Zl, a combination of transmissive and reflective light in a third zone Z2, and reflective light only in a second zone Z2. For example, different illumination methods might also apply different colors of light. For some inspections, even dark field kinds of illumination in the reflective or transmissive regime can be employed as well. Therein, the zones Zl, Z2, Z3,... of the field of view FOV are arranged behind each other in the transport direction TD. Correspondingly, the images IMA(t(i) ) can be divided in the same number of zones. Potential event signals SIG(il) , SIG(i2) , SIG(i3) detected from the respective zones Zl, Z2, Z3, ... in the field of view FOV of the DVS camera 210 with different illumination conditions can be combined or merged by making use of the transport velocity VEL of the object 10 to be inspected, e.g. by the velocity VEL of the object 10 or the conveyor belt or a robotic system which transports the object 10 and / or the DVS camera 210. For example, the velocity VEL, which might be available from a control system of the mass production facility 100, can be used to derive which sensor element SE(i2) in a second one Z2 of the zones Zl, Z2, Z3 corresponds to a first sensor element SE(il) in a first one Zl of the zones Zl, Z2, Z3. Such sensor elements SE(il) , SE(i2) in different zones Zl, Z2 correspond to each other in case their mutual distance DIST from each other in transport direction TD is essentially proportional to the distance, which any point on the surface of the object travels relative to the field of view FOV between points tl and t2 in time, i.e. DIST=PROP*VEL* ( t2-t 1 ) . Therein, the proportionality factor PROP depends on the imaging characteristics of the DVS camera and its optics, e.g. scaling of the imaged object.

[0080] With that, the availability of different zones Zl, Z2, ... as described above can be used to make the detection of an irregularity IRR even more reliable. In case a first event signal SIG(il) of a first event type of the group GRP, e.g. a positive event, is generated for the first sensor element SE(il) in the first zone Zl at a first point tl in time and, additionally, a second event signal SIG(i2) of the same event type as the first event type is generated for a second sensor element SE(i2) in the second zone Z2 at a second point t2 in time, presence of an irregularity IRR can be assumed in case the second sensor element SE(i2) "corresponds" to the first sensor element SE(il) as per the above definition. Therewith, data and information from di f ferent zones can be considered to be fused . For a more robust detection and / or classi fication of a potential irregularity, it is advantageous to use the fused inspection data for a respective lateral position of the obj ect 10 . Preferably, some coordinate axis like the transported length of the plane or a generic length coordinate of a conveyor can be used to properly fuse the respective data also based on an actual velocity information as registered from the transport system . The same concept can also be used together with a robotic system .

[0081] In principle , the usage of events instead of a broad range of values representing fluxes or similar acts as a first kind of data or event filter to reduce the amount of measured data right at the source , i . e . at the imaging system itsel f . In a very broad and general perspective , the approach based on usage of a DVS camera 210 helps to directly reduce the data created to the relevant portion without long chains of image processing and thus also to improve the inspection system 200 performance with respect to sustainability aspects like energy consumption, materials involved due to less powerful computing and data transmission systems , and overall ICT infrastructure and also related carbon footprint . Despite these optimi zations , the core of the inspection in finding potential defect sites is much more ef ficient and detailed than by employing conventional cameras .

[0082] The following considerations sometimes again use the example of the obj ect 10 being a fabric, as exemplarily applied in the first embodiment of the invention, for illustration purposes . However, the corresponding considerations and benefits are typically applicable for other reali zations of the obj ect 10 as well , possibly with adopted concrete figures and numbers , as the case may be , and not limited to the obj ect 10 being a fabric . However, the concrete example of the obj ect 10 being a fabric is beneficial since it can unambiguously demonstrate the positive aspects of the invention, e.g. exploiting that a fabric 10 is typically at least semi-transparent. Thus, it can be inspected with transmitted and / or reflected light in bright field or dark field conditions and it has a high demand for lateral resolution, since the single fibers or thread might be as small as 10 pm and below. A typical inspection task for a fabric 10 is to check the quality of a panel of e.g. Im or 2m in width with a lateral sampling accuracy of e.g. 20pm. This requires a number of 50.000 points or pixels per meter width. With a conventional camera, this would mean to have approx, forty such cameras per meter. The fabric 10 typically moves at a velocity of 200 to 800mm / s. When the same lateral sampling is applied, this means to have 10.000 to 40.000 pixels or 14 to 55 images. With higher velocity, it would mean to have 55 Mpxl images for 40 cameras, or just 2200 camera frames per second. With 8 Bit resolution in the image, it would already account for 2.2GByte / s or almost 20Gbit / s. For 120 db resolution, a 12 or even 16 bit image representation would be required which in turn would double the amount of data to about 40 Gbit / s as a ballpark figure for a conventional camera. Corresponding, similar considerations are applicable for other object 10 types both in the first and in the second embodiment.

[0083] From a data point of view and independent from the concrete embodiment of the object 10, the amount of data created from the inspection system 200 with a DVS camera 210 is decreased significantly as compared to a conventional camera, since all regions of the object 10 with no anomality or defect do not create an event because the changes in light flux of the respective illumination modalities do not exceed the predefined threshold THRES . Typically, an irregularity area is only a few mm2or cm2in size, which is very low in comparison to, for example, the 100-500m2of a fabric 10. One hundred defects of 1cm2, which would be a very large defect, would make up 100cm2, so the area is only 20 to lOOppm of the object's 10 surface. On the other side, as the irregularity area transverses the illuminated field of view of the DVS camera 210 , every irregularity IRR creates multiple events when traversing the entire height in transport direction TD . At a given instant of time , when the irregularity IRR is completely inside one field of view FOV and also in one illumination regime , at least a full contour view of the irregularity IRR is created, since most of the changes happen at the rim area of the respective irregularity IRR . An additional advantage with regard to the signi ficant saving of amount of data to be processed comes from the basic principle of the DVS camera which only reacts on changes of flux . While a conventional camera provides a comprehensive, full area image of an obj ect or a component , including data and information about areas without any contrast , the DVS camera only reacts on changes so that only high contrast areas like moving edges etc . might contribute to the amount of data to be processed .

[0084] Typically, and as shown in FIG 3 in a simpli fied manner, the sensor chip 211 is a rectangular chip 211 with its sensor elements arranged in rows and columns perpendicular to each other . To improve the sampling accuracy of the irregularity IRR, one can make use of the fact that typically the transport direction TD for the obj ect 10 and the rows of the DVS camera' s 210 sensor elements SE are not necessarily perfectly parallel , i . e . 0 °<ALPHA<90 ° is applicable for the angle ALPHA between the rows or, correspondingly, the columns , and the transport direction TD . Therefore , the multiple events from di f ferent rows of the sensor chip 211 can be used to fuse the data with respect to the feed motion to create a higher resolved ef fective irregularity detection which achieves a sub-pixel level . In turn, this can be used to reduce the designed physical resolution which means that the DVS camera 210 with a given si ze of the sensor chip 211 can capture a larger area of the obj ect 10 and still meet the required resolution criteria . This technical concept is essentially known and established for conventional cameras , e . g . from pixel shi fting imaging cameras used in the past , when the number of sensor elements on a chip was limited . With the DVS camera 210 applied in an inspection system 200 as introduced herein, it can signi ficantly contribute to optimi ze system parameters , since an ef fective sampling is possible with valuable gains in optical resolution by 3 or 5 shi ft intervals to cover one ef fective sensor element SE width . This factor could be used to improve the optical resolution and thus reduce the number of DVS cameras 210 required to still perform high resolution quality inspection tasks . Reducing the native optical resolution of the imaging system of the camera 210 also helps to smoother out the information, i . e . the maximally achievable optical resolution, and thus also to reduce false events , e . g . for obj ect 10 surfaces with textures with a lower filling factor or grid or gaze like obj ects 10 to be inspected . For the inspection system 200 , a zoom lens might be employed in order to adapt the native optical resolution to the obj ect 10 properties or obj ect 10 surface properties .

[0085] The benefit enabled by introducing the DVS camera 210 for such inspection tasks comes from the much faster event based imaging capabilities plus the fact , that only changes to an earlier state make up events . For the exemplary fabric 10 it could be any irregularity IRR in the texture , locally changing the optical appearance of the surface' s texture . Such change might be triggered by irregularities IRR such as , for example , di f ferent spacings between threads , holes , knots , inclusions from other fibers or particles and the like . Coming from the fact that the vast maj ority of the fabric 10 has no such or other irregularities , it is of importance not to miss the irregularities and potential irregularities for a proper quality inspection . During the quality inspection under stage INSPECT , irregularities IRR can be graded in the respective severity from non critical to critical in grades dependent on the general quality criteria in the respective inspection domain . For fabrics 10 of a given length, a speci fication of the total number of irregularities in the fabric 10 might be suf ficient . For example , in case the total number of irregularities IRR is below some speci fied number, the quality of the obj ect 10 might be graded "good" or " first class" or similar . I f it is below a second, higher number, it might be graded "acceptable" or " second class" . I f the second, higher number is exceeded, the whole obj ect 10 might get rej ected . The same is of course applicable for other obj ects 10 to be inspected, both in the first and in the second embodiment as described below . The quality criteria and numbers , respectively, might be based upon inspection by humans . Their vision system is di f ferent to cameras which can and will detect many more potential defects . Thus , a good match between human inspection and DVS camera 210 based inspection is crucial . DVS cameras 210 are very suited for this task, since their operational concept is to a certain extent adopted to the human vision system in that only deviations from an actual normal state are detected as events and can be classi fied afterwards .

[0086] To create such a quality information for the obj ect 10 under inspection, the inspection system 200 might comprise a plurality of DVS cameras 210 , each one configured as described above . Data IMA from di f ferent DVS cameras can be aggregated and fused, e . g . to show a full digital view or digital twin of the obj ect 10 as a kind of digital twin information and all known computer and machine vision based data evaluation strategies could be applied either to the event data or a conventional camera view as derived from the event data or by employing a "DAVIS" kind of DVS camera 210 type .

[0087] Another advantage of the DVS camera 210 based inspection system 200 is the much higher speed of event output from the DVS camera 210 , being a factor of 1 . 000 or even 10 . 000 and more faster than a conventional camera . This helps to directly mark or label the obj ect 10 under inspection at or very close to the position of the potential irregularity without reducing the transport velocity for the product under inspection. This cannot be achieved with today's conventional camera based inspection system in a satisfactory kind of operation. Typically, the transport has to be stopped and a rewind has to be done back to the potential irregularity position or coordinate of the transport system. E.g. for fabrics 10, it is state of the art to position a physical marker like a sticker at one of the rims of the fabric 10. On sheet metals, also some marking is used by color spots sprayed on the respective position or also at the rim.

[0088] In case the inspection should be done in a color mode, the lateral separate regions of illuminations can be multiplied, so that there are separate regions for each main color. Powerful LED illumination might be used to even switch camera sensitivity patterns adopted to the object features and characteristics to e.g. analyze a color print on a panel to have it visible in one color and not visible for another color .

[0089] Regarding the method METH itself, it is possible that the stage ADOPT is executed immediately before execution of the stage INSPECT, each time an object 10 has to be inspected. However, the stage ADOPT might also be executed at a much earlier time, e.g. in the context of a general installation or commissioning, e.g. when the object 10 to be inspected is produced for the first time, or in the context of training the potentially applied AT system. The data and threshold THRES, respectively, determined in the stage ADOPT can then be stored, e.g. in the memory 223 of the evaluation system 220. As soon as the same kind of object 10 is produced again at a later stage, the stored threshold THRES can be re-used instead of executing the stage ADOPT again.

[0090] In the exemplary situation shown in FIG 1 and assumed in the subsequent explanations in the context of the first and second embodiment foresees that the object 10 is moving while the DVS camera 210 is motionless. However, the approach introduced herein only requires a relative movement between the obj ect 10 to be inspected and the DVS camera 210 with a relative velocity VEL . Thus , the above explanations can be applied without further ado to situations with a motionless obj ect and a moving camera as well as to situations with both the obj ect 10 and the DVS camera 210 being moved .

[0091] While the present invention has been described above by reference to various embodiments , it should be understood that many changes and modi fications can be made to the described embodiments . It is therefore intended that the foregoing description be regarded as illustrative rather than limiting, and that it be understood that all equivalents and / or combinations of embodiments are intended to be included in this description . Thus , the invention is not restricted to the above illustrated embodiments , but variations can be derived by a person skilled in the art without deviation from the scope of the invention .

Claims

Claims1. Method METH for camera (210) based inspection of an object (10) for detecting a presence of an irregularity IRR of the object (10) , wherein the object (10) and the camera (210) are moving relative to each other during the inspection in a transport direction TD,- wherein the camera (210) is a DVS camera-- comprising a sensor chip (211) with a plurality of sensor elements SE(j) and-- being configured to generate for each sensor element SE(j) individually an event signal SIG(j) from a group GRP of event types including at least a positive event and a negative event, in case a light flux FX impinging onto the respective sensor element SE(j) changes by more than a given threshold THRES, and- wherein the method METH comprises an irregularity detection stage INSPECT, during which the presence of an irregularity IRR of the object (10) is assumed in case one or more event signals SIG(j) generated by the DVS camera (210) during the relative movement of a section of the object (10) through a field of view FOV of the DVS camera (210) fulfill a predefined condition.

2. Method according to claim 1, wherein a presence of an irregularity is assumed in case one or more event signals SIG(j) are generated for one or more sensor elements SE(j) .

3. Method according to any one of claims 1 to 2, wherein the presence of an irregularity is assumed in case a constellation of those one or more sensor elements SE(j) , for which respective event signals SIG(j) are generated during the irregularity detection stage INSPECT, forms a pattern in time and / or in space which either corresponds to or deviates from a predefined reference pattern.

4. Method according to claim 3, wherein the reference pattern is a pattern in space, wherein the reference pattern is sucha spatial constellation of sensor elements SE(j) momentarily generating an event signal SIG(j) , which would be caused by a certain section of the object (10) momentarily positioned in the field of view FOV in case no irregularities of the object (10) are present in the certain section, wherein the presence of an irregularity IRR is assumed in case a constellation of the one or more sensor elements SE(j) , for which respective event signals SIG(j) are momentarily generated, forms a pattern in space, which deviates from the predefined reference pattern.

5. Method according to claim 4, wherein the method METH comprises a stage ADOPT of adoption which is executed before the irregularity detection stage INSPECT, wherein in the stage ADOPT of adoption at least one spatial reference pattern is automatically determined, wherein, preferably,- the DVS camera (210) captures the object (10) during its relative movement through the field of view FOV to generate at least one spatial constellation of event signal SIG(j) generating sensor elements SE ( j ) ,- the so generated at least one spatial constellation of event signal SIG(j) generating sensor elements SE(j) is used as the at least one spatial reference pattern.

6. Method according to any one of claims 1 to 5, wherein a presence of an irregularity IRR is assumed in case during a given time span dTl, for the same sensor element SE ( j ) the generated event signal SIG(j) changes from a first event type of the group GRP to a second, different event type of the group GRP .

7. Method according to any one of claims 1 to 6, wherein a presence of an irregularity IRR is assumed in case during a time span dT2, event signals SIG(jl) , SIG(j2) , SIG(j3) of the same event type are generated for a plurality of N>2 different sensor elements SE(jl) , SE(j2) , SE(j3) for different points tl, t2, t3 in time, wherein the differentsensor elements SE(jl) , SE(j2) , SE(j3) are lying on an essentially straight line.

8. Method according to any one of claims 1 to 7, wherein a presence of an irregularity IRR is assumed in case,- for a first sensor element SE(jl) an event signal SIG(jl) of a first event type of the group GRP is generated and, essentially at the same time tO,- for a second sensor element SE(j2) , which is located adjacent to the first sensor element SE(jl) , an event signal SIG(j2) of a different, second event type of the group GRP is generated.

9. Method according to any one of claims 1 to 8, wherein the method METH is applied with an object (10) , the surface of which is essentially homogeneous, wherein the given threshold THRES is set based on a roughness of a texture at the surface .

10. Method according to any one of claims 1 to 9, wherein the method comprises a stage ADOPT of adoption which is executed before the irregularity detection stage INSPECT, wherein in the stage ADOPT of adoption the given threshold THRES is automatically set, wherein, preferably,- the threshold THRES is varied until a relationship REL=f (NGEN, REFPARA) depending on the number NGEN of generated event signals related to a reference parameter REFPARA reaches an expectation value X,- the momentary threshold THRES is selected to be the set threshold THRES when such relationship REL reaches the expectation value X.

11. Method according to any one of claims 1 to 10, wherein different methods of illuminating the object (10) are applied for different zones (Zl, Z2, Z3) of the DVS camera's (210) field of view FOV, wherein the differently illuminated zones (Zl, Z2, Z3) of the field of view FOV are arranged behind each other in the transport direction TD.

12. Method according to claim 11, wherein, in case a first event signal SIGl (jl) of a first event type of the group GRP is generated for a first sensor element SE(jl) in a first one (Zl) of the differently illuminated zones (Zl, Z2, Z3) at a first point tl in time, presence of an irregularity is assumed in case a second event signal SIG2 (j2) of the corresponding event type as the first event type is generated for a second sensor element SE(j2) at a second point t2 in time and in case the second sensor element SE(j2)- is located in a second one (Z2, Z3) of the differently illuminated zones (Zl, Z2, Z3) and- corresponds to the first sensor element SE(jl) , wherein sensor elements in different zones (Zl, Z2, Z3) correspond to each other in case their mutual distance DIST from each other in transport direction TD is essentially proportional to the distance, which a point on the surface of the object (10) moves relative to the field of view FOV between the points tl and t2 in time.

13. System (200) for camera (210) based inspection of an object (10) for detecting a presence of an irregularity IRR of the object (10) when the object (10) and the camera (210) are moving relative to each other during the inspection in a transport direction TD, comprising the camera (210) being embodied as a DVS camera (210) and an evaluation system (220) configured to execute a method according to any one of claims 1 to 12.

14. System (200) according to claim 13, wherein the DVS camera (210) is arranged such that rows of sensor elements SE(j) of its sensor chip (211) are aligned under an angle 0 °<ALPHA<90 ° , preferably ALPHA<10°, with regard to the transportation direction TD.

15. System (200) according to any one of claims 13 to 14, wherein the system (200) comprises an illumination system with a plurality of light sources (231, 232, 233) forilluminating the field of view FOV, wherein the light sources (231, 232, 233) are arranged such that different light sources (231, 232, 233) of the illumination system illuminate different zones (Zl, Z2, Z3) of the field of view FOV and / or illuminate the same zone of the field of view FOV with different angles of incidence on the object (10) .

Images