Inspection system and inspection method

Abstract

To accurately detect a defect in a surface of an inspection object.SOLUTION: An inspection system 1 comprises: a lighting unit 10 that illuminates a moving object 2 with diffused lighting in an illumination pattern having a bright area and a dark area; and an imaging apparatus 16 that picks up an image of the moving object 2 illuminated in the illumination pattern. The imaging apparatus 16 includes an event-based image pickup device 17 that detects a change in luminance value for every pixel and outputs the changes in luminance value as event data. The inspection system 1 further comprises a control unit (image processing unit) 18 that analyzes the event data from the image pickup device 17 to inspect a surface 3 of the object 2.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an inspection system and an inspection method for inspecting the appearance of an inspection object using a captured image.

Background Art

[0002] Many inspection systems for inspecting the appearance of an inspection object (hereinafter simply referred to as "object") using image processing technology have been proposed. Many inspection systems include a lighting device for illuminating the object and an imaging device for imaging the object.

[0003] Optical systems used for imaging an object (referred to as "imaging optical systems") are roughly classified into a diffuse reflection optical system and a specular reflection optical system. As methods for illuminating and imaging an inspection object using a diffuse reflection system, a dark field illumination method, a light section method, a phase shift method (diffusion), an illuminance difference stereo method, etc. are known. On the other hand, as methods for illuminating and imaging an inspection object using a specular reflection optical system, a bright field illumination method, a phase shift method (specular reflection), etc. are available.

[0004] For example, Utility Model Registration No. 3197766 (Patent Document 1) discloses a reflection type phase shift method. In the reflection type phase shift method, an object is irradiated with light having a light and dark pattern while shifting it by one cycle, and the object is imaged. Stripes can be seen in the image obtained by imaging. Since the luminance change differs depending on the presence or absence of a defect, the defect can be detected by checking the presence or absence of the luminance change.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The above method has been proposed for inspecting the presence or absence of defects on the surface of an object. However, depending on the surface condition of the object, inspection using the above imaging optical system may not be easy. Therefore, there is a need for a new inspection system and inspection method that can detect surface defects of an object with high sensitivity.

[0007] As mentioned above, imaging optical systems can be broadly classified into diffuse reflection optical systems and specular reflection optical systems. Known imaging methods using diffuse reflection optical systems include dark-field illumination, light section method, phase-shift method (diffuse), and illuminance difference stereo method. On the other hand, methods for illuminating and imaging objects using specular reflection optical systems include bright-field illumination and phase-shift method (specular reflection).

[0008] As an example, let's explain the challenges of inspecting dents formed on a glossy surface (or mirror surface) using a conventional imaging optical system. A dent is, for example, a depression on the surface of an object formed by being pressed by a tool or other component. In dark-field illumination, the object is illuminated from an oblique angle so that the illumination light specularly reflected by the object does not enter the lens of the imaging device, and the light scattered by the object is received by the imaging device. However, in dark-field illumination, it is not easy to adjust the angle of illumination so that dents can be detected. This is because the change in the angle of light reflection before and after the formation of a dent on the surface is small, so the reflectance hardly changes. Similarly, in bright-field illumination, it is not easy to adjust the angle of illumination so that dents can be detected.

[0009] The light section method is a method of illuminating an object with a linear beam of light. However, shape measurement techniques based on triangulation are difficult to apply to mirrored surfaces. Therefore, the light section method is difficult to apply to mirrored surfaces. For similar reasons, the phase shift method (diffuse) and the illuminance difference stereo method are also difficult to apply to mirrored surfaces. Furthermore, the phase shift method (specular reflection) is difficult to apply to glossy surfaces other than mirrored surfaces.

[0010] Thus, all conventionally known imaging optical systems face the challenge of difficulty in imaging defects on glossy (or mirrored) surfaces. Therefore, there is a need for new inspection systems and methods that can detect surface defects of objects with high sensitivity.

[0011] The purpose of this disclosure is to provide a solution that enables the accurate detection of surface defects in an object being inspected. [Means for solving the problem]

[0012] According to one example of this disclosure, an inspection system for inspecting the surface of an object comprises an illumination device that illuminates a moving object with a lighting pattern having bright and dark regions using diffuse illumination, and an imaging device that images the moving object illuminated by the lighting pattern. The imaging device includes an event-based image sensor that detects changes in brightness values ​​for each pixel and outputs the changes in brightness values ​​as event data. The inspection system further comprises an image processing unit that inspects the surface of the object by analyzing the event data from the image sensor.

[0013] According to the above disclosure, an object that is illuminated by a lighting pattern having bright and dark regions and is in motion is imaged using an event-based image sensor. The event-based image sensor detects the change in brightness caused by the movement of the object illuminated by the above lighting pattern. If there is a defect on the surface of the object being inspected, a difference in how the brightness value changes occurs between the defective part and other parts. The event-based image sensor can detect this difference in brightness value change with high sensitivity. Therefore, defects on the surface of the object being inspected can be detected with high accuracy.

[0014] Furthermore, according to the above disclosure, even if the surface of the object to be inspected is glossy or mirrored, by using an event-based image sensor, changes in brightness values ​​on the surface of the object to be inspected can be detected with high accuracy. Defects formed on glossy or mirrored surfaces can be detected with high accuracy.

[0015] Furthermore, according to the disclosure described above, event data is selectively read from pixels where the brightness has changed. Therefore, the imaging time can be shortened.

[0016] "The surface of the object" refers to a surface of the object that is exposed to the outside of the object. For example, if the object has a housing, the outer surface of that housing corresponds to "the surface of the object" in this disclosure.

[0017] In the above disclosure, the image processing unit applies motion compensation to the event data from the image sensor according to the movement speed and direction of the object on the image, and generates pixel-corresponding event data, which is event data for each pixel.

[0018] "Event data" refers to data that captures changes in brightness values ​​at each pixel as events, including the coordinates of the pixel where the change occurred, the brightness change value of each pixel, and the polarity of the brightness change (increase or decrease) of each pixel. As the object moves, reflected light from the same part of the object moves along the direction of the object's movement. By applying motion compensation to the event data from the image sensor, changes in brightness at a certain part of the object can be detected as event data at the same pixel of the image sensor. In this specification, motion-compensated event data is referred to as "pixel-corresponding event data."

[0019] In the above disclosure, the image processing unit calculates the amplitude of the luminance value and the phase difference of the luminance values ​​between pixels based on the pixel-corresponding event data. Based on the amplitude and phase difference of the luminance values, the image processing unit determines whether or not there are defects on the surface of the object.

[0020] The "amplitude of the luminance value" corresponds to the difference between the maximum value of the luminance and the minimum value of the luminance. When the luminance change value is output as event data from the image sensor, the amplitude can be calculated by calculating the luminance value from the luminance change value in the pixel-corresponding event data. When the polarity of the luminance change is output as event data from the image sensor, the amplitude can be calculated based on the number of occurrences of the pixel-corresponding event data. The "phase difference of the luminance value" corresponds to the difference in the occurrence times of the pixel-corresponding event data between a plurality of pixels. The amplitude and phase difference of the luminance value reflect the reflection characteristics of the surface of the object. If there is a defect in a part of the surface of the object, the reflection characteristics of that part are different from those of other normal parts. Therefore, based on the amplitude and phase difference of the luminance value, by determining the presence or absence of a defect on the surface of the object, the defect on the surface of the inspection object can be accurately detected.

[0021] In the above disclosure, the image processing unit determines whether a candidate corresponding to a defect is included in the image based on the amplitude and phase difference of the luminance value. When a candidate is included in the image, the presence or absence of a defect is determined based on the number and distribution of the candidates in the image.

[0022] If the amplitude and phase difference of the luminance value at a certain pixel are different from the amplitude and phase difference of other pixels, it is considered that the pixel reflects a defect on the surface of the object. On the other hand, the defect on the surface of the object has a certain size. By determining whether there is a defect on the surface of the defective object based on the number and distribution of the candidate pixels, an accurate determination can be made.

[0023] In the above disclosure, the image processing unit generates a phase difference image showing the distribution of the phase difference of the luminance value in the image.

[0024] The phase difference reflects the shape of the defect. By generating a phase difference image, the shape defect can be imaged.

[0025] In the above disclosure, the image processing unit generates a reflectance image showing the distribution of the amplitude of the luminance value in the image.

[0026] The amplitude of the luminance value indicates the reflectivity of the surface of the object. For example, when the surface of the object is a glossy surface or a mirror surface, dirt, foreign matter, and scratches reduce the reflectivity of the surface of the object. Therefore, by generating a reflectivity image, the above-mentioned defects can be imaged.

[0027] In the above disclosure, the image processing unit calculates the temporal change rate of the luminance value and generates a reflection angle image showing the distribution of the temporal change rate of the luminance value in the image.

[0028] The temporal change rate (gradient) of the luminance value indicates the reflection angle on the surface of the object. For example, when the surface of the object is a glossy surface or a mirror surface, dents, dirt, foreign matter, and scratches change the reflection angle of the surface of the object. Therefore, by generating a reflection angle image, the above-mentioned defects can be imaged.

[0029] In the above disclosure, the imaging device is arranged in the specular reflection direction with respect to the irradiation direction of light from the illumination device to the object.

[0030] By arranging the imaging device in the specular reflection direction with respect to the irradiation direction of light from the illumination device to the object, defects formed on a glossy surface or a mirror surface can be detected.

[0031] According to an example of the present disclosure, an inspection method for inspecting the surface of an object includes a step in which an illumination device illuminates a moving object with an illumination pattern having bright regions and dark regions by diffuse illumination, and a step in which an imaging device images the object illuminated with the illumination pattern and moving. The imaging device includes an event-based imaging element that detects a change in luminance value for each pixel and outputs the change in luminance value as event data. The inspection method further includes a step in which an image processing unit analyzes the event data from the imaging element.

[0032] According to the above disclosure, an object that is illuminated by a lighting pattern having bright and dark regions and is in motion is imaged using an event-based image sensor. The event-based image sensor detects the change in brightness caused by the movement of the object illuminated by the above lighting pattern. If there is a defect on the surface of the object being inspected, a difference in how the brightness value changes occurs between the defective part and other parts. The event-based image sensor can detect this difference in brightness value change with high sensitivity. Therefore, defects on the surface of the object being inspected can be detected with high accuracy. [Effects of the Invention]

[0033] According to this disclosure, surface defects of an object being inspected can be detected with high sensitivity. [Brief explanation of the drawing]

[0034] [Figure 1] This is a schematic diagram showing the overall configuration of the visual inspection system according to this embodiment. [Figure 2] This is a schematic diagram showing the configuration of a lighting device. [Figure 3] This is a schematic diagram illustrating an example of a lighting pattern according to this embodiment. [Figure 4] This figure shows an example of the hardware configuration of a control device. [Figure 5] This is a diagram explaining reflectance. [Figure 6] This is a diagram illustrating the angle of reflection. [Figure 7] This diagram shows an example of the positional relationship between a specific part of an object and the pixels of an image sensor at a given point in time. [Figure 8] Figure 7 shows examples of height and reflectivity characteristics at various points on the surface of the object shown. [Figure 9] Figure 3(A) shows the change in brightness value of a specific part of an object when the object is illuminated and imaged with a normal camera. [Figure 10]Figure 3(A) schematically shows the data output from an event-based camera when the object is illuminated and the event-based camera captures an image of the object. [Figure 11] This diagram illustrates the candidate defects to be detected in this embodiment. [Figure 12] This diagram schematically illustrates the changes in surface reflectivity and reflection angle caused by dirt and abrasion, respectively. [Figure 13] This is a flowchart illustrating the basic flow of data analysis processing according to this embodiment. [Figure 14] This is a flowchart illustrating the display processing flow according to this embodiment. [Figure 15] This figure illustrates the surface of an object to be processed using the image processing method of this embodiment. [Figure 16] This figure shows the shape image (phase difference image) generated from the image data of the object shown in Figure 15. [Figure 17] This figure shows a reflectance image generated from the image data of the object shown in Figure 15. [Figure 18] This figure shows the reflection angle image generated from the image data of the object shown in Figure 15. [Figure 19] This figure shows another example configuration of the inspection system according to this embodiment. [Figure 20] This is a flowchart illustrating a first modified example of the data analysis process according to this embodiment. [Figure 21] This is a flowchart illustrating a second modified example of the data analysis process according to this embodiment. [Figure 22] This figure shows another example configuration for defect detection according to this embodiment. [Modes for carrying out the invention]

[0035] Embodiments of the present invention will be described in detail with reference to the drawings. Note that identical or corresponding parts in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[0036] §1 Examples of Application An example of the application of the present invention will be described with reference to Figure 1. Figure 1 is a schematic diagram showing the overall configuration of the inspection system 1 according to this embodiment. The inspection system 1 inspects the surface 3 of the object 2. The surface 3 of the object 2 is a surface of the object 2 that is exposed to the outside of the object 2. If the object 2 has a housing, the outer surface of the housing corresponds to the surface 3 of the object 2. The object 2 is, for example, an industrial product flowing on a production line. The object 2 can include, for example, an object with a glossy surface or an object with a mirror surface. The inspection system 1 is, for example, incorporated into a production line and inspects the surface 3 of the object 2 for defects. Examples of defects include dents, scratches, dirt, foreign matter adhesion, and abrasions. In this embodiment, the appearance of the object 2 being transported is inspected.

[0037] As shown in Figure 1, the inspection system 1 comprises, as its main components, an illumination device 10, an imaging device 16, and a control device 18.

[0038] The lighting device 10 is a device for illuminating the object 2. Under the control of the control device 18, the lighting device 10 can illuminate the object 2 with any lighting pattern. In this embodiment, the lighting device 10 illuminates the moving object 2 with a rectangular pattern in which bright and dark areas are separated by a boundary, using diffuse illumination.

[0039] The imaging device 16 receives light reflected from the surface 3 of the object 2. In this embodiment, the imaging device 16 is positioned such that the direction of its optical axis is the direction in which the light reflected from the illumination device 10 to the surface 3 of the object 2 travels, i.e., the direction of specular reflection. Figure 1 shows a typical arrangement of the illumination device 10, specifically a side illumination arrangement. However, the arrangement of the illumination device 10 is not limited to this, and for example, a coaxial incident illumination arrangement may also be used (see Figure 19).

[0040] The imaging device 16 includes an optical system such as a lens, as well as an image sensor 17 divided into multiple pixels. In this embodiment, an event-based image sensor is used as the image sensor 17. Such an imaging device is generally called an event-based camera.

[0041] An event-based camera is a luminance value difference output camera. An event-based camera is configured to sense changes in pixel luminance and output only the changed data, combined with pixel coordinate and time information. The event-based camera outputs only the information of the portion of the subject that has changed over time as image data (hereinafter also referred to as "event data"). This enables high-speed and low-latency data output. Furthermore, the reduced data size also reduces the computational load on the control device 18.

[0042] The control device 18 controls the lighting device 10 and the imaging device 16. Furthermore, the control device 18 analyzes the event data output from the imaging device 16. Based on the analysis results of the event data, the control device 18 inspects the surface 3 of the object 2 for defects. Therefore, the control device 18 performs the functions of both the image analysis unit and the visual inspection unit.

[0043] If object 2 is an object with a glossy or mirrored surface, most of the light irradiated onto surface 3 is specularly reflected. However, if surface 3 has a defect, a difference in reflection characteristics occurs between the defective area and the surrounding area, resulting in a difference in how the brightness value changes between the defective area and the other areas. In this embodiment, object 2 is illuminated with an illumination pattern having bright and dark regions and is in motion, and is imaged using an event-based image sensor. The event-based camera can detect defects on surface 3 of object 2 with high sensitivity by sensing changes in pixel brightness. In addition, the event-based camera selectively reads only the information from pixels whose brightness has changed. Therefore, according to this embodiment, the imaging time of object 2 can be shortened.

[0044] §2 Specific Examples <A. Example of Lighting Device> FIG. 2 is a diagram schematically showing the configuration of the lighting device 10. As shown in FIG. 2, the lighting device 10 has a light emitting surface 11. A plurality of light sources 101 are two-dimensionally arranged on the light emitting surface 11, and the light emitting surface 11 emits light in a planar shape. By controlling the lighting and extinguishing of each of the light sources 101, the lighting device 10 can illuminate the object 2 in an arbitrary lighting pattern. In FIG. 2, the light sources 101 in the extinguished state are shown by applying hatching.

[0045] Each of the light sources 101 is, for example, diffused illumination. Therefore, the lighting device 10 can illuminate the object 2 by diffused illumination from the light emitting surface 11. The light source 101 may be a point light source or a surface light source. For convenience of illustration, the plurality of light sources 101 are discretely arranged on the light emitting surface 11. However, it is also possible to apply a lighting device having a plurality of pixels arranged two-dimensionally (such as a liquid crystal display or an organic EL display, etc.) as the lighting device 10. In addition, in the present embodiment, since the lighting pattern may be a fixed pattern, a diffusion plate and a light shielding plate may be arranged on the light emitting surface 11. In this case, each of the light sources 101 does not necessarily have to be diffused illumination. Ideally, it is only necessary to supply light from the light emitting surface 11 so as to obtain a uniform illuminance with respect to the diffusion plate. The diffusion plate corresponds to the bright region of the lighting pattern, and the light shielding plate corresponds to the dark region of the lighting pattern.

[0046] FIG. 3 is a schematic diagram for explaining an example of the lighting pattern according to the present embodiment. As shown in FIG. 3(A), when an object 2 illuminated by a rectangular lighting pattern is photographed with an ordinary camera, an image of the object 2 in which the bright region 21 and the dark region 22 are reflected can be obtained. As shown in FIG. 3(A), the bright region 21 and the dark region 22 are separated by a boundary 23. For clarity of the figure, in FIG. 3(A), only the illumination state of the object 2 is shown, and the illustration of the illumination state of the background other than the object 2 is omitted.

[0047] In the example shown in Figure 3(A), if the direction of movement of object 2 is the X direction, the image changes as the image of object 2, which reflects the light region 21 and the dark region 22, moves along the image in the X direction.

[0048] Note that the examples of lighting patterns are not limited to those shown in Figure 3(A). For example, the arrangement of the light region 21 and the dark region 22 in the lighting pattern may be reversed from the arrangement shown in Figure 3(A). Also, the lighting pattern may contain multiple boundaries 23 between the light region 21 and the dark region 22. In that case, the change in luminance value will be in the opposite direction (i.e., monotonically decreasing) to the change (monotonically increasing) shown in the graph of Figure 9, which will be described later.

[0049] Alternatively, the illumination pattern may be generated such that it includes, for example, a change from a dark region 22 to a bright region 21, and a change from that bright region 21 to the next dark region 22 (i.e., it includes two boundaries 23). In this case, in the change of luminance value shown in Figure 9, an increase in luminance value and a decrease in luminance value occur once each. The processing method by the control device 18 can calculate feature quantities by obtaining feature quantities such as phase differences and combining them (for example, by averaging, maximizing, or minimizing).

[0050] Alternatively, the illumination pattern 20 may be generated such that multiple dark regions 22 and multiple bright regions 21 are arranged alternately.

[0051] Furthermore, in the example shown in Figure 3(A), the boundary 23 is oriented perpendicular to the direction of movement of the object 2. However, as shown in Figures 3(B) to 3(D), the direction of the boundary 23 may be oblique to the direction of movement of the object 2 (for example, the X direction shown in Figure 3(A)), such as +45 degrees or -45 degrees, and a combination of these may also be used as lighting patterns.

[0052] In the example shown in Figure 3(A), the direction of movement of object 2 is the X direction. However, the direction of movement of object 2 is not limited to the X direction shown in Figure 3(A).

[0053] <B. Configuration Example of Control Device> FIG. 4 is a diagram showing an example of the hardware configuration of the control device 18. As shown in FIG. 4, the control device 18 includes a processor 180 such as a CPU (Central Processing Unit) or MPU (Micro-Processing Unit), a RAM (Random Access Memory) 181, a display controller 182, a system controller 183, an I / O (Input Output) controller 184, a hard disk 185, a camera interface 186, an input interface 187, a lighting controller 188, a communication interface 189, and a memory card interface 190. These components are connected to each other so as to be capable of data communication with the system controller 183 as the center.

[0054] The processor 180 exchanges programs (codes) and the like with the system controller 183 and executes them in a predetermined order, thereby realizing the target arithmetic processing.

[0055] The system controller 183 is connected to the processor 180, the RAM 181, the display controller 182, and the I / O controller 184 via buses, respectively, exchanges data and the like with each component, and controls the processing of the entire control device 18.

[0056] The RAM 181 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory), and holds programs read from the hard disk 185, captured images received from the imaging device 16, processing results for the captured images, and work data.

[0057] The display controller 182 is connected to the display device 5, and outputs a signal for displaying various information to the display device 5 according to an internal command from the system controller 183. As an example, the display device 5 includes a liquid crystal display or an organic EL (Electro Luminescence) display.

[0058] The I / O controller 184 controls data exchange between the control device 18 and the recording medium and external devices connected to it. More specifically, the I / O controller 184 is connected to the hard disk 185, the camera interface 186, the input interface 187, the communication interface 189, and the memory card interface 190.

[0059] The hard disk 185 is typically a non-volatile magnetic storage device that stores analysis programs 191 executed by the processor 180. The analysis program 191 installed on the hard disk 185 is distributed in a state stored on a memory card 6 or similar. Furthermore, the hard disk 185 also stores captured images. Note that a semiconductor storage device such as flash memory may be used instead of the hard disk 185.

[0060] The camera interface 186 corresponds to an input unit that receives the captured image generated by imaging the object 2, and mediates data transmission between the processor 180 and the imaging device 16. More specifically, the processor 180 outputs an imaging instruction to the imaging device 16 via the camera interface 186. As a result, the imaging device 16 images the subject and outputs the generated captured image to the processor 180 via the camera interface 186.

[0061] The input interface 187 mediates data transmission between the processor 180 and input devices 7 such as a keyboard, mouse, touch panel, or dedicated console. In other words, the input interface 187 receives operation commands given by the user when they operate the input devices 7.

[0062] The lighting controller 188 is connected to the lighting device 10 and outputs signals to the lighting device 10 to control the lighting device 10 (controlling the on / off of the light source 101) according to commands from the system controller 183.

[0063] The communication interface 189 mediates data transmission between the processor 180 and other personal computers or server devices (not shown). The communication interface 189 typically consists of Ethernet (registered trademark) or USB (Universal Serial Bus). As will be described later, instead of installing the program stored on the memory card 6 into the control device 18, the program may be downloaded from a distribution server or the like and installed into the control device 18 via the communication interface 189.

[0064] The memory card interface 190 mediates data transmission between the processor 180 and the memory card 6, which is the recording medium. Specifically, the memory card 6 stores the analysis program 191 executed by the control device 18, and the memory card interface 190 reads the analysis program 191 from the memory card 6. The memory card interface 190 also writes the captured images acquired by the imaging device 16 and / or the processing results from the control device 18 to the memory card 6 in response to internal commands from the processor 180. The memory card 6 consists of a general-purpose semiconductor storage device such as an SD (Secure Digital) memory card or a CF (CompactFlash) card.

[0065] When using a computer having a structure that conforms to the general-purpose computer architecture described above, an OS (Operating System) for providing the basic functions of the computer may be installed in addition to the application for providing the functions according to this embodiment. In this case, the program according to this embodiment may call the necessary modules from among the program modules provided as part of the OS in a predetermined order and / or timing to execute processing. That is, the program according to this embodiment itself may not include the above-mentioned modules, and processing may be executed in cooperation with the OS.

[0066] Furthermore, the analysis program 191 according to the present embodiment may be provided by being incorporated into a part of another program. Even in this case, the program itself does not include the modules included in the other programs to be combined as described above, and the processing is executed in cooperation with the other program. That is, the analysis program 191 according to the present embodiment may be in a form incorporated into such other programs.

[0067] Alternatively, a part or all of the functions provided by the execution of the analysis program 191 may be implemented as a dedicated hardware circuit.

[0068] <C. Reflection Characteristics> Prior to specifically describing the present embodiment, the surface reflection characteristics related to the detection of defects according to the present embodiment will be described. Specifically, as the reflection characteristics, the "reflectance" and the "reflection angle" will be described below. The following description is an example of the definitions of reflectance and reflection intensity, and is not intended to limit the present embodiment.

[0069] FIG. 5 is a diagram for explaining the reflectance. In FIG. 5, a normal surface is a plane that can be regarded as having no defects. It is assumed that there is no light absorption on the normal surface. Light incident at an angle θ with respect to the normal n of the normal surface is reflected by the normal surface. The traveling direction of the reflected light (i.e., the specular reflection direction) is a direction that forms an angle θ with respect to the normal n of the normal surface.

[0070] The reflected light has an intensity distribution 30 centered on the specular reflection direction. The intensity distribution may be regarded as the spatial distribution of the energy of the reflected light. The reflectance is defined as the ratio of the integral value of the intensity distribution 30 of the reflected light to the energy of the incident light.

[0071] FIG. 6 is a diagram for explaining the reflection angle. In FIG. 6, a characteristic curve showing the relationship between the angle in the reflection direction and the intensity of the reflected light is shown. Note that the intensity of the reflected light is assumed to be normalized. The "reflection angle" means the range (θa in FIG. 6) from the angle at which the intensity of the reflected light becomes the peak intensity to the angle at which the intensity becomes X% (for example, 10%) of the peak intensity. Note that the angle at which the intensity of the reflected light peaks is the angle in the specular reflection direction.

[0072] <D. Outputs of a Normal Camera and an Event-Based Camera> FIG. 7 is a diagram showing an example of the positional relationship between a specific part of the object 2 and the pixels of the imaging device 17 at a certain point in time. In FIG. 7, a state where the pixels 40A to 40C of the imaging device 17 overlap with the specific parts 3A to 3C (hereinafter referred to as "part 3A", etc.) of the object 2 is shown. Since the object 2 moves, the positional relationship between the pixels 40A to 40C of the imaging device 17 and the parts 3A to 3C of the object 2 changes with time.

[0073] FIG. 8 is a diagram showing examples of the height and reflection characteristics at each part of the surface of the object 2 shown in FIG. 7. In FIG. 8, the height information is information in the Z direction (the direction perpendicular to the X direction and the Y direction shown in FIG. 3), and is information regarding the direction of the normal line to each part of the surface 3.

[0074] The stronger the specularly reflected light (in other words, the narrower the reflection angle), the higher the gloss of the surface 3. On the other hand, when the specularly reflected light is weak, the diffused light is strong, and the reflection angle is wide, the gloss of the surface 3 becomes low. In the example shown in FIG. 8, the normal direction of part 3B is inclined with respect to the normal directions of parts 3A and 3C. However, the reflection characteristics of parts 3A and 3B are substantially the same. On the other hand, the reflection angle at part 3C is narrower than the reflection angles at parts 3A and 3B. That is, part 3C has a higher gloss than parts 3A and 3B.

[0075] Figure 9 shows the change in brightness value of specific parts of object 2 when the object is illuminated as shown in Figure 3(A) and object 2 is imaged with a normal camera. In Figure 9, the change in brightness value over time is shown for each of the parts 3A to 3C shown in Figure 7.

[0076] Areas 3A, 3B, and 3C are initially dark. As object 2 moves, when the image of boundary 23 reflected on the object is captured, the image of boundary 23 appears to pass over object 2. Therefore, the brightness values ​​of each area 3A, 3B, and 3C change from the minimum value B1 to the maximum value B2.

[0077] In the case of part 3A, the luminance value increases from B1 at time t1, and reaches its maximum value B2 at time t2. In the graph shown in Figure 9, "amplitude" is the difference between the maximum luminance value B2 and the minimum luminance value B1, and reflects the reflectance of the surface of the object.

[0078] The "slope of the luminance value" is the rate of change of the luminance value over time. In part 3A, the slope of the luminance value is a1 (= (B2-B1) / (t2-t1)). The slope of the luminance value reflects the reflective properties (i.e., glossiness) of the corresponding part on surface 3. In other words, a large slope of the luminance value indicates high glossiness of the surface.

[0079] For part 3A, time t3 in the brightness value graph is the time when it reaches a predetermined value (e.g., an intermediate value) between the minimum value B1 and the maximum value B2. In the brightness value graph for part 3B, the same time is shown as t3'. Time t3' is a different time from time t3. In this embodiment, the difference between pixels at the time when they reach a predetermined brightness value is called the "phase difference" and is represented by Δφ. The phase difference occurs because the normal of part 3B is tilted relative to the normal of part 3A (see Figure 6). For example, such a phase difference occurs when part 3B is a recessed area from its surroundings.

[0080] Regarding part 3C, the luminance value rises from B1 at time t1', and reaches the maximum value B2 at time t2'. In the case of part 3C, the slope of the luminance value is a2. The slope a2 is larger than the slope a1. This is because the reflection angle of part 3C is narrower than that of part 3A. That is, as shown in Fig. 3(A), when the object is illuminated and the object 2 moves, the brightness changes more steeply at part 3C than at part 3A. This means that part 3C has a stronger gloss than part 3A.

[0081] Fig. 10 is a diagram schematically showing the data output from an event-based camera when an object is illuminated as shown in Fig. 3(A) and the object 2 is imaged with the event-based camera. As described above, when the luminance of a pixel changes, the event-based camera outputs only the changed data, combined with the pixel coordinates and time information, as event data. Therefore, for each of parts 3A, 3B, and 3C, event data is generated triggered by the luminance value changing from the original value by a predetermined threshold.

[0082] The occurrence frequency of event data for each pixel corresponds to the time change rate (slope) of the luminance value. As understood from the comparison between the luminance value graph of part 3A and the luminance value graph of part 3C, the larger the change rate of the luminance value, the higher the occurrence frequency of event data per unit time. Also, as understood from the comparison between the luminance value graph of part 3A and the luminance value graph of part 3B, there is a difference in the occurrence time of event data. The difference in the occurrence time of event data corresponds to the phase difference.

[0083] Thus, the occurrence frequency and occurrence time of event data have a certain relationship with the reflection characteristics of the surface of the inspection object. In this embodiment, this relationship is utilized to inspect the presence or absence of defects on the surface of the object.

[0084] <E. Types of Defects> Figure 11 illustrates the candidate defects to be detected in this embodiment. As shown in Figure 11, examples of defects include dents, scratches, dirt and foreign matter, and abrasions.

[0085] An indentation 51 is a depression in the surface 3 of an object 2 formed by pressing an object with a rounded tip, such as a tool 50, against the surface 3 of the object 2. Although the reflectivity of the surface does not substantially change before and after the formation of the indentation 51, the angle of reflection tends to become narrower.

[0086] The scratch 52 is a scratch with a V-shaped cross-section. The scratch represents a state where the surface 3 has been scraped, and because the surface condition is hardly changed, the reflectivity and reflection angle of the surface hardly change before and after the formation of the scratch 52.

[0087] Scratch 53 is a very shallow scratch formed by friction on the surface 3 of object 2.

[0088] Dirt or foreign matter refers to the state in which dirt (or foreign matter) 54 is attached to the surface 3 of object 2.

[0089] Figure 12 schematically illustrates the changes in surface reflectance and reflection angle due to dirt and abrasion, respectively. Referring to Figures 11 and 12, in the case of abrasion, the reflectance hardly changes because there is no absorption by the surface 3 of object 2. On the other hand, the distribution of reflection intensity widens slightly spatially. Therefore, the reflection angle widens slightly from the reflection angle in the non-abrasion state. Consequently, the angle θb shown in Figure 12 is larger than the angle θa shown in Figure 6.

[0090] When dirt (or foreign matter) adheres to the surface 3 of object 2, the distribution of reflectivity becomes more spatially spread due to the dirt. Furthermore, a portion of the energy of the incident light is absorbed by the dirt. As a result, the reflectivity decreases and the reflection angle widens. The angle θc shown in Figure 12 is greater than the angle θb. Note that when dirt adheres to the surface 3 of object 2, the reflection of incident light at the location of the dirt can be considered to be close to diffuse reflection.

[0091] <F. Defect Inspection> FIG. 13 is a flowchart for explaining the basic flow of data analysis processing according to the present embodiment. Mainly, the control device 18 shown in FIG. 1 executes an analysis program. Thereby, the processing shown in FIG. 13 is executed.

[0092] The lighting device 10 illuminates the moving object 2 with a rectangular lighting pattern by diffused lighting. The imaging device 16 (event-based camera) images the object 2 that is illuminated by the lighting pattern and moves. The control device 18 receives event data (see FIG. 10) output from the imaging device 16 (event-based camera) (step S10).

[0093] Since the object 2 moves, the reflected light from the same part on the object 2 moves along the moving direction of the object 2. Therefore, the control device 18 performs motion compensation on the event data output from the imaging element 17 so that the change in luminance at a certain part on the object 2 is detected as event data in the same pixel of the imaging element 17 (step S11).

[0094] The method of motion compensation is not particularly limited. For example, the control device 18 can calculate the moving speed and moving direction on the image from the conveyance speed of the object 2 (the moving speed of the object 2) and the working distance of the imaging element 17, and generate a pixel column based on the calculation result. In addition, known devices and methods can be applied to detect the conveyance speed of the object 2. A motor is generally used for a line (for example, a belt conveyor) that conveys the object. Therefore, the conveyance speed of the object 2 can be detected by detecting the rotation speed of the motor by a rotary encoder or the like.

[0095] Alternatively, the control device 18 may detect feature points of object 2 (for example, corners or edges of object 2) from the captured image of object 2 and obtain a mapping function. The control device 18 may use this mapping function to perform motion compensation on the event data. For example, if four feature points are obtained, the control device 18 may perform a projection transformation from those feature points.

[0096] Alternatively, the control device 18 may find the mapping function using an energy minimization method. For example, the energy function may reflect the fact that the movement vector is smooth in the spatial and time series directions, and that the time series data on the movement vector is smooth in the time direction, and the solution that minimizes its energy may be calculated using an iterative method or other technique.

[0097] The control device 18 calculates the phase, amplitude, and slope of the brightness value of each pixel based on the motion-compensated event data, i.e., "pixel-corresponding event data". Specifically, when the polarity of brightness change is output as event data from the image sensor 17, the control device 18 calculates the amplitude of the brightness value based on the number of occurrences of the pixel-corresponding event data and calculates the phase of each pixel based on the timing of occurrence of the pixel-corresponding event data (step S12). However, the method for calculating the phase, amplitude, and slope of the brightness value of each pixel is not limited to this. A camera that outputs time t, coordinates (x,y), and brightness change amount p as event data from the image sensor 17 may be applied to the imaging device 16. In such a case, the control device 18 calculates the brightness value from the brightness change value of the pixel-corresponding event data and calculates the amplitude of the brightness value.

[0098] Next, the control device 18 calculates the phase difference between the luminance value of each pixel and the luminance values ​​of the surrounding pixels (step S13). For example, the phase difference of luminance values ​​is calculated between a given pixel and the pixels adjacent to it above, below, to the left, and to the right.

[0099] Finding the phase difference between two adjacent pixels can be considered equivalent to finding the first derivative of the phase change. Note that the method for finding the phase difference is not limited to the above. For example, the phase difference may be found using the second derivative. Alternatively, a phase image may be generated, and the phase difference in phase space may be found by applying a spatial filter to that image data.

[0100] Next, the control device 18 determines candidate defective pixels (step S14). Specifically, the control device 18 determines whether or not a candidate corresponding to a defect is included in the image based on the amplitude and phase difference of the brightness value. The control device 18 designates pixels whose amplitude is greater than or equal to a threshold and whose absolute value of the phase difference is greater than or equal to a threshold as candidate defective pixels.

[0101] If a candidate for a defective pixel is included in the image, the control device 18 determines an image region in which defective pixels appear at a high frequency as a defective region (step S15). "Image region" refers to one of the imaging regions of the imaging device 16 that are partitioned to a predetermined size. "Frequency" refers to, for example, the proportion of defective pixels among the pixels in the image region. The control device 18 can determine the presence or absence of a defective region by comparing the frequency with a threshold value.

[0102] If a defective area is included in the image, the control device 18 determines whether the area of ​​the defective area is greater than or equal to a threshold, and whether the number of defective areas is also greater than or equal to a threshold (step S16). This process determines whether the defect criteria are met. If the area of ​​the defective area is less than the threshold, or if the number of defective areas is less than the threshold (NO in step S16), the control device 18 determines that there are no defects on the surface of the object 2. Therefore, the result of the visual inspection is "OK" (step S17).

[0103] On the other hand, if the area of ​​the defective region is greater than or equal to the threshold, and the number of defective regions is also greater than or equal to the threshold (YES in step S16), the control device 18 determines that there is a defect on the surface of the object 2. Therefore, the result of the visual inspection is "NG" (step S18).

[0104] When the amplitude and phase difference of the luminance value in a certain pixel are different from those of other pixels, it is considered that the pixel reflects a defect on the surface of the object. On the other hand, the defect on the surface of the object has a certain size. By determining whether there is a defect on the surface of the defective object based on the number and distribution of candidate pixels, an accurate determination can be made.

[0105] <G. Display Processing> A process of displaying the result of the image processing shown in FIG. 13 to the user may be added. In order to display the result of the image processing to the user, the control device 18 generates an image for providing the result of the image processing to the user. This process may be executed after the process of step S17 or after the process of step S18 shown in FIG. 13.

[0106] FIG. 14 is a flowchart for explaining the flow of the display processing according to the present embodiment. Mainly, the processing shown in FIG. 14 is executed by the control device 18 shown in FIG. 1 executing an analysis program.

[0107] The control device 18 generates a shape image based on the phase difference of the luminance values between each pixel included in the captured image (step S21). The shape image is an image showing the distribution of the phase difference between pixels in the image.

[0108] Similarly, the control device 18 generates a reflectance image of the amplitude of the luminance value of each pixel in the image based on the amplitude of each pixel (step S22). The control device 18 generates a reflection angle image based on the inclination of the luminance value of each pixel in the image (step S23). Note that the order of generating the shape image (phase difference image), the reflectance image, and the reflection angle image is not limited as described above, and these images can be generated in any order. Alternatively, the shape image, the reflectance image, and the reflection angle image may be generated simultaneously.

[0109] The control device 18 displays the generated image on the display device 5 (see Figure 4) (step S23). The control device 18 may display all of the shape image (phase difference image), reflectance image, and reflection angle image on the display device 5. Alternatively, the control device 18 may display only the image selected by the user from among the shape image (phase difference image), reflectance image, and reflection angle image on the display device 5.

[0110] Figure 15 illustrates the surface of an object to be image-processed according to this embodiment. Referring to Figure 15, the surface 3 of object 2 is a glossy surface, for example, a metallic surface with a reflection angle. In Figure 15, defects on surface 3 are shown as dents 51, scratches 52, abrasions 53, and dirt (or foreign matter) 54.

[0111] Figure 16 shows a shape image (phase difference image) generated from the image data of the object shown in Figure 15. Referring to Figure 16, the shape image 61 is a grayscale image with grayscale values ​​corresponding to the magnitude of the phase difference. The brightly displayed areas in the shape image are areas on the surface 3 of the object 2, as captured by the event-based camera, where the phase change is large. In the image captured by the event-based camera, the phase change of the brightness value between pixels is large due to the impact marks 51 and scratches 52. Therefore, defect areas 62 representing the impact marks 51 and defect areas 63 representing the scratches 52 appear in the shape image 61. In this way, shape defects such as impact marks 51 and scratches 52 can be visualized by generating a shape image.

[0112] Figure 17 shows a reflectance image generated from the image data of the object shown in Figure 15. Referring to Figure 17, the reflectance image 64 is a grayscale image with grayscale values ​​corresponding to the magnitude of the reflectance. In the reflectance image 64, brightly displayed areas are areas of the surface 3 of the object 2 with high reflectance. On the other hand, darkly displayed areas in the reflectance image 64 are areas of the surface 3 of the object 2 with low reflectance. When dirt (foreign matter) 54 adheres to the surface, the reflectance of the surface 3 decreases in that area. Therefore, a defect area 65 representing the dirt 54 appears in the reflectance image 64. Furthermore, if the decrease in reflectance due to abrasion 53 is large (for example, if the area of ​​abrasion 53 is large), the probability of a defect area representing abrasion 53 appearing in the reflectance image 64 increases. Therefore, depending on the degree of abrasion, it is possible to detect abrasion 53 using the reflectance image 64.

[0113] Figure 18 shows a reflection angle image generated from the image data of the object shown in Figure 15. Referring to Figure 18, the reflection angle image 66 is a grayscale image with grayscale values ​​corresponding to the magnitude of the reflection angle. The areas that appear bright in the reflection angle image 66 are areas with a narrow reflection angle on the surface 3 of the object 2 captured by the event-based camera. In other words, areas of the surface 3 with high gloss appear bright in the reflection angle image 66. On the other hand, the areas that appear dark in the reflection angle image are areas of the surface 3 of the object 2 captured by the event-based camera with a wide reflection angle and low gloss.

[0114] When a dent 51 is formed on the surface 3 of object 2, the reflection angle of surface 3 narrows. As a result, the defect area 67 corresponding to the dent 51 is displayed brightly in the reflection angle image 66. On the other hand, in the case of abrasion 53 or dirt (foreign matter) 54, the reflection angle of surface 3 widens, so in the reflection angle image 66, the defect area 68 corresponding to abrasion 53 and the defect area 69 corresponding to dirt 54 are displayed darkly. Note that the widening of the reflection angle is greater for dirt 54 than for abrasion 53. For this reason, the defect area 69 is displayed darker than the defect area 68 in the reflection angle image 66.

[0115] <Another Configuration Example of the Inspection System> The inspection system 1 according to the present embodiment is not limited as described above. Hereinafter, various configuration examples applicable to the present embodiment will be described.

[0116] FIG. 19 is a diagram showing another configuration example of the inspection system according to the present embodiment. Referring to FIG. 19, the inspection system 1A is different from the inspection system 1 shown in FIG. 1 in that it has a coaxial optical system. That is, in FIG. 19, a configuration for coaxial epi-illumination is shown.

[0117] Specifically, the inspection system 1A includes an illumination device 10A instead of the illumination device 10. The illumination device 10A includes a light source 12 and a half mirror 13. The light source 12 is arranged such that the optical axis of the light emitted from the light source 12 is orthogonal to the normal of the surface 3 of the object 2. The half mirror 13 is arranged such that the angle formed by the optical axis of the emitted light and the reflection surface of the half mirror 13 is 45 degrees in order to direct the light emitted from the light source 12 toward the object 2. Thereby, the light emitted from the light source 12 is reflected by the half mirror 13 and illuminates the object 2 from the direction of the normal of the surface 3 of the object 2.

[0118] The reflected light from the object 2 travels in the direction of the normal of the surface 3 of the object 2. The imaging device 16 is arranged such that the angle formed by the optical axis of the imaging device 16 and the half mirror 13 is 45 degrees. Thereby, the reflected light from the object 2 passes through the half mirror 13 and enters the imaging device 16. Thus, the imaging optical system may be arranged such that the incident light on the object 2 and the reflected light from the object 2 are coaxial.

[0119] FIG. 20 is a flowchart for explaining a first modification of the data analysis process according to the present embodiment. The flowchart shown in FIG. 20 is different from the flowchart shown in FIG. 13 in that the process of step S19 is added after step S18. Therefore, FIG. 19 particularly shows the points different from the flowchart of FIG. 13.

[0120] In step S16, the control device 18 determines whether the area of ​​the defect region is greater than or equal to a threshold, and whether the number of defect regions is also greater than or equal to a threshold. In step S18, the control device 18 determines that there is a defect on the surface of the object 2. In this case, in step S19, the control device 18 outputs information about the defect region. The information about the defect region is not particularly limited, but may include, for example, the coordinate position of the defect region, the area of ​​the defect region, and the number of defect regions. This allows the user to obtain more information about defects on the surface 3 of the object 2, such as the type of defect, the location of the defect, the size of the defect, and the number of defects.

[0121] Figure 21 is a flowchart illustrating a second modified example of the data analysis process according to this embodiment. The flowchart in Figure 21 differs from the flowchart in Figure 13 in that the processes in steps S20 and S21 are added after step S18. Therefore, Figure 21 specifically highlights the differences from the flowchart in Figure 13. In step S20, the glossiness is estimated from the slope of the change in brightness of each pixel. The slope of the change in brightness of each pixel is obtained by the process in step S12, so the explanation will not be repeated.

[0122] In step S21, the control device 18 estimates the type of defect by comparing the glossiness with a threshold. As described above, the slope of the change in brightness is related to the reflection angle. Depending on the type of defect, the reflection angle widens or narrows. That is, depending on the type of defect, the glossiness becomes higher or lower. Therefore, by appropriately determining the glossiness threshold, the control device 18 can identify the type of defect.

[0123] In the flowchart shown in Figure 14, the user determines the type of defect based on the image generated by image processing. By executing the flowchart shown in Figure 20, the control device 18 can automatically determine the type of defect. In addition, in the flowchart shown in Figure 21, the process of step S19 shown in Figure 20 may be added at an appropriate stage after the process of step S18.

[0124] Figure 22 shows another configuration example for defect detection according to this embodiment. Referring to Figure 22, the control device 18 has a hard disk 185A that stores the analysis program 191A and the learning model 192. Note that Figure 20 only shows the processor 180, RAM 181, and hard disk 185A as the main components for defect detection. The configuration of the other parts of the control device 18 is the same as the configuration of the corresponding parts of the control device 18 shown in Figure 4, so the explanation will not be repeated.

[0125] The learning model 192 is a pre-trained estimation model that has been trained to determine the presence or absence of defects by machine learning event data output from the imaging device 16 (event-based camera). The learning model 192 may also perform deep learning using, for example, a neural network (for example, a recurrent neural network). A supervised learning algorithm may be used to train the learning model 192. The data after motion compensation has been applied to the output of the imaging device 16 (event-based camera) (pixel-corresponding event data) may be used as data for machine learning the learning model 192.

[0126] During inspection of the target product, the processor 180 refers to the learning model 192 and determines the presence or absence of defects based on event data from the imaging device 16. The learning model 192 may be trained to determine the presence or absence of defects for each pixel. Alternatively, the learning model 192 may be trained to determine the presence or absence of defects using the entire image or an image region of a certain size as input.

[0127] §3 Addendum As described above, this embodiment includes the following disclosures.

[0128] (Composition 1) An inspection system (1,1A) for inspecting the surface (3) of an object (2), A lighting device (10,10A) that illuminates a moving object (2) with a lighting pattern having bright and dark regions using diffuse illumination, The system includes an imaging device (16) that is illuminated by a lighting pattern and captures images of a moving object (2), and the imaging device (16) is Includes an event-based image sensor (17) that detects changes in brightness value for each pixel and outputs the changes in brightness value as event data, An inspection system (1,1A) further comprising an image processing unit (18) that inspects the surface (3) of an object (2) by analyzing event data from an image sensor (17).

[0129] (Configuration 2) The inspection system (1,1A) described in Configuration 1, wherein the image processing unit (18) applies motion compensation to the event data from the image sensor (17) according to the movement speed and direction of the object on the image to generate pixel-corresponding event data, which is event data for each pixel.

[0130] (Composition 3) The image processing unit (18) is: The inspection system (1,1A) described in Configuration 1 calculates the amplitude of the brightness value and the phase difference of the brightness values ​​between pixels based on pixel-corresponding event data, and determines whether or not there are defects on the surface (3) of the object (2) based on the amplitude and phase difference of the brightness values.

[0131] (Composition 4) The inspection system (1,1A) described in configuration 3, wherein the image processing unit (18) determines whether or not a candidate corresponding to a defect is included in the image based on the amplitude and phase difference of the brightness values, and if a candidate is included in the image, it determines whether or not a defect is present based on the number and distribution of the candidates.

[0132] (Composition 5) The inspection system (1,1A) according to configuration 3 or configuration 4, wherein the image processing unit (18) generates a phase difference image (61) showing the distribution of phase differences of brightness values ​​within the image.

[0133] (Composition 6) The inspection system (1,1A) according to configuration 3 or configuration 4, wherein the image processing unit (18) generates a reflectance image (64) showing the distribution of the amplitude of brightness values ​​within the image.

[0134] (Composition 7) The inspection system (1,1A) according to configuration 3 or configuration 4, wherein the image processing unit (18) calculates the rate of change of brightness values ​​over time and generates a reflection angle image (66) showing the distribution of the rate of change of brightness values ​​over time within the image.

[0135] (Composition 8) The inspection system (1,1A) according to any one of configurations 1 to 7, wherein the imaging device (16) is positioned in a specular reflection direction with respect to the direction of light irradiation from the illumination device (10) to the object (2).

[0136] (Composition 9) An inspection method for inspecting the surface (3) of an object (2), Step (S11) of the lighting device (10,10A) illuminating the moving object (2) with a lighting pattern having a light area (21) and a dark area (22) by diffuse illumination, The imaging device (16) includes the step (S11) of imaging an object (2) that is illuminated by a lighting pattern and is moving, and the imaging device (16) includes an event-based image sensor (17) that detects changes in brightness value for each pixel and outputs the changes in brightness value as event data. An inspection method comprising the steps (S11-S21) of an image processing unit (18) analyzing event data from an image sensor (17).

[0137] (Composition 10) The inspection method according to configuration 9, further comprising the step (S10) of applying motion compensation to the event data from the image sensor (17) according to the movement speed and direction of movement of the object (2) on the image, prior to the analysis step (S11 to S21), to generate pixel-corresponding event data which is event data for each pixel.

[0138] (Composition 11) The analysis steps (S11-S21) are: The image processing unit (18) performs the step (S12) of calculating the amplitude of the brightness value and the phase difference of the brightness values ​​between pixels based on the pixel-corresponding event data, The inspection method according to configuration 10, comprising the steps (S14-S18) of an image processing unit (18) determining whether or not there are defects on the surface (3) of an object (2) based on the amplitude and phase difference of the brightness values.

[0139] (Composition 12) The step (S14-S18) of determining whether or not there are defects on the surface (3) is, Step (S14) is to determine whether or not the image contains candidates corresponding to defects based on the amplitude and phase difference of the brightness values. If the image contains candidates corresponding to defects, the step (S15) is to determine the number and distribution of candidates in the image, Step (S15) to identify defective regions in the image based on the number and distribution of candidates, The inspection method according to configuration 11, comprising the step (S16) of determining whether the number of defect regions and the area of ​​the defect regions meet the criteria for determining defects.

[0140] (Composition 13) The inspection method according to configuration 11 or configuration 12, further comprising the step (S21) of an image processing unit (18) generating a phase difference image (61) showing the distribution of phase differences of luminance values ​​within an image.

[0141] (Composition 14) The inspection method according to configuration 11 or configuration 12, further comprising the step (S22) of an image processing unit (18) generating a reflectance image (64) showing the distribution of the amplitude of brightness values ​​within the image.

[0142] (Composition 15) The image processing unit (18) performs the step (S12) of calculating the time rate of change of the brightness value, The inspection method according to configuration 11 or configuration 12, further comprising the step (S23) of an image processing unit (18) generating a reflection angle image (66) showing the distribution of the rate of change of brightness values ​​over time within an image.

[0143] While embodiments of the present invention have been described, the embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0144] 1,1A Inspection system, 2 Object, 3 Surface, 3A,3B,3C Part, 5 Display device, 6 Memory card, 7 Input device, 10,10A Illumination device, 11 Light-emitting surface, 12,101 Light source, 13 Half mirror, 16 Imaging device, 17 Image sensor, 18 Control device, 21 Bright area, 22 Dark area, 23 Boundary, 30 Intensity distribution, 40 Image data, 40A,40B,40C Pixel, 50 Tool, 51 Indentation, 52 Linear scratch, 53 Scuff, 54 Dirt (foreign matter), 61 Shape image, 62,63,65,67,68,69 Defect area, 64 Reflectance image, 66 Reflection angle image, 180 Processor, 181 RAM, 182 Display controller, 183 System controller, 184 I / O controller, 185,185A Hard disk, 186 Camera interface, 187 Input interface, 188 Lighting controller, 189 Communication interface, 190 Memory card interface, 191,191A Analysis program, 192 Learning model, B1 Minimum brightness value, B2 Maximum brightness value, S11~S23 Steps, a1,a2 Slope of brightness value, n Normal, t1,t2,t3,t3' Time.

Claims

1. An inspection system for inspecting the surface of an object, A lighting device that illuminates the moving object with a lighting pattern having light and dark regions using diffuse illumination, The imaging device comprises an imaging device that images the object being illuminated by the aforementioned illumination pattern and moving, and the imaging device is Includes an event-based image sensor that detects changes in brightness value for each pixel and outputs the changes in brightness value as event data, The system further includes an image processing unit that inspects the surface of the object by analyzing the event data from the image sensor, The image processing unit performs motion compensation on the event data from the image sensor to generate pixel-corresponding event data, which is event data for each pixel, as an inspection system.

2. The inspection system according to claim 1, wherein the image processing unit applies motion compensation to the event data from the image sensor according to the movement speed and direction of movement of the object on the image to generate the pixel-corresponding event data.

3. The aforementioned image processing unit, The inspection system according to claim 2, comprising calculating the amplitude of the brightness value and the phase difference of the brightness value based on the pixel-corresponding event data, and determining whether or not there are defects on the surface of the object based on the amplitude of the brightness value and the phase difference.

4. The inspection system according to claim 3, wherein the image processing unit determines whether or not a candidate corresponding to the defect is included in the image based on the amplitude and phase difference of the brightness value, and if the candidate is included in the image, determines whether or not the defect is present based on the number and distribution of the candidates.

5. The inspection system according to claim 3 or 4, wherein the image processing unit generates a phase difference image showing the distribution of the phase difference of the luminance values ​​within the image.

6. The inspection system according to claim 3 or 4, wherein the image processing unit generates a reflectance image showing the distribution of the amplitude of the luminance values ​​within the image.

7. The inspection system according to claim 3 or 4, wherein the image processing unit calculates the rate of change over time of the luminance value and generates a reflection angle image showing the distribution of the rate of change over time of the luminance value within the image.

8. The inspection system according to claim 1, wherein the imaging device is arranged in a specular reflection direction with respect to the direction of light irradiation from the illumination device to the object.

9. An inspection method for inspecting the surface of an object, The lighting device illuminates the moving object with a lighting pattern having a light region and a dark region using diffuse illumination. The imaging device includes the step of imaging an object that is illuminated by the illumination pattern and is moving, and the imaging device includes an event-based image sensor that detects changes in brightness value for each pixel and outputs the changes in brightness value as event data. The image processing unit performs motion compensation on the event data from the image sensor to generate pixel-corresponding event data, which is event data for each pixel. An inspection method further comprising the step of the image processing unit analyzing the pixel-corresponding event data.

Images