Image processing device, image processing method, and program

Abstract

The purpose is to identify the cause of changes in illumination for the object being inspected during visual inspection. [Solution] The image processing device sequentially lights up multiple light sources to image a reference reflector and obtains the brightness of the resulting image. It then obtains the brightness of a reference corresponding to the brightness of the image and determines whether the state of each light source included in the multiple light sources is normal or abnormal based on the brightness of the image and the brightness of the reference. If the state of a light source is abnormal, the type of abnormality is identified.

Description

Technical Field

[0001] The present invention relates to an inspection technique using images.

Background Art

[0002] For the appearance inspection of industrial products, a technique is known in which an object is imaged while sequentially lighting a plurality of light sources, and defects are detected based on a plurality of captured images with different lighting directions. In such an inspection technique, if the illuminance of each light source is not appropriate, it may affect the defect detection accuracy. For example, in the case of an LED light source, it is known that the illuminance decreases with the lighting time. If such a light source is included in the inspection, the defect detection accuracy may decrease. As a technique for detecting deterioration of a light source, for example, there is Patent Document 1. Patent Document 1 discloses a technique in which the illuminance due to the light emission of an LED element is recorded using an illuminance sensor, and when the measured illuminance during light emission falls below a set value, it notifies that it is the life of the light source.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, the cause of the change in illuminance for the inspection object is not necessarily only the deterioration of the light source. For example, even when the direction of the light source changes due to reasons such as touching the installed light source, the illuminance may change. When the light source deteriorates, it can be dealt with by replacing it with a new light source, but when the direction of the light source changes, measures to correct the light source direction are required. Since the countermeasures differ depending on the cause of the illuminance change, it is important to identify the cause of the illuminance change. However, in Patent Document 1, the cause of the change in illuminance could not be identified.

[0005] Therefore, the present invention aims to identify the cause of changes in illuminance relative to the object being inspected during visual inspection. [Means for solving the problem]

[0006] To solve the above problems, the image processing apparatus according to the present invention comprises: a first acquisition means for acquiring the brightness of an image obtained by sequentially lighting up a plurality of light sources and imaging a reference reflector; a second acquisition means for acquiring the brightness of a reference corresponding to the brightness of the image; and a determination means for determining whether the state of each light source included in the plurality of light sources is normal or abnormal based on the brightness of the image and the brightness of the reference, wherein the determination means identifies the type of abnormality when the state of the light source is abnormal. [Effects of the Invention]

[0007] According to the present invention, it is possible to identify the cause of changes in illuminance relative to the object being inspected during visual inspection. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of the hardware configuration of the inspection system. [Figure 2] External view of the inspection system [Figure 3] Block diagram showing the functional configuration of the image processing device. [Figure 4] A flowchart showing the processes performed by the image processing unit. [Figure 5] A diagram illustrating the geometric conditions between the reference reflector, light source, and imaging device. [Figure 6] A diagram showing an example of a user interface. [Figure 7] Diagram illustrating the process of determining the state of a light source. [Figure 8] Flowchart showing the process for determining the state of the light source [Figure 9] A diagram showing an example of a user interface. [Figure 10] Flowchart showing the process for determining the state of the light source [Figure 11] A diagram showing an example of a judgment area. [Figure 12] A diagram illustrating the geometric conditions between the reference reflector, light source, and imaging device. [Figure 13] Diagram illustrating the process of determining the state of a light source. [Figure 14] Flowchart showing the process for determining the state of the light source [Modes for carrying out the invention]

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all combinations of features described in each embodiment are essential to the solution of the present invention.

[0010] [First Embodiment] <Hardware configuration of the image processing device> An example of the hardware configuration of the inspection system according to this embodiment will be explained using the block diagram in Figure 1. The inspection system according to this embodiment includes an image processing device 1, a display device 115, an input device 110, an imaging device 111, a light source device 116, and a storage device 113.

[0011] The CPU 101 executes various processes using computer programs and data stored in the RAM 103. In this way, the CPU 101 controls the operation of the image processing device 1 and executes or controls various processes described as those performed by the image processing device 1. The ROM 102 stores setting data for the image processing device 1, computer programs and data related to the startup of the image processing device 1, and computer programs and data related to the basic operation of the image processing device 1. The RAM 103 has areas for storing computer programs and data loaded from the ROM 102 and the storage device 113, and areas for storing captured images output from the imaging device 111. Furthermore, the RAM 103 has a work area used by the CPU 101 when executing various processes. In this way, the RAM 103 can provide various areas as appropriate.

[0012] The display device 115 is connected to the video card (VC) 104. For example, the CPU 101 can output the processing result by the CPU 101 to the display device 115 via the VC 104, so that the processing result can be displayed on the display device 115 as an image, characters, etc. The display device 115 is a display device having a liquid crystal screen or a touch panel screen. Incidentally, the display device 115 may be a projection device such as a projector. The input device 110, the imaging device 111, and the light source device 116 are connected to the general-purpose I / F (interface) 105.

[0013] The input device 110 is a user interface (UI) such as a keyboard, a mouse, or a touch panel, and various instructions and information can be input to the image processing device 1 by the user's operation. The imaging device 111 is a device that images an inspection target object. The imaging device 111 may be an imaging device that periodically or irregularly images a still image, or may be an imaging device that images a moving image. The light source device 116 is a device that irradiates light on the inspection target object and has a plurality of light sources. The storage device 113 is connected to the SATA (Serial ATA) I / F 106. The CPU 101 reads and writes computer programs and data to and from the storage device 113 via the SATA I / F 106. The storage device 113 is a non-volatile storage device such as a hard disk drive. The storage device 113 stores an OS, computer programs and data for causing the CPU 101 to execute or control various processes described as the processes performed by the image processing device 1.

[0014] Furthermore, the image processing device 1 can be connected to a network such as a LAN or the Internet via a NIC (Network Interface Card) 107, and can communicate data with devices on the network. The image processing device 1 may acquire some or all of the information used in each of the processes described later from devices on the network via the NIC 107. The CPU 101, ROM 102, RAM 103, VC 104, general-purpose I / F 105, SATA I / F 106, and NIC 107 are all connected to the system bus 108. Note that the image processing device 1 can be a computer device such as a PC (personal computer), smartphone, or tablet terminal. Also, the configuration of the inspection system shown in Figure 1 is merely an example, and for example, a system may be configured by combining two or more of the devices shown in Figure 1.

[0015] Next, an example of the arrangement of the imaging device 111 and light source device 116, which are positioned to inspect the gloss, color, and unevenness of the surface of the object to be inspected, will be explained using Figure 2. The light source device 116 has a gloss inspection light source 203-1 that irradiates light onto the object to be inspected in order to inspect the gloss of the object to be inspected, and a color / unevenness inspection light source 203-2 that irradiates light onto the object to be inspected in order to inspect the color and unevenness of the object to be inspected.

[0016] In this embodiment, as shown in Figure 2(a), there are two objects to be inspected: object 202a and object 202b. Object 202a is placed at position C1, and object 202b is placed at position C2. The gloss inspection light source 203-1 and the color / surface texture inspection light source 203-2 are arranged to surround the two objects to be inspected and irradiate the two objects with light. The imaging device 111 simultaneously images the two objects to be inspected that are irradiated with light by the gloss inspection light source 203-1 and the color / surface texture inspection light source 203-2. Here, simultaneous imaging means that imaging is performed while the two objects to be inspected are contained within the field of view of the imaging device 111. The image processing device 1 inspects the gloss, color, and surface texture of the two objects to be inspected based on the images obtained by imaging.

[0017] In this embodiment, an LED is used as the light source in the light source device 116, but the type of light source is not limited to a specific type; for example, another type of light source such as a xenon lamp may be used. Alternatively, a surface light source with multiple LED elements arranged in a row may also be used. Furthermore, in the inspection of an object to be inspected using a shadow image, the method of light irradiation may be changed according to the appearance inspection items of the object to be inspected.

[0018] When inspecting the color and surface texture of an object, it is necessary to illuminate the object from a direction that does not capture specularly reflected light from the inspection surface. The color and surface texture inspection light source 203-2 is positioned such that the angle between the incident vector of the light irradiated onto the object and the normal vector of the object to which the light is irradiated is relatively large. In other words, the color and surface texture inspection light source 203-2 is positioned so that the imaging device 111 receives diffusely reflected light. The color and surface texture inspection light sources 203-2 are turned on one by one in sequence, and imaging is performed in synchronization with the timing of the lights being turned on.

[0019] When inspecting the gloss of an object, it is necessary to illuminate the inspection surface of the object from a direction that allows for imaging of reflected light near specular reflection. The gloss inspection light source 203-1 is positioned such that the angle between the incident vector of the light irradiated onto the object and the normal vector of the object to which the light is irradiated is relatively small. In other words, the gloss inspection light source 203-1 is positioned so that the imaging device 111 receives specularly reflected light. The light sources constituting the gloss inspection light source 203-1 are turned on simultaneously, and imaging is performed in synchronization with the timing of their illumination.

[0020] By combining images of an object to be inspected, illuminated by light sources from multiple directions, using a known photometric stereo method, an inspection image can be generated consisting of normal information representing surface irregularities and color information corresponding to reflectance.

[0021] Figure 2(b) is a top view of the light source arrangement in this embodiment. As shown in Figure 2(b), in this embodiment, 16 light sources 203-1 for gloss inspection are used, with 8 light sources evenly distributed on both the left and right sides of the imaging center C0. In the following description, the 16 light sources for gloss inspection will be considered as a single light source. In addition, 31 light sources 203-2 for color and surface texture inspection are used, arranged to surround the object to be inspected 202. The number of light sources used for inspection is not limited to this. For example, the number of light sources for gloss inspection may be increased or decreased, or the number of light sources for color and surface texture inspection may be increased or decreased. Also, the method of arranging the light sources is not limited to this, as long as three or more light sources are arranged.

[0022] <Image Processing Device Functional Configuration> Figure 3 shows the functional configuration of the image processing device 1. The image processing device 1 includes a light source control unit 301, an imaging control unit 302, a state determination unit 303, and an inspection unit 304. The state determination unit 303 includes a comparison unit 3031, a display control unit 3032, an imaging brightness acquisition unit 3033, a reference brightness acquisition unit 3034, and a reference holding unit 3035. The light source control unit 301 controls the light source device 116. The imaging control unit 302 controls the imaging device 111. The state determination unit 303 determines the state of the light source used for inspection. The inspection unit 304 performs inspection processing based on the captured image.

[0023] <Processing performed by the image processing unit> In this embodiment, the processing flow performed by the image processing device 1 will be explained using the flowchart in Figure 4. The processing shown in the flowchart in Figure 4 begins when the user inputs an instruction via the input device 110 and the CPU 101 receives the input instruction. Hereafter, each step (process) will be represented by adding an S before the symbol.

[0024] In S401, the imaging control unit 302 images a reference reflector installed for determining the state of the light source, in conjunction with the light source control unit 301 turning on the light source. Figure 5 shows the geometric conditions of the reference reflector, light source, and imaging device 111 in this embodiment. As shown in Figure 5, the reference reflector 501 is installed at the imaging center C0. It is desirable that the reference reflector 501 be relatively large, and in this embodiment, a reference reflector having a size of 75% of the field of view is used.

[0025] In S402, the state determination unit 303 determines the state of the light source used for visual inspection of the object based on the image obtained by imaging in S401. In this embodiment, the state of the light source is determined to be either "normal," meaning that inspection can be performed normally, or "abnormal," meaning that inspection cannot be performed normally. Furthermore, regarding abnormalities, the unit determines whether the abnormality is "deterioration," where the illuminance has decreased due to aging and inspection cannot be performed normally, or "abnormal orientation," where the angle of the light source is different from what was expected due to factors such as touching the light source. Details of S402 will be described later. In S403, the state determination unit 303 determines whether the state determined in S402 is normal or not. If the state of the light source is normal, the process proceeds to S404; otherwise, the process proceeds to S405.

[0026] In S404, the inspection unit 304 performs an inspection on the object to be inspected 202. Specifically, first, the light source control unit 301 simultaneously lights up the gloss inspection light sources 203-1 to irradiate the object to be inspected 202 with light. The imaging control unit 302 causes the imaging device 201 to image the object to be inspected 202 in accordance with the timing of the light source lighting. Next, the light source control unit 301 sequentially lights up the color and surface texture inspection light sources 203-2 to irradiate the object to be inspected 202 with light. The imaging control unit 302 causes the imaging device 201 to image the object to be inspected 202 in accordance with the timing of each light source lighting. In this embodiment, since there are 31 color and surface texture inspection light sources 203-2, imaging is performed once for the gloss inspection light sources and 31 times for the color and surface texture inspection light sources, for a total of 32 imagings. In this embodiment, imaging is performed using all light sources, but the number of imagings is not limited to this. For example, to speed up the inspection, the number of imaging cycles may be reduced by pre-setting an effective light source according to the object 202 under inspection. An effective light source is, for example, a light source that reduces the shadowed area according to the height of the object 202 under inspection, or a light source that easily receives reflected light near specular reflection. Also, for example, to reduce the effects of noise, imaging may be performed multiple times while a single light source is lit.

[0027] Next, the inspection unit 304 synthesizes 31 images captured using the color and surface texture inspection light source 203-2 using a known photometric stereo method to generate a normal image representing the surface texture of the object to be inspected and a color image corresponding to the reflectance. The inspection unit 304 also generates a gloss image representing the gloss intensity at each position of the object to be inspected 202 based on a single image captured using the gloss inspection light source 203-1. The inspection unit 304 detects defects by applying spatial filtering to the normal image, color image, and gloss image, respectively. In this embodiment, the inspection unit 304 calculates an abnormality score by integrating and quantifying the response values ​​to the spatial filtering on the inspection images. Depending on the magnitude of the calculated abnormality score, error notifications may be sent to the user via the display device 115.

[0028] In S405, the display control unit 3032 notifies the user of the light source determined to be "abnormal" via the display device 115. An example of the UI displayed in S405 is shown in FIG. 6. As shown in FIG. 6, the UI in FIG. 6 presents the numbers of the light sources determined to be "abnormal" to the user for each type of abnormality.

[0029] <Details of the processing of S402> First, the concept of the processing of S402 will be described using FIG. 7. FIG. 7 is an example of an image when the brightness of the light source changes in the first embodiment. FIG. 7(a) is an imaging image obtained by lighting a normal light source without deterioration or abnormal orientation of the light source and imaging an object 502. As shown in FIG. 7(a), the imaging image is dark in the upper left and bright in the lower right. FIG. 7(b) is an imaging image obtained by lighting a light source installed at the same position as in FIG. 7(a) and having deteriorated over time and imaging the object 502. It can be seen that the image in FIG. 7(b) is darker overall compared to the image in FIG. 7(a), but the relative brightness in the image is the same as in FIG. 7(a), with the upper left being dark and the lower right being bright. FIG. 7(c) is an imaging image obtained by lighting a light source installed at the same position as in FIG. 7(a) but with its orientation not at the imaging center C0 but slightly tilted and imaging the object 502. It can be seen that the image in FIG. 7(c) is bright in the upper right and bright in the lower left, and the spatial distribution of the relative brightness is different from the image in FIG. 7(a). S402 is a process of determining the state including the cause of the change in the brightness of the light source based on the above characteristics.

[0030] Figure 8 is a flowchart showing the light source state determination process in S402. In S801, the comparison unit 3031 sets the variable i, which represents the light source number, to an initial value. In this embodiment, i is set to 1, which indicates the first light source number. As described above, the gloss inspection light source 203-1 is considered as one light when all lights are on. In S802, the imaging brightness acquisition unit 3033 acquires an imaging brightness image Ii, which is a two-dimensional map of brightness values ​​obtained by imaging the object 502 with the i-th light source lit. In S803, the reference brightness acquisition unit 3034 acquires a reference brightness image Ri corresponding to the i-th light source from the reference holding unit 3035. In this embodiment, the reference brightness image corresponding to the i-th light source is the image taken when the object 502 is first imaged with the i-th light source lit, i.e., a two-dimensional map of brightness values. In this embodiment, the imaging image is an 8-bit grayscale image, but is not limited to this. For example, it may be 16-bit or a color image. For color images, one can convert them to luminance images using known luminance conversion methods. Alternatively, one can consider the pixel values ​​of a specific channel (e.g., the green channel) as luminance.

[0031] In S804, the comparison unit 3031 calculates the relative value Ii_ref for all pixel positions (x,y) of the captured brightness image Ii acquired in S802 using equation (1). Ii_ref(x,y)=Ii(x,y) / max(Ii(x,y))...Equation (1)

[0032] Furthermore, max(I(x,y)) is the maximum pixel value among all pixel positions in image I. In addition, the comparison unit 3031 calculates the relative value Ri_ref for all pixel positions (x,y) of the reference luminance image Ri acquired in S803 using equation (2). Ri_ref(x,y)=Ri(x,y) / max(Ri(x,y))...Equation (2)

[0033] The comparison unit 3031 then calculates the difference ΔRef(x,y) between the relative value Ii_ref(x,y) and the relative value Ri_ref(x,y) using equation (3). In equation (3), |a| represents the absolute value of a. The processing in S804 corresponds to the process of calculating the difference in relative pixel values ​​in order to compare the spatial brightness of the captured brightness image I and the reference brightness image R. ΔRef(x,y)=|Ii_ref(x,y)-Ri_ref(x,y)|·····································································································································ΔRef(x,y)=|Ii_ref(x,y)−Ri_ref(x,y)|

[0034] In S805, the comparison unit 3031 determines whether the ΔRef(x,y) calculated in S804 is less than a preset threshold. If it is less than the threshold, the spatial distribution of pixel values ​​is considered to be similar, and the process proceeds to S806. Otherwise, the process proceeds to S807. In S806, the comparison unit 3031 calculates the difference image ΔI between the captured brightness image I and the reference brightness image R using equation (4). ΔI(x,y)=|Ii(x,y)-Ri(x,y)|...Equation (4)

[0035] In S807, the comparison unit 3031 determines that the orientation of the i-th light source is different because the spatial brightness distribution is different between the captured brightness image I and the reference brightness image R, and sets the state of the i-th light source to "abnormal orientation". Then proceeds to S811. In S808, the comparison unit 3031 determines whether the difference in relative brightness values ​​calculated in S806 is less than a threshold. If it is less than the threshold, proceeds to S809; otherwise, proceeds to S810. In S809, the comparison unit 3031 determines that the brightness is approximately the same at all positions in the captured brightness image I and the reference brightness image R, and sets the state of the i-th light source to "normal". Then proceeds to S811. In S810, the comparison unit 3031 determines that the overall brightness of the captured brightness image I and the reference brightness image R is different, and sets the state of the i-th light source to "degraded". Then proceeds to S811. In S811, the comparison unit 3031 determines whether processing has been performed for all light source numbers. If processing has been performed for all light source numbers, the process ends; otherwise, it proceeds to S812. In S812, the comparison unit 3031 updates the variable i representing the light source number and returns to S802.

[0036] As described above, the image processing device 1 according to this embodiment determines the state of each light source by comparing the captured brightness image corresponding to each light source with a reference brightness image. This makes it possible to perform inspection without reducing the defect detection accuracy of the object to be inspected.

[0037] In this embodiment, the display control unit 3032 displayed the UI shown in Figure 6. However, the display method is not limited to this. For example, the status of each light source may be illustrated as shown in Figure 9. In Figure 9, black squares represent "deterioration," and diagonal squares represent "abnormal orientation." This display makes it possible to graphically understand the status of light sources located at each position.

[0038] Still, in this embodiment, the reference luminance image R was the captured image when the light source was lit for the first time and the object 502 was imaged. However, the method for generating the reference luminance image is not limited to this. For example, it is possible to theoretically calculate the luminance value based on the distance between each light source and the object 502, the angle formed by each light source and the object 502, and the luminance of each light source. At that time, each threshold value may be set in consideration of the error between the theoretical value and the actual value.

[0039] Still, in this embodiment, the states of all the light sources were determined, but it is not always necessary to determine all the light sources. For example, when it is known that only a specific light source is used for the inspection of a certain inspection target object, the light source to be determined may be selected so that only the said light source is determined in the determination of the state of the light source.

[0040] [Second Embodiment] In the first embodiment, the entire captured luminance image was compared with the reference luminance image to determine whether the light source was normal or abnormal. However, if a comparison based on the difference in the spatial distribution of luminance is to be performed, it is not always necessary to compare the entire image. For example, in order to perform the determination at high speed, a plurality of discrete determination regions may be set in advance, and the state of the light source may be determined based on the comparison result of the luminance values at the same positions of the reference reflector and the captured luminance image. In this embodiment, the state of the light source is determined using a plurality of preset regions. Still, since the hardware configuration and functional configuration of the image processing apparatus 1 in this embodiment are the same as those of the first embodiment, the description is omitted. In the following, the parts different between this embodiment and the first embodiment will be mainly described. Still, the same components as those in the first embodiment will be described with the same reference numerals.

[0041] <Details of the processing of S402> Figure 10 is a flowchart of the light source state determination process in S402. S1001 and S1002 are the same as S801 and S802 in the first embodiment, so their explanation is omitted. In S1003, the comparison unit 3031 calculates the average value of the pixel values ​​of the captured brightness image Ii acquired in S1002 for each of the pre-set determination areas. In this embodiment, as shown in Figure 11, processing is performed using six pre-set determination areas for the captured image. In S1004, the comparison unit 3031 sets the variable n, which represents the number of the determination area, to an initial value of 1. In S1005, the reference brightness acquisition unit 3034 acquires the reference brightness value in determination area n for the reference brightness image R of the i-th light source. The reference holding unit 3035 holds the average value of the pixel values ​​for each determination area calculated based on the captured image when the object 502 was first captured by the i-th light source. The reference holding unit 3035 also holds a value that shows the ratio of the average pixel value of a given area to the maximum value of the average pixel value of that area among all areas. In S1005, the reference brightness acquisition unit 3034 obtains the average value Ref_An corresponding to the nth region, and the ratio Ref_Rn of the average pixel value of that region to the maximum value of the average pixel value, from the reference holding unit 3035.

[0042] In S1006, the comparison unit 3031 calculates the ratio S_Rn of the average value of the determination area n to the maximum value of the average pixel value for all areas calculated in S1003 using equation (5). Note that in equation (5), Vave _ n is the average pixel value in the nth region of the captured brightness image, and max(Vave _ n) is the maximum value of the average pixel value across all judgment regions. Note that S_Rn is a value corresponding to the relative brightness. S_Rn=Vave _ n / max(Vave _ n)...Equation (5)

[0043] Next, the comparison unit 3031 calculates the difference ΔRn between S_Rn and Ref_Rn, the ratio of the average value of the reference brightness in the judgment region n acquired in S1005 to the maximum value, using equation (6). ΔRn=|S_Rn-Ref_Rn| Formula (6)

[0044] In S1007, the comparison unit 3031 determines whether ΔRn is less than a threshold. If it is less than the threshold, the process proceeds to S1008; otherwise, it proceeds to S1009. In S1008, the comparison unit 3031 calculates the difference ΔVn between the average value Vave_n in the determination region n of the captured brightness image and Ref_An obtained in S1005 using equation (7). This makes it possible to compare the same position between the reference reflector and the captured brightness image. ΔVn=|Vave_n-Ref_An|...Equation (7)

[0045] S1009 is the same as S807, so its explanation is omitted. In S1010, the comparison unit 3031 determines whether the value of ΔVn calculated in S1008 is less than the threshold. If the value of ΔVn is less than the threshold, the process proceeds to S1011; otherwise, it proceeds to S1012.

[0046] In S1011, the comparison unit 3031 determines whether or not processing has been performed for all determination areas n. If all processing has been performed, the process proceeds to S1013; otherwise, it proceeds to S1014. S1012 is the same as S810, so its explanation is omitted. In S1014, the comparison unit 3031 updates the variable n, which represents the number of the determination area, and returns to S1004. S1013, S1015, and S1016 are the same as S809, S811, and S812, so their explanations are omitted.

[0047] As described above, the image processing apparatus according to this embodiment determines the state of the light source based on the difference in the spatial distribution of brightness by comparing the captured brightness and the reference brightness at the same position using discrete determination regions. This makes it possible to determine the state of the light source faster than determining the light source for the entire image.

[0048] [Embodiment 3] In the above-described embodiment, the state of the light source is determined using one reference reflector. However, it is not necessarily required to determine the state using one reference reflector. In this embodiment, the state of the light source is determined based on the characteristics of a plurality of differences at the same position of a plurality of discretely arranged reference plates. Since the hardware configuration and functional configuration of the image processing apparatus 1 in this embodiment are the same as those in the first embodiment, the description thereof will be omitted. In the following, the parts that are different between this embodiment and the first embodiment will be mainly described. For the same configuration as that in the first embodiment, the same reference numerals will be used for the description.

[0049] <Details of the processing of S402> FIG. 12 is a diagram showing the geometric conditions among the reference reflector, the light source, and the imaging device 111 in this embodiment. As shown in FIG. 12, in this embodiment, two reference reflectors are used, and the reference reflector 1201 and the reference reflector 1202 are installed at equal intervals with respect to the central position C0 of imaging. An example of the captured image obtained by imaging under the geometric conditions of FIG. 12 is shown in FIG. 13. In FIG. 13, the square surrounded by the dotted line represents the determination region. The determination region is a region within the range set inside each of the left and right reference reflectors. FIG. 13(a) shows a captured image in a normal state. The captured image is bright as a whole, and the pixel values in the determination region of the left reference reflector tend to be higher than those on the right side. FIG. 13(b) shows a captured image when imaging is performed in a state where the light source at the same position as in FIG. 13(a) has deteriorated over time and become darker. In FIG. 13(b), the determination region of the right reference reflector tends to be bright and the left side tends to be dark, and the ratio of the average pixel value of the left reference reflector to the average pixel value of the right reference reflector (that is, the relative pixel value relationship) is the same as that in FIG. 13(a). On the other hand, it can be seen that the overall brightness of the left and right reference reflectors is darker than that in FIG. 13(a).

[0050] Figure 13(c) shows an image captured when the orientation of a light source, located in the same position as in Figure 13(a), is changed. In Figure 13(c), the pixel values ​​in the determination area of ​​the left reference reflector are smaller than the pixel values ​​in the determination area of ​​the right reference reflector, indicating that the ratio of the left side to the right side's pixel values ​​differs from that of normal conditions or aging deterioration. In this embodiment, the state of the light source is determined using two reference reflectors and two regions based on the above characteristics.

[0051] Figure 14 is a flowchart showing the light source state determination process in S402. S1401 is the same as S801, so its explanation is omitted. In S1402, the comparison unit 3031 calculates the average pixel value for each of the left and right determination regions of the i-th captured brightness image. The average pixel value of the left determination region is denoted as Iave_left_i, and the average pixel value of the right determination region is denoted as Iave_right_i. In S1403, the comparison unit 3031 calculates the ratio SRi of the average value of the left captured brightness to the average value of the right captured brightness calculated in S1402. The ratio is calculated using equation (5).

[0052] In S1404, the reference luminance acquisition unit 3034 acquires a reference luminance value corresponding to the i-th light source. In this embodiment, the reference holding unit 3033 holds the average value of the luminance values ​​in the left and right determination regions calculated based on the captured image when objects 1201 and 1202 are first imaged with the i-th light source. Furthermore, the reference holding unit 3033 holds a value indicating the ratio of the average luminance value of the left determination region to the right determination region. The reference luminance acquisition unit 3034 acquires the left average luminance value Ref_left_i and the right average luminance value Ref_right_i corresponding to the i-th light source from the reference holding unit 3033. The reference luminance acquisition unit 3034 also acquires the ratio Ref_Ratio_i of the average luminance value of the region to the maximum value of the average pixel value from the reference holding unit 3033.

[0053] In S1405, the comparison unit 3031 compares the ratio SRi of the imaging luminance calculated in S1403 and the ratio Ref_Ratio_i of the reference luminance according to Equation (8). Note that the process of S1405 corresponds to a process for grasping the relationship of the relative luminance values between the imaging luminance image and the reference luminance image. Score = |SRi - Ref_Ratio_i| ··· Equation (8)

[0054] In S1406, the comparison unit 3031 determines whether the Score calculated in S1405 is less than the threshold value. If it is less than the threshold value, the process proceeds to S1407; otherwise, it proceeds to S1408. In S1407, the comparison unit 3031 determines whether Equation (9) is satisfied for Iave_left_i and Iave_right_i calculated in S1402 and Ref_left_i and Ref_right_i acquired in S1404. When Equation (9) is satisfied, the process proceeds to S1409; otherwise, it proceeds to S1410. |Iave_left_i - Ref_left_i| < Th or |Iave_right_i - Ref_right_i| < Th ··· Equation (9) Since S1408 to S1412 are the same as S807 and S809 to S812, the description thereof is omitted.

[0055] As described above, the image processing apparatus according to the present embodiment determines the state of the light source using the reference reflectors arranged discretely at two locations. Thereby, it is possible to determine the state of the light source with the same accuracy with a configuration different from the above-described embodiment.

[0056] [Other Embodiments] In the embodiments described above, the state of the light source was classified into three states: "normal," "degraded," and "abnormal orientation." However, the state of the light source is not limited to these three states. For example, if dust or other debris adheres to the light source, the overall brightness of the light source decreases, similar to "degraded." In this case, it may also be classified into three states: "normal," "dirty," and "abnormal orientation." Furthermore, if the light source does not light up due to a broken electrical cable or the like, the image may appear completely black. In this case, it is possible to detect this by determining whether all pixel values ​​are smaller than a predetermined pixel value. In this case, it may also be classified into four states: "normal," "degraded," "abnormal orientation," and "not lit."

[0057] In the embodiment described above, the state of the light source was classified into three states: "normal," "degraded," and "abnormal orientation." However, since the state of degradation changes over time, the determination of "degradation" may be made in stages. For example, a warning may be displayed when the difference between the captured brightness image and the reference brightness image exceeds a predetermined threshold Th1, and "degradation" may be determined when it exceeds a threshold Th2 which is even larger than the predetermined threshold Th1.

[0058] In the embodiment described above, the status of each light source was displayed according to the type of abnormality, but the display method is not limited to the above example. For example, the number of a normal light source may be displayed, or the numbers of all abnormal light sources may be displayed together without displaying them according to the type of abnormality.

[0059] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of symbols]

[0060] 1 Image processing device 303 State determination unit 3033 Image Brightness Acquisition Unit 3034 Reference Brightness Acquisition Unit

Claims

1. A first acquisition means for sequentially illuminating multiple light sources and capturing an image of a reference reflector to obtain the brightness of the image, A second acquisition means for acquiring the brightness of a reference corresponding to the brightness of the aforementioned image, The system includes a determination means for determining whether the state of each light source included in the plurality of light sources is normal or abnormal, based on the brightness of the image and the brightness of the reference. The image processing apparatus is characterized in that the determination means identifies the type of abnormality when the state of the light source is abnormal.

2. The image processing apparatus according to claim 1, characterized in that the determination means determines whether the state of each light source included in the plurality of light sources is normal or abnormal based on the difference between the brightness of the image and the brightness of the reference.

3. The image processing apparatus according to claim 1, characterized in that the type of abnormality is any of the following: deterioration of the light source, abnormal orientation of the light source, dirt, or non-illumination.

4. The image processing apparatus according to claim 1, characterized in that the determination means performs a comparison based on the difference between the brightness of the image and the brightness of the reference at the same position on the reference reflector.

5. The image processing apparatus according to claim 1, further comprising inspection means for performing inspection processing based on an image obtained by imaging an object to be inspected.

6. The image processing apparatus according to claim 1, further comprising a display control means for displaying the type of the identified abnormality.

7. A program for causing a computer to function as an image processing device according to any one of claims 1 to 6.

8. A first acquisition step involves sequentially illuminating multiple light sources to image a reference reflector and obtaining the brightness of the resulting image, A second acquisition step involves obtaining the brightness of a reference corresponding to the brightness of the aforementioned image, The system includes a determination step of determining whether the state of each light source included in the plurality of light sources is normal or abnormal, based on the brightness of the image and the brightness of the reference. An image processing method characterized in that, in the determination step, if the state of the light source is abnormal, the type of abnormality is identified.

Images