Image processing system and image processing method

Abstract

To provide an image processing system capable of selectively detecting a foreign object on a surface of an inspection object.SOLUTION: An image processing system comprises: one or more lighting devices that irradiate an inspection object surface with first light at a first incident angle; a first lighting device that irradiates the inspection object surface with second light of a different color at a second incident angle smaller than the first incident angle; an imaging apparatus that detects a received light amount of reflectance at each position on the inspection object surface in a state of being irradiated with the first and second light; and a generation unit that generates an inspection image representing a component corresponding to a difference between the received light amount in the first wavelength range corresponding to the color of the first light and the received light amount in the second wavelength range corresponding to the color of the second light for each position on the inspection object surface based on the detection results of the imaging apparatus.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an image processing system and an image processing method. [Background technology]

[0002] Conventionally, techniques have been developed for inspecting the appearance of the surface of an object using images obtained by photographing the object. For example, Japanese Patent Laid-Open Publication No. 2012-251929 (Patent Document 1) discloses a technique for detecting defects in the uneven shape of an object by irradiating the object with illumination light of different colors from multiple directions. A similar technique is also disclosed in Japanese Patent Laid-Open Publication No. 2018-36175 (Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-251929 [Patent Document 2] Japanese Patent Application Publication No. 2018-36175 Summary of the Invention [Problem to be solved by the invention]

[0004] The techniques described in Patent Documents 1 and 2 aim to detect defects in the uneven shape of an object. However, defects in the appearance of the surface of an object may include foreign matter (e.g., metal pieces, lint, etc.) on the surface. For example, foreign matter may adhere to an uneven pouch. In such cases, users want to detect the foreign matter on the pouch separately from the unevenness.

[0005] The present disclosure has been made in view of the above circumstances, and its purpose is to provide an image processing system and an image processing method that are capable of selectively detecting foreign matter on an inspection target surface. [Means for solving the problem]

[0006] An image processing system according to one aspect of the present disclosure includes one or more illumination devices, a first illumination device, an image capture device, and a generation unit. The one or more illumination devices irradiate a surface to be inspected with first light at a first angle of incidence. The first illumination device irradiates the surface to be inspected with second light of a different color from the first light at a second angle of incidence smaller than the first angle of incidence. The image capture device detects the amount of light received for reflected light at each position on the surface to be inspected while the first light and second light are irradiated. The generation unit generates an inspection image representing a component corresponding to the difference between the amount of light received in a first wavelength range corresponding to the color of the first light and the amount of light received in a second wavelength range corresponding to the color of the second light, for each position on the surface to be inspected, based on the detection result of the image capture device.

[0007] According to this disclosure, the imaging device is likely to receive light diffusely reflected by foreign matter on the inspection target surface, out of the first light irradiated at a relatively large first incident angle. Therefore, the amount of light received in the first wavelength range corresponding to the color of the first light is greater in areas where foreign matter is present. However, if the inspection target surface has an uneven surface, the uneven surface may have a surface that specularly reflects the first light toward the imaging device. Therefore, the imaging device may receive light in the first wavelength range even in the uneven surface. Furthermore, depending on the spectral sensitivity of the imaging device, the first wavelength range may partially overlap with the second wavelength range. Therefore, the imaging device may receive a portion of the second light specularly reflected from flat areas of the inspection target surface as light in the first wavelength range. In this way, the imaging device mainly receives light in the first wavelength range in areas where foreign matter is present, but may also receive a small amount of light in the first wavelength range in areas other than the foreign matter. On the other hand, the amount of light received in the second wavelength range corresponding to the color of the second light irradiated at a relatively small second incident angle is greater in areas other than the foreign matter (especially the uneven surface). Therefore, in an inspection image that represents a component corresponding to the difference between the amount of received light in the first wavelength range and the amount of received light in the second wavelength range for each position on the inspection surface, the difference in brightness between the location where a foreign substance exists and the area other than the foreign substance is greater than in an image that represents the amount of received light in the first wavelength range. As a result, the image processing system can selectively detect foreign substances on the inspection surface by using the inspection image.

[0008] In the above disclosure, the component indicates an amount obtained by subtracting an amount obtained by multiplying an amount of received light in a second wavelength range by a second coefficient from an amount obtained by multiplying an amount of received light in a first wavelength range by a first coefficient. The image processing system further includes an adjustment unit that adjusts at least one of the first coefficient and the second coefficient.

[0009] According to this disclosure, the first coefficient and the second coefficient are adjusted according to the condition of the surface to be inspected, thereby generating an inspection image suitable for detecting foreign matter.

[0010] In the above disclosure, the component indicates an amount obtained by subtracting an amount of received light in the second wavelength range from an amount of received light in the first wavelength range. The image processing system further includes a control unit that controls one or more lighting devices to change the intensity of the first light by a first factor and controls the first lighting device to change the intensity of the second light by a second factor, and an adjustment unit that adjusts at least one of the first factor and the second factor.

[0011] According to this disclosure, the first coefficient and the second coefficient are adjusted according to the condition of the surface to be inspected, thereby generating an inspection image suitable for detecting foreign matter.

[0012] In the above disclosure, the adjustment unit adjusts at least one of the first coefficient and the second coefficient in response to an input to a user interface screen. The user interface screen includes an image display area. The adjustment unit displays, in the image display area, an image selected from a first image representing the amount of received light in a first wavelength range for each position on the inspection surface, a second image representing the amount of received light in a second wavelength range for each position on the inspection surface, a third image representing a first product obtained by multiplying the amount of received light in the first wavelength range by the first coefficient for each position on the inspection surface, a fourth image representing a second product obtained by multiplying the amount of received light in the second wavelength range by the second coefficient for each position on the inspection surface, and a fifth image representing an amount obtained by subtracting the second product from the first product for each position on the inspection surface.

[0013] According to this disclosure, the user can appropriately set the first coefficient and the second coefficient by checking the image displayed in the image display area.

[0014] In the above disclosure, the one or more lighting devices include a second lighting device and a third lighting device. The optical axis of the imaging device is perpendicular to the surface to be inspected. The second lighting device and the third lighting device are disposed symmetrically with respect to the optical axis of the imaging device. According to this disclosure, the amount of diffusely reflected light from foreign matter received by the imaging device is stabilized.

[0015] In the above disclosure, the color of the first light emitted from the second lighting device may be different from the color of the first light emitted from the third lighting device.

[0016] In the above disclosure, the imaging device is a prism spectroscopic camera. According to this disclosure, the overlapping portion between the first wavelength range and the second wavelength range is small. This makes it easier for the generation unit to generate an inspection image with a large difference in brightness between the foreign matter and the non-fine matter area.

[0017] An image processing method according to one aspect of the present disclosure includes: irradiating a surface to be inspected with a first light at a first incident angle; irradiating the surface to be inspected with a second light of a different color from the first light at a second incident angle smaller than the first incident angle; detecting an amount of received light of the reflected light at each position on the surface to be inspected while the first and second lights are irradiated; and generating, for each position on the surface to be inspected, an inspection image representing a component corresponding to a difference between an amount of received light in a first wavelength range corresponding to the color of the first light and an amount of received light in a second wavelength range corresponding to the color of the second light, based on the detection result. According to this disclosure, by using the inspection image generated by the image processing method, foreign matter on the surface to be inspected can be selectively detected. [Effects of the Invention]

[0018] According to the present disclosure, an image processing system and an image processing method are capable of selectively detecting foreign matter on an inspection target surface. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an example of an image processing system according to an embodiment. [Figure 2]1 is a flowchart showing an example of the flow of an image processing method according to the present embodiment. [Figure 3] 10A and 10B are diagrams illustrating an example of generation of an inspection image of a sample product of an object. [Figure 4] FIG. 1 is a diagram illustrating an example of the configuration of an imaging system. [Figure 5] FIG. 2 is a diagram illustrating an example of the internal configuration of the imaging device. [Figure 6] FIG. 1 is a diagram illustrating the spectral sensitivity of a prism spectroscopic camera. [Figure 7] FIG. 1 is a diagram showing the spectral sensitivity of a Bayer camera. [Figure 8] 2 is a schematic diagram illustrating an example of a hardware configuration of the image processing device illustrated in FIG. 1. FIG. [Figure 9] 10 is a flowchart illustrating an example of a processing flow of an adjustment unit. [Figure 10] FIG. 10 is a diagram illustrating an example of a user interface screen. DETAILED DESCRIPTION OF THE INVENTION

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings, in which the same or corresponding parts in the drawings are designated by the same reference numerals and the description thereof will not be repeated.

[0021] §1 Application Examples An image processing system according to an embodiment will be described with reference to FIGS. 1 to 3. FIG. 1 is a diagram showing an example of an image processing system according to an embodiment. The image processing system 1 shown in FIG. 1 is applied to visual inspection of an inspection target surface 2a of an object 2. The object 2 is not particularly limited, but is, for example, a pouch. Examples of pouches include cell packs of lithium ion batteries. The inspection target surface 2a has a glossy appearance. In other words, the specular reflection light is strong on the inspection target surface 2a, and the spread angle of the reflected light is narrow.

[0022] The image processing system 1 includes an imaging system 9 for imaging the inspection target surface 2a. The imaging system 9 includes one or more lighting devices 3 that irradiate the inspection target surface 2a with light 6 at an incident angle θ1, and an illumination device 4 that irradiates the inspection target surface 2a with light 7 of a different color from the light 6 at an incident angle θ2 that is smaller than the incident angle θ1.

[0023] The incident angle θ1 is an example of a "first incident angle" in the present disclosure. The incident angle θ1 is preferably 60° to 85°, for example, 72.5°. The light 6 is an example of a "first light" in the present disclosure. In the example shown in FIG. 1, the one or more lighting devices 3 include a lighting device 3a that emits red (R) light 6a and a lighting device 3b that emits blue (B) light 6b.

[0024] The illumination device 4 is an example of a "first illumination device" according to the present disclosure. The incident angle θ2 corresponds to the position of a "second incident angle" according to the present disclosure. The incident angle θ2 is ideally 0°, but is preferably 5° to 30°, for example 20°, taking into account the physical overlap with the imaging device 5. The light 7 is an example of a "second light" according to the present disclosure, and is green (G) in the example shown in FIG. 1.

[0025] The photographing system 9 further includes a photographing device 5 that detects the amount of reflected light at each position on the inspection target surface 2a while the lights 6 and 7 are irradiated. The photographing device 5 is preferably positioned so that its optical axis 5a coincides with the normal 2b of the inspection target surface 2a. The normal 2b is the normal to the inspection target surface 2a at the position where the lights 6 and 7 are irradiated. This reduces fluctuations in the photographing position on the inspection target surface 2a taken by the photographing device 5, even if the position of the object 2 moves up or down, and makes it easier for the photographing device 5 to focus on the inspection target surface 2a. However, the photographing device 5 may be positioned at an angle to more easily receive the light 7 specularly reflected from the inspection target surface 2a. For example, the photographing device 5 may be positioned on the opposite side of the illumination device 4 with respect to the normal 2b, with an angle θ3 between the optical axis 5a and the normal 2b in the range of 0° to θ2. In other words, the lighting device 4 and the image capturing device 5 may be disposed on opposite sides of the normal line 2b such that the difference between the angle θ3 and the angle θ2 is within the range of ±30°.

[0026] The image capturing device 5 outputs a color image representing the intensity and color of reflected light at each position on the inspection target surface 2a. The color image represents the amount of light received for each component of RGB for each pixel. The R component of each pixel is in the wavelength range λ R1 ~λ R2 The G component of each pixel indicates the amount of light received in the wavelength range λ G1 ~λ G2 The B component of each pixel is in the wavelength range λ B1 ~λ B2 The received light amount for the wavelength range λ R1 ~λ R2 , wavelength range λ G1 ~λ G2 and wavelength range λ B1 ~λ B2 is determined according to the spectral characteristics of the image capturing device 5.

[0027] The image processing system 1 further includes an image processing device 100. The image processing device 100 is, for example, a general-purpose computer. The image processing device 100 includes a generation unit 11 that generates, for each position on the inspection target surface 2a, an inspection image representing a component corresponding to the difference between the amount of received light in a first wavelength range corresponding to the color of light 6 and the amount of received light in a second wavelength range corresponding to the color of light 7, based on the detection result of the imaging device 5. When the light 6 includes red (R) light 6a and blue (B) light 6b, the wavelength range λ R1 ~λ R2 and wavelength range λ B1 ~λ B2 When the light 7 is green (G), the wavelength range λ G1 ~λ G2 is set as the second wavelength range.

[0028] 2 is a flowchart showing an example of the flow of an image processing method according to this embodiment. The flow shown in FIG.

[0029] 2, in step S1, one or more lighting devices 3 irradiate the inspection target surface 2a with light 6 at an incident angle θ1. In step S2, the lighting device 4 irradiates the inspection target surface 2a with light 7 of a different color from the light 6 at an incident angle θ2 that is smaller than the incident angle θ1. Note that step S1 may be started before step S2, after step S2, or simultaneously with step S2.

[0030] In the next step S3, the image capturing device 5 detects the amount of light received for the reflected light at each position on the inspection target surface 2a while the lights 6 and 7 are irradiated. In the next step S4, the generation unit 11 of the image processing device 100 generates, for each position on the inspection target surface 2a, an inspection image representing a component corresponding to the difference between the amount of light received in a first wavelength range corresponding to the color of light 6 and the amount of light received in a second wavelength range corresponding to the color of light 7, based on the detection result of the image capturing device 5. After step S4, the processing ends.

[0031] FIG. 3 is a diagram illustrating an example of an inspection image generated for a sample product of an object. The color image 50 is obtained by capturing images using the image capture device 5 and represents the intensity and color of reflected light at each position on the inspection target surface 2a. As shown in FIG. 3, lint 61 and numerous metal powder particles 62 are attached to the surface of the sample product as foreign matter. Furthermore, the surface of the sample product has dents 63, wrinkles 64, and scratches 65 as uneven portions. The surface of the sample product is flat in areas where dents 63, wrinkles 64, and scratches 65 are not formed. As shown in FIG. 3, the lint 61, metal powder particles 62, dents 63, wrinkles 64, and scratches 65 are all visible in the color image 50. Therefore, when using the color image 50, it is difficult to distinguish between uneven portions and foreign matter.

[0032] As described above, when a foreign object may adhere to a pouch having an uneven surface, a user may want to selectively detect the foreign object on the pouch. To meet this need, the image processing system 1 and image processing method according to this embodiment generate an inspection image 53 suitable for detecting the foreign object.

[0033] The image 51 represents, for each position on the inspection target surface 2a, the amount of received light in a first wavelength range corresponding to the color of the light 6. The image 51 is generated by extracting only the components in the first wavelength range from the color image 50. For example, if the light 6 includes red (R) light 6a and blue (B) light 6b, the generation unit 11 applies a known color extraction filter to the color image 50 to generate the image 51 representing the R component and the B component.

[0034] The image 52 represents, for each position on the inspection target surface 2a, the amount of received light in a second wavelength range corresponding to the color of the light 7. The image 52 is generated by extracting only the components in the second wavelength range from the color image 50. For example, if the light 7 is green (G), the generation unit 11 applies a known color extraction filter to the color image 50 to generate the image 52 representing the G component.

[0035] The light 6 and 7 are specularly reflected from the glossy inspection target surface 2a, but are diffusely reflected from areas where foreign matter such as lint 61 and metal powder 62 is present.

[0036] Light 6 is irradiated onto inspection target surface 2a at a relatively large angle of incidence θ1. Therefore, light 6 that is specularly reflected from flat inspection target surface 2a does not enter image capturing device 5. On the other hand, part of light 6 that is diffusely reflected by lint 61 and metal powder 62 enters image capturing device 5. Furthermore, uneven portions where dents 63, wrinkles 64, and scratches 65 are formed may have surfaces that specularly reflect light 6 toward image capturing device 5. Therefore, image capturing device 5 may receive light 6 that is specularly reflected from parts of dents 63, wrinkles 64, and scratches 65.

[0037] Light 7 is irradiated onto the inspection target surface 2a at a relatively small angle of incidence θ2. Therefore, the light beam irradiated from the lighting device 4 includes light that is specularly reflected from the flat inspection target surface 2a toward the imaging device 5. That is, some of the light 7 specularly reflected by the flat inspection target surface 2a is incident on the imaging device 5. Furthermore, the uneven portions where the dents 63, wrinkles 64, and scratches 65 are formed may have surfaces that specularly reflect light 7 traveling along the optical axis of the lighting device 4 toward the imaging device 5. Therefore, the amount of light 7 specularly reflected from some of the dents 63, wrinkles 64, and scratches 65 toward the imaging device 5 may be greater than the amount of light 7 specularly reflected from the flat inspection target surface 2a toward the imaging device 5. On the other hand, light 7 is diffusely reflected in areas where lint 61 and metal powder 62 are present. Therefore, the amount of light 7 reflected in the direction of the image capturing device 5 from the area where the lint 61 and metal powder 62 are present is less than the amount of light 7 specularly reflected in the direction of the image capturing device 5 from the flat inspection target surface 2a.

[0038] 3, image 51 shows that the amount of light received in the first wavelength range is relatively large in areas where lint 61 and metal powder 62 are present. Furthermore, image 51 shows that light in the first wavelength range is received slightly in areas other than areas where lint 61 and metal powder 62 are present. This is due to the spread of reflected light in areas other than areas where lint 61 and metal powder 62 are present.

[0039] The distribution of the radiation direction of the reflected light from the lint 61 and metal powder 62 depends on the shape and orientation of the lint 61 and metal powder 62. The lint 61 and metal powder 62 may have various shapes and may adhere to the inspection target surface 2a in various orientations. Therefore, as shown in FIG. 1, the image processing system 1 preferably includes multiple illumination devices 3a and 3b that emit light in the first wavelength range. This stabilizes the amount of diffusely reflected light from the lint 61 and metal powder 62 received by the image capture device 5, regardless of the shape and orientation of the lint 61 and metal powder 62. Furthermore, the illumination devices 3a and 3b are preferably positioned symmetrically to each other with respect to the optical axis 5a of the image capture device 5. This further stabilizes the amount of diffusely reflected light from the lint 61 and metal powder 62 received by the image capture device 5.

[0040] On the other hand, image 52 mainly shows the uneven shape of inspection target surface 2a. Specifically, image 52 shows that light in the second wavelength range is received on flat inspection target surface 2a. Furthermore, image 52 shows that light in the second wavelength range is received more in particular at locations where dents 63, wrinkles 64, and scratches 65 are present. That is, in image 52, dents 63, wrinkles 64, and scratches 65 appear brighter than on flat inspection target surface 2a.

[0041] Inspection image 53 represents a component corresponding to the difference between the amount of light received in the first wavelength range and the amount of light received in the second wavelength range for each position on inspection target surface 2a. Typically, inspection image 53 is generated by performing a difference calculation between image 51 and image 52. That is, the luminance value of pixel (x, y) in inspection image 53 is obtained by subtracting the luminance value of pixel (x, y) in image 52 from the luminance value of pixel (x, y) in image 51.

[0042] As described above, image 51 shows that the amount of light received in the first wavelength range is relatively large in areas where lint 61 and metal powder 62 are present, and that light in the first wavelength range is also received slightly in areas other than areas where lint 61 and metal powder 62 are present. Therefore, when inspecting the appearance of inspection target surface 2a using image 51, not only lint 61 and metal powder 62 but also dents 63, wrinkles 64, and scratches 65 may be detected. Alternatively, lint 61 and metal powder 62 may not be detected accurately. However, in inspection image 53 generated by performing a subtraction operation between images 51 and 52, the brightness values ​​of areas other than areas where lint 61 and metal powder 62 are present are lower than those in image 51. Therefore, by using inspection image 53, a user can selectively detect foreign matter such as lint 61 and metal powder 62 on inspection target surface 2a.

[0043] Note that foreign matter on the inspection target surface 2a may be detected by visual inspection using the inspection image 53. Alternatively, as shown in Fig. 1, the image processing device 100 may include an inspection unit 12 that detects foreign matter on the inspection target surface 2a by performing known image processing on the inspection image 53. For example, the inspection unit 12 identifies the position of the foreign matter by performing processing such as comparing the brightness value of each pixel in the inspection image 53 with a threshold value.

[0044] The specular reflection characteristics of the surface of the inspection target surface 2a differ depending on the material and smoothness of the object 2. Therefore, the image processing device 100 may include an adjustment unit 13 that adjusts the calculation method of the component according to the difference between the amount of received light in the first wavelength range and the amount of received light in the second wavelength range, in order to generate an inspection image 53 that is advantageous for detecting foreign matter.

[0045] The amount of received light in the first wavelength range and the amount of received light in the second wavelength range depend on the intensities of the lights 6 and 7. Therefore, the image processing device 100 may include a control unit 14 that controls one or more of the lighting devices 3 and 4 to change the intensities of the lights 6 and 7 in order to generate an inspection image 53 that is advantageous for detecting foreign matter. Note that the control unit 14 may control the one or more of the lighting devices 3 and 4 to change the intensities of the lights 6 and 7 via a programmable logic controller (PLC) (not shown).

[0046] §2 Specific examples <Configuration of the imaging system> Fig. 4 is a diagram showing an example of the configuration of an imaging system. As shown in Fig. 4, the imaging system 9 includes two elongated plate-like members 91a and 91b arranged to face each other. The plate-like members 91a and 91b are arranged at the same height so that their longitudinal directions are parallel to the horizontal direction.

[0047] The photographing system 9 further includes holding members 92a to 94a, 92b to 94b, and 95 fixed to the plate-like members 91a and 91b.

[0048] The holding members 92a and 92b are fixed to one longitudinal end of the plate-shaped members 91a and 91b, respectively, and hold the lighting device 3a therebetween. The holding members 93a and 93b are fixed to the other longitudinal ends of the plate-shaped members 91a and 91b, respectively, and hold the lighting device 3b therebetween. The holding members 94a and 94b are fixed near the longitudinal centers of the plate-shaped members 91a and 91b, respectively, and hold the lighting device 4 therebetween.

[0049] Each of the illumination devices 3a, 3b, and 4 is a line illuminator that emits light in a line shape along a direction perpendicular to the longitudinal direction of the plate-like members 91a and 91b. The illumination devices 3a, 3b, and 4 irradiate the light onto a linear inspection target region 96. The arrangement of the illumination devices 3a, 3b, and 4 is not limited to the example shown in FIG.

[0050] The holding member 95 is fixed to the plate-like members 91a and 91b between the holding members 92a and 92b and the holding members 94a and 94b, and holds the image capturing device 5. The image capturing device 5 is a line sensor camera having imaging elements arranged in a line. The image capturing device 5 is arranged above the linear inspection target area 96 so that the optical axis is parallel to the vertical direction, and captures a linear color image of the inspection target area 96.

[0051] The object 2 is transported by a transport mechanism (not shown) along a direction D that is perpendicular to the linear inspection area 96 and parallel to the horizontal plane so as to pass through the inspection area 96. By connecting linear color images obtained by photographing the object 2 with the imaging device 5 while the object 2 is being transported, a two-dimensional color image including the inspection surface 2a of the object 2 is obtained.

[0052] The image capturing device 5 for capturing color images may be a Bayer camera or a prism camera. A Bayer camera uses a color filter array called a Bayer array to obtain color information from received light. A prism camera uses a prism to obtain color information from received light.

[0053] Fig. 5 is a diagram showing an example of the internal configuration of an imaging device. Fig. 5 shows the internal configuration of imaging device 5, which is a prism spectroscopic camera. Imaging device 5 shown in Fig. 5 includes a light receiving lens 501, a prism 502, and a plurality of sensor sets 510 arranged in a line.

[0054] Light receiving lens 501 guides light received from the outside to prism 502. Prism 502 disperses the light. Sensor set 510 includes, of the light emitted from prism 502, sensor 511 that detects the amount of received R component light, sensor 512 that detects the amount of received G component light, and sensor 513 that detects the amount of received B component light.

[0055] The imaging device 5, which is a Bayer camera, has a color filter array instead of the prism 502. In this case, sensor 511 detects the amount of R component light received by receiving light that has passed through the red filter portion of the color filter array. Sensor 512 detects the amount of G component light received by receiving light that has passed through the green filter portion of the color filter array. Sensor 513 detects the amount of B component light received by receiving light that has passed through the blue filter portion of the color filter array.

[0056] FIG. 6 is a diagram showing the spectral sensitivity of a prism spectroscopic camera. FIG. 7 is a diagram showing the spectral sensitivity of a Bayer camera. In FIGS. 6 and 7, line 81 indicates the relative sensitivity of sensor 511 corresponding to the R component. Line 82 indicates the relative sensitivity of sensor 512 corresponding to the G component. Line 83 indicates the relative sensitivity of sensor 513 corresponding to the B component. The region of line 81 where the relative sensitivity is significant is within the wavelength range λ that the image capture device 5 can detect as R component light. R1 ~λ R2 The region of the line 82 where the relative sensitivity is significant is within the wavelength range λ that the image capture device 5 can detect as G component light. G1 ~λ G2 The region of the line 83 where the relative sensitivity is significant is within the wavelength range λ that the image capture device 5 can detect as B component light. B1 ~λ B2 Shows.

[0057] The wavelength range λ that can be detected as R component light R1 ~λ R2 and the wavelength range λ that can be detected as G component light G1 ~λ G2 The overlapping area between the wavelengths λ and λ is smaller in a prism camera than in a Bayer camera. B1 ~λ B2 and the wavelength range λ that can be detected as G component light G1 ~λ G2The overlapping portion between the lint 61 and the metal powder 62 is smaller in a prism camera than in a Bayer camera. Therefore, by using a prism camera as the image capture device 5, the amount of light 7 detected as light in the first wavelength range corresponding to the color of light 6 is reduced compared to when a Bayer camera is used as the image capture device 5. The reduction in the amount of light 7 detected as light in the first wavelength range reduces the brightness values ​​of areas other than the lint 61 and the metal powder 62 in the image 51 shown in FIG. 3. As a result, based on the images 51 and 52, the generation unit 11 of the image processing device 100 can easily create an inspection image 53 in which there is a large difference in brightness between the foreign objects, such as the lint 61 and the metal powder 62, and areas other than the foreign objects. For these reasons, it is preferable to use a prism camera as the image capture device 5.

[0058] <Example of hardware configuration of image processing device> Image processing device 100 is typically a computer having a general-purpose architecture, and executes a pre-installed program (instruction code) to perform image processing according to this embodiment. Such a program is typically distributed in a state stored on various recording media, or is installed in image processing device 100 via a network, etc.

[0059] When using such a general-purpose computer, an OS (Operating System) for executing basic computer processing may be installed in addition to the application for executing the image processing according to the present embodiment. In this case, the program according to the present embodiment may execute processing by calling necessary modules from among program modules provided as part of the OS in a predetermined sequence at a predetermined timing. In other words, the program according to the present embodiment itself may not include the above-mentioned modules, and may execute processing in cooperation with the OS. The program according to the present embodiment may also be in a form that does not include some of these modules.

[0060] Furthermore, the program according to the present embodiment may be provided by being incorporated into a part of another program. In this case, the program itself does not include the modules included in the other program to be combined as described above, and executes processing in cooperation with the other program. In other words, the program according to the present embodiment may be in a form incorporated into such other program. Note that some or all of the functions provided by the execution of the program may be implemented as dedicated hardware circuits.

[0061] Fig. 8 is a schematic diagram showing an example of the hardware configuration of the image processing device shown in Fig. 1. As shown in Fig. 8, the image processing device 100 includes a CPU (Central Processing Unit) 110, which is an example of a processor, a main memory 112, a hard disk 114, a camera interface 116, an input interface 118, a display controller 120, a communication interface 124, and a data reader / writer 126. These components are connected to each other via a bus 128 so as to be able to communicate data with each other.

[0062] The CPU 110 loads the programs 115 installed on the hard disk 114 into the main memory 112 and executes them in a predetermined order to perform various calculations. The main memory 112 typically includes a volatile storage device such as a dynamic random access memory (DRAM), and stores the programs 115 read from the hard disk 114 as well as color images acquired from the image capturing device 5. The hard disk 114 also stores various data, as will be described later. Note that in addition to the hard disk 114, or instead of the hard disk 114, a semiconductor storage device such as a flash memory may be used.

[0063] The generation unit 11, the inspection unit 12, and the adjustment unit 13 shown in FIG.

[0064] The camera interface 116 mediates data transmission between the CPU 110 and the image capturing device 5. That is, the camera interface 116 is connected to the image capturing device 5. The camera interface 116 issues an image capturing command to the image capturing device 5 in accordance with an internal command generated by the CPU 110. The image capturing command may be output in response to a detection signal from a photoelectric sensor. Alternatively, the image capturing command may be output in response to an external command from a programmable logic controller (PLC).

[0065] The camera interface 116 includes an image buffer 116a for temporarily storing color images received from the image capturing device 5. In the example shown in Fig. 8, the image capturing device 5 is external to the image processing device 100. However, the image capturing device 5 may also be built into the image processing device 100.

[0066] The input interface 118 mediates data transmission between the CPU 110 and the input device 160. That is, the input interface 118 accepts input information input to the input device 160 by the user.

[0067] The display controller 120 is connected to the display 150 and controls the screen of the display 150 so as to notify the user of the processing results of the CPU 110 and the like.

[0068] The communication interface 124 mediates data transmission between the CPU 110 and an external device (for example, a PLC). The communication interface 124 is typically implemented by an Ethernet (registered trademark) or a Universal Serial Bus (USB).

[0069] The data reader / writer 126 mediates data transmission between the CPU 110 and the memory card 106, which is a recording medium. That is, the memory card 106 stores and distributes programs to be executed by the image processing device 100, and the data reader / writer 126 reads the programs from the memory card 106. In addition, in response to an internal command from the CPU 110, the data reader / writer 126 writes images received from the photographing device 5 and / or processing results in the image processing device 100 to the memory card 106. The memory card 106 may be a general-purpose semiconductor storage device such as an SD (Secure Digital), a magnetic storage medium such as a flexible disk, or an optical storage medium such as a CD-ROM (Compact Disk Read Only Memory).

[0070] <Example of adjustment section processing> An example of processing by the adjustment unit 13 will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is a flowchart showing an example of the flow of processing by the adjustment unit.

[0071] First, in step S11, the adjustment unit 13 acquires a color image to be used to adjust the calculation method for the component corresponding to the difference between the amount of received light in the first wavelength range and the amount of received light in the second wavelength range. Specifically, the adjustment unit 13 may acquire the most recent color image acquired from the image capture device 5. Alternatively, the adjustment unit 13 acquires a color image designated by the user from among multiple color images previously acquired from the image capture device 5. In this case, the user designates a color image that shows the inspection target surface 2a having foreign matter such as lint 61 and metal powder 62 attached thereto.

[0072] In the next step S12, the adjustment unit 13 applies a known color extraction filter (also called a "color gray filter") to the color image to generate an image for each RGB color component (hereinafter called a "processing unit image"). In the processing unit image, the luminance value of each pixel represents the amount of light received by the corresponding color component.

[0073] In the next step S13, the adjustment unit 13 selects a processing unit image corresponding to the first wavelength range according to user input, and uses the selected processing unit image to generate a first image representing the amount of light received in the first wavelength range for each position on the inspection surface 2a.

[0074] The first wavelength range corresponds to the color of light 6 irradiated onto the inspection target surface 2a at a relatively large incident angle θ1. In the example shown in FIG. 1, the light 6 includes red light 6a and blue light 6b. Therefore, in step S13, the user simply inputs an instruction to select a processing unit image of the R component and a processing unit image of the B component. The adjustment unit 13 generates a first image by combining the processing unit image of the R component and the processing unit image of the B component. Note that, when one processing unit image is selected in step S13, the adjustment unit 13 simply determines the selected processing unit image as the first image.

[0075] In the next step S14, the adjustment unit 13 provisionally sets a first coefficient for the first image in accordance with the user input.

[0076] In the next step S15, the adjustment unit 13 selects a processing unit image corresponding to the second wavelength range according to user input, and uses the selected unit image to generate a second image representing the amount of light received in the second wavelength range for each position on the inspection surface 2a.

[0077] The second wavelength range corresponds to the color of light 7 irradiated onto the inspection target surface 2a at a relatively small angle of incidence θ2. In the example shown in FIG. 1, the light 7 is green. Therefore, in step S14, the user inputs an instruction to select a processing unit image of the G component. The adjustment unit 13 determines the processing unit image of the G component as the second image. Note that, when two processing unit images are selected in step S15, the adjustment unit 13 generates the second image by combining the two selected processing unit images.

[0078] In the next step S16, the adjustment unit 13 provisionally sets second coefficients for the second image in accordance with the user input.

[0079] In the next step S17, the adjustment unit 13 uses the first image, the second image, the first coefficient, and the second coefficient to generate a processed image representing, for each position on the inspection surface 2a, an amount obtained by subtracting a second product obtained by multiplying the amount of received light in the second wavelength range by the second coefficient from a first product obtained by multiplying the amount of received light in the first wavelength range by the first coefficient. Specifically, the adjustment unit 13 calculates the value of pixel (x, y) in the processed image as the value obtained by multiplying the value of pixel (x, y) in the first image by the first coefficient minus the value obtained by multiplying the value of pixel (x, y) in the second image by the second coefficient.

[0080] In the next step S18, the adjustment unit 13 displays an image selected from the first image, the second image, the third image, the fourth image, and the processed image on the display 150 in accordance with the user input. The third image represents a first product obtained by multiplying the amount of light received in the first wavelength range by a first coefficient for each position on the inspection surface 2a, and is generated by multiplying the first image by the first coefficient. The fourth image represents a second product obtained by multiplying the amount of light received in the second wavelength range by a second coefficient for each position on the inspection surface 2a, and is generated by multiplying the second image by the second coefficient. The processed image corresponds to the "fifth image" in this disclosure.

[0081] In the next step S19, the adjustment unit 13 determines whether or not a reset instruction has been received. If the answer is YES in step S19, the process returns to step S13. If the answer is NO in step S19, the adjustment unit 13 officially sets the first coefficient and the second coefficient in step S20. After step S20, the process ends.

[0082] The generation unit 11 generates an inspection image using the first coefficient and the second coefficient finally set by the adjustment unit 13. The generation unit 11 generates the inspection image in the same manner as for the processed image. That is, the generation unit 11 generates an inspection image that represents, for each position on the inspection target surface 2a, the amount obtained by subtracting the amount of light received in the second wavelength range multiplied by the second coefficient from the amount of light received in the first wavelength range multiplied by the first coefficient.

[0083] 10 is a diagram showing an example of a user interface screen. A user interface screen 70 shown in FIG. 10 is generated by the adjustment unit 13 and displayed on the display 150 shown in FIG.

[0084] The user interface screen 70 includes an area 71 for inputting information about the first image, an area 72 for inputting information about the second image, an area 73 showing the calculation mode for generating the inspection image, an image display area 74, radio buttons 75, and a button 76.

[0085] The area 71 includes an input field 71a for selecting a processing unit image corresponding to the first wavelength range and an input field 71b for inputting a first coefficient. The input field 71a is used in step S13 shown in FIG. 9. That is, the adjustment unit 13 selects a processing unit image corresponding to the first wavelength range in accordance with an input to the input field 71a. The input field 71b is used in steps S14 and S20 shown in FIG. 9. That is, the adjustment unit 13 provisionally or officially sets the value input to the input field 71b as the first coefficient.

[0086] The area 72 includes an input field 72a for selecting a processing unit image corresponding to the second wavelength range and an input field 72b for inputting a second coefficient. The input field 72a is used in step S15 shown in FIG. 9. That is, the adjustment unit 13 selects a processing unit image corresponding to the second wavelength range in accordance with the input into the input field 72a. The input field 72b is used in steps S16 and S20 shown in FIG. 9. That is, the adjustment unit 13 provisionally or officially sets the value input into the input field 72b as the second coefficient.

[0087] As described above, the inspection image represents components corresponding to the difference between the amount of light received in the first wavelength range and the amount of light received in the second wavelength range for each position on the inspection target surface 2a. Therefore, region 73 indicates "subtraction" as the arithmetic operation mode for generating the inspection image.

[0088] The radio buttons 75 are used to select one of the first image, second image, third image, fourth image, and processed image in step S16 shown in Fig. 9. In Fig. 10, the "first image (coefficient)" and "second image (coefficient)" correspond to the third image and the fourth image, respectively. The adjustment unit 13 displays the image selected in accordance with the input to the radio buttons 75 in the image display area 74.

[0089] The button 76 is operated to permanently set the first coefficient and the second coefficient. In response to the operation of the button 76, the adjustment unit 13 determines YES in step S19 shown in FIG.

[0090] By using the user interface screen 70, the user can check the processed image according to the first coefficient and the second coefficient, and therefore the user can appropriately set the first coefficient and the second coefficient so that the difference in luminance between the foreign substance and the area other than the foreign substance in the processed image becomes larger.

[0091] <Modification> In the above description, the light 6a and 6b irradiated onto the inspection target surface 2a at a relatively large incident angle θ1 are red and blue, respectively. Furthermore, the light 7 irradiated onto the inspection target surface 2a at a relatively small incident angle θ2 is green. However, the color combination of the light 6a, 6b, and 7 is not limited to this, and it is sufficient that the color of the light 6 (light 6a and 6b) and the color of the light 7 are different. For example, the light 6a may be red, the light 6b may be green, and the light 7 may be blue. Furthermore, the color of the light 6a may be the same as the color of the light 6b. For example, the light 6a and 6b may be red, and the light 7 may be blue.

[0092] The amount of light received in the first wavelength range and the amount of light received in the second wavelength range depend on the intensities of light 6 and 7. Therefore, the first and second coefficients set by the adjustment unit 13 may be used to control the intensities of light 6 and 7. That is, the control unit 14 controls one or more lighting devices 3 to increase the intensity of light 6 by the first coefficient and controls the lighting device 4 to increase the intensity of light 7 by the second coefficient. In this case, the generation unit 11 generates an inspection image representing the amount of light received in the first wavelength range minus the amount of light received in the second wavelength range for each position on the inspection surface 2a. This also allows the first and second coefficients for the intensities of light 6 and 7 to be adjusted according to the conditions of the inspection surface 2a, thereby generating an inspection image suitable for detecting foreign objects. Furthermore, the user can appropriately set the first and second coefficients so that the brightness difference between foreign objects and non-foreign object areas is greater in the processed image displayed on the user interface screen 70.

[0093] The shapes and arrangement of the one or more lighting devices 3 and lighting device 4 are not limited to the example shown in FIG. 4. For example, when viewed from above the inspection target surface 2a, multiple lighting devices may be arranged radially. Specifically, when viewed from above the inspection target surface 2a, four lighting devices are arranged on all four sides. In this case, some of the four lighting devices are used as lighting devices 4, and the remaining are used as one or more lighting devices 3. Furthermore, the lighting devices 3 and 4 are not limited to line lighting devices having a linear light source, but may also be lighting devices having a point light source or a planar light source.

[0094] §3 Supplementary Note As described above, the present embodiment includes the following disclosures.

[0095] (Configuration 1) An image processing system (1), one or more lighting devices (3, 3a, 3b) for irradiating a first light (6, 6a, 6b) onto an inspection target surface (2a) at a first incident angle (θ1); a first lighting device (4) that irradiates the inspection target surface (2a) with second light (7) having a color different from that of the first light (6, 6a, 6b) at a second incident angle (θ2) smaller than the first incident angle (θ1); an imaging device (5) that detects the amount of light received with respect to reflected light at each position on the inspection target surface (2a) in a state where the first light (6, 6a, 6b) and the second light (7) are irradiated; and a generation unit (11, 110) that generates, for each position on the inspection surface (2a) based on the detection result of the photographing device (5), an inspection image (53) that represents a component corresponding to the difference between the amount of received light in a first wavelength range corresponding to the color of the first light (6, 6a, 6b) and the amount of received light in a second wavelength range corresponding to the color of the second light (7).

[0096] (Configuration 2) the component indicates an amount obtained by subtracting an amount obtained by multiplying an amount of received light in the second wavelength range by a second coefficient from an amount obtained by multiplying an amount of received light in the first wavelength range by a first coefficient, The image processing system (1) 2. The image processing system (1) according to configuration 1, further comprising an adjustment unit (13, 110) that adjusts at least one of the first coefficient and the second coefficient.

[0097] (Configuration 3) the component indicates an amount obtained by subtracting the amount of light received in the second wavelength range from the amount of light received in the first wavelength range, The image processing system (1) a control unit (14, 110) that controls the one or more lighting devices (3, 3a, 3b) to change the intensity of the first light (6, 6a, 6b) by a first factor and controls the first lighting device (4) to change the intensity of the second light (7) by a second factor; The image processing system (1) according to configuration 1 further comprises an adjustment unit (13, 110) that adjusts at least one of the first coefficient and the second coefficient.

[0098] (Configuration 4) the adjustment unit (13, 110) adjusts at least one of the first coefficient and the second coefficient in response to an input on a user interface screen (70); The user interface screen (70) includes an image display area (74); The image processing system (1) according to configuration 2 or 3, wherein the adjustment unit (13, 110) displays in the image display area (74) an image selected from a first image representing the amount of light received in the first wavelength range for each position on the inspection surface (2a), a second image representing the amount of light received in the second wavelength range for each position on the inspection surface (2a), a third image representing a first product obtained by multiplying the amount of light received in the first wavelength range by the first coefficient for each position on the inspection surface (2a), a fourth image representing a second product obtained by multiplying the amount of light received in the second wavelength range by the second coefficient for each position on the inspection surface (2a), and a fifth image representing an amount obtained by subtracting the second product from the first product for each position on the inspection surface (2a).

[0099] (Configuration 5) the one or more lighting devices (3, 3a, 3b) include a second lighting device (3a) and a third lighting device (3b); The optical axis (5a) of the imaging device (5) is perpendicular to the inspection target surface (2a), The image processing system (1) according to any one of configurations 1 to 4, wherein the second lighting device (3a) and the third lighting device (3b) are arranged in positions symmetrical to each other with respect to the optical axis (5a) of the photographing device (5).

[0100] (Configuration 6) The image processing system (1) according to configuration 5, wherein the color of the first light (6a) emitted from the second lighting device (3a) is different from the color of the first light (6b) emitted from the third lighting device (3b).

[0101] (Configuration 7) The image processing system (1) according to any one of configurations 1 to 6, wherein the photographing device (5) is a prism spectroscopic camera.

[0102] (Configuration 8) 1. An image processing method, comprising: irradiating a first light (6, 6a, 6b) onto an inspection target surface (2a) at a first incident angle (θ1); Irradiating the inspection target surface (2a) with second light (7) having a color different from that of the first light (6, 6a, 6b) at a second incident angle (θ2) smaller than the first incident angle (θ1); detecting the amount of light received for reflected light at each position on the inspection target surface (2a) while the first light (6, 6a, 6b) and the second light (7) are irradiated; and generating, for each position on the inspection surface (2a) based on the detection results, an inspection image (53) representing a component corresponding to the difference between the amount of received light in a first wavelength range corresponding to the color of the first light (6, 6a, 6b) and the amount of received light in a second wavelength range corresponding to the color of the second light (7).

[0103] Although the embodiments of the present invention have been described, the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, and it is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0104] 1 image processing system, 2 object, 2a inspection target surface, 3, 3a, 3b, 4 lighting device, 5 imaging device, 5a optical axis, 6, 6a, 6b, 7 light, 9 imaging system, 11 generation unit, 12 inspection unit, 13 adjustment unit, 14 control unit, 50 color image, 51, 52 image, 53 inspection image, 61 lint, 62 metal powder, 63 dent, 64 wrinkle, 65 scratch, 70 user interface screen, 71, 72, 73 area, 71a, 71b, 72a, 72b input field, 74 image display area, 75 radio button, 76 button, 91a, 91b plate-shaped member, 92a to 94a, 92b to 94b, 95 holding member, 96 inspection target area, 100 image processing device, 106 memory card, 110 CPU, 112 main memory, 114 hard disk, 115 program, 116 camera interface, 116a image buffer, 118 input interface, 120 display controller, 124 communication interface, 126 data reader / writer, 128 bus 150 display, 160 input device, 501 light receiving lens, 502 prism, 510 sensor set, 511, 512, 513 sensors.

Claims

1. 1. An image processing system, comprising: one or more illumination devices configured to irradiate a first light onto the surface to be inspected at a first angle of incidence; a first lighting device that irradiates the inspection target surface with second light having a different color from the first light at a second incident angle that is smaller than the first incident angle; an imaging device that detects the amount of light received with respect to reflected light at each position on the inspection target surface while the first light and the second light are irradiated; an image processing system comprising: a generation unit that generates, for each position on the surface to be inspected, an inspection image representing a component corresponding to the difference between the amount of light received in a first wavelength range corresponding to the color of the first light and the amount of light received in a second wavelength range corresponding to the color of the second light, based on the detection result of the imaging device.

2. the component indicates an amount obtained by subtracting an amount obtained by multiplying an amount of received light in the second wavelength range by a second coefficient from an amount obtained by multiplying an amount of received light in the first wavelength range by a first coefficient, The image processing system includes: The image processing system according to claim 1 , further comprising an adjustment unit that adjusts at least one of the first coefficient and the second coefficient.

3. the component indicates an amount obtained by subtracting the amount of light received in the second wavelength range from the amount of light received in the first wavelength range, The image processing system includes: a control unit that controls the one or more lighting devices to change the intensity of the first light by a first factor and controls the first lighting device to change the intensity of the second light by a second factor; The image processing system according to claim 1 , further comprising an adjustment unit that adjusts at least one of the first coefficient and the second coefficient.

4. the adjustment unit adjusts at least one of the first coefficient and the second coefficient in response to an input on a user interface screen; the user interface screen includes an image display area; 4. The image processing system of claim 2 or 3, wherein the adjustment unit displays in the image display area an image selected from a first image representing the amount of light received in the first wavelength range for each position on the surface to be inspected, a second image representing the amount of light received in the second wavelength range for each position on the surface to be inspected, a third image representing a first product obtained by multiplying the amount of light received in the first wavelength range by the first coefficient for each position on the surface to be inspected, a fourth image representing a second product obtained by multiplying the amount of light received in the second wavelength range by the second coefficient for each position on the surface to be inspected, and a fifth image representing an amount obtained by subtracting the second product from the first product for each position on the surface to be inspected.

5. the one or more lighting devices include a second lighting device and a third lighting device; the optical axis of the imaging device is perpendicular to the inspection target surface; The image processing system according to claim 1 , wherein the second illumination device and the third illumination device are disposed at positions symmetrical to each other with respect to an optical axis of the image capturing device.

6. The image processing system according to claim 5 , wherein a color of the first light emitted from the second lighting device is different from a color of the first light emitted from the third lighting device.

7. 4. The image processing system according to claim 1, wherein the image capturing device is a prism spectroscopic camera.

8. 1. An image processing method, comprising: irradiating a surface to be inspected with a first light at a first incident angle; irradiating the inspection target surface with second light having a different color from the first light at a second incident angle smaller than the first incident angle; detecting an amount of received light for reflected light at each position on the inspection target surface while the first light and the second light are irradiated; and generating, for each position on the surface to be inspected, an inspection image representing a component corresponding to the difference between the amount of light received in a first wavelength range corresponding to the color of the first light and the amount of light received in a second wavelength range corresponding to the color of the second light, based on the detection results.

Images