Imaging device, inspection device, and imaging method

Abstract

The photographing device (15) of the present application is provided with a camera (30), light sources (21-23), a band-pass filter (31), and a conversion unit (60). The camera (30) is provided with an image sensor (33) having a color filter of N bands (N is a natural number of 3 or more). The light sources (21-23) of M types (M is a natural number of 2≦M≦N) have emission spectrum characteristics each having a peak in a different wavelength region from each other within a visible light region and a near-infrared region. The conversion unit (60) performs matrix operation on a photographing signal (S1) of N bands obtained by the image sensor (33) when the camera (30) photographs an article (12), thereby generating image signals of M bands each having a spectral sensitivity in a different wavelength region from each other. The light directions and the emission intensities of the light sources (21-23) are selected respectively.

Description

Technical Field

[0001] This invention relates to a photographing device, an inspection device, and a photographing method for photographing a subject. Background Technology

[0002] For example, Patent Documents 1-3 disclose a photographing device that uses a camera to capture an image of light illuminating a bottle or other object from a light source. Patent Documents 1-3 also disclose an inspection device that, for example, inspects text, defects, labels, or other defects attached to an object based on an image of the object captured by a camera.

[0003] In situations where multiple objects are being inspected, such as text, defects, or labels, the optical characteristics of these objects differ. Using a common light source and camera to inspect these diverse objects becomes difficult. Therefore, to obtain images suitable for inspection, different shooting conditions are required, necessitating a shooting system with multiple light sources and cameras.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2015-92141

[0007] Patent Document 2: Japanese Patent Application Publication No. 2011-33641

[0008] Patent Document 3: Japanese Patent Application Publication No. 2007-209206 Summary of the Invention

[0009] The technical problem to be solved by the invention

[0010] However, in shooting systems with multiple light sources and cameras, when multiple cameras are configured, staggered shooting times and individual image processing are required, thus increasing inspection time at multiple points. Furthermore, when images from multiple cameras are processed together, the different positions of the subjects necessitate position and perspective corrections for the images obtained from each camera. This also necessitates mechanisms or light emission controls to prevent interference between the illumination from multiple light sources.

[0011] The purpose of this invention is to provide a photographing device and method that, through a simple structure, can capture images of multiple locations with different optical properties on a subject using a single camera from the same perspective. Furthermore, another purpose of this invention is to provide an inspection device that, through a simple structure and simple processing, can perform inspections by photographing multiple locations with different optical properties on a subject.

[0012] Technical solutions adopted to solve technical problems

[0013] The following describes the methods used to solve the above problems and their effects.

[0014] The imaging device camera that solves the above-mentioned problems includes an image sensor having an N-band spectral filter with different spectral transmittance characteristics and sensitivity in the visible and near-infrared regions, where N is a natural number greater than or equal to 3; M types of light sources, each having emission spectral characteristics such as peaks in different wavelength regions in the visible and near-infrared regions, where M is a natural number satisfying 2 ≤ M ≤ N; a filter disposed in the optical path between the image sensor and the subject, capable of transmitting light from the different wavelength regions of the M types of light sources; and a conversion unit that converts... When the camera captures the subject, the image signal obtained by the image sensor is separated into N-band image signals. By performing matrix operations on the separated N-band image signals, M-band image signals with spectral sensitivity in different wavelength regions are generated. The M types of light sources illuminate the subject area at point M. The illumination direction and luminous intensity of the subject are selected according to the subject area. The illumination direction of the subject determines whether the image of the subject area at point M captured by the camera is transmitted light or reflected light.

[0015] According to this structure, an M-band image signal obtained by matrix operation on an N-band imaging signal can be generated into M images, each with spectral sensitivity in its own independent wavelength region. Therefore, when illuminating the M-band object region with M types of light sources, each with a emission spectrum peaking in a different wavelength region, from a selected illumination direction, to capture the M-band object region with different optical characteristics, the illumination intensity of the M-band light sources for that M-band object region can be set separately. As a result, even if the M-band object region is simultaneously captured by a single camera from the same viewing angle, M images can still be obtained where one of the M-band objects is captured more clearly than the others. Therefore, with this simple structure, multiple locations with different optical characteristics of a subject can be simultaneously captured by a single camera from the same viewing angle.

[0016] The aforementioned shooting device can also be: the camera is a general-purpose color camera after removing the infrared light cutoff filter.

[0017] Based on this structure, since the camera uses a general-purpose color camera, the structure of the shooting device can be simple.

[0018] The above-mentioned imaging device can also be: the M-type light source includes two types of light sources, and the two types of light sources have the following emission spectrum characteristics: they have peaks in two different wavelength regions in the visible light region, the M-band image signal is an image signal with more than 3 bands, including a first image signal, a second image signal and a third image signal, the first image signal has spectral sensitivity in the first wavelength region of one of the two wavelength regions, the second image signal has spectral sensitivity in the second wavelength region of the other, and the third image signal has spectral sensitivity in a third wavelength region that is different from both the first and second wavelength regions.

[0019] Based on this structure, it is possible to obtain image signals with more than 3 frequency bands from multiple locations with different optical characteristics of the subject in a single shot.

[0020] The above-mentioned imaging device may also be: the light source of type M includes the following light source: having emission spectral characteristics in a predetermined wavelength region in the near-infrared region, and the image signal of the M band includes the following image signal: having spectral sensitivity in the predetermined wavelength region in the near-infrared region.

[0021] According to this structure, it is possible to acquire an M-band image signal containing image signals of parts of the subject that can be photographed with near-infrared light illumination in a single shot.

[0022] The above-mentioned shooting device may also be: the subject being photographed by the camera includes a translucent area, and the M-type light source illuminates the subject at the following time: during the period when the subject area at point M is simultaneously illuminated, the camera takes a single shot of the subject.

[0023] Based on this structure, it is possible to capture images of multiple locations with different optical properties of the subject in a single shot using a single camera, thanks to a simple design.

[0024] The aforementioned shooting device may also be: the M-type light source includes at least two of a first light source, a second light source, and a third light source; the first light source is positioned on the side opposite to the camera relative to the subject; the second light source is positioned on the same side as the camera relative to the subject; and the third light source is positioned on the side opposite to the camera relative to the subject. The camera captures at least two of a first image, a second image, and a third image. The first image is an image of transmitted light from the first light source through the subject; the second image is an image of reflected light from the second light source reflected by the subject; and the third image is an image of transmitted light from the third light source through the subject.

[0025] According to this structure, at least two images can be captured: a first image of transmitted light formed by light from a first light source passing through the subject; a second image of reflected light formed by light from a second light source being reflected by the subject; and a third image of transmitted light formed by light from a third light source passing through the subject. Therefore, images of at least two locations with different optical properties in the subject can be effectively obtained.

[0026] The inspection apparatus for solving the above-mentioned problem includes: the imaging device; and an inspection processing unit that inspects the photographed object based on the image signal of the M-band output by the imaging device.

[0027] Based on this configuration, a simple imaging device can be used to capture images of multiple locations with different optical properties on a subject from the same angle using a single camera. When capturing images of these locations, the M-type light source can be adjusted to an appropriate intensity with minimal consideration for its impact on other areas of the subject. Therefore, it is possible to inspect multiple locations with different optical properties on a subject with simple processing.

[0028] The inspection device described above may also be: the object being inspected is a container containing liquid, including a translucent area, the inspection processing unit inspects the text attached to the outer peripheral surface of the container, and the inspection device performs at least two of the following inspections: inspection of the text attached to the part of the container that overlaps with the liquid; inspection of the text on the part of the container that does not overlap with the liquid; and inspection of the text on the label affixed to the outer peripheral surface of the container.

[0029] Based on this structure, it is possible to properly inspect multiple locations with different optical properties in the photographed object using a simple method.

[0030] The shooting method for solving the above-mentioned problems generates an image signal by photographing the subject with a camera. It includes an illumination step, a shooting step, and a conversion step. In the illumination step, the subject is illuminated by M types of light sources, each of which has the following emission spectrum characteristics: peaks in different wavelength regions within the visible and near-infrared regions. M is a natural number satisfying 2 ≤ M ≤ N, and N is a natural number greater than or equal to 3. In the shooting step, the camera captures the subject area at point M using a filter. The camera has an image sensor with a spectral filter in the N-band having different spectral transmittance characteristics in the visible and near-infrared regions. The outer region has sensitivity; the filter can transmit light from the different wavelength regions of the M types of light sources. In the conversion step, the image signal captured by the image sensor is separated into N-band image signals, and matrix operations are performed on the separated N-band image signals to generate M-band image signals with spectral sensitivity in the different wavelength regions. The M types of light sources illuminate the M-region of the subject. The illumination direction and luminous intensity of the subject are selected according to the subject region. The illumination direction of the subject determines whether the image of the M-region captured by the camera is transmitted light or reflected light.

[0031] Invention Effects

[0032] According to the present invention, since the illumination intensity can be set separately for multiple subject areas with different optical properties in the subject, it is possible to simultaneously capture images of multiple subject areas from the same angle using a single camera with a simple structure. Attached Figure Description

[0033] Figure 1 This is a schematic diagram illustrating an inspection system equipped with an inspection device according to the first embodiment.

[0034] Figure 2This is a schematic diagram showing the structure of the camera device.

[0035] Figure 3 It is a schematic side view of an object.

[0036] Figure 4 (a) is a diagram showing the structure of a general-purpose color camera and the relationship between wavelength and relative sensitivity. Figure 4 (b) is a diagram showing the structure of a color camera after the IR cutoff filter has been removed, and the relationship between wavelength and relative sensitivity.

[0037] Figure 5 This is a schematic diagram showing the structure of a camera.

[0038] Figure 6 (a) is a graph showing the light transmittance characteristics of a bandpass filter. Figure 6 (b) is a graph representing the relative sensitivity of the image sensor to various colors.

[0039] Figure 7 Figures (a) to (c) show the emission spectra of the three types of light sources.

[0040] Figure 8 (a) is a graph showing the emission spectrum of a combination of three types of light sources. Figure 8 (b) is a graph showing the relative sensitivity of various colors of the image sensor after passing through the bandpass filter.

[0041] Figure 9 (a) is a graph showing the relative sensitivity of the image sensor for various colors when three types of light sources are used as light sources. Figure 9 (b) is a graph showing the relative output characteristics of various frequency bands of the second captured signal.

[0042] Figure 10 It is a block diagram representing the functional structure of the inspection device.

[0043] Figure 11 It is a picture that shows an object photographed using natural light.

[0044] Figure 12 (a) to (c) are graphs representing the X-image, Y-image, and Z-image, respectively.

[0045] Figure 13 (a) to (c) are images representing the X-contour image, Y-contour image, and Z-contour image, respectively.

[0046] Figure 14 (a) is a graph showing the light transmittance characteristics of the bandpass filter in the second embodiment. Figure 14(b) is a graph showing the relative output characteristics of the second captured signal using a near-infrared light source across various frequency bands.

[0047] Figure 15 This is a diagram representing an X-ray image obtained by taking a picture with a near-infrared light source set as a single light source. Detailed Implementation

[0048] (First Embodiment)

[0049] The following description refers to the inspection system equipped with a camera.

[0050] Figure 1 The inspection system 10 shown uses images obtained by photographing an item 12, one example of the object being photographed, to inspect the condition of the item 12. The inspection system 10 includes a transport device 13 and an inspection device 11. The transport device 13 transports the item 12, and the inspection device 11 inspects the condition of the item 12 based on the photographing results (photographing signals) captured by the camera 30. The inspection device 11 includes a photographing device 15 and an inspection processing unit 70. The photographing device 15 generates a multiband image (image signal) containing an M-band image (image signal) based on the N-band (where N is a natural number greater than or equal to 3) photographing signal output by the camera 30 photographing the item 12. The inspection processing unit 70 uses the multiband image for inspection. Here, a multiband image refers to the total number of M photographed images based on the M-band image signal.

[0051] The imaging device 15 includes M types of light sources 21-23 (where M satisfies 2≦M≦N) (e.g., 3 types) that illuminate the article 12, a camera 30 for imaging the article 12, and a control processing unit 40 electrically connected to the camera 30. At least a portion of the control processing unit 40 is composed of a computer. The computer includes an input device and a display unit. In this embodiment, the transport device 13 is driven by the control unit of the transport system, which is communicatively connected to the control system of the inspection device 11. The control processing unit 40 can also control the transport device 13.

[0052] like Figure 1As shown, the handling device 13 includes a conveyor 16, a sensor 17, and a removal device (not shown in the diagram). The conveyor 16 transports the item 12, the sensor 17 detects the item 12 transported by the conveyor 16, and the removal device removes items 12 deemed defective based on the inspection results of the inspection device 11 from the good product production line. The conveyor 16 can be a belt conveyor, a roller conveyor, or a device that transports the item 12 by holding it or by suspending it. The removal device can be a structure that removes the item 12 by squeezing it or by blowing it away with air.

[0053] The inspection device 11 includes an imaging device 15, an inspection processing unit 70, and a display unit 41. The imaging device 15 generates images in multiple frequency bands, the inspection processing unit 70 uses the images in multiple frequency bands to inspect the quality of the item 12, and the display unit 41 displays the multi-frequency images and the inspection results.

[0054] The item 12 is inspected based on multiple images XI, YI, and ZI obtained by signal processing of the second imaging signal S2 with multiple frequency bands output when the imaging device 15 images the item 12 to be inspected. The display unit 41 can be a screen connected to a computer or a monitor installed on the control panel.

[0055] The imaging device 15 includes three types of light sources 21-23 that illuminate the object 12, a camera 30 for photographing the object 12, and a control processing unit 40. The three types of light sources 21-23 each illuminate the object 12 from different predetermined directions and each illuminates a different area of ​​the object 12. That is, the locations and directions in which the three types of light sources 21-23 illuminate the object 12 are different from each other.

[0056] In this embodiment, the three types of light sources 21 to 23 are a first light source 21, a second light source 22, and a third light source 23. Each of these light sources illuminates different areas of the object 12 from different positions, allowing for simultaneous imaging of multiple locations on the object 12 with different optical characteristics in a single shot. The light source characteristics of the three types of light sources 21 to 23 should meet the following two conditions. Specifically, (a) the overlapping area (area) of the luminous characteristics of each of the three types of light sources 21 to 23 is as follows: Figure 8 (a) is small enough; (b) the three types of light sources 21-23 have luminescent properties in the visible light region.

[0057] like Figure 1The three types of light sources 21-23 shown are light sources with different emission spectrum characteristics in the visible light region. Each of the three types of light sources 21-23 has multiple light-emitting parts 21a, 22a, and 23a with different emission spectrum characteristics in the visible light wavelength region VA. In this example, the first light source 21 has a first light-emitting part 21a that emits red light. The second light source 22 has a second light-emitting part 22a that emits green light. Furthermore, the third light source 23 has a third light-emitting part 23a that emits blue light. Each light-emitting part 21a, 22a, and 23a is, for example, composed of an LED. In this example, the first light-emitting part 21a is composed of a red LED, the second light-emitting part 22a is composed of a green LED, and the third light-emitting part 23a is composed of a blue LED. That is, the first light source 21 is a red light source, illuminating red light (hereinafter referred to as "R light") with an emission spectrum in the red wavelength region. The second light source 22 is a green light source, illuminating green light (hereinafter referred to as "G light") with an emission spectrum in the green wavelength region. The third light source 23 is a blue light source, which illuminates blue light (hereinafter referred to as "B light") with a emission spectrum having a blue wavelength region.

[0058] like Figure 1 , Figure 2 As shown, the first light source 21 is positioned on the side opposite to the camera 30 relative to the object 12. The second light source 22 is positioned on the same side as the camera 30 relative to the object 12. The third light source 23 is positioned on the opposite side of the camera 30 relative to the object 12. The M-type light sources 21 to 23 have the following emission spectrum characteristics: they each have peaks in different M-type wavelength regions within the visible light region.

[0059] like Figure 1 , Figure 2 As shown, the camera 30 includes a bandpass filter 31 (which serves as an example of a filter), a lens 32, and a color image sensor 33 (hereinafter referred to as "image sensor 33"). The bandpass filter 31 is disposed in the optical path between the article 12 and the image sensor 33. Figure 1 In the example shown, the bandpass filter 31 is positioned between the article 12 and the lens 32, but it can also be positioned between the lens 32 and the image sensor 33.

[0060] Image sensor 33 receives light from the image of object 12 through bandpass filter 31 and lens 32, and outputs a first imaging signal S1 corresponding to the light reception result. The first imaging signal S1 output by image sensor 33 is input to conversion unit 60. Conversion unit 60 converts the first imaging signal S1 into a second imaging signal S2 that displays an image with multiple frequency bands.

[0061] The control processing unit 40 includes a control unit 50 and a conversion unit 60. The control unit 50 controls M types of light sources 21 to 23 and a camera 30. The conversion unit 60 converts multiple frequency bands of shooting signals from the camera 30 into images of multiple frequency bands.

[0062] The control unit 50 controls the illumination of three types of light sources 21-23. When a detection signal from the sensor 17 that detects the items 12 being transported by the conveyor 16 is input as a trigger signal, the control unit 50 causes the three types of light sources 21-23 to emit light simultaneously. Therefore, the light from the three types of light sources 21-23 simultaneously illuminates the arriving items. Figure 1 , Figure 2 The item 12 is shown in the inspection position. Therefore, multiple different inspection areas of item 12 are illuminated by R-light, G-light, and B-light from three types of light sources 21-23. Furthermore, the control unit 50 outputs a shooting command signal to the camera 30 to perform a single shooting action based on a trigger signal. As a result, the camera 30 performs a single shot of item 12, where multiple different areas of item 12 are illuminated by various lights from the three types of light sources 21-23. Alternatively, the control unit 50 may not control the illumination of the three types of light sources 21-23, but rather ensure that they are always emitting light.

[0063] As described above, the object 12 photographed by camera 30 includes areas that are translucent. Light sources 21-23 of type M are applied at times during which different areas of the object 12 are simultaneously illuminated by the aforementioned type M light. Camera 30 takes a single photograph of the object 12.

[0064] The conversion unit 60 converts the first image signal S1 captured by the camera 30 into a second image signal S2 that displays an image across multiple frequency bands. In this example, the second image signal S2 is a signal that displays an image across three frequency bands. The second image signal S2 includes a first image signal XS, a second image signal YS, and a third image signal ZS that constitute the three frequency bands.

[0065] The conversion unit 60 separates the image signal output by the image sensor 33 into N frequency bands (where N is a natural number greater than 3), and generates a 3-band image signal XS, a first image signal YS, and a third image signal ZS with spectral sensitivity in the visible light region by performing matrix operations on the separated N frequency band image signals.

[0066] The first image signal XS, the second image signal YS, and the third image signal ZS are input to the inspection processing unit 70. The inspection processing unit 70 inspects the quality of the article 12 based on the first image signal XS, the second image signal YS, and the third image signal ZS. The inspection processing unit 70 displays the X image XI based on the first image signal XS, the Y image YI based on the second image signal YS, and the Z image ZI based on the third image signal ZS as the imaging results on the display unit 41. In addition, the inspection results of inspecting the quality of the article 12 based on the images XI, YI, and ZI of the above three channels are displayed on the display unit 41. In the X image XI, Y image YI, and Z image ZI displayed on the display unit 41, defects detected by the inspection processing can also be highlighted by using methods such as overlay display marks or color assignment. In this embodiment, the object of inspection is the text attached to the outer peripheral surface of the article 12. Defects such as distortion, wear, leakage, and dead pixels of the text are inspected.

[0067] Figure 1 The inspection processing unit 70 shown inspects the quality of the item 12 based on the N-band image signals XS, YS, and ZS generated by the conversion unit 60. The inspection processing unit 70 inspects the item 12 using visible light images from three channels simultaneously captured by a single camera 30. To ensure simultaneous capture by a single camera 30, the capture time and viewing angle of the three visible light channels are identical. To effectively implement the processing of the conversion unit 60 and the inspection processing unit 70, a structure integrating the conversion unit 60 and the inspection processing unit 70 can be adopted, employing a processing structure that does not output the N-band image signals XS, YS, and ZS.

[0068] As described above, the shooting device 15 has M types (e.g., 3 types) of light sources 21 to 23, a camera 30, and a conversion unit 60 as its main components.

[0069] The camera 30 includes an image sensor 33, which has a color filter 34 as an example of a spectral filter and is sensitive in the visible light and near-infrared regions. The color filter 34 is a spectral filter with N-band (where N is a natural number greater than or equal to 3) of different spectral transmittance characteristics. In this embodiment, N = 3, and the color filter 34 is a spectral filter with 3-band (3-band) of different spectral transmittance characteristics.

[0070] The imaging device 15 has M types of light sources (where M is a natural number satisfying 2≦M≦N). In this embodiment, M=3, and the imaging device 15 has 3 types of light sources 21 to 23. The 3 types of light sources 21 to 23 have the following emission spectrum characteristics: each of the M types of wavelength regions in the visible light region has a peak.

[0071] Furthermore, the imaging device 15 includes a conversion unit 60, which inputs the imaging signal obtained by the camera 30 photographing the object 12 and passing it through the image sensor 33 to the conversion unit 60 for conversion. The imaging signal S1 from the image sensor 33 is input to the conversion unit 60 and separated into an N-band imaging signal. The conversion unit 60 generates an M-band image signal from the N-band imaging signal. The conversion unit 60 performs matrix operations on the imaging signal S1 separated into N-band imaging signals, thereby generating an M-band image signal with spectral sensitivity in M ​​types of wavelength regions. The M-band image signal is an image signal with 3 or more bands, including a first image signal, a second image signal, and a third image signal. The first image signal has spectral sensitivity in a first wavelength region of one of the two types of wavelength regions, the second image signal has spectral sensitivity in a second wavelength region of the other, and the third image signal has spectral sensitivity in a third wavelength region different from both the first and second wavelength regions.

[0072] Here, refer to Figure 3 A description is provided for item 12, which is the subject of the inspection.

[0073] like Figure 3 As shown, article 12 is, for example, container 12A. Container 12A is formed of a light-transmitting material. The material of container 12A is, for example, synthetic resin (plastic) or glass. Article 12 can also be colorless and transparent, or colored transparent or translucent. Article 12 is, for example, a container 12A containing liquid LQ. Article 12 includes a container 12A containing liquid LQ and a cap 12B that seals the opening of container 12A. In addition, article 12 can also have a label 12L affixed to the outer peripheral surface of container 12A. Label 12L includes, for example, a substrate made of a colorless and transparent film and a printed layer on the surface of the film containing text and patterns. In label 12L, the substrate, i.e., the film, is attached to the outer peripheral surface of container 12A by heat welding or heat shrinking, or by an adhesive. In addition, label 12L can also be: made of paper that is not permeable to visible light and attached to the surface of container 12A using glue. Furthermore, Figure 3 In the example, label 12L is, for example, cylindrical and affixed to the entire circumferential area of ​​container 12A, or it may be affixed only to a portion of the circumferential direction.

[0074] like Figure 3 As shown, text is affixed to the outer periphery of container 12A. The text affixed to container 12A includes the first character C1, the second character C2, and the third character C3. The first character C1 is affixed to the first region A1 below the liquid level LS in container 12A, the second character C2 is affixed to the second region A2 of the label 12L on the outer periphery of container 12A, and the third character C3 is affixed to the third region A3 above the liquid level LS in container 12A.

[0075] The first and third characters, C1 and C3, are printed with ink on the outer surface of article 12. The first and third characters, C1 and C3, contain manufacturing-related information about article 12, such as the product name, manufacturing number, batch number, manufacturing location, year, month, day, and time of manufacture. Additionally, the second character, C2, is printed with ink on label 12L. While the second character, C2, differs depending on whether it is printed on container 12A or label 12L, it still contains manufacturing-related information about article 12. Furthermore, the second character, C2, may also include text originally printed on label 12L. Label 12L contains product-related information such as barcodes, QR codes, product names, ingredients (liquid), volume, weight, storage methods, precautions, manufacturing plant address, and customer contact information. The second character, C2, may also include product-related information. Here, the text can be in any language, such as Japanese, English, or Chinese. In addition, the text includes Chinese characters, letters, hiragana, katakana, numbers, symbols, and barcodes.

[0076] The object of inspection is not limited to text; it can also be a defect in either the container or the label. Defects referred to here can include impurities mixed into the container, scratches, whisker-like protrusions, air bubbles, etc. Furthermore, when item 12 is a container, the inspection location is not limited to the container and label; it can also be the bottle cap. For example, the text containing manufacturing information attached to the bottle cap can be used as the inspection object, as can defects in the bottle cap itself. Additionally, areas A1 to A3 are the illumination areas respectively illuminated by light from the three light sources 21 to 23, and are also the shooting areas of camera 30. When the object of inspection is text, the small areas within each area A1 to A3 containing the characters C1 to C3 can also be used as the respective illumination areas and shooting areas of each light source 21 to 23.

[0077] In this embodiment, the multiple locations in article 12 with different optical properties are designated as M locations, corresponding to regions A1 to A3 of M types (3 types in this example). In this example, the optical properties of light from each light source 21 to 23 are different in the first region A1 where light passes through container 12A and liquid LQ, the second region A2 where light is reflected from the surface of label 12L, and the third region A3 where light passes through container 12A and air. Furthermore, the optical properties also vary depending on the conditions on the subject side, such as the material of container 12A, the presence or absence of liquid LQ, the composition of liquid LQ, the presence or absence of label 12L, and the material of label 12L. In addition, the optical properties will differ depending on whether the image being photographed is an image of transmitted light penetrating the subject or an image of reflected light reflected from the surface of the subject. Furthermore, the optical properties will also differ depending on the material and thickness of the ink constituting the characters C1 to C3.

[0078] Furthermore, container 12A only needs to be a structure that includes at least a light-transmitting area; the entire container may be light-transmitting, or only a portion of it may be light-transmitting. This light-transmitting portion may be a single location or multiple locations situated at different points within container 12A. Additionally, liquid LQ may or may not be light-transmitting. Here, "not light-transmitting" means a light transmittance of less than 10%. The light transmittance referred to here is defined as the value of the light transmittance of the liquid LQ in the area where the object to be inspected is located when the container 12A is contained within it.

[0079] Figure 1 The light sources 21-23 of type M shown illuminate areas A1-A3 of the object 12, which are examples of the photographed area at point M. The direction of illumination and the intensity of light emitted by each area A1-A3 are selected according to their respective characteristics. The direction of illumination determines whether the image of area A1-A3 at point M captured by the camera 30 is transmitted light or reflected light. Furthermore, Figure 1 The imaging device 15 shown has an image sensor 33, and the image sensor 33 has a color filter 34, which is an example of a beam splitter filter. Figure 5 (Refer to) and has sensitivity in the visible light and near-infrared regions, wherein the color filter 34 has N-bands with different spectral transmittance characteristics per pixel unit (where N is a natural number greater than 3). The color filter 34 is configured as follows: multiple R filters 34R, G filters 34G, and B filters 34B with different transmittance characteristics are arranged in pixel units.

[0080] The conversion unit 60 separates the first image signal S1 captured by the image sensor 33 into signals of N frequency bands. Then, the conversion unit 60 generates a first image signal XS, a second image signal YS, and a third image signal ZS with N frequency bands (where N is a natural number of 3 or more) that have spectral sensitivity in the visible light region by performing matrix operations on the separated N frequency band image signals.

[0081] Next, refer to Figure 4 Sections (a) and (b) describe the structure of the image sensor 33 of the camera 30. Figure 4(a) is a general-purpose color camera 200 for capturing RGB images. The color camera 200 includes a lens 32 mounted on a lens barrel 30a, a near-infrared cutoff filter 201 (hereinafter referred to as the IR cutoff filter 201) that blocks near-infrared light, and an image sensor 33. The image sensor 33 includes an R light-receiving element 33R, a G light-receiving element 33G, and a B light-receiving element 33B. The R light-receiving element 33R receives red light passing through the R filter 34R and outputs an R-image signal corresponding to the received light amount. The G light-receiving element 33G receives green light passing through the G filter 34G and outputs a G-image signal corresponding to the received light amount. The B light-receiving element 33B receives blue light passing through the B filter 34B and outputs a B-image signal corresponding to the received light amount. In the image sensor 33, the R light-receiving element 33R, the G light-receiving element 33G, and the B light-receiving element 33B are arranged in a predetermined configuration.

[0082] The aforementioned image sensor 33 has RGB imaging characteristics after near-infrared light is cut off. The R light-receiving element 33R, G light-receiving element 33G, and B light-receiving element 33B are for... Figure 4 The light in each wavelength band shown in Figure (a) has sensitivity. In this figure, the horizontal axis represents wavelength, and the vertical axis represents relative sensitivity. The light-receiving element 33R is sensitive to... Figure 4 The light in the red (R) band shown in Figure (a) has high sensitivity. The G light-receiving element 33G is for... Figure 4 The light in the green (G) band shown in Figure (a) has high sensitivity. The light-receiving element 33B is for... Figure 4 The blue (B) band of light shown in Figure (a) has high sensitivity.

[0083] Figure 4 (b) is from Figure 4 (a) shows a general-purpose color camera 200 with the IR cutoff filter 201 removed, and a color camera 250 thereafter. The image sensor 33 built into the color camera 250 does not cut off near-infrared light, but has RGB imaging characteristics that include the wavelength band of near-infrared light. The R light-receiving element 33R, G light-receiving element 33G, and B light-receiving element 33B are for... Figure 4 The light in the visible wavelength region VA and the near-infrared wavelength region NIRA (especially the near-infrared wavelength region) shown in Figure (b) has sensitivity.

[0084] Image sensor 33 itself, such as Figure 4 As shown in Figure (b), it exhibits sensitivity in the visible light wavelength region (VA) and the near-infrared wavelength region (NIRA). Camera 30 is a general-purpose color camera with the infrared cutoff filter removed. Specifically, camera 30 is configured, for example, to: Figure 4In the general color camera 200 shown in (a), after removing the IR cutoff filter 201, a bandpass filter 31 is installed in the optical path. The camera 30 is not limited to a structure based on the general color camera 200.

[0085] The color filter 34 constituting the image sensor 33 is an RGB primary color filter, or it can be a complementary color filter of Mg, Ye, and Cy. In addition to RGB filters or complementary color filters, it can also include a NIR filter that selectively transmits near-infrared light. Furthermore, it can also have the following structure: the RGB filter is an R, G1, G2, B filter, or the color filter 34 is a combination of complementary color filters and primary color filters. Furthermore, the combined filters can be of three or more types.

[0086] Figure 5 This diagram illustrates the structure of the camera 30 in this embodiment. (Example:) Figure 5 As shown, camera 30 has a bandpass filter 31 in the optical path between image sensor 33 and object 12. Camera 30 does not have... Figure 4 The IR cutoff filter 201 is shown in (a). Image sensor 33 and Figure 4 The general-purpose color camera 200 shown in (a) has the same structure.

[0087] Figure 5 The image sensor 33 within the camera 30 shown has Figure 8 The relative sensitivity characteristics are shown in (b). That is, the R light-receiving element 33R constituting the image sensor 33 has... Figure 8 In diagram (b), R represents the sensitivity of the frequency band, G (receiving element 33G) has the sensitivity of the frequency band represented by G, and B (receiving element 33B) has the sensitivity of the frequency band represented by B. Each receiving element 33R, 33G, and 33B receives light from the light passing through the bandpass filter 31 and the light passing through the color filter 34R, 34G, and 34B, in a manner corresponding to its respective sensitivity. The image sensor 33 outputs a first imaging signal S1, which is formed by serially arranging the imaging signals having R, G, and B values ​​corresponding to the amount of light received by each receiving element 33R, 33G, and 33B in a predetermined pattern.

[0088] The bandpass filter 31 has the following spectral transmittance characteristics: it has one or more blocking regions in the visible light wavelength region VA, and one or more transmitting regions in the near-infrared wavelength region NIRA. The bandpass filter 31 has these spectral transmittance characteristics.

[0089] It can be composed of one piece or multiple pieces. When the bandpass filter 31 is composed of multiple pieces, the light transmittance is reduced, so the light intensity of each light source 21 to 23 needs to be increased. Therefore, from the point of view of saving power, it is better to have fewer pieces.

[0090] The bandpass filter 31 uses a device with the following spectral transmittance characteristics: it is appropriately inspected in accordance with the difference in spectral reflectance characteristics between the object to be identified (e.g., text, defects, etc.) in the item 12 to be inspected.

[0091] Figure 6 The figure shown in (a) represents the transmission band of bandpass filter 31. Figure 6 Figure (b) shows the relative sensitivity characteristics of image sensor 33 when the illumination emission spectrum is an ideal light source with a relative value of 1 in the entire wavelength region.

[0092] Figure 6 The transmittance characteristic curve F1 of the bandpass filter 31 shown in (a) has multiple peaks in the transmission band. It has the following spectral transmittance characteristics: blocking regions with transmittance below 10% in multiple bands, and transmission regions with transmittance above 70% in multiple bands. The bandpass filter 31 has the following spectral transmittance characteristics: one or more blocking regions in the visible light wavelength region VA, and a transmission region in the near-infrared wavelength region NIRA.

[0093] In addition, from Figure 6 As shown in the transmittance characteristic curve F1 in Figure (a), the bandpass filter 31 exhibits multiple transmission peaks in the visible light wavelength region VA, approximately 400–510 nm, one transmission peak in the visible light wavelength region VA, and multiple transmission peaks in the visible light wavelength region VA. Furthermore, the bandpass filter 31 also exhibits transmission bands in the near-infrared wavelength region NIRA, specifically in the wavelength regions approximately 820–870 nm and above approximately 880 nm.

[0094] Therefore, even if the image sensor 33 itself has Figure 6 The relative sensitivity of the RGB 3-band as shown in (b) is obtained by... Figure 6 The spectral transmittance characteristics of the bandpass filter 31 shown in (a) are as follows: Figure 5 The image sensor 33 shown has Figure 8 The actual relative sensitivity is shown in Figure (b).

[0095] Figure 5 The relative sensitivity characteristics of the image sensor 33 in the camera 30 shown are as follows: When illuminated by light from an ideal light source with a relative value of 1 across the entire wavelength range, due to the transmission... Figure 6The light in the transmission band of the bandpass filter 31 shown in (a) is filtered by a filter with... Figure 6 As shown in (b), the image sensor 33 with relative sensitivity is exposed to light; therefore, the relative sensitivity of the R, G, and B signals of the image sensor 33 in the camera 30 is... Figure 8 As shown in Figure (b), the relative sensitivity of the image sensor 33 when passing through the bandpass filter 31 depends on... Figure 6 The transmission band of the bandpass filter 31 shown in (a) and Figure 6 The relative sensitivity of image sensor 33 shown in (b) is as follows, while... Figure 8 As shown in (b).

[0096] Figure 7 This represents the emission spectrum of the first light source 21, the second light source 22, and the third light source 23. Figure 7 (a) represents the emission spectrum LR of the first light source 21. Figure 7 (b) represents the emission spectrum LG of the second light source 22. Figure 7 (c) represents the emission spectrum LB of the third light source 23. Figure 7 The emission spectrum LR of the first light source 21 shown in (a) has one peak in the visible light wavelength region VA of about 600 to 670 nm. Figure 7 The emission spectrum LG of the second light source 22 shown in (b) has one peak in the visible light wavelength region of about 470 to 620 nm. Figure 7 The emission spectrum LB of the third light source 23 shown in (c) has one peak in the visible light wavelength region of about 430 to 550 nm. Although the three emission spectra LR, LG and LB have some overlap in the bands below 0.1, their peaks are located in almost different bands and are independent of each other when the luminous intensity is above 0.2.

[0097] Here, Figure 8 (a) represents the emission spectra LR, LG, and LB of the first to third light sources 21 to 23. The emission spectra LR, LG, and LB have one peak in the visible light wavelength region of about 600 to 670 nm, one peak in the visible light wavelength region of about 470 to 620 nm, and one peak in the visible light wavelength region of about 430 to 550 nm.

[0098] like Figure 8As shown in (a), the emission spectra LR, LG, and LB of the three types of light sources 21-23 have overlapping peaks in the visible light wavelength region VA within the multiple transmission bands of the bandpass filter 31. Specifically, the emission spectrum LR of the first light source 21 has a peak in the visible light wavelength region of approximately 600-670 nm, the emission spectrum LG of the second light source 22 has a peak in the visible light wavelength region of approximately 470-620 nm, and the emission spectrum LB of the third light source 23 has a peak in the visible light wavelength region of approximately 430-550 nm.

[0099] Figure 9 (a) represents the relative output characteristics of the image sensor 33 when illuminated by three types of light sources 21-23, each with peaks in wavelength regions distinct from each other across the entire wavelength range. Furthermore, Figure 9 (b) represents the input with Figure 9 The relative sensitivity of the R, G and B signals of (a) is converted into the relative output characteristics of the X, Y and Z signals by the conversion unit 60.

[0100] when Figure 8 When light with emission spectral characteristics of three types of light sources 21-23 is incident as shown in (a), Figure 8 The luminescence intensity characteristics shown in (a) are similar to Figure 8 The relative sensitivity characteristics of the first captured signal S1 shown in (b) are synthesized, resulting in the image sensor 33 outputting... Figure 9 The RGB signal of the 3-band frequency shown in Figure (a) is used as the first shooting signal S1. The conversion unit 60 converts... Figure 9 The 3-band RGB signal shown in Figure (a) is converted into a signal with different spectral sensitivity characteristics than the RGB signal. Figure 9 The XYZ signal with three frequency bands is shown in (b).

[0101] like Figure 9 As shown in Figure (b), the X band has peaks at approximately 600–670 nm, the Y band has peaks at approximately 470–510 nm and approximately 530–570 nm, and the Z band has a peak at approximately 430–510 nm. The sensitivity peaks of the XYZ bands indicate that the sensitivity of each band is higher than that of the other two bands in its respective wavelength region, and the XYZ bands are separated. The X, Y, and Z bands represent mutually separated visible light frequency bands.

[0102] Next, refer to Figure 10 The detailed structure of the conversion unit 60 and the inspection and processing unit 70 will be explained.

[0103] like Figure 10As shown, the image of the object 12 passes through the bandpass filter 31 and the lens 32 and is imaged onto the imaging surface of the image sensor 33. The image sensor 33 outputs a first imaging signal S1, which is the imaging result of the object 12, to the conversion unit 60. The first imaging signal S1 is a serial signal that includes the R imaging signal (red signal), G imaging signal (green signal), and B imaging signal (blue signal) from each of the light-receiving elements 33R, 33G, and 33B. The R imaging signal, G imaging signal, and B imaging signal are simply referred to as the R signal, G signal, and B signal.

[0104] like Figure 10 As shown, the conversion unit 60 includes an RGB separation unit 61 and an XYZ conversion unit 62. The RGB separation unit 61 separates the first image signal S1 input from the image sensor 33 into an R signal, a G signal, and a B signal.

[0105] The XYZ conversion unit 62 converts the R, G, and B signals input from the RGB separation unit 61 into X, Y, and Z signals. Specifically, the XYZ conversion unit 62 performs matrix operations on the RGB values, which are the signal values ​​of the R, G, and B signals, to convert them into X, Y, and Z signals. Matrix coefficients are assigned to the XYZ conversion unit 62. Here, the matrix used for the matrix operation is a 3×3 matrix. The coefficients of the 3×3 matrix are assigned to the XYZ conversion unit 62.

[0106] The XYZ conversion unit 62 performs matrix operations, wherein a specific 3×3 matrix using matrix coefficients is multiplied by the RGB values ​​of the first captured signal S1, and then converted into a second captured signal S2 having different spectral characteristics than the RGB values ​​of the first captured signal S1 and expressed in XYZ. Matrix coefficients are coefficients used to split the RGB values ​​of the first captured signal S1 into multiple frequency bands in the XYZ of the second captured signal S2.

[0107] Here, the formula for converting the RGB signal of the first shooting signal S1 into the XYZ signal of the second shooting signal S2 can be summarized as follows (1).

[0108] Number 1

[0109]

[0110] Here, a1~a3, b1~b3, and c1~c3 are matrix coefficients.

[0111] The XYZ conversion unit 62 performs the operation in equation (1) above, that is, multiplies the 3×3 matrix by the RGB values. The XYZ conversion unit 62 outputs the XYZ values, and further outputs the values ​​after being amplified by the amplification unit.

[0112] Here, the number of colors in the color filter 34 of the image sensor 33 is set to n (where N is a natural number greater than or equal to 3). The matrix operation performed between the n captured signals is an m×n matrix operation (where m is a natural number greater than or equal to 2). The m×n matrix contains matrix coefficients that can separate the captured signals of various colors in the first captured signal S1 into wavelength regions of n frequency bands. In this example, the captured signals of various colors in the first captured signal S1 are R signal, G signal, and B signal, and the number of colors n is "3" (n = 3). In addition, the second captured signal S2 is a 3-band X signal, Y signal, and Z signal, and m = 3. That is, the m×n matrix is ​​a 3×3 matrix. Furthermore, the 3×3 matrix contains matrix coefficients that can improve the separation of the 3 frequency bands.

[0113] For example, when using with Figure 6 The transmittance characteristic curve F1 of (a) shows the transmittance characteristics of bandpass filter 31. Figure 9 (a) transformed into Figure 9 In the matrix operation of (b), the coefficients of the 3×3 matrix are given as follows. In detail, as shown in the calculation formula (2) below, the coefficients of the 3×3 matrix are: a1=1, a2=-0.05, a3=-0.1, b1=-0.27, b2=1, b3=-0.35, c1=-0.1, c2=-0.1, c3=1.2, and the XYZ values ​​can be arranged as follows (2).

[0114] Number 2

[0115]

[0116] This can also be an m×n (m≠n) matrix operation. When the number of colors is "3", it is not limited to a 3×3 matrix operation. A 3×4 matrix operation can also be performed to generate a 4-band multi-band image, or a 3×2 matrix operation can be performed to generate a multi-band image with fewer bands than the number of colors n.

[0117] Alternatively, an amplification unit (figure omitted) can be set up to multiply the XYZ values ​​from the XYZ conversion unit 62 by the X amplification factor Gx, the Y amplification factor Gy, and the Z amplification factor Gz, respectively. The amplification unit can also normalize the frequency band of XYZ calculated by the above formula (1). The normalization process can also be performed by fixing one signal level and adjusting the other two signal levels. For example, the Y signal can be fixed and the X and Z signals can be adjusted.

[0118] In this manner, the conversion unit 60 sequentially performs RGB separation processing and XYZ conversion processing on the input first imaging signal S1, thereby outputting the second imaging signal S2. The conversion unit 60 outputs the second imaging signal S2, a three-band multi-band image of the display item 12, to the inspection processing unit 70. The first imaging signal S1 is composed of N-band R, G, B imaging signals. The second imaging signal S2 is composed of N-band X, Y, Z image signals. The conversion unit 60 converts the N-band R, G, B imaging signals into N-band X, Y, Z image signals. In this example, the three-band R, G, B imaging signals are converted into three-band X, Y, Z image signals. A three-band multi-band image is generated in this way. The conversion unit 60 generates a first image signal XS, a second image signal YS, and a third image signal ZS with spectral sensitivity in the visible light region from the first imaging signal S1.

[0119] The first signal processing unit 66 outputs the first image signal XS to the inspection processing unit 70. The second signal processing unit 67 outputs the second image signal YS to the inspection processing unit 70. The third signal processing unit 68 outputs the third image signal ZS to the inspection processing unit 70. Each of the signal processing units 66-68 may also perform nonlinear processing such as brightness adjustment or contour enhancement as needed, including gamma correction.

[0120] Next, the inspection and processing unit 70 will be explained.

[0121] The inspection processing unit 70 inspects the article 12 based on the first image signal XS, the second image signal YS, and the third image signal ZS, which have spectral sensitivity in the visible light region and are output by the imaging device 15 in the N-band (e.g., 3-band). The inspection processing unit 70 includes a first inspection unit 71, a second inspection unit 72, and a third inspection unit 73. The first inspection unit 71 uses an X-image generated based on the first image signal XS to inspect the first area A1 of the article 12. The second inspection unit 72 uses a Y-image generated based on the second image signal YS to inspect the second area A2 of the article 12. The third inspection unit 73 uses a Z-image generated based on the third image signal ZS to inspect the third area A3 of the article 12. For example, each inspection unit 71 to 73 checks for the presence or absence of printing errors, smudges, dead pixels, and other text errors in the text C1, C2, and C3 in their respective inspection target areas in the first area A1, the second area A2, and the third area A3. In this way, the inspection and processing department 70 determines the quality of item 12 by checking for any textual errors.

[0122] Next, the functions of the imaging device 15 and the inspection device 11 will be explained.

[0123] like Figure 1As shown, item 12 is transported by conveyor 16 of transport device 13. When sensor 17 detects item 12, a trigger signal is input to control unit 50. Control unit 50, which has received the trigger signal, causes the three types of light sources 21-23 to emit light by outputting a light emission command signal, and performs image capture by outputting an image capture command signal. At this time, the first light source 21 illuminates the first area A1 with a light intensity suitable for capturing the text C1 of the first area A1 below the liquid surface LS of container 12A with transmitted light. The second light source 22 illuminates the second area A2 with a light intensity suitable for capturing the image of the reflected light of the text C2 of label 12L affixed to container 12A by illuminating it from its front. The third light source 23 illuminates the third area A3 with a light intensity suitable for capturing the text C3 of the third area A3 above the liquid surface LS of container 12A with transmitted light.

[0124] In camera 30, the image of red light passing through area A1 of item 12, the image of reflected light reflected from the surface of area A2 of label 12L of item 12, and the image of blue light passing through area A3 of item 12 are incident through bandpass filter 31 and lens 32. Each area A1 to A3 is illuminated with light of an amount suitable for capturing the attached text C1 to C3. Therefore, the image of transmitted light passing through area A1 is prone to contain light other than red, the image of reflected light reflected from area A2 is prone to contain light other than green, and the image of transmitted light passing through area A3 is prone to contain light other than blue. When the amount of light from one of the light sources 21 to 23 increases, neighboring areas outside the target area are also illuminated by a lot of light. In addition, other colors of light are sometimes mixed in as they pass through item 12 due to reflection, refraction, and diffusion. The image sensor 33 within the camera 30 forms images of each region A1, A2, and A3 of the object 12 on its imaging surface. The image sensor 33 takes a single shot of the images of each region A1, A2, and A3 of the object 12 formed by light passing through the bandpass filter 31. The image sensor 33 outputs a first image signal S1 of the shooting result to the conversion unit 60.

[0125] The conversion unit 60 performs RGB separation on the first captured signal S1 and multiplies the separated RGB values ​​by a 3×3 matrix, thereby converting the RGB values ​​into XYZ values. For example, each signal processing unit 66-68 performs predetermined signal processing including normalization by multiplying the XYZ values ​​by the amplification rates Gx, Gy, and Gz. The conversion unit 60 generates an N-band XYZ image signal with spectral sensitivity in the visible light region that constitutes the second captured signal S2 from the RGB image signal constituting the first captured signal S1. In this way, a multi-band image of three frequency bands is displayed, and the XYZ image signals constituting the second captured signal S2 (first image signal XS, second image signal YS, and third image signal ZS) are output from the conversion unit 60 to the inspection and processing unit 70. Here, by means of Figure 9 The output characteristics shown allow for the acquisition of an X-image unaffected by G and B rays based on the first image signal XS, and a Z-image unaffected by R and G rays based on the third image signal ZS. The Y-image generated based on the second image signal YS is unaffected by R rays but affected by B rays. However, since B rays illuminate from the back of the object 12 and G rays illuminate from the front, the effect of B rays can be ignored.

[0126] In the inspection processing unit 70, the first inspection unit 71 inspects the item 12 based on the first image signal XS. Furthermore, the second inspection unit 72 inspects the item 12 based on the second image signal YS. Additionally, the third inspection unit 73 inspects the item 12 based on the third image signal ZS of the visible light image.

[0127] Here, the X image XI generated from the first image signal XS, the Y image YI generated from the second image signal YS, and the Z image ZI generated from the third image signal ZS are displayed. Figure 11 An image of item 12 taken under natural light. A container 12A containing liquid LQ is prepared as item 12, and the inspection area of ​​the item is photographed using an imaging device. Figure 11 As shown, a label 12L is affixed to the outer periphery of container 12A. Furthermore, on the outer periphery of container 12A, text is affixed to area A1 below the liquid level LS, and text is printed to area A3 above the liquid level LS. Text is also affixed to label 12L. The item 12 described above is photographed using photographing device 15.

[0128] Figure 12 (a) to (c) respectively represent the images captured by the shooting device 15. Figure 11 The X-image XI, Y-image YI, and Z-image ZI of the photographic results obtained when the item 12 is shown.

[0129] Figure 12Image X1 shown in (a) is an image generated based on the first image signal XS. In image X1, the text in the area below the liquid surface LS of item 12, that is, the first area A1 overlapping with the liquid, is clearly captured. In this example, the light transmittance of the text printed on container 12A is low in the visible light region and high in the infrared region. On the other hand, the light transmittance of the liquid contents of container 12A is low in the blue and green regions and increases with longer wavelengths above the red region. Therefore, since the transmittance of the text and the liquid differs in the red region, the text can be easily identified.

[0130] also, Figure 12 The Y image YI shown in (b) is an image generated based on the second image signal YS. The Y image YI generated based on the second image signal YS is not affected by the R light, but is affected by the B light. However, since the B light is emitted from the back of the article 12 and the G light is emitted from the front, the effect of the B light can be ignored. Therefore, in the Y image YI, the text of the second area A2 attached to the label 12L is clearly captured.

[0131] Figure 12 The Z-image ZI shown in (c) is an image generated based on the third image signal ZS. The Z-image ZI generated based on the third image signal ZS is unaffected by R-light and G-light. That is, there is no influence from R-light, which easily transmits text, and the G-light from the front, reflected from the container surface (illumination), does not hinder text recognition. Therefore, in the Z-image ZI, the text in region A3, the area that does not overlap with the liquid, is clearly captured. Figure 12 (a)~ Figure 12 The images XI, YI, and ZI obtained from the inspection shown in (c) are displayed on the display unit 41.

[0132] In addition, such as Figure 10 As indicated by the double-dotted arrow, two or more of the image signals XS, YS, and ZS from the conversion unit 60 are input to each of the inspection units 71 to 73 of the inspection processing unit 70, and an image for inspection can be generated based on multiple image signals. For example, each of the inspection units 71 to 73 can also acquire a differential image or a contour image formed by multiple image signals. Figure 13 (a) to (c) represent the contour images obtained by each inspection unit 71 to 73 from various image signals XS, YS, and ZS. The first inspection unit 71 generates... Figure 13 The X-contour image XE shown in (a) is used to inspect the text in the first region A1. The second inspection unit 72 generates... Figure 13 The Y-contour image YE shown in (b) is used to inspect the text in the second region A2. The third inspection unit 73 generates... Figure 13The Z-contour image ZE shown in (c) is used to inspect the text in region A3 of the third region.

[0133] In the inspection and processing unit 70, each inspection unit 71 to 73 determines whether the item 12 is good or bad. When the inspection results of each inspection unit 71 to 73 determine that the item 12 is a defective product, the inspection and processing unit 70 drives the removal device to remove the defective item 12 from the conveyor 16.

[0134] This embodiment employs a photographing method. The photographing method includes an illumination step, a photographing step, and a conversion step. In the illumination step, M types of light sources 21 to 23 (where M is a natural number satisfying 2≦M≦N) illuminate the object 12 with visible light of different wavelength regions in the visible light region. In the photographing step, a camera 30 equipped with an image sensor 33 photographs the object 12 using a bandpass filter 31 disposed in the optical path between the image sensor 33 and the object 12. The image sensor 33 has a color filter 34, which is an example of a spectrophotometer with N frequency bands (where N is a natural number of 3 or more) having different spectral transmittance characteristics, and has sensitivity in the visible light region and the near-infrared region. In the conversion step, the image signal captured by the image sensor 33 is separated into N frequency band signals, and matrix operations are performed on the separated N frequency band image signals to generate a 3-band signal formed by a first image signal XS, a second image signal YS, and a third image signal ZS, which has spectrophotometer sensitivity in the visible light region. Light sources 21-23 of type M illuminate areas A1-A3 at point M in item 12 respectively. The direction of illumination and the intensity of light are selected for each area according to the target area. The direction of illumination on item 12 determines whether the image of area A1-A3 at point M captured by camera 30 is transmitted light or reflected light.

[0135] The effects described above are as follows, based on the first embodiment.

[0136] (1) The imaging device 15 includes a camera 30, M types of light sources 21-23 (where M is a natural number satisfying 2≦M≦N), a bandpass filter 31, and a conversion unit 60. The camera 30 includes an image sensor 33, which has a color filter 34 as an example of a spectral filter with N-band (where N is a natural number greater than 3) of different spectral transmittance characteristics and has sensitivity in the visible light and near-infrared regions. The M types of light sources 21-23 have the following emission spectrum characteristics: they have peaks in different wavelength regions in the visible light and near-infrared regions. The bandpass filter 31 is disposed in the optical path between the image sensor 33 and the article 12 and is able to transmit light from the different wavelength regions of the M types of light sources 21-23. The conversion unit 60 separates the image signal obtained by the camera 30 from the object 12 via the image sensor 33 into N-band image signals, and generates M-band image signals with spectral sensitivity in different wavelength regions by performing matrix operations on the separated N-band image signals. M types of light sources 21-23 illuminate regions A1-A3 of the object 12, which are examples of the M-band image areas. The illumination direction and intensity of each region A1-A3 are selected according to their respective illumination directions. The illumination direction determines whether the image of regions A1-A3 at the M-band image captured by the camera 30 is transmitted light or reflected light. M images of regions A1-A3 at the M-band image signals are generated separately. Therefore, since the illumination intensity can be set separately for multiple regions A1-A3 with different optical characteristics in the object 12, images of multiple regions A1-A3 can be captured simultaneously from the same angle using a single camera 30 with a simple structure.

[0137] (2) The camera 30 is a general-purpose color camera 30 with the infrared light cutoff filter removed. Therefore, since the camera 30 is a general-purpose color camera 30, the structure of the shooting device 15 can be simple.

[0138] (3) The M light sources 21 to 23 include two types of light sources, each with the following emission spectrum characteristics: peaks in two different wavelength regions within the visible light region. The M-band image signal is an image signal with three or more bands, including a first image signal, a second image signal, and a third image signal. The first image signal has spectral sensitivity in the first wavelength region of one of the two wavelength regions, the second image signal has spectral sensitivity in the second wavelength region of the other, and the third image signal has spectral sensitivity in a third wavelength region different from both the first and second wavelength regions. Therefore, it is possible to obtain image signals with three or more bands from multiple locations in the item 12 with different optical characteristics captured by illumination of different intensities in a single shot.

[0139] (4) The object 12 photographed by camera 30 includes areas that are translucent. Light sources of type M, 21 to 23, illuminate the object 12 at the same time when different areas of the object 12 are simultaneously illuminated by light of type M. Camera 30 takes a single photograph of the object 12. Therefore, it is possible to take a single photograph of multiple locations of the object 12 with different optical properties using a single camera 30 with a simple structure.

[0140] (5) The M-type light sources 21-23 include at least two of the following: a first light source 21, a second light source 22, and a third light source 23. The first light source 21 is positioned on the side opposite to the camera 30 relative to the article 12; the second light source 22 is positioned on the same side as the camera 30 relative to the article 12; and the third light source 23 is positioned on the side opposite to the camera 30 relative to the article 12. The camera 30 includes at least two of the following: a first image, a second image, and a third image. The first image is an image of transmitted light from the first light source 21 passing through the article 12; the second image is an image of reflected light from the second light source 22 reflected by the article 12; and the third image is an image of transmitted light from the third light source 23 passing through the article 12. Therefore, it is possible to effectively acquire images of at least two locations on the article 12 with different optical characteristics.

[0141] (6) The inspection device 11 includes an imaging device 15 and an inspection processing unit 70 that inspects the article 12 based on the M-band image signal output by the imaging device 15. Therefore, with a simple structure, images of multiple locations with different optical characteristics on the article 12 can be simultaneously captured by a single camera 30 from the same viewing angle. At this time, the M-type light sources 21-23 can be adjusted to appropriate light intensity with almost no consideration of the influence on the light of other areas of the subject. Therefore, multiple locations with different optical characteristics on the article 12 can be inspected with simple processing.

[0142] (7) Article 12 is a container 12A for containing liquid LQ, and includes a light-transmitting area. The inspection processing unit 70 inspects the text attached to the outer peripheral surface of container 12A. At least two of the following inspections are performed: inspection of the text C1 attached to the part of container 12A that overlaps with liquid LQ, inspection of the text C2 of label 12L affixed to the outer peripheral surface of container 12A, and inspection of the text C3 of the part of container 12A that does not overlap with liquid LQ. Therefore, multiple locations with different optical properties in article 12 can be appropriately inspected with a simple structure.

[0143] (8) The shooting method for generating an image signal by photographing the object 12 with the camera 30 includes a light illumination step, a shooting step, and a conversion step. Therefore, according to this shooting method, the same effect as that of the shooting device 15 can be obtained.

[0144] (Second Implementation)

[0145] Next, regarding the second embodiment, refer to Figure 14 and Figure 15 The following is an example of the second implementation method: Figure 1 and Figure 2 In this embodiment, one of the M-type light sources 21 to 23 is configured as a near-infrared light source that irradiates near-infrared light. In this example, either the first light source 21 or the third light source 23 is configured as a near-infrared light source that irradiates the article 12. The example described below is a configuration where the first light source 21, capable of illuminating the first region A1, is configured as a near-infrared light source, allowing the first region A1 to capture images of impurities in the liquid LQ within the container 12A. The camera 30 has the same structure as in the first embodiment described above.

[0146] Figure 1 , Figure 2 In this setup, the near-infrared light source of the first light source 21 is located on the opposite side of the camera 30, across from the object 12 which is at the shooting position. The first light source 21 illuminates a first region A1 of the object 12 with near-infrared light. The camera 30 captures an image of the near-infrared light transmitted through the first region A1 of the object 12. By using the transmitted near-infrared light to photograph the object 12, the presence of impurities within the object 12 can be inspected.

[0147] The first light source 21, serving as a near-infrared light source, has an emission spectrum in the near-infrared wavelength region (NIRA) and no emission spectrum in the visible light region. The first light source 21 has a first light-emitting part 21a that emits near-infrared light. The first light-emitting part 21a is, for example, composed of an LED.

[0148] The emission spectrum of the first light source 21 has a peak in the transmission band of the bandpass filter 31. According to Figure 14 The transmittance characteristic curve F2 of the bandpass filter 31 shown in (a) has a peak in the near-infrared wavelength region (NIRA) at approximately 820–870 nm, and a transmission region with approximately 100% transmittance above approximately 880 nm. Furthermore, the emission spectrum of the first light source 21 in this embodiment has, for example, a peak in the range of approximately 820–870 nm.

[0149] Figure 14 (b) represents the X, Y, and Z signals, which are input to the conversion unit 60 and converted and output after having the aforementioned relative sensitivities of the R, G, and B signals. When light combining the emission spectra of the three types of light sources 21-23 is incident, the image sensor 33 outputs a three-band RGB signal synthesized from the emission intensity characteristics of the three types of light sources 21-23 and the relative sensitivity characteristics of the image sensor 33. The conversion unit 60, which receives the first image capture signal S1, converts it into... Figure 14 Figure (b) shows the 3-band XYZ signal.

[0150] like Figure 14 As shown in the figure, the X band has a peak at approximately 830–860 nm, the Y band has a peak at approximately 600–670 nm, and the Z band has a peak at approximately 530–550 nm. The sensitivity peaks of the XYZ bands are higher than the relative sensitivity of the other two bands in their respective wavelength regions, and the XYZ bands are mutually separated. The Y and Z bands are mutually separated visible light bands, and the X band is a near-infrared light band separated from the two visible light bands.

[0151] Figure 15 This represents an X-ray image (XI) captured using near-infrared light. For example... Figure 15 As shown, image X11 is an image of the first region A1 below the liquid surface LS in the container 12. The text in the third region A3, located above the liquid surface LS, is almost indistinguishable. The image is transmitted through the liquid in the first region A1 below the liquid surface LS in the item 12, but the label 12L is not visible to near-infrared light and appears as a shadow (black). Therefore, impurities in the liquid can be inspected in the area of ​​the item 12 that does not overlap with the label 12L.

[0152] According to the second embodiment, in addition to obtaining the same effects as the first embodiment described above, the following effects can also be obtained. (9)

[0154] The M-type light sources 21-23 are light sources with emission spectral characteristics within a predetermined wavelength region in the near-infrared region. The M-band image signal is an image signal with spectroscopic sensitivity within a predetermined wavelength region in the near-infrared region. Therefore, it is possible to obtain an M-band image signal containing image signals of the parts of the article 12 that can be photographed with near-infrared light illumination in a single shot.

[0155] <Example of Change>

[0156] The implementation method is not limited to the above, and may also be modified to the following forms.

[0157] • The first embodiment described above uses three types of light sources 21 to 23, but it can also be a structure that uses two types of light sources. For example, it can also be a combination of the first light source 21 and the second light source 22, a combination of the first light source 21 and the third light source 23, or a combination of the second light source 22 and the third light source 23.

[0158] The second embodiment described above is an example of a combination of the arrangement positions of the three light sources. In a structure where one of the three light sources is a near-infrared light source, other combinations related to the arrangement positions of the three light sources can also be used. For example, it could be a combination where the first light source 21 is a near-infrared light source, or a combination where the third light source 23 is a near-infrared light source, etc.

[0159] The second embodiment described above uses three light sources, but a structure using only two light sources is also possible. For example, it could be a combination of the first light source 21 and the second light source 22, a combination of the first light source 21 and the third light source 23, or a combination of the second light source 22 and the third light source 23. Furthermore, one of the two light sources can be a visible light source and the other a near-infrared light source. For example, it could be a combination of the first light source 21 being a visible light source and the second light source 22 being a near-infrared light source; a combination of the second light source 22 being a near-infrared light source and the third light source 23 being a visible light source; or a combination of the first light source 21 being a visible light source and the third light source 23 being a near-infrared light source.

[0160] • In the second embodiment described above, the first near-infrared light source 21 is disposed on the back side of the article 12, but the second light source 22 disposed on the front side of the article 12 can also be a near-infrared light source. The ink printed on the outer peripheral surface of the article 12 or on the label 12L can also include ink that reflects near-infrared light. The inspection device 11 with the imaging device 15 can also be a device for inspecting the ink that reflects infrared light printed on the object to be photographed. At this time, the camera 30 can also be a structure for photographing the near-infrared light reflected from the surface of the article 12. The surface of the article 12 can also be configured to have a printing area printed using ink that is a mixture of ink that reflects visible light and ink that reflects infrared light, or it can be configured to have a first printing area printed using ink that reflects visible light and a second printing area printed using ink that reflects near-infrared light.

[0161] • The M types (e.g., 3 types) of light sources 21 to 23 illuminate the different areas A1 to A3 of the subject separately, but as long as the area that is the subject can be illuminated, the other areas can also be illuminated.

[0162] • The first embodiment described above is as follows Figure 9 As shown in (b), there are other frequency bands Y in the Z band with a relative sensitivity of 0.2 or higher. However, as long as it does not adversely affect the image sharpness of each region A1 to A3, other frequency bands with a relative sensitivity of 0.2 or higher can also exist in the X or Y bands. In addition, the characteristics or matrix coefficients of the bandpass filter 31 can be selected so that the relative sensitivity of other frequency bands is less than 0.1.

[0163] In the above embodiments and modifications, the light source is not limited to three types. As long as it can irradiate visible light or near-infrared light with different emission spectra, it can also be four or more types or two types. In addition, the types of light-emitting parts constituting the near-infrared light source can also be one type, two types, or three or more types.

[0164] The value of N is not limited to N=3; it can also be 4 or higher. For example, in a color camera with a primary color filter, it can use 4 colors: R, G1, G2, and B (N=4). In addition, a color camera with a complementary color filter can also be used, and the complementary colors can be 4 colors: yellow, cyan, magenta, and green.

[0165] The image data (e.g., RGB image data) of the first captured signal obtained by the image sensor 33 using the camera 30 through the bandpass filter 31 can be stored in a removable memory such as a USB memory. Alternatively, the image data stored in this removable memory can be read by a personal computer, and the CPU (conversion unit 60) of the personal computer can perform conversion processing including matrix operations to generate a multi-band image with multiple frequency bands. In other words, the device performing the capturing step and the device performing the conversion step can be separate devices. Even with this capturing method, multi-band images with multiple frequency bands can be obtained.

[0166] • The conversion unit 60 may also omit matrix operations. For example, when the optimal matrix coefficients are an identity matrix (100, 010, 001), in the above embodiment, matrix operations are performed for conversion. However, since the result obtained from the conversion is the same as before the conversion, it is equivalent to no conversion, so the matrix operations can be omitted. In other words, the imaging device may also be configured such that, although a conversion unit is provided, matrix operations are omitted when the matrix coefficients are, for example, an identity matrix, in order to reduce the burden of computational processing.

[0167] • The imaging device 15 is not limited to inspection. For example, it can also be configured as the robot's eye, allowing the robot to distinguish and recognize images of visible light and infrared light.

[0168] • It can also be a structure that is inspected by visual confirmation of the image output by the imaging device 15.

[0169] The number of colors in the color filter 34 constituting the image sensor 33 is not limited to 3 or 4 colors, but can also be 5, 6, 7, or 8 colors. It can also be a filter in which at least one color does not transmit visible light but transmits non-visible light. For example, the image sensor 33 can also be equipped with a color filter that includes a NIR filter that transmits near-infrared light.

[0170] • Item 12, as an example of the object being photographed or inspected, is not limited to containers such as PET bottles or jars. Item 12 can also be, for example, food, beverages, electronic components, electrical products, daily necessities, parts, components, powders, or liquids. Item 12 is any item that can be inspected for quality using multi-band imaging.

[0171] • Liquids can also be opaque. For example, a liquid can be an opaque liquid of a specific color. Such liquids can also be black sauces or other condiments, specific colored beverages, specific colored medicines, or specific colored oils. In addition, liquids can also contain non-flowing gels.

[0172] • The array pattern of the color filter constituting the image sensor 33 is not limited to an RGB Bayer array, but can also be any array pattern such as a stripe array.

[0173] • At least one of the control unit 50, the conversion unit 60 and the inspection and processing unit 70 may be composed, in part or in whole, of software formed by a computer executing the program, or of hardware such as electronic circuits.

[0174] • The camera device 15 can also be used for purposes other than inspection.

[0175] Symbol Explanation

[0176] 10 Inspection System

[0177] 11 Inspection Device

[0178] 12 items (one example of the items being photographed)

[0179] 12A container

[0180] 12B cover

[0181] 12L label

[0182] 13. Handling device

[0183] 15 camera devices

[0184] 16 conveyors

[0185] 17 sensors

[0186] 21 First Light Source

[0187] 21a First Light-Emitting Section

[0188] 22 Second Light Source

[0189] 22a Second Light-Emitting Section

[0190] 23 Third Light Source

[0191] 23a Third Light-Emitting Part

[0192] 30 cameras

[0193] 30a lens tube

[0194] 31. Bandpass filter (an example of a filter)

[0195] 32 lenses

[0196] 33 Color Image Sensor (Image Sensor)

[0197] 33R R light-receiving element

[0198] 33G G light receiving element

[0199] 33B B light-receiving element

[0200] 34. Color filter (an example of a spectrophotometer)

[0201] 34R R filter

[0202] 34G G filter

[0203] 34B B filter

[0204] 40 Control Processing Unit

[0205] 41 Display Section

[0206] 50 Control Department

[0207] 60 conversion unit

[0208] 61RGB Separator

[0209] 62XYZ converter

[0210] 70 Inspection and Processing Department

[0211] 71 First Inspection Department

[0212] 72nd Inspection Department

[0213] 73 Third Inspection Department

[0214] 201 Infrared Cut-off Filter (IR Cut-off Filter)

[0215] LR emission spectrum

[0216] LG Emission Spectrum

[0217] LB emission spectrum

[0218] Area A1, Zone 1 (An example of the area to be photographed)

[0219] Area A2, Section 2 (An example of the area to be photographed)

[0220] Area 3 of A3 (an example of the area to be photographed)

[0221] LQ liquid

[0222] C1 1st text

[0223] C2 2nd text

[0224] C3 3rd text

[0225] S1 First Shooting Signal

[0226] S2 Second Shooting Signal

[0227] XS First Image Signal

[0228] YS second image signal

[0229] ZS third image signal

[0230] VA visible light wavelength region

[0231] NIRA near-infrared wavelength region

Claims

1. A photographing apparatus, comprising: have: The camera is equipped with an image sensor having an N-band spectral filter with different spectral transmittance characteristics and sensitivity in the visible and near-infrared regions, wherein N is a natural number greater than 3. M types of light sources each have the following emission spectrum characteristics: they have peaks in different wavelength regions in the visible light region and the near-infrared region, where M is a natural number that satisfies 2≦M≦N; A filter is disposed in the optical path between the image sensor and the subject, and is capable of transmitting light from the different wavelength regions of the M types of light sources; as well as The conversion unit separates the image signal obtained by the image sensor when the camera captures the subject into N-band image signals, and generates M-band image signals with spectral sensitivity in different wavelength regions by performing matrix operations on the separated N-band image signals. The M types of light sources illuminate the subject area at point M of the subject, and the illumination direction and luminous intensity of the subject are selected according to the subject area. The illumination direction of the subject determines whether the image of the subject area at point M captured by the camera is transmitted light or reflected light.

2. The imaging device as described in claim 1, wherein, The camera is a general-purpose color camera with the infrared cutoff filter removed.

3. The imaging device as described in claim 1, wherein, The M-type light source includes two types of light sources, each with the following emission spectral characteristics: peaks are present in two distinct wavelength regions within the visible light region. The M-band image signal is an image signal with more than 3 bands, including a first image signal, a second image signal, and a third image signal. The first image signal has a splitting sensitivity in the first wavelength region of one of the two wavelength regions, the second image signal has a splitting sensitivity in the second wavelength region of the other region, and the third image signal has a splitting sensitivity in a third wavelength region that is different from both the first and second wavelength regions.

4. The imaging device as described in claim 1, wherein, The M-type light source includes light sources that have emission spectral characteristics in a predetermined wavelength region within the near-infrared region. The image signal in the M-band includes the following image signal: it has spectral sensitivity in the predetermined wavelength region of the near-infrared region.

5. The imaging device as claimed in claim 1, wherein, The object being photographed by the camera includes areas that are translucent. The M-type light source illuminates the subject area at the M-type location during the period when the subject is simultaneously illuminated. The camera takes a single shot of the subject.

6. The imaging device as described in claim 5, wherein, The M-type light source includes at least two of a first light source, a second light source, and a third light source. The first light source is positioned on the side opposite to the camera relative to the subject, the second light source is positioned on the same side as the camera relative to the subject, and the third light source is positioned on the side opposite to the camera relative to the subject. The camera captures at least two of the following images: a first image, a second image, and a third image. The first image is an image of transmitted light from the first light source through the subject. The second image is an image of reflected light from the second light source reflected by the subject. The third image is an image of transmitted light from the third light source through the subject.

7. An inspection apparatus wherein, have: The imaging device according to any one of claims 1 to 6; and The inspection and processing unit inspects the photographed object based on the image signal of the M-band output by the imaging device.

8. The inspection apparatus as claimed in claim 7, wherein, The object being photographed is a container holding liquid, which includes a translucent area. The inspection and processing unit inspects the text attached to the outer circumference of the container. The inspection device performs at least two of the following inspections: Inspection of the text attached to the portion of the container that overlaps with the liquid; inspection of the text on the portion of the container that does not overlap with the liquid; and inspection of the text on the label affixed to the outer periphery of the container.

9. A shooting method, wherein an image signal is generated by photographing a subject with a camera, wherein, It includes lighting, shooting, and conversion steps. In the illumination step, the subject is illuminated by M types of light sources, each of which has the following emission spectrum characteristics: peaks in different wavelength regions within the visible light and near-infrared regions. M is a natural number satisfying 2 ≤ M ≤ N, and N is a natural number greater than or equal to 3. In the shooting step, the camera captures the subject area at point M of the subject through a filter. The camera is equipped with an image sensor that has a spectroscopic filter with different spectral transmittance characteristics in the N-band and has sensitivity in the visible and near-infrared regions. The filter can transmit light from the different wavelength regions of the M-type light sources. In the conversion step, the image signal captured by the image sensor is separated into N-band image signals, and matrix operations are performed on the separated N-band image signals to generate M-band image signals with spectral sensitivity in different wavelength regions. The M types of light sources illuminate the subject area at point M of the subject, and the illumination direction and luminous intensity of the subject are selected according to the subject area. The illumination direction of the subject determines whether the image of the subject area at point M captured by the camera is transmitted light or reflected light.

Images