Optical member, processing device, processing method, and program

Abstract

This optical member comprises a first part having an attachment portion that is attached to a lens device having an optical system, and a plurality of second parts that are provided on the first part. Each of the second parts has a first opening and a second opening. The plurality of second parts include a third part in which the first opening is provided at an angle to the second opening.

Description

Optical member, processing device, processing method, and program

[0001] The technology of the present disclosure relates to an optical member, a processing device, a processing method, and a program.

[0002] Japanese Patent Application Laid-Open Publication No. 2016-183957 discloses a spectrometer for measuring the spectral characteristics of an object. The spectrometer includes a front optical element, a birefringent optical element, a camera, and a processor in combination. The front optical element includes a diffuser that receives light from the object. The birefringent optical element receives the light from the diffuser and generates interference fringes. The camera receives the interference fringes. The processor generates the spectral characteristics of the object.

[0003] Japanese Patent Laid-Open Publication No. 08-285688 discloses a spectral image analysis device including a band-pass interference filter, an imaging means, a first optical means, a second optical means, a subtraction means, and a display means. The band-pass interference filter is capable of changing the light transmission band. The imaging means has a plurality of light-receiving cells. The first optical means forms an image of an object on the interference filter. The second optical means forms an image of the interference filter on each of the light-receiving cells of the imaging means. The subtraction means calculates, for each corresponding light-receiving cell, the difference between a first output of each light-receiving cell when the interference filter is set to a first transmission band and a second output of each light-receiving cell when the interference filter is set to a second transmission band different from the first transmission band. The display means visually displays the output of the subtraction means.

[0004] Japanese Patent Application Laid-Open Publication No. 2008-165806 discloses an image processing system including an external illumination unit, an image capture unit, and an image processing unit. The external illumination unit is equipped with multiple illumination light sources having different spectral distribution characteristics. The image capture unit includes an imaging optical system for capturing an image of a subject, an image sensor unit for acquiring a subject signal from the subject, a capture operation unit for performing image capture operations, a connection contact unit for interlocking with the external illumination unit, and a spectrum detection unit for detecting the spectrum of light from the external illumination unit. The image capture unit synchronizes the multiple illumination light sources with the exposure timing of the image sensor unit and selectively turns on the multiple illumination light sources to obtain multiple subject spectral images. The image processing unit includes an image memory unit for storing the subject spectral images captured by the image capture unit, and performs desired image calculations based on the subject spectral images stored in the image memory unit and the spectral data acquired by the spectrum detection unit. The external illumination unit is detachably attached to the image capture unit.

[0005] International Publication No. 2022 / 102295 discloses an imaging device including a light source spectroscopic sensor that performs wavelength separation of light from a light source that illuminates a measurement target and detects light for each wavelength, and a multispectral camera that performs wavelength spectroscopic imaging of the measurement target. The light source spectroscopic sensor is configured to be able to separate more wavelengths than the multispectral camera.

[0006] One embodiment of the technology disclosed herein provides an optical element, a processing device, a processing method, and a program that can contribute to efficient measurement of the spectrum of a light source, compared to, for example, sequentially measuring the spectrum at multiple angles relative to the light source.

[0007] A first aspect of the technology of the present disclosure is an optical element comprising a first part having an attachment portion to be attached to a lens device having an optical system, and a plurality of second parts provided on the first part, each of the second parts having a first opening and a second opening, and the plurality of second parts including a third part in which the first opening is angled relative to the second opening.

[0008] A second aspect of the technology of the present disclosure is an optical member comprising a first member having an attachment portion to be attached to a lens device having an optical system, and a plurality of opening portions provided in the first member, each opening portion having a first opening and a second opening, and the plurality of opening portions including an opening portion in which the first opening is angled relative to the second opening.

[0009] A third aspect of the technique of the present disclosure is an optical element according to the first aspect, wherein the plurality of second portions include a fourth portion in which the first opening opens in the optical axis direction of the lens device.

[0010] A fourth aspect of the technology of the present disclosure is an optical element according to the third aspect, wherein the plurality of second portions include a plurality of third portions, and the plurality of third portions are arranged around the fourth portion.

[0011] A fifth aspect of the technique of the present disclosure is the optical member according to the third or fourth aspect, wherein the fourth portion is disposed in a central portion of the first portion.

[0012] A sixth aspect of the technology of the present disclosure is an optical element according to any one of the third to fifth aspects, wherein the plurality of second portions include a plurality of third portions, and the plurality of third portions have a plurality of fifth portions arranged in a ring shape around the fourth portion, and a plurality of sixth portions arranged in a ring shape around the fourth portion outside the plurality of fifth portions.

[0013] A seventh aspect of the technique of the present disclosure is the optical member according to the sixth aspect, wherein the angle of the sixth portion is larger than the angle of the fifth portion.

[0014] An eighth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to seventh aspects, wherein each second portion is an optical element formed in a cylindrical shape.

[0015] A ninth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to eighth aspects, wherein the first opening is open on a side other than the lens device side, and the second opening is open on the lens device side.

[0016] A tenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to ninth aspects, wherein the third portion is an optical element that is inclined with respect to the optical axis direction of the lens device.

[0017] An eleventh aspect of the technology of the present disclosure is an optical element according to any one of the third to seventh aspects and the eighth to tenth aspects dependent on the third aspect, wherein the fourth portion is an optical element extending in the optical axis direction of the lens device.

[0018] A twelfth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to eleventh aspects, wherein the angle is an angle corresponding to at least one of the zenith angle and the azimuth angle.

[0019] A thirteenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to twelfth aspects, wherein the plurality of second portions are arranged corresponding to at least one of the zenith angle and the azimuth angle.

[0020] A fourteenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to thirteenth aspects, wherein a diffusion plate that diffuses light is disposed in the second opening.

[0021] A fifteenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to fourteenth aspects, wherein the plurality of second portions include adjacent seventh portions, and a portion of the range of incident angles of light limited by one of the seventh portions overlaps with a portion of the range of incident angles of light limited by the other of the seventh portions.

[0022] A sixteenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to fifteenth aspects, wherein the first shape, which is the shape of the first opening, is a rectangular shape or an arc shape.

[0023] A seventeenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to sixteenth aspects, wherein the first shape, which is the shape of the first opening, is similar to the second shape, which is the shape of the second opening.

[0024] An 18th aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to seventeenth aspects, wherein the range of incident angles of light limited by the plurality of second portions is set to a first incident angle range.

[0025] A nineteenth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to eighteenth aspects, wherein the lens device is an optical element that is a lens device of a spectroscopic imaging device.

[0026] A twentieth aspect of the technology of the present disclosure is an optical element according to the first aspect and any one of the third to nineteenth aspects, comprising a first optical element having a first portion and a plurality of second portions, and a second optical element that fixes the first optical element at a position away from the object side of the lens device.

[0027] A 21st aspect of the technology of the present disclosure is a processing device that includes a processor, which acquires a subject image obtained by capturing an image of the subject using a first imaging device, acquires a light source image obtained by capturing an image of the light source using a second imaging device having an optical element related to any one of the first to twentieth aspects, and derives the spectral reflectance of the subject when light is irradiated from the light source at a first angle and the subject is captured at a second angle.

[0028] A 22nd aspect of the technology of the present disclosure is a processing device according to the 21st aspect, wherein the spectral reflectance is derived based on an image of the subject, an image of the light source, and a bidirectional reflectance distribution function of the subject.

[0029] A 23rd aspect of the technology of the present disclosure is a processing device according to the 21st or 22nd aspect, in which the light source image includes information regarding the wavelength and intensity of light for at least one of the zenith angle and the azimuth angle.

[0030] A 24th aspect of the technology of the present disclosure is a processing device according to the 22nd aspect and the 23rd aspect dependent on the 22nd aspect, in which the processor derives a first reflection coefficient, which is the reflection coefficient of the subject, based on an image of the subject, and derives bidirectional reflectance distribution function coefficients included in the bidirectional reflectance distribution function based on the first reflection coefficient depending on the type of the subject, and deriving the spectral reflectance of the subject includes deriving a second reflection coefficient, which is the reflection coefficient of the subject at a second angle when light is irradiated onto the subject from a first angle, based on a light source spectral function based on the spectrum of the light source determined based on the light source image and the bidirectional reflectance distribution function coefficients.

[0031] A 25th aspect of the technology of the present disclosure is a processing method comprising: acquiring a subject image obtained by capturing an image of the subject using a first imaging device; acquiring a light source image obtained by capturing an image of a light source using a second imaging device having an optical element according to any one of the first to twentieth aspects; and deriving the spectral reflectance of the subject when irradiating the subject with light from the light source at a first angle and capturing an image of the subject from a second angle.

[0032] A 26th aspect of the technology of the present disclosure is a program for causing a computer to execute processing including: acquiring a subject image obtained by capturing an image of the subject using a first imaging device; acquiring a light source image obtained by capturing an image of the light source using a second imaging device having an optical element according to any one of the first to twentieth aspects; and deriving the spectral reflectance of the subject when irradiating the subject with light from the light source at a first angle and capturing an image of the subject from a second angle.

[0033] 5 is a block diagram showing an example of an imaging system. FIG. 6 is a perspective view showing an example of a subject camera. FIG. 7 is a perspective view showing an example of a light source camera. FIG. 8 is a perspective view showing an example of an attachment. FIG. 9 is a plan view showing an example of an attachment. FIG. 10 is a cross-sectional view taken along line F6-F6 in FIG. 5. FIG. 11 is a cross-sectional view showing an example of a first incident angle range and a second incident angle range. FIG. 12 is a longitudinal cross-sectional view showing an example of an attachment and a spacer. FIG. 13 is a block diagram showing an example of a hardware configuration of a multispectral camera. FIG. 14 is an exploded perspective view showing an example of a pupil division filter. FIG. 15 is an exploded perspective view showing an example of a portion of a photoelectric conversion element. FIG. 16 is a block diagram showing an example of a functional configuration of a multispectral camera. FIG. 17 is a block diagram showing an example of the operation of an output value acquisition unit and an interference removal processing unit. FIG. 18 is a block diagram showing an example of the functional configuration of a processing device. FIG. 19 is a block diagram showing an example of the operation of an image acquisition unit. FIG. 19 is a block diagram showing an example of the operation of a light source spectral distribution derivation unit. FIG. 19 is a block diagram showing an example of the operation of a spectral reflectance derivation unit. FIG. 19 is a block diagram showing an example of detailed operation of the spectral reflectance derivation unit. FIG. 19 is a flowchart showing an example of the flow of a spectral image generation process. FIG. 19 is a flowchart showing an example of the flow of a spectral reflectance derivation process. FIG. 19 is a flowchart showing an example of the flow of a reflection coefficient derivation process. FIG. 19 is a plan view showing an example of a modified example of a cylindrical portion.

[0034] Hereinafter, examples of embodiments of an optical member, a processing device, a processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0035] First, the terms used in the following description will be explained.

[0036] LED is an abbreviation for "light emitting diode". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". I / F is an abbreviation for "Interface". RAM is an abbreviation for "Random Access Memory". CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". HDD is an abbreviation for "Hard Disk Drive". BRDF is an abbreviation for "Bidirectional Reflectance Distribution Function". RGB is an abbreviation for "Red Green Blue". TPU is an abbreviation for "Tensor processing unit". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus." ASIC is an abbreviation for "Application Specific Integrated Circuit." FPGA is an abbreviation for "Field-Programmable Gate Array." PLD is an abbreviation for "Programmable Logic Device." SoC is an abbreviation for "System-on-a-Chip." IC is an abbreviation for "Integrated Circuit."

[0037] In the description of this specification, "same" refers to the same in the sense of including, in addition to being completely the same, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure. In the description of this specification, "orthogonal" refers to the same in the sense of including, in addition to being completely orthogonal, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure. In the description of this specification, "straight line" refers to the same in the sense of including, in addition to being a perfect straight line, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure. In the description of this specification, "plane" refers to the same in the sense of including, in addition to being a perfect plane, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure.

[0038] 1 , an imaging system S includes a subject camera 1, a light source camera 2, a light source 3, and a processing device 4. The imaging system S is a system in which the processing device 4 derives the spectral reflectance of a subject 5 when the subject 5 is irradiated with light from the light source 3 at a first angle and the subject 5 is imaged by the subject camera 1 at a second angle. The first angle is defined by a zenith angle θ1 and an azimuth angle φ1, and the second angle is defined by a zenith angle θ2 and an azimuth angle φ2.

[0039] The Ax-axis direction, the Ay-axis direction, and the Az-axis direction are perpendicular to each other. The Ax-axis direction, the Ay-axis direction, and the Az-axis direction may be either a horizontal direction or a vertical direction. ０ is the normal to the surface of the object 5, and the line L １ is a line connecting the light source 3 and the subject 5, and line L ２ is a line connecting the captured camera 1 and the subject 5. ０ In the example shown in FIG. ０ , line L １ , and line L ２ intersect at point P on the surface of the subject 5. The coordinates of point P in the XY coordinate system are expressed as coordinates (x, y). Hereinafter, point P on the surface of the subject 5 will be referred to as "subject position (x, y)."

[0040] The zenith angle θ1 is the angle of the line L when viewed from the Ay axis direction. ０ and line L １ The azimuth angle φ1 is the angle between the line L １ The zenith angle θ2 is the angle around the line L when viewed from the Ay axis direction. ０ and line L ２ The azimuth angle φ2 is the angle between the line L ２ The zenith angle θ1, the azimuth angle φ1, the zenith angle θ2, and the azimuth angle φ2 are all angles at the subject position (x, y).

[0041] The captured camera 1 and the light source 3 are arranged at positions spaced apart in the Az-axis direction from the subject 5 when viewed from the Ay-axis direction. The captured camera 1 and the light source 3 are also arranged at positions spaced apart from each other in the Ax-axis direction when viewed from the Ay-axis direction. The light source camera 2 is, for example, positioned at a position along the line L １ is placed on top.

[0042] The light source camera 2 is located along the line L １ The light source camera 2 may be located at a position away from above, but in the vicinity of the subject 5. The vicinity of the subject 5 is, for example, a position where the light source spectral distribution (see FIG. 16 ), which will be described later, acquired by the light source camera 2 falls within an error range when the light source camera 2 is located at the subject position (x, y) and when the light source camera 2 is located in the vicinity of the subject 5.

[0043] The light source 3 may be any type of light source. Examples of the light source 3 include an LED light source, a laser light source, and an incandescent light bulb. The light source 3 may be the sun or a reflective member that reflects light emitted from another light source. The light emitted from the light source 3 is unpolarized.

[0044] The object camera 1 is an example of a "first imaging device" according to the technology of the present disclosure. The light source camera 2 is an example of a "second imaging device" according to the technology of the present disclosure. The light source 3 is an example of a "light source" according to the technology of the present disclosure. The processing device 4 is an example of a "processing device" according to the technology of the present disclosure.

[0045] 2, the photographic camera 1 includes a lens device 12 and an image capture device body 14. The lens device 12 has a pupil division filter 16 that splits incident light into multiple wavelength bands. The photographic camera 1 is a multispectral camera that captures light split into multiple wavelength bands by the pupil division filter 16, thereby generating and outputting spectral images 72A to 72C.

[0046] In this embodiment, as an example, spectral images 72A to 72C generated based on light dispersed into three wavelength bands will be described as multiple spectral images. Three wavelength bands is merely an example, and four or more wavelength bands may be used. In other words, the subject camera 1 may be a multispectral camera capable of capturing an image of a subject with a higher wavelength resolution than a multispectral camera capable of capturing light dispersed into three wavelength bands.

[0047] Furthermore, the spectral images 72A to 72C may include images obtained by capturing light in the visible light band, and may also include images that visualize light in wavelength bands that cannot be perceived by the human eye (e.g., near-infrared band and / or ultraviolet band, etc.).

[0048] 3, the light source camera 2 includes a lens device 12, an image capture device body 14, and an optical member 130. The lens device 12 and the image capture device body 14 have the same configuration as the lens device 12 and the image capture device body 14 of the object camera 1 (see FIG. 2). The optical member 130 includes an attachment 132 and a spacer 134. The spacer 134 is attached to the object-side end of the lens device 12, and the attachment 132 is attached to the object-side end of the spacer 134.

[0049] The optical member 130 is an example of an "optical member" according to the technology of the present disclosure. The attachment 132 is an example of a "first optical member" according to the technology of the present disclosure. The spacer 134 is an example of a "second optical member" according to the technology of the present disclosure. The light source camera 2 is an example of a "spectroscopic imaging device" according to the technology of the present disclosure. The lens device 12 is an example of a "lens device" according to the technology of the present disclosure.

[0050] As shown in FIGS. 4 and 5 as an example, the attachment 132 has a base portion 136 and a plurality of cylindrical portions 138. As an example, the base portion 136 is formed in a flat plate shape. When viewed from above in the axial direction of the base portion 136, the base portion 136 is formed in a circular shape. The plurality of cylindrical portions 138 are provided on the base portion 136. The plurality of cylindrical portions 138 extend from the base portion 136 toward the object side. As an example, the plurality of cylindrical portions 138 are formed integrally with the base portion 136. Note that the plurality of cylindrical portions 138 may be separate from the base portion 136. Each cylindrical portion 138 is formed in a cylindrical shape. The base portion 136 is an example of a "first portion" and a "first member" according to the technology of the present disclosure. The plurality of cylindrical portions 138 are an example of a "plurality of second portions" and a "plurality of openings" according to the technology of the present disclosure.

[0051] The plurality of cylindrical portions 138 are arranged corresponding to the zenith angle θ1 and the azimuth angle φ1 (see FIG. 1 ). Specifically, the plurality of cylindrical portions 138 include a cylindrical portion 138A arranged in the center of the base portion 136, a plurality of cylindrical portions 138B arranged in a ring shape around the cylindrical portion 138A on the outside of the cylindrical portion 138A, and a plurality of cylindrical portions 138C arranged in a ring shape around the cylindrical portion 138A on the outside of the plurality of cylindrical portions 138B.

[0052] Cylindrical portion 138A, cylindrical portion 138B, and cylindrical portion 138C are arranged side by side in the direction of zenith angle θ1 (i.e., the radial direction of base portion 136). Furthermore, the multiple cylindrical portions 138B are arranged in an annular shape in the direction of azimuth angle φ1 (i.e., the circumferential direction of base portion 136). Similarly, the multiple cylindrical portions 138C are also arranged in an annular shape in the direction of azimuth angle φ1.

[0053] As an example, the number of cylindrical portions 138A is 1, the number of the plurality of cylindrical portions 138B arranged around the cylindrical portion 138A is 6, and the number of the plurality of cylindrical portions 138C arranged around the plurality of cylindrical portions 138B is 8. Note that the number and arrangement of the plurality of cylindrical portions 138 may be other than those described above.

[0054] 6 is a cross-sectional view taken along line F6-F6 in FIG. 5. As shown in FIG. 6 as an example, the base portion 136 has a first surface 142 and a second surface 144. The first surface 142 is the surface on the object side of the base portion 136, and the second surface 144 is the surface on the image side of the base portion 136. As an example, the first surface 142 and the second surface 144 are flat surfaces. The first surface 142 and the second surface 144 are each surfaces that are perpendicular to the central axis AC of the base portion 136. The central axis AC is an axis that passes through the center of the base portion 136 and extends in the axial direction of the base portion 136. The axial direction of the base portion 136 coincides with the optical axis direction of the lens device 12 (see FIG. 3).

[0055] The cylindrical portion 138A extends in the axial direction of the base portion 136. Specifically, the cylindrical portion 138A is formed along the central axis AC of the base portion 136. The angle formed between the central axis A1 of the cylindrical portion 138A and the central axis AC is set to 0°.

[0056] The cylindrical portions 138B and 138C are inclined with respect to the axial direction of the base portion 136. Specifically, the cylindrical portions 138B and 138C are inclined radially outward from the base portion 136 with respect to the central axis AC of the base portion 136. The angle γ1 between the central axis A2 of the cylindrical portion 138B and the central axis AC is set to be greater than 0°. Similarly, the angle γ2 between the central axis A3 of the cylindrical portion 138C and the central axis AC is also set to be greater than 0°. Furthermore, the angle γ2 is set to be greater than the angle γ1.

[0057] The cylindrical portion 138A is an example of a "fourth portion" according to the technology of the present disclosure. The multiple cylindrical portions 138B and the multiple cylindrical portions 138C are an example of a "multiple third portions" according to the technology of the present disclosure. The multiple cylindrical portions 138B are an example of a "multiple fifth portions" according to the technology of the present disclosure. The multiple cylindrical portions 138C are an example of a "multiple sixth portions" according to the technology of the present disclosure.

[0058] Each cylindrical portion 138 has a first opening 146 and a second opening 148. The first opening 146 is formed at the object-side end of the cylindrical portion 138, and the second opening 148 is formed at the image-side end of the cylindrical portion 138. The first opening 146 opens toward the object side, and the second opening 148 opens toward the image side. The first opening 146 and the second opening 148 communicate with each other through a hole 140 formed inside the cylindrical portion 138. The first opening 146 of the cylindrical portion 138A is perpendicular to the central axis A1 of the cylindrical portion 138A, the first opening 146 of the cylindrical portion 138B is perpendicular to the central axis A2 of the cylindrical portion 138B, and the first opening 146 of the cylindrical portion 138C is perpendicular to the central axis A3 of the cylindrical portion 138C.

[0059] The first opening 146 of the cylindrical portion 138A opens in the axial direction of the base portion 136 and is parallel to the first surface 142. On the other hand, the first opening 146 of the cylindrical portion 138B is inclined with respect to the first surface 142 and has an angle with respect to the first surface 142. Similarly, the first opening 146 of the cylindrical portion 138C is inclined with respect to the first surface 142 and has an angle with respect to the first surface 142. The second opening 148 of each cylindrical portion 138 opens in a direction facing the first surface 142 and is parallel to the first surface 142.

[0060] The first opening 146 of the cylindrical portion 138B has an angle relative to the second opening 148 of the cylindrical portion 138B that corresponds to the zenith angle θ1 and the azimuth angle φ1. Specifically, in response to the cylindrical portion 138B being inclined in the directions of the zenith angle θ1 and the azimuth angle φ1 with respect to the central axis AC of the base portion 136, the angle formed between the first opening 146 and the second opening 148 of the cylindrical portion 138B is set to an angle δ1.

[0061] Similarly, the first opening 146 of the cylindrical portion 138C has an angle relative to the second opening 148 of the cylindrical portion 138C that corresponds to the zenith angle θ1 and the azimuth angle φ1. Specifically, in response to the cylindrical portion 138C being inclined in the directions of the zenith angle θ1 and the azimuth angle φ1 with respect to the central axis AC of the base portion 136, the angle formed between the first opening 146 and the second opening 148 of the cylindrical portion 138C is set to an angle δ2. The angle δ2 is set to an angle greater than the angle δ1. The angles δ1 and δ2 are examples of "angles" according to the technology of the present disclosure.

[0062] As an example, the shape of the first opening 146 is rectangular. Similarly, the shape of the second opening 148 is also rectangular. As an example, the rectangular shapes of the first opening 146 and the second opening 148 are square. The shape of the first opening 146 is similar to the shape of the second opening 148. The opening area of ​​the first opening 146 is larger than the opening area of ​​the second opening 148. The shape of the first opening 146 is an example of a "first shape" according to the technology of the present disclosure. The shape of the second opening 148 is an example of a "second shape" according to the technology of the present disclosure.

[0063] A diffuser plate 150 that diffuses light is disposed in each second opening 148. The diffuser plate 150 has a size and shape that covers the second opening 148. The diffuser plate 150 is disposed perpendicular to the central axis AC of the base portion 136 and covers the second opening 148. The diffuser plate 150 diffuses the light emitted from the light source 3 (see FIG. 1 ), thereby functioning as a secondary light source when the light source 3 is used as the primary light source.

[0064] As shown in FIG. 7 as an example, each cylindrical portion 138 has a range R of light incidence angles limited by the cylindrical portion 138 (hereinafter referred to as the "incident angle range R"). The incident angle range R is defined as the range between a first line La and a second line Lb. The first line La is a line connecting a first side 146A of the multiple sides of the first opening 146 to a first side 148A of the multiple sides of the second opening 148 that is opposite to the first side 146A, extended toward the object side. Similarly, the second line Lb is a line connecting a second side 146B of the multiple sides of the first opening 146 to a second side 148B of the multiple sides of the second opening 148 that is opposite to the second side 146B, extended toward the object side.

[0065] The plurality of cylindrical portions 138 includes adjacent cylindrical portions 138. As an example, adjacent cylindrical portions 138 refer to all of the plurality of cylindrical portions 138 that are adjacent to each other in the circumferential direction and / or radial direction of the base portion 136. The incident angle range R of each cylindrical portion 138 is set to a range in which a portion of the incident angle range R1 of one of the adjacent cylindrical portions 138 overlaps with a portion of the incident angle range R2 of the other of the adjacent cylindrical portions 138. In other words, the incident angle range R1 and the incident angle range R2 have an overlapping range R3.

[0066] Furthermore, the incident angle range R of light limited by each cylindrical portion 138 is set to the same incident angle range. In other words, the angle formed by the first line La and the second line Lb that define the incident angle range R is constant. The same incident angle range is an example of a "first incident angle range" according to the technology of the present disclosure. Adjacent cylindrical portions 138 are an example of a "seventh portion" according to the technology of the present disclosure.

[0067] As shown in FIG. 8 as an example, the attachment 132 has a mounting portion 152. The mounting portion 152 extends from the second surface 144 toward the image side. The spacer 134 is formed in a cylindrical shape. The mounting portion 152 is attached to the object-side end of the spacer 134. The spacer 134 has a mounting portion 154 attached to the object-side end of the lens apparatus 12. The spacer 134 fixes the attachment 132 at a position away from the object side of the lens apparatus 12. When the attachment 132 is attached to the lens apparatus 12 via the spacer 134, the first opening 146 opens on a side different from the lens apparatus 12 (i.e., the object side), and the second opening 148 opens on the lens apparatus 12 side (i.e., the image side). The mounting portion 152 is an example of an "mounting portion" according to the technology of the present disclosure.

[0068] 9 shows an example of the configuration of a multispectral camera 10 that is used as the object camera 1 and the light source camera 2. As shown in FIG. 9, the multispectral camera 10 includes a lens device 12 and an image capture device body 14. The lens device 12 has the pupil division filter 16 described above.

[0069] As an example, as shown in FIG. 10, the pupil division filter 16 has a frame 18, spectral filters 20A to 20C, and polarizing filters 22A to 22C.

[0070] The frame 18 has openings 24A to 24C. The openings 24A to 24C are formed in a line around the optical axis OA. Hereinafter, when there is no need to distinguish between the openings 24A to 24C, each opening 24A to 24C will be referred to as an "opening 24." The spectral filters 20A to 20C are provided in the openings 24A to 24C, respectively, and are thereby arranged in a line around the optical axis OA. As a result, the center of gravity of each of the spectral filters 20A to 20C is located at a position different from the optical axis OA.

[0071] Each of the spectral filters 20A to 20C is a bandpass filter that transmits light in a specific wavelength band. The spectral filters 20A to 20C have different wavelength bands. Specifically, the spectral filter 20A transmits light in a first wavelength band λ １ and the spectral filter 20B has a second wavelength band λ２ and the spectral filter 20C has a third wavelength band λ ３ It has.

[0072] Hereinafter, when there is no need to distinguish between the spectral filters 20A to 20C, each of the spectral filters 20A to 20C will be referred to as a "spectral filter 20." １ , second wavelength band λ ２ , and the third wavelength band λ ３ When there is no need to distinguish between the first wavelength band λ １ , second wavelength band λ ２ , and the third wavelength band λ ３ are referred to as "wavelength band λ."

[0073] Polarizing filters 22A to 22C are provided corresponding to spectral filters 20A to 20C, respectively. Specifically, polarizing filter 22A is provided in opening 24A and overlaps spectral filter 20A. Polarizing filter 22B is provided in opening 24B and overlaps spectral filter 20B. Polarizing filter 22C is provided in opening 24C and overlaps spectral filter 20C.

[0074] Each of the polarizing filters 22A to 22C is an optical filter that transmits light that vibrates in a specific direction. The polarizing filters 22A to 22C have polarization axes with different polarization angles. Specifically, the polarizing filter 22A has a first polarization angle α １ and the polarizing filter 22B has a second polarization angle α ２ and the polarizing filter 22C has a third polarization angle α ３ The polarization axis may be referred to as a transmission axis. For example, the first polarization angle α １ is set to 0°, and the second polarization angle α ２ is set to 45°, and the third polarization angle α ３ is set to 90°.

[0075] Hereinafter, when there is no need to distinguish between the polarizing filters 22A to 22C, each of the polarizing filters 22A to 22C will be referred to as a "polarizing filter 22." １ , the second polarization angle α２ , and the third polarization angle α ３ When it is not necessary to distinguish between the first polarization angle α １ , the second polarization angle α ２ , and the third polarization angle α ３ are referred to as "polarization angles α."

[0076] 10 , the number of the openings 24 is three, corresponding to the number of the wavelength bands λ, but the number of the openings 24 may be greater than the number of the wavelength bands λ (i.e., the number of the spectral filters 20). Furthermore, unused openings 24 of the openings 24 may be blocked by a shielding member (not shown). Furthermore, although the spectral filters 20 have different wavelength bands λ in the example shown in FIG. 10 , the spectral filters 20 may include spectral filters 20 having the same wavelength band λ.

[0077] 9 , the lens device 12 includes an optical system 26, and the imaging device body 14 includes an image sensor 28. The optical system 26 includes the pupil division filter 16, a first lens 30, and a second lens 32.

[0078] The first lens 30 causes light from the object side to be incident on the pupil division filter 16. The second lens 32 causes the light that has passed through the pupil division filter 16 to form an image on a light receiving surface 34A of a photoelectric conversion element 34 provided in the image sensor 28.

[0079] The pupil division filter 16 is disposed at the pupil position of the optical system 26. The pupil position refers to the diaphragm surface that limits the brightness of the optical system 26. The pupil position here includes nearby positions, and nearby positions refer to the range from the entrance pupil to the exit pupil. The configuration of the pupil division filter 16 is as described using Figure 10. For convenience, Figure 9 shows a state in which a plurality of spectral filters 20 and a plurality of polarizing filters 22 are linearly arranged in a direction perpendicular to the optical axis OA.

[0080] The image sensor 28 includes a photoelectric conversion element 34 and a signal processing circuit 36. The image sensor 28 is, for example, a CMOS image sensor. In the present embodiment, a CMOS image sensor is used as the image sensor 28, but the technology of the present disclosure is not limited thereto. For example, the technology of the present disclosure can be applied even if the image sensor 28 is another type of image sensor, such as a CCD image sensor.

[0081] As an example, Fig. 9 shows a schematic configuration of a photoelectric conversion element 34. Also, as an example, Fig. 11 specifically shows the configuration of a portion of the photoelectric conversion element 34. The photoelectric conversion element 34 has a pixel layer 38, a polarizing filter layer 40, and a spectral filter layer 42. Note that the configuration of the photoelectric conversion element 34 shown in Fig. 11 is just an example, and the technology of the present disclosure is applicable even if the photoelectric conversion element 34 does not have the spectral filter layer 42.

[0082] The pixel layer 38 has a plurality of pixels 44. The plurality of pixels 44 are arranged in a matrix and form the light receiving surface 34A of the photoelectric conversion element 34. Each pixel 44 is a physical pixel having a photodiode (not shown), which photoelectrically converts received light and outputs an electrical signal according to the amount of received light.

[0083] Hereinafter, the pixels 44 provided in the photoelectric conversion element 34 will be referred to as "physical pixels 44" to distinguish them from pixels that form the spectral image. Also, the pixels that form the spectral image 72 will be referred to as "image pixels."

[0084] The photoelectric conversion element 34 outputs the electrical signals output from the plurality of physical pixels 44 as imaging data to the signal processing circuit 36. The signal processing circuit 36 ​​digitizes the analog imaging data input from the photoelectric conversion element 34. The imaging data is image data representing a captured image 70.

[0085] The plurality of physical pixels 44 form a plurality of pixel blocks 46. Each pixel block 46 is formed by two vertical and two horizontal rows, for a total of four physical pixels 44. For convenience, in Fig. 9, the four physical pixels 44 forming each pixel block 46 are shown as being linearly arranged in a direction perpendicular to the optical axis OA, but the four physical pixels 44 are arranged adjacent to each other in the vertical and horizontal directions of the photoelectric conversion element 34 (see Fig. 11).

[0086] The polarizing filter layer 40 has multiple types of polarizers 48A to 48D. Each of the polarizers 48A to 48D is an optical filter that transmits light vibrating in a specific direction. The polarizers 48A to 48D have polarization axes with different polarization angles. Specifically, the polarizer 48A has a first polarization angle β １ and the polarizer 48B has a second polarization angle β ２ and the polarizer 48C has a third polarization angle β ３ and the polarizer 48D has a fourth polarization angle β ４ As an example, the first polarization angle β １ is set to 0°, and the second polarization angle β ２ is set to 45°, and the third polarization angle β ３ is set to 90°, and the fourth polarization angle β ４ is set to 135°.

[0087] Hereinafter, when there is no need to distinguish between the polarizers 48A to 48D, each of the polarizers 48A to 48D will be referred to as a "polarizer 48." １ , second polarization angle β ２ , third polarization angle β ３ , and the fourth polarization angle β ４ When it is not necessary to distinguish between the first polarization angle β １ , second polarization angle β ２ , third polarization angle β ３ , and the fourth polarization angle β ４ are referred to as "polarization angles β."

[0088] The spectral filter layer 42 has a B filter 50A, a G filter 50B, and an R filter 50C. The B filter 50A is a blue-pass filter that transmits the most light in the blue wavelength band among light in a plurality of wavelength bands. The G filter 50B is a green-pass filter that transmits the most light in the green wavelength band among light in a plurality of wavelength bands. The R filter 50C is a red-pass filter that transmits the most light in the red wavelength band among light in a plurality of wavelength bands. The B filter 50A, G filter 50B, and R filter 50C are assigned to each pixel block 46.

[0089] For convenience, Fig. 9 shows the B filter 50A, G filter 50B, and R filter 50C arranged in a line in a direction perpendicular to the optical axis OA, but as an example, as shown in Fig. 11, the B filter 50A, G filter 50B, and R filter 50C are arranged in a matrix in a predetermined pattern arrangement. In the example shown in Fig. 11, the B filter 50A, G filter 50B, and R filter 50C are arranged in a matrix in a Bayer array, which is an example of a predetermined pattern arrangement. Note that the predetermined pattern arrangement may be an RGB stripe array, an R / G checkerboard array, an X-Trans (registered trademark) array, a honeycomb array, or the like, in addition to the Bayer array.

[0090] Hereinafter, when it is not necessary to distinguish between the B filter 50A, the G filter 50B, and the R filter 50C, they will each be referred to as "filters 50."

[0091] 9 , the imaging device body 14 includes, in addition to the image sensor 28, a control driver 52, an input / output I / F 54, a computer 56, and a communication device 58. The signal processing circuit 36, the control driver 52, the computer 56, and the communication device 58 are connected to the input / output I / F 54.

[0092] The computer 56 has a processor 60, a storage 62, and a RAM 64. The processor 60 controls the entire multispectral camera 10. The processor 60 is, for example, a processing unit including a CPU and a GPU, and the GPU operates under the control of the CPU and is responsible for executing image processing. Here, a processing unit including a CPU and a GPU is given as an example of the processor 60, but this is merely one example, and the processor 60 may be one or more CPUs that integrate a GPU function, or one or more CPUs that do not integrate a GPU function.

[0093] The processor 60, storage 62, and RAM 64 are connected via a bus 66, which is connected to the input / output I / F 54. The storage 62 is a non-transitory storage medium and stores various parameters and programs. For example, the storage 62 is a flash memory (e.g., an EEPROM). However, this is merely an example, and an HDD or the like may also be used as the storage 62 in addition to the flash memory. The RAM 64 temporarily stores various information and is used as a work memory. Examples of the RAM 64 include a DRAM and / or an SRAM.

[0094] The processor 60 reads out a necessary program from the storage 62 and executes the read program on the RAM 64. The processor 60 controls the control driver 52 and the signal processing circuit 36 ​​in accordance with the program executed on the RAM 64. The control driver 52 controls the photoelectric conversion element 34 under the control of the processor 60.

[0095] The communication device 58 is connected to the processor 60 via the input / output I / F 54 and the bus 66. The communication device 58 is also connected to the processing device 4 so as to be able to communicate with it via a wired or wireless connection. The communication device 58 is responsible for exchanging information with the processing device 4. For example, the communication device 58 transmits data to the processing device 4 in response to a request from the processor 60. The communication device 58 also receives data transmitted from the processing device 4 and outputs the received data to the processor 60 via the bus 66.

[0096] 12 , a spectral image generation program 80 is stored in the storage 62. The processor 60 reads the spectral image generation program 80 from the storage 62 and executes the read spectral image generation program 80 on the RAM 64. The processor 60 executes a spectral image generation process for generating a plurality of spectral images 72 in accordance with the spectral image generation program 80 executed on the RAM 64. The spectral image generation process is realized by the processor 60 operating as an output value acquisition unit 82 and an interference removal processing unit 84 in accordance with the spectral image generation program 80.

[0097] 13 , when imaging data output from the image sensor 28 is input to the processor 60, the output value acquisition unit 82 acquires an output value Y of each physical pixel 44 based on the imaging data. The output value Y of each physical pixel 44 corresponds to the luminance value of each pixel included in the captured image 70 represented by the imaging data.

[0098] Here, the output value Y of each physical pixel 44 is a value including interference (i.e., crosstalk). １ , second wavelength band λ ２ , and the third wavelength band λ ３ Since light of each wavelength band λ is incident, the output value Y is １ The value according to the amount of light in the second wavelength band λ ２ and the third wavelength band λ ３ The value is a mixture of values ​​according to the amount of light.

[0099] To obtain the spectral image 72, the processor 60 needs to perform a process of separating and extracting values ​​corresponding to each wavelength band λ from the output value Y for each physical pixel 44, that is, an interference removal process that removes interference, on the output value Y. Therefore, in this embodiment, to obtain the spectral image 72, the interference removal processing unit 84 performs the interference removal process on the output value Y of each physical pixel 44 acquired by the output value acquisition unit 82.

[0100] Here, the interference removal process will be described. The output value Y of each physical pixel 44 includes, for red, green, and blue, the luminance values ​​for each polarization angle β as components of the output value Y. The output value Y of each physical pixel 44 is expressed by equation (1).

[0101] However, Y β１＿Ｒ is the red output value Y, and the polarization angle is the first polarization angle β １ The luminance value of the component, Y β２＿Ｒ is the red output value Y, and the polarization angle is the second polarization angle β ２ The luminance value of the component Y β３＿Ｒ is the red output value Y, and the polarization angle is the third polarization angle β ３ The luminance value of the component Y β４＿Ｒ is the output value Y of red, and the polarization angle is the fourth polarization angle β ４ is the luminance value of the component.

[0102] Also, Y β１＿Ｇ is the green output value Y, and the polarization angle is the first polarization angle β １ The luminance value of the component Y β２＿Ｇ is the green output value Y, and the polarization angle is the second polarization angle β ２ The luminance value of the component, Y β３＿Ｇ is the green output value Y, and the polarization angle is the third polarization angle β ３ The luminance value of the component Y β４＿Ｇ is the green output value Y, and the polarization angle is the fourth polarization angle β ４ is the luminance value of the component.

[0103] Also, Y β１＿Ｂ is the blue output value Y, and the polarization angle is the first polarization angle β １ The luminance value of the component Y β２＿Ｂ is the blue output value Y, and the polarization angle is the second polarization angle β ２ The luminance value of the component Y β３＿Ｂ is the blue output value Y, and the polarization angle is the third polarization angle β ３ The luminance value of the component Y β４＿Ｂ is the blue output value Y, and the polarization angle is the fourth polarization angle β ４ is the luminance value of the component.

[0104] The pixel value X of each image pixel forming the spectral image 72 is determined by the first polarization angle α １ A first waveband λ having １ The brightness value X of the polarized light (hereinafter referred to as "first wavelength band polarized light") λ１ and the second polarization angle α ２ A second waveband λ having ２ The brightness value X of the polarized light (hereinafter referred to as "second wavelength band polarized light") λ２ and the third polarization angle α ３ A third waveband λ having ３ The brightness value X of the polarized light (hereinafter referred to as the "third wavelength band polarized light") λ３ and as components of the pixel value X. The pixel value X of each image pixel is expressed by equation (2).

[0105] The output value Y of each physical pixel 44 is expressed by equation (3).

[0106] In equation (3), A is an interference matrix. The interference matrix A (not shown) is a matrix that indicates the characteristics of interference. The interference matrix A is determined in advance based on a plurality of known values, such as the spectrum of the incident light, the spectral transmittance of the first lens 30, the spectral transmittance of the second lens 32, the spectral transmittances of the plurality of spectral filters 20, and the spectral sensitivity of the image sensor 28.

[0107] The interference cancellation matrix, which is the generalized inverse matrix of the interference matrix A, is defined as A ＋ In this case, the pixel value X of each image pixel is expressed by equation (4).

[0108] Interference cancellation matrix A ＋ Similarly to the interference matrix A, the interference cancellation matrix A is a matrix defined based on the spectrum of the incident light, the spectral transmittance of the first lens 30, the spectral transmittance of the second lens 32, the spectral transmittances of the plurality of spectral filters 20, the spectral sensitivity of the image sensor 28, etc. ＋ is stored in advance in the storage 62.

[0109] The interference cancellation processor 84 uses the interference cancellation matrix A stored in the storage 62 ＋ and the output value Y of each physical pixel 44 acquired by the output value acquisition unit 82, and＋ and the output value Y of each physical pixel 44, the pixel value X of each image pixel is output according to equation (4).

[0110] Here, as described above, the pixel value X of each image pixel is the luminance value X of the first wavelength band polarized light. λ１ and the brightness value X of the second wavelength band polarized light. λ２ and the brightness value X of the third wavelength band polarized light. λ３ and are included as components of the pixel value X.

[0111] The spectral image 72A of the captured image 70 is a first wavelength band λ １ The brightness value of the light X λ１ (i.e., the brightness values ​​X λ１ The spectral image 72B of the captured image 70 is an image based on the second wavelength band λ ２ The brightness value of the light X λ２ (i.e., the brightness values ​​X λ２ The spectral image 72C of the captured image 70 is an image based on the third wavelength band λ ３ The brightness value of the light X λ３ (i.e., the brightness values ​​X λ３ (Image based on ).

[0112] In this way, the interference removal processing is performed by the interference removal processing unit 84, and the captured image 70 is converted into a first wavelength band polarized light luminance value X λ１ and a spectral image 72A corresponding to the second wavelength band polarized light intensity value X λ２ and a spectral image 72B corresponding to the third wavelength band polarized light intensity value X λ３ That is, the captured image 70 is separated into spectral images 72 for each wavelength band λ of the plurality of spectral filters 20.

[0113] 14 , the processing device 4 includes a computer 92. The computer 92 includes a processor 94, a storage 96, and a RAM 98. The processor 94, the storage 96, and the RAM 98 are realized by hardware similar to the above-described processor 60, the storage 62, and the RAM 64 (see FIG. 9 ). The processor is an example of a “processor” according to the technology of the present disclosure.

[0114] A spectral reflectance derivation program 100 is stored in the storage 96. The spectral reflectance derivation program 100 is an example of a "program" according to the technology of the present disclosure. The processor 94 reads the spectral reflectance derivation program 100 from the storage 96 and executes the read spectral reflectance derivation program 100 on the RAM 98. The processor 94 executes a spectral reflectance derivation process in accordance with the spectral reflectance derivation program 100 executed on the RAM 98. The spectral reflectance derivation process is realized by the processor 94 operating as an image acquisition unit 102, a light source spectral distribution derivation unit 104, and a spectral reflectance derivation unit 106 in accordance with the spectral reflectance derivation program 100. The spectral reflectance derivation process is data processing of the imaging data transmitted from the object camera 1 and the light source camera 2 and received by the processing device 4.

[0115] 15 , the image acquisition unit 102 acquires a subject image 120 obtained by capturing an image of the subject 5 using the subject camera 1. The subject image 120 includes a plurality of spectral images 72 obtained by capturing an image of the subject 5. The image acquisition unit 102 also acquires a light source image 122 obtained by capturing an image of the light source 3 using the light source camera 2. The light source image 122 includes a plurality of spectral images 72 obtained by capturing an image of the light source 3.

[0116] 16 , each spectral image 72 included in the light source image 122 includes a plurality of images 124. Each image 124 corresponds to light that has passed through each cylindrical portion 138 provided on the attachment 132 and formed an image on the light-receiving surface 34A (see FIG. 9 ). The position of the image 124 within the spectral image 72 corresponds to the position of the cylindrical portion 138, i.e., the zenith angle θ1 and the azimuth angle φ1 (see FIG. 1 ). The brightness value of the image 124 corresponds to the intensity of the light that has passed through the cylindrical portion 138.

[0117] In this way, each spectral image 72 included in the light source image 122 includes a plurality of images 124, and therefore includes the spectral distribution of the light source 3 for each zenith angle θ1 and azimuth angle φ1 (hereinafter referred to as the "light source spectral distribution") as information regarding the wavelength and intensity of light for each zenith angle θ1 and azimuth angle φ1. The light source spectral distribution refers to the light intensity distribution for each zenith angle θ1 and azimuth angle φ1 in each wavelength band λ. The light source spectral distribution derivation unit 104 derives the light source spectral distribution corresponding to each wavelength band λ based on the plurality of images 124 included in the spectral image 72.

[0118] 17 as an example, the storage 96 stores the bidirectional reflectance distribution function (hereinafter referred to as "subject BRDF") of the subject 5. The subject BRDF is a function that represents the angular distribution characteristics of the intensity of light reflected by the subject 5 when incident light is incident on the subject 5 at a specific angle. The subject BRDF is stored in the storage 96 for each wavelength band λ.

[0119] The spectral reflectance derivation unit 106 derives the spectral reflectance of the subject 5 (hereinafter referred to as the "specific spectral reflectance") when light is irradiated onto the subject 5 from the light source 3 at a first angle and the subject 5 is imaged from a second angle (see Figure 1) based on the spectral image 72 included in the subject image 120, the light source spectral distribution, and the subject BRDF for each wavelength band λ.

[0120] 18 , the spectral reflectance derivation unit 106 performs a reflection coefficient derivation process to derive a specific spectral reflectance. The spectral reflectance derivation unit 106 includes a first reflection coefficient derivation unit 108, a subject type identification unit 110, a BRDF coefficient derivation unit 112, and a second reflection coefficient derivation unit 114. The reflection coefficient derivation process is realized by the first reflection coefficient derivation unit 108, the subject type identification unit 110, the BRDF coefficient derivation unit 112, and the second reflection coefficient derivation unit 114. The reflection coefficient derivation process is performed for each wavelength band λ.

[0121] The first reflection coefficient derivation unit 108 derives a first reflection coefficient, which is the reflection coefficient of the subject 5 in the wavelength band λ, based on the spectral image 72 included in the subject image 120 acquired by the image acquisition unit 102. The first reflection coefficient is expressed by a function h(x, y; λ). x corresponds to the x-coordinate of the subject position (x, y) and represents the horizontal position of the spectral image 72. y corresponds to the y-coordinate of the subject position (x, y) and represents the vertical position of the spectral image 72. Hereinafter, the first reflection coefficient may be referred to as the "first reflection coefficient h(x, y; λ)."

[0122] The subject type identification unit 110 identifies the type of subject 5 based on, for example, information indicating the type of subject 5 given to the processing device 4 by the user and / or the result of image processing executed on the spectral image 72 included in the subject image 120. For example, if the subject 5 is a plant, the type of subject 5 may be the specific name of the plant, or the specific type of subject 5, such as whether the subject 5 is a plant or soil.

[0123] The storage 96 stores BRDF coefficient information that indicates the relationship between the first reflection coefficient h(x, y; λ), the type of subject 5, and the BRDF coefficient. The BRDF coefficient is a coefficient included in the subject BRDF and is expressed by the function ci(x, y; λ). Hereinafter, the BRDF coefficient may be referred to as the "BRDF coefficient ci(x, y; λ)."

[0124] The BRDF coefficient derivation unit 112 derives the BRDF coefficients ci(x, y; λ) based on the BRDF coefficient information and the first reflection coefficient h(x, y; λ) according to the type of the subject 5 .

[0125] The second reflection coefficient derivation unit 114 derives a second reflection coefficient, which is the reflection coefficient of the subject 5 at a second angle when light is irradiated from the light source 3 onto the subject 5 from a first angle, based on the light source spectral distribution derived by the light source spectral distribution derivation unit 104 and the BRDF coefficients ci(x, y; λ) derived by the BRDF coefficient derivation unit 112.

[0126] The light source spectral distribution is expressed by a function f(θ1, φ1; λ). Hereinafter, the light source spectral distribution may be referred to as "light source spectral distribution f(θ1, φ1; λ)."

[0127] If the zenith angle θ1 and azimuth angle φ1 that represent the first angle are defined as zenith angle θ'1 and azimuth angle φ'1, the spectrum of light irradiated from light source 3 onto subject 5 at the first angle is expressed by the function f'(θ'1, φ'1; λ). Hereinafter, the spectrum of light irradiated from light source 3 onto subject 5 at the first angle may be referred to as "specific spectrum f'(θ'1, φ'1; λ)."

[0128] Furthermore, when the zenith angle θ2 and azimuth angle φ2 representing the second angle are set to zenith angle θ'2 and azimuth angle φ'2, the second reflection coefficient is expressed by the function h'[f'](x, y; θ'2, φ'2; λ). Hereinafter, the second reflection coefficient may be referred to as "second reflection coefficient h'[f'](x, y; θ'2, φ'2; λ)."

[0129] The object BRDF is expressed by a function g(θ1, φ1; θ2, φ2; λ). Hereinafter, the object BRDF may be referred to as "object BRDFg(θ1, φ1; θ2, φ2; λ)."

[0130] A specific method for determining the second reflection coefficient h'[f'](x, y; θ'2, φ'2; λ) will be described below.

[0131] The relationship between the first reflection coefficient h(x, y; λ), the light source spectral distribution f(θ1, φ1; λ), and the subject BRDF g(θ1, φ1; θ2, φ2; λ) is expressed by equation (5).

[0132] The subject BRDFg(θ1, φ1; θ2, φ2; λ) is set for each type of subject 5, and is expressed as a linear combination of n BRDF coefficients ci(x, y; λ) (where i = 0, ..., n-1) and preset basis functions gi(θ1, φ1; θ2, φ2; λ), as shown in equation (6).

[0133] When n BRDF coefficients ci are individually set for each object position (x, y) and wavelength band λ, the first reflection coefficient h(x, y; λ) is expressed by equations (7) and (8).

[0134] The BRDF coefficients c i (x, y; λ) are estimated from the first reflection coefficients h (x, y; λ) calculated by the formulas (7) and (8). Details will be described later for each case.

[0135] Once the BRDF coefficients ci(x, y; λ) are estimated, the subject BRDF g(θ1, φ1; θ2, φ2; λ) is estimated. As a result, the second reflection coefficients h'[f'](x, y; θ'2, φ'2; λ) are obtained as calculated by equations (9) and (10). Then, the specific spectral reflectance is derived based on the second reflection coefficients.

[0136] Next, estimation of the BRDF coefficients ci(x, y; λ) will be explained for each case.

[0137] Case 1 is a case where n = 2 and c0 = 1. Case 1 is a case where the reflection characteristics of the object 5 are an object BRDF that does not depend on the state of the object 5. ０ (θ1, φ1; θ2, φ2; λ) and the object BRDFg, which is an object BRDF whose degree of reflection changes depending on the state of the object 5. １ (θ1, φ1; θ2, φ2; λ). Case 1 is, for example, a case where the BRDF characteristics of an object are expressed by the superposition of reflections on the surface and the interior of the object, and the state of the interior of the object does not change. In case 1, reflections inside the object are expressed as subject BRDFg ０ (θ1, φ1; θ2, φ2; λ), and the reflection on the surface of the object is the subject BRDFg １ In Case 1, the unknown BRDF coefficient ci(x, y; λ) is one of c1, and therefore c1 is estimated from the first reflection coefficient h(x, y; λ) derived at one second angle using Equation (11).

[0138] Case 2 is a case where n=2, c0 does not depend on the wavelength band λ, and there is a wavelength band where H1 is 0. Case 2 is a case where the reflection characteristic of the object 5 is a constant object BRDF that does not depend on the wavelength band λ, object BRDFg ０ (θ1, φ1; θ2, φ2), and object BRDFg, which is an object BRDF whose reflectance is 0 in a specific wavelength band λ. １ (θ1, φ1; θ2, φ2; λ). The subject 5 may be, for example, a plant. Since plants are green, there are wavelengths in the blue or red wavelength band where the spectral reflectance is approximately 0. In this case, the subject BRDF is subject BRDFg. １ (θ1, φ1; θ2, φ2). Also, the shine on the surface of a plant has a constant reflectance independent of wavelength. In this case, the subject BRDF is subject BRDFg. ０ In Case 2, the wavelength band λ (hereinafter referred to as "wavelength band λ") where H1 is 0 is determined by Equation (12). ｒｅｆ ") is used to estimate c0, which is independent of the wavelength band λ, and then c1 in other wavelength bands λ is estimated using equation (13).

[0139] Case 3 is a case in which the angle of the captured camera 1 is changed and the subject 5 is imaged multiple times. In Case 3, first reflection coefficients h(x, y; λ) are derived at multiple second angles, and unknown BRDF coefficients c(x, y; λ) are estimated using simultaneous equations. Here, different second angles are defined as (θ2j(x, y), φ2j(x, y), the first reflection coefficients h(x, y; λ) derived at the different second angles are defined as hj(x, y; λ) (where j = 0, ..., m-1), and Hi(x, y; λ) corresponding to hj(x, y; λ) is defined as Hji(x, y; λ). The first reflection coefficients hj(x, y; λ) are expressed by Equation (14).

[0140] Let h = {hj} be the vector of hj(x, y; λ) and c = {ci} be the vector of Hji(x, y; λ). Then, the estimated solution c of the unknown BRDF coefficient ci(x, y; λ) is ＊ is expressed by equation (15). Here, H ＋ represents the generalized inverse matrix of Moore-Penrose. ＊ In order to improve the accuracy of m, it is preferable that m≧n.

[0141] Next, the operation of the imaging system S according to this embodiment will be described. First, a spectral image generation process executed by each of the captured camera 1 and the light source camera 2 according to this embodiment will be described. Fig. 19 shows an example of the flow of the spectral image generation process according to this embodiment.

[0142] 19 , first, in step ST10, the output value acquisition unit 82 acquires the output value Y of each physical pixel 44 based on the imaging data output from the image sensor 28 (see FIG. 13 ). After the processing of step ST10 is executed, the spectral image generation processing proceeds to step ST12.

[0143] In step ST12, the interference cancellation processor 84 uses the interference cancellation matrix A stored in the storage 62 ＋ and the output value Y of each physical pixel 44 acquired in step ST10 are acquired, and the acquired interference cancellation matrix A ＋ and the output value Y of each physical pixel 44, the pixel value X of each image pixel is output (see FIG. 13). By performing the interference removal process in step ST12, the captured image 70 is converted into a luminance value X of the first wavelength band polarized light. λ１ and a spectral image 72A corresponding to the second wavelength band polarized light intensity value X λ２ and a spectral image 72B corresponding to the third wavelength band polarized light luminance value X λ３ After the processing of step ST12 is executed, the spectral image generation processing ends.

[0144] Next, a description will be given of the spectral reflectance derivation process executed by the processing device 4 according to this embodiment. Fig. 20 shows an example of the flow of the spectral reflectance derivation process according to this embodiment.

[0145] 20 , first, in step ST20, image acquisition unit 102 acquires subject image 120 obtained by capturing an image of subject 5 with subject camera 1 and light source image 122 obtained by capturing an image of light source 3 with light source camera 2 (see FIG. 15 ). After the processing of step ST20 is executed, the spectral reflectance derivation processing proceeds to step ST22.

[0146] In step ST22, the light source spectral distribution deriving unit 104 derives a light source spectral distribution corresponding to each wavelength band λ based on each spectral image 72 included in the light source image 122 acquired in step ST20 (see FIG. 16 ). After the processing of step ST22 is performed, the spectral reflectance deriving process proceeds to step ST24.

[0147] In step ST24, the spectral reflectance deriving unit 106 derives a specific spectral reflectance for each wavelength band λ based on the spectral image 72 included in the object image 120 acquired in step ST20, the light source spectral distribution derived in step ST22, and the object BRDF stored in the storage 96 (see FIG. 17 ). The specific spectral reflectance is derived based on the second reflection coefficient derived by the reflection coefficient derivation process (see FIG. 18 ). After the process of step ST24 is executed, the spectral reflectance derivation process ends.

[0148] Next, a reflection coefficient derivation process for deriving the second reflection coefficient will be described. Fig. 21 shows an example of the flow of the reflection coefficient derivation process according to this embodiment.

[0149] 21 , first, in step ST30, the first reflection coefficient derivation unit 108 derives a first reflection coefficient, which is the reflection coefficient of the object 5 in the wavelength band λ, based on the spectral image 72 included in the object image 120 acquired in step ST20 (see FIG. 18 ). After the processing of step ST30 is executed, the reflection coefficient derivation processing proceeds to step ST32.

[0150] In step ST32, the subject type identification section 110 identifies the type of subject 5 (see FIG. 18). After the process of step ST32 is executed, the reflection coefficient derivation process proceeds to step ST34.

[0151] In step ST34, the BRDF coefficient derivation unit 112 derives BRDF coefficients based on the BRDF coefficient information stored in the storage 96 and the first reflection coefficient derived in step ST30, in accordance with the type of subject 5 identified in step ST32 (see FIG. 18 ). After the process of step ST34 is executed, the reflection coefficient derivation process proceeds to step ST36.

[0152] In step ST36, the second reflection coefficient derivation unit 114 derives a second reflection coefficient, which is the reflection coefficient of the subject 5 at the second angle when light is irradiated from the light source 3 at the first angle, based on the light source spectral distribution derived in step ST22 and the BRDF coefficient derived in step ST34 (see FIG. 18 ). This allows the specific spectral reflectance to be derived based on the second reflection coefficient. After the processing of step ST34 is executed, the reflection coefficient derivation processing ends. The processing method described above as the operation of the processing device 4 is an example of a "processing method" according to the technology of the present disclosure.

[0153] Next, the effects of this embodiment will be described.

[0154] As described above, the attachment 132 according to this embodiment (see FIGS. 6 to 8 ) includes a base portion 136 and a plurality of cylindrical portions 138. Each cylindrical portion 138 has a first opening 146 and a second opening 148. The plurality of cylindrical portions 138 includes a plurality of cylindrical portions 138B and a plurality of cylindrical portions 138C, in which the first opening 146 is angled relative to the second opening 148. Therefore, the spectrum for each of the zenith angle θ1 and the azimuth angle φ1 can be measured simultaneously based on the light passing through the plurality of cylindrical portions 138B and the plurality of cylindrical portions 138C. This contributes to more efficient measurement of the spectrum of the light source 3 than, for example, sequentially measuring the spectrum for each of the plurality of angles.

[0155] The plurality of cylindrical portions 138 also includes a cylindrical portion 138A in which a first opening 146 opens in the optical axis direction of the lens device 12. Therefore, the spectrum in the optical axis direction of the lens device 12 can be measured based on the light that has passed through the cylindrical portion 138A.

[0156] Furthermore, the cylindrical portion 138A extends in the optical axis direction of the lens device 12. Therefore, the spectrum in the optical axis direction of the lens device 12 can be measured based on the light that has passed through the cylindrical portion 138A.

[0157] Furthermore, the cylindrical portion 138A is disposed in the center of the base portion 136. Therefore, it is possible to measure the spectrum in the optical axis direction of the lens device 12 on the optical axis OA of the lens device based on the light that has passed through the cylindrical portion 138A.

[0158] Furthermore, the plurality of cylindrical portions 138 include a plurality of cylindrical portions 138B and a plurality of cylindrical portions 138C, which are arranged around the cylindrical portion 138A. Therefore, it is possible to measure the spectrum for each zenith angle θ1 and azimuth angle φ1 based on the light that has passed through the plurality of cylindrical portions 138B and the plurality of cylindrical portions 138C.

[0159] Furthermore, the multiple cylindrical portions 138B are arranged in a ring shape around the cylindrical portion 138A, and the multiple cylindrical portions 138C are arranged in a ring shape around the cylindrical portion 138A outside the multiple cylindrical portions 138B. Therefore, using the position of the cylindrical portion 138A as a reference, it is possible to measure the spectrum for each zenith angle θ1 and azimuth angle φ1 based on the light that has passed through the multiple cylindrical portions 138B and the multiple cylindrical portions 138C.

[0160] Furthermore, the multiple cylindrical portions 138B are arranged corresponding to the zenith angle θ1 and the azimuth angle φ1. Therefore, it is possible to measure the spectrum for each of the zenith angle θ1 and the azimuth angle φ1 based on the light that has passed through the multiple cylindrical portions 138B.

[0161] Similarly, the multiple cylindrical portions 138C are arranged corresponding to the zenith angle θ1 and the azimuth angle φ1. Therefore, it is possible to measure the spectrum for each of the zenith angle θ1 and the azimuth angle φ1 based on the light that has passed through the multiple cylindrical portions 138C.

[0162] Furthermore, cylindrical portion 138B and cylindrical portion 138C are inclined with respect to the optical axis direction of lens device 12. Therefore, it is possible to measure the spectrum in a direction inclined with respect to the optical axis direction of lens device 12 based on the light that has passed through cylindrical portion 138B and cylindrical portion 138C.

[0163] Furthermore, the first opening 146 of the cylindrical portion 138B has an angle δ1 relative to the second opening 148 of the cylindrical portion 138B that corresponds to the zenith angle θ1 and the azimuth angle φ1. Therefore, it is possible to measure the spectrum in the direction of the zenith angle θ1 and the azimuth angle φ1 that correspond to the angle δ1.

[0164] Similarly, the first opening 146 of the cylindrical portion 138C has an angle δ2 with respect to the second opening 148 of the cylindrical portion 138C that corresponds to the zenith angle θ1 and the azimuth angle φ1. Therefore, it is possible to measure the spectrum in the direction of the zenith angle θ1 and the azimuth angle φ1 that correspond to the angle δ2.

[0165] Furthermore, the angle δ2 of the cylindrical portion 138C is larger than the angle δ1 of the cylindrical portion 138B. Therefore, based on the light that has passed through the cylindrical portion 138C, it is possible to measure the spectrum in the direction of the zenith angle θ1 that is larger than the zenith angle θ1 corresponding to the cylindrical portion 138B.

[0166] Furthermore, the base portion 136 is formed in a flat plate shape, and each cylindrical portion 138 is formed in a cylindrical shape extending from the base portion 136. Therefore, the amount of material used for the attachment 132 can be reduced compared to when the base portion 136 is formed in a dome shape, for example.

[0167] Furthermore, the first opening 146 opens to a side different from the lens device 12 side, and the second opening 148 opens to the lens device 12 side. Therefore, by causing light that has entered the cylindrical portion 138 from a side different from the lens device 12 side to exit toward the lens device 12 side, it is possible to limit the light that passes through the cylindrical portion 138.

[0168] Furthermore, a diffuser plate 150 that diffuses light is disposed in the second opening 148. Therefore, when the light source 3 is used as a primary light source, the light diffused by the diffuser plate 150 can be used as a secondary light source. This makes it possible to prevent the intensity of light that passes through the cylindrical portion 138 and forms an image on the light receiving surface 34A from being locally saturated.

[0169] Furthermore, the multiple cylindrical portions 138 include adjacent cylindrical portions 138, and a portion of the incident angle range R1 of one of the adjacent cylindrical portions 138 overlaps with a portion of the incident angle range R2 of the other of the adjacent cylindrical portions 138. Therefore, compared to a case where the incident angle range R1 and the incident angle range R2 do not overlap, it is possible to measure the spectrum for each zenith angle θ1 and azimuth angle φ1 without omission.

[0170] Furthermore, the first shape, which is the shape of the first opening 146, is a rectangle. Therefore, the image 124 corresponding to the light that passes through each cylindrical portion 138 and is imaged on the light receiving surface 34A can also be a rectangle. This makes it possible to improve the accuracy when deriving the light source spectral distribution based on the multiple images 124, for example, compared to when the images 124 have a shape other than a rectangle.

[0171] Furthermore, the first shape, which is the shape of the first opening 146, is similar to the second shape, which is the shape of the second opening 148. Therefore, the image 124 corresponding to the light that passes through the first opening 146 and the second opening 148 and is formed on the light receiving surface 34A can be made to have a shape that corresponds to the first shape and the second shape.

[0172] Furthermore, the incident angle ranges R of light limited by the multiple cylindrical portions 138 are set to the same incident angle range. Therefore, for example, the accuracy of deriving the light source spectral distribution based on the multiple images 124 can be improved compared to when the incident angle ranges R are set to different incident angle ranges.

[0173] Furthermore, object camera 1 is a multispectral camera. Therefore, a plurality of spectral images 72 can be acquired by capturing an image of object 5 once. Similarly, light source camera 2 is also a multispectral camera. Therefore, a plurality of spectral images 72 can be acquired by capturing an image of light source 3 once.

[0174] The optical member 130 also includes an attachment 132 and a spacer 134, and the attachment 132 is attached to the object-side end of the lens device 12 via the spacer 134. Therefore, the attachment 132 can be fixed at a position away from the object side of the lens device 12 by the spacer 134. This makes it easier to focus on the attachment 132 compared to, for example, when the attachment 132 is directly attached to the object-side end of the lens device 12.

[0175] Furthermore, processor 94 of processing device 4 acquires subject image 120 obtained by capturing an image of subject 5 using subject camera 1, and acquires light source image 122 obtained by capturing an image of light source 3 using light source camera 2. Processor 94 then derives the specific spectral reflectance, which is the spectral reflectance of subject 5 when light is irradiated onto subject 5 from light source 3 at a first angle and subject 5 is captured at a second angle. Therefore, the specific spectral reflectance can be obtained without directly measuring the specific spectral reflectance.

[0176] Furthermore, the specific spectral reflectance is derived based on the subject image 120, the light source image 122, and the subject BRDF. Therefore, the specific spectral reflectance can be derived with higher accuracy than when the specific spectral reflectance is derived without using the subject image 120, the light source image 122, and the subject BRDF.

[0177] The light source image 122 also includes information about the wavelength and intensity of light for each zenith angle θ1 and azimuth angle φ1. This makes it possible to derive the light source spectral distribution corresponding to each wavelength band λ based on the light source image 122.

[0178] Next, a modification of this embodiment will be described.

[0179] In the above embodiment, the shape of the first opening 146 is rectangular, but as an example, as shown in Figure 22, the shape of the first opening 146 in the cylindrical portions 138B and 138C may be an arc shape centered on the central portion of the base portion 136 (see Figure 5, etc.). Similarly, the shape of the second opening 148 in the cylindrical portions 138B and 138C may also be an arc shape. Furthermore, the shape of the first opening 146 and the second opening 148 in the cylindrical portion 138A may be circular.

[0180] In the above embodiment, the multiple cylindrical portions 138 include a cylindrical portion 138A arranged in the center of the base portion 136, multiple cylindrical portions 138B arranged in a ring shape around the cylindrical portion 138A, and multiple cylindrical portions 138C arranged in a ring shape around the cylindrical portion 138A outside the multiple cylindrical portions 138B, corresponding to the zenith angle θ1 and the azimuth angle φ1. However, the multiple cylindrical portions 138 may be arranged linearly in the radial direction of the base portion 136 in correspondence with the zenith angle θ1. Alternatively, the multiple cylindrical portions 138 may be arranged in a ring shape around the center of the base portion 136 in correspondence with the azimuth angle φ1. Furthermore, in each cylindrical portion 138, the first opening 146 may have an angle relative to the second opening 148 that corresponds to either the zenith angle θ1 or the azimuth angle φ1.

[0181] Furthermore, in the above embodiment, the multispectral camera 10 is used as the object camera 1 and the light source camera 2, but a hyperspectral camera may also be used, or an RGB camera with a spectral filter may also be used.

[0182] In addition, in the above embodiment, the processor 60 is exemplified for the multispectral camera 10, but instead of the processor 60, or together with the processor 60, at least one other CPU, at least one GPU, and / or at least one TPU may be used.

[0183] In addition, in the above embodiment, the processor 94 is exemplified as the processing device 4, but instead of the processor 94, or together with the processor 94, at least one other CPU, at least one GPU, and / or at least one TPU may be used.

[0184] In the above embodiment, the multispectral camera 10 has been described with reference to an example in which the spectral image generation program 80 is stored in the storage 62. However, the technology of the present disclosure is not limited to this. For example, the spectral image generation program 80 may be stored in a portable, non-transitory, computer-readable storage medium (hereinafter simply referred to as a "non-transitory storage medium") such as an SSD or a USB memory. The spectral image generation program 80 stored in the non-transitory storage medium may be installed in the computer 56 of the multispectral camera 10.

[0185] Alternatively, the spectral image generation program 80 may be stored in a storage device such as another computer or server device connected to the multispectral camera 10 via a network, and the spectral image generation program 80 may be downloaded and installed on the computer 56 of the multispectral camera 10 in response to a request from the multispectral camera 10.

[0186] Furthermore, it is not necessary to store the entire spectral image generation program 80 in a storage device such as another computer or server device connected to the multispectral camera 10, or in the storage 62; only a part of the spectral image generation program 80 may be stored therein.

[0187] In the above embodiment, the processing device 4 has been described with reference to an example in which the spectral reflectance derivation program 100 is stored in the storage 96. However, the technology of the present disclosure is not limited to this. For example, the spectral reflectance derivation program 100 may be stored in a non-transitory storage medium. The spectral reflectance derivation program 100 stored in the non-transitory storage medium may be installed in the computer 92 of the processing device 4.

[0188] Alternatively, the spectral reflectance derivation program 100 may be stored in a storage device such as another computer or server device connected to the processing device 4 via a network, and the spectral reflectance derivation program 100 may be downloaded in response to a request from the processing device 4 and installed in the computer 92 of the processing device 4.

[0189] Furthermore, it is not necessary to store the entire spectral reflectance derivation program 100 in a storage device such as another computer or server device connected to the processing device 4, or in the storage 96; only a portion of the spectral reflectance derivation program 100 may be stored therein.

[0190] Furthermore, although the multispectral camera 10 has a built-in computer 56 , the technology of the present disclosure is not limited to this. For example, the computer 56 may be provided outside the multispectral camera 10 .

[0191] Furthermore, although the processing device 4 has a built-in computer 92 , the technology of the present disclosure is not limited to this, and for example, the computer 92 may be provided outside the processing device 4 .

[0192] In addition, in the above embodiment, the computer 56 including the processor 60, the storage 62, and the RAM 64 is exemplified for the multispectral camera 10, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 56. Furthermore, instead of the computer 56, a combination of a hardware configuration and a software configuration may be used.

[0193] In addition, in the above embodiment, the processing device 4 is exemplified as a computer 92 including a processor 94, a storage 96, and a RAM 98, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 92. Furthermore, instead of the computer 92, a combination of a hardware configuration and a software configuration may be used.

[0194] Furthermore, the following various processors can be used as hardware resources for executing the various processes described in the above embodiments. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing various processes by executing software, i.e., a program. Examples of processors include dedicated electronic circuits, such as FPGAs, PLDs, and ASICs, which are processors with a circuit configuration designed specifically for executing specific processes. Each processor has built-in or connected memory, and each processor uses the memory to execute various processes.

[0195] The hardware resources that execute various processes may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resources that execute various processes may be a single processor.

[0196] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute various processes, on a single IC chip, as typified by SoC. In this way, various processes are realized using one or more of the above-mentioned various processors as hardware resources.

[0197] Furthermore, the hardware structure of these various processors can be, more specifically, electronic circuits that combine circuit elements such as semiconductor devices. The above-described gaze detection process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0198] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0199] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0200] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A first part having a mounting portion to be attached to a lens device having an optical system, A plurality of second parts provided on the first part, Comprising, Each of the second parts has a first opening and a second opening, The plurality of second parts include a third part in which the first opening has an angle with respect to the second opening Optical member.

2. A first member having a mounting portion to be attached to a lens device having an optical system, A plurality of opening portions provided on the first member, Comprising, Each of the opening portions has a first opening and a second opening, The plurality of opening portions include an opening portion in which the first opening has an angle with respect to the second opening Optical member.

3. The plurality of second parts include a fourth part in which the first opening opens in the optical axis direction of the lens device The optical member according to claim 1.

4. The plurality of second parts include a plurality of the third parts, The plurality of third parts are arranged around the fourth part The optical member according to claim 3.

5. The fourth part is arranged at the center of the first part The optical member according to claim 3.

6. The plurality of second parts include a plurality of the third parts, The plurality of third parts are, A plurality of fifth parts arranged annularly around the fourth part, And a plurality of sixth parts arranged annularly around the fourth part outside the plurality of fifth parts The optical member according to claim 3.

7. The angle of the sixth part is larger than the angle of the fifth part The optical member according to claim 6.

8. Each of the second parts is formed in a cylindrical shape The optical member according to claim 1.

9. The first opening opens on a side different from the lens device side, The second opening opens on the lens device side The optical member according to claim 1.

10. The third part is inclined with respect to the optical axis direction of the lens device The optical member according to claim 1.

11. The fourth part extends in the optical axis direction of the lens device The optical member according to claim 3.

12. The angle is an angle corresponding to at least one of the zenith angle and the azimuth angle The optical member according to claim 1.

13. The plurality of second parts are arranged corresponding to at least one of the zenith angle and the azimuth angle The optical member according to claim 1.

14. A diffusion plate for diffusing light is arranged in the second opening The optical member according to claim 1.

15. The plurality of second portions includes adjacent seventh portions, A part of the range of the incident angle of light restricted by one of the seventh portions and a part of the range of the incident angle of light restricted by the other of the seventh portions overlap The optical member according to claim 1.

16. The first shape, which is the shape of the first opening, is a square shape or an arc shape The optical member according to claim 1.

17. The first shape, which is the shape of the first opening, is similar to the second shape, which is the shape of the second opening The optical member according to claim 1.

18. The range of the incident angle of light restricted by the plurality of second portions is set to a first incident angle range The optical member according to claim 1.

19. The lens device is a lens device of a spectroscopic imaging device The optical member according to claim 1.

20. A first optical member including the first portion and the plurality of second portions, A second optical member that fixes the first optical member at a position away from the object side with respect to the lens device, are provided The optical member according to claim 1.

21. Comprising a processor, The processor, Acquires a subject image obtained by imaging a subject with a first imaging device, Acquires a light source image obtained by imaging a light source with a second imaging device having the optical member according to claim 1, Derives the spectral reflectance of the subject when the subject is irradiated with light from a first angle by the light source and the subject is imaged from a second angle Processing device.

22. The spectral reflectance is derived based on the subject image, the light source image, and the bidirectional reflectance distribution function of the subject, The processing device according to claim 21.

23. The light source image includes information regarding the wavelength and intensity of light for each angle of at least one of the zenith angle and the azimuth angle The processing device according to claim 21.

24. The processor, Derives a first reflection coefficient, which is the reflection coefficient of the subject, based on the subject image, Derives a bidirectional reflectance distribution function coefficient included in the bidirectional reflectance distribution function based on the first reflection coefficient according to the type of the subject, Deriving the spectral reflectance of the subject includes deriving a second reflection coefficient, which is the reflection coefficient of the subject at the second angle when the subject is irradiated with light from the first angle, based on a light source spectral function based on the spectrum of the light source determined based on the light source image and the bidirectional reflectance distribution function coefficient. The processing device according to claim 22. **Claim 25** Obtaining a subject image obtained by imaging a subject with a first imaging device, Obtaining a light source image obtained by imaging a light source with a second imaging device having an optical member according to any one of claims 1 to 20, and Deriving the spectral reflectance of the subject when the subject is irradiated with light from a first angle by the light source and the subject is imaged from a second angle A processing method comprising the steps of: **Claim 26** Obtaining a subject image obtained by imaging a subject with a first imaging device, Obtaining a light source image obtained by imaging a light source with a second imaging device having an optical member according to any one of claims 1 to 20, and Deriving the spectral reflectance of the subject when the subject is irradiated with light from a first angle by the light source and the subject is imaged from a second angle A program for causing a computer to execute a process including the steps of: