Processing device, inspection device, processing method, and program

Abstract

This processing device comprises a processor. The processor: acquires first imaging data that is obtained by imaging a subject having a first surface and a second surface, the subject being imaged through use of an imaging device; acquires, from the first imaging data, reference imaging data in a reference region that spans the first and second surfaces; and, on the basis of the reference imaging data, corrects second imaging data that is obtained by imaging through use of the imaging device.

Description

Processing device, inspection device, processing method, and program

[0001] The technology disclosed herein relates to a processing device, an inspection device, a processing method, and a program.

[0002] Japanese Patent Application Laid-Open No. 2018-004509 discloses a color measurement system including an inspection device having an illumination device that irradiates a measurement object with light and an image capture device that acquires spectral information of the measurement object. The inspection device has a contact part that comes into contact with the measurement object and whose positional relationship with the illumination device and the image capture device is known, and the image capture device acquires a two-dimensional distribution of the spectral information of the measurement object in one shot.

[0003] Japanese Patent Application Laid-Open Publication No. 2015-194368 discloses a defect inspection method. The defect inspection method acquires a surface image of an inspection object by illuminating the inspection object with an illumination device selected from a plurality of illumination devices capable of illuminating the inspection object from different directions. The defect inspection method also compares, for each pixel, the luminance of the acquired surface image with the luminance of a reference image of an inspection object without defects, and corrects the position of the inspection object by moving the inspection object to a position where the imaging positions of the surface image and the reference image coincide. The defect inspection method then acquires a surface image of the inspection object after the correction, compares the luminance of the surface image with the luminance of the reference image for each pixel, and determines that a defect exists in the specified area if the degree of coincidence of the luminance in the specified area is lower than a threshold value.

[0004] Japanese Patent Laid-Open Publication No. 08-240535 discloses an appearance inspection device including a light source, a reflector, an imaging unit, and a defect determination unit. The light source irradiates light onto an inspection object being conveyed from diagonally above. Reflectors are provided below the inspection object and diagonally above it, facing the light source across the inspection object, and diffusely reflect the light. The imaging unit is provided vertically above the inspection object. The defect determination unit performs A / D conversion of the output signal from the imaging unit to obtain an image signal, binarizes the image signal, and integrates the density values, thereby determining chip defects on the inspection object, streak-like coating defects on the surface of the inspection object that occur in the same direction as the conveyance direction, and band-like coating defects on the surface of the inspection object that occur in the same direction as the conveyance direction.

[0005] One embodiment of the technology of the present disclosure provides a processing device, an inspection device, a processing method, and a program that can improve measurement accuracy compared to measuring the spectral reflectance of a subject by directly using imaging data obtained by imaging the subject with an imaging device.

[0006] A first aspect of the technology of the present disclosure is a processing device that includes a processor, which acquires first imaging data obtained by imaging a subject having a first surface and a second surface using an imaging device, acquires reference imaging data in a reference area spanning the first surface and the second surface from the first imaging data, and corrects second imaging data obtained by imaging using the imaging device based on the reference imaging data.

[0007] A second aspect of the technique of the present disclosure is the processing device according to the first aspect, wherein the first imaging data includes second imaging data.

[0008] A third aspect of the technique of the present disclosure is the processing device according to the first or second aspect, wherein the second imaging data is processed by a processing device different from that for the first imaging data.

[0009] A fourth aspect of the technology of the present disclosure is a processing device according to any one of the first to third aspects, wherein correcting the second imaging data includes correcting data relating to non-uniformity in the spectral reflectance of the subject.

[0010] A fifth aspect according to the technique of the present disclosure is the processing device according to any one of the first to fourth aspects, wherein the imaging device is a spectroscopic imaging device.

[0011] A sixth aspect of the technology of the present disclosure is a processing device according to the fifth aspect, wherein the spectroscopic imaging device includes a plurality of spectral filters, and the center of gravity of the plurality of spectral filters is located at a position different from the optical axis.

[0012] A seventh aspect of the technology of the present disclosure is a processing device according to any one of the first to sixth aspects, wherein the processor acquires reference imaging data when a calculation result is obtained based on the first imaging data, indicating that the first surface and the second surface have an angle.

[0013] An eighth aspect of the technology of the present disclosure is a processing device according to the seventh aspect, wherein the imaging device is a spectral imaging device, the spectral imaging device includes a plurality of spectral filters, the centers of gravity of which are located at positions different from the optical axis, and the calculation result is a processing device in which a first area, which is the area of ​​an image region corresponding to a first surface derived based on a first wavelength band image corresponding to a first spectral filter among the plurality of spectral filters, is different from a second area, which is the area of ​​an image region corresponding to the first surface derived based on a second wavelength band image corresponding to a second spectral filter among the plurality of spectral filters.

[0014] A ninth aspect of the technology of the present disclosure is a processing device according to the eighth aspect, wherein the first area is an area derived based on the luminance distribution of the first wavelength band image, and the second area is an area derived based on the luminance distribution of the second wavelength band image.

[0015] A tenth aspect of the technology of the present disclosure is a processing device relating to any one of the first to ninth aspects, wherein the reference area is an area determined based on the boundary between the first surface and the second surface.

[0016] An eleventh aspect of the technology of the present disclosure is a processing device according to any one of the first to ninth aspects, wherein correcting the second imaging data includes calculating an average value of the spectral reflectance in the reference area, and correcting data relating to the non-uniformity of the spectral reflectance in a first evaluation area on the first surface that is different from the reference area based on the average value.

[0017] A twelfth aspect of the technology of the present disclosure is a processing device according to the eleventh aspect, wherein correcting the second imaging data includes correcting data relating to the non-uniformity of the spectral reflectance of a second evaluation area on the second surface that is different from the reference area based on an average value.

[0018] A thirteenth aspect of the technology of the present disclosure is a processing device according to the eleventh or twelfth aspect, wherein the processor outputs a first evaluation result regarding the degree of unevenness in the spectral reflectance of the first evaluation area obtained by correction.

[0019] A fourteenth aspect of the technology of the present disclosure is a processing device according to the twelfth aspect or a thirteenth aspect dependent on the twelfth aspect, wherein the processor outputs a second evaluation result regarding the degree of unevenness in the spectral reflectance of the second evaluation area obtained by correction.

[0020] A fifteenth aspect of the technology of the present disclosure is a processing device according to any one of the first to fourteenth aspects, in which light is irradiated to the first surface and the second surface under different conditions.

[0021] A sixteenth aspect of the technique of the present disclosure is an inspection device including the processing device according to any one of the first to fifteenth aspects and an imaging device.

[0022] A seventeenth aspect of the technology of the present disclosure is a processing method comprising: acquiring first imaging data obtained by imaging a subject having a first surface and a second surface using an imaging device; acquiring reference imaging data in a reference area spanning the first surface and the second surface from the first imaging data; and correcting, based on the reference imaging data, second imaging data obtained by imaging using the imaging device.

[0023] An 18th aspect of the technology of the present disclosure is a program for causing a computer to execute processing including acquiring first imaging data obtained by imaging a subject having a first surface and a second surface using an imaging device, acquiring reference imaging data in a reference area spanning the first surface and the second surface from the first imaging data, and correcting, based on the reference imaging data, the second imaging data obtained by imaging using the imaging device.

[0024] 1 is a block diagram showing an example of an inspection device. FIG. 2 is a perspective view showing an example of an imaging device. FIG. 3 is an exploded perspective view showing an example of a pupil division filter. FIG. 4 is a block diagram showing an example of the hardware configuration of an imaging device. FIG. 5 is an exploded perspective view showing an example of a part of a photoelectric conversion element. FIG. 6 is a block diagram showing an example of the functional configuration of an imaging device. FIG. 7 is a block diagram showing an example of the operation of an output value acquisition unit and an interference removal processing unit. FIG. 8 is a block diagram showing an example of the functional configuration of a processing device. FIG. 9 is a block diagram showing an example of the operation of an imaging data acquisition unit and an image region setting unit. FIG. 10 is a block diagram showing an example of the operation of an area derivation unit and an area determination unit. FIG. 11 is a block diagram showing an example of the operation of a boundary setting unit and a reference image region setting unit. FIG. 12 is a block diagram showing an example of the operation of an average value derivation unit and an evaluation image region setting unit. FIG. 13 is a block diagram showing an example of the operation of an evaluation value derivation unit and an evaluation result output unit. FIG. 14 is a flowchart showing an example of the flow of a spectral image generation process. FIG. 15 is a flowchart showing an example of the flow of a measurement process.

[0025] Hereinafter, exemplary embodiments of a processing device, an inspection device, a processing method, and a program according to the techniques of the present disclosure will be described with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] RGB is an abbreviation for "Red Green Blue". LED is an abbreviation for "light emitting diode". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". I / F is an abbreviation for "Interface". RAM is an abbreviation for "Random Access Memory". CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". HDD is an abbreviation for "Hard Disk Drive". EL is an abbreviation for "Electro Luminescence". TPU is an abbreviation for "Tensor processing unit". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus." ASIC is an abbreviation for "Application Specific Integrated Circuit." FPGA is an abbreviation for "Field-Programmable Gate Array." PLD is an abbreviation for "Programmable Logic Device." SoC is an abbreviation for "System-on-a-Chip." IC is an abbreviation for "Integrated Circuit."

[0028] In the description of this specification, "identical" refers to identicalness in the sense of including, in addition to complete identicalness, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs, and that does not contradict the spirit of the technology of the present disclosure. In the description of this specification, "orthogonal" refers to orthogonality in the sense of including, in addition to complete orthogonality, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs, and that does not contradict the spirit of the technology of the present disclosure. In the description of this specification, "straight line" refers to a straight line in the sense of including, in addition to a perfect straight line, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs, and that does not contradict the spirit of the technology of the present disclosure.

[0029] As an example, as shown in FIG. 1 , an inspection device 130 according to this embodiment includes a first light source 132A, a second light source 132B, a housing 134, an imaging device 10, and a processing device 90. The inspection device 130 is an example of an "inspection device" according to the technology of the present disclosure. The imaging device 10 is an example of an "imaging device" according to the technology of the present disclosure. The processing device 90 is an example of a "processing device" according to the technology of the present disclosure. The first light source 132A and the second light source 132B are examples of a "light source" according to the technology of the present disclosure.

[0030] The first light source 132A and the second light source 132B are, for example, an LED light source, a laser light source, or an incandescent light bulb. The light emitted from the first light source 132A and the second light source 132B is unpolarized. As an example, the first light source 132A and the second light source 132B are arranged at an upper part inside the housing 134. As an example, the number of the first light source 132A and the second light source 132B is two. The first light source 132A and the second light source 132B are arranged on both sides of the imaging device 10. There may be any number of the first light source 132A and the second light source 132B. Instead of the first light source 132A and the second light source 132B, a single light source may be used.

[0031] The housing 134 is configured to cover the imaging space 136. A first light source 132A, a second light source 132B, an incident portion 10A of the imaging device 10, and a subject 200 are arranged in the imaging space 136. The imaging device 10 is provided on a ceiling portion 134A of the housing 134. The subject 200 is, for example, a triangular prism. The subject 200 has a first surface 202A, a second surface 202B, and a third surface 202C. The first surface 202A, the second surface 202B, and the third surface 202C are side surfaces of the triangular prism.

[0032] The subject 200 is placed on the bottom 134B of the housing 134 with the third surface 202C in contact with the bottom 134B and the first surface 202A and the second surface 202B facing the image capture device 10. The subject 200 is placed, for example, on the optical axis OA of the image capture device 10. The first surface 202A and the second surface 202B are flat surfaces. The first surface 202A is angled with respect to the second surface 202B. A ridge line 204 extending in the height direction of the triangular prism is formed between the first surface 202A and the second surface 202B.

[0033] The imaging device 10 is, for example, a multispectral camera. Here, an example is given in which the imaging device 10 is a multispectral camera, but the imaging device 10 may be a spectral camera such as a hyperspectral camera. The imaging device 10 may also be an RGB camera with a spectral filter. The following description will be given taking an example in which the imaging device 10 is a multispectral camera. The imaging device 10 is an example of a "spectral imaging device" according to the technology of the present disclosure.

[0034] The imaging device 10 includes an optical system 26 and an image sensor 28. The optical system 26 includes a first lens 30, a pupil division filter 16, and a second lens 32. The first lens 30, the pupil division filter 16, and the second lens 32 are arranged in this order along the optical axis OA from the subject 200 side to the image sensor 28 side.

[0035] The pupil division filter 16 has spectral filters 20A to 20C. Each of the spectral filters 20A to 20C is a bandpass filter that transmits light in a specific wavelength band. The spectral filters 20A to 20C have different wavelength bands. Specifically, the spectral filter 20A transmits light in a first wavelength band λ １ and the spectral filter 20B has a second wavelength band λ ２ and the spectral filter 20C has a third wavelength band λ ３ It has.

[0036] Hereinafter, when there is no need to distinguish between the spectral filters 20A to 20C, each of the spectral filters 20A to 20C will be referred to as a "spectral filter 20." １ , second wavelength band λ ２ , and the third wavelength band λ ３ When there is no need to distinguish between the first wavelength band λ １ , second wavelength band λ ２ , and the third wavelength band λ ３ The first wavelength band λ is referred to as a "wavelength band λ." The spectral filter 20 is an example of a "spectral filter" according to the technology of the present disclosure. １ The spectral filter 20A corresponding to the second wavelength band λ is an example of the "first spectral filter" according to the technology of the present disclosure. ２ The spectral filter 20B corresponding to is an example of a "second spectral filter" according to the technology of the present disclosure.

[0037] As will be described in detail later, the imaging device 10 generates spectral images 72A to 72C corresponding to the respective wavelength bands λ based on a captured image 70 obtained by capturing an image of the subject 200. The spectral image 72A is generated in the first wavelength band λ. １ The spectral image 72B corresponds to the second wavelength band λ ２ The spectral image 72C corresponds to the third wavelength band λ ３ Hereinafter, when there is no need to distinguish between the spectral images 72A to 72C, each of the spectral images 72A to 72C will be referred to as a "spectral image 72."

[0038] In this embodiment, as an example, a case will be described in which three spectral images 72 are generated based on light dispersed into three wavelength bands λ. However, three wavelength bands λ are merely an example, and two or more wavelength bands λ may be used.

[0039] The imaging device 10 has a zoom function. When the subject 200 is imaged by the imaging device 10, the zoom function adjusts the angle of view of the imaging device 10. The angle of view of the imaging device 10 is set to an angle of view that includes the subject 200 within the imaging range of the imaging device 10.

[0040] The processing device 90 is communicatively connected to the imaging device 10. The processing device 90 is, for example, an information processing device such as a personal computer or a server. The processing device 90 includes a display device 122. The display device 122 is, for example, a liquid crystal display or an EL display. As will be described in detail later, the processing device 90 measures the spectral reflectances of the first surface 202A and the second surface 202B of the subject 200 based on the multiple spectral images 72, and displays the measurement results 124 on the display device 122.

[0041] Next, the imaging device 10 according to this embodiment will be described in detail.

[0042] 2, the imaging device 10 includes a lens device 12 and an imaging device body 14. The lens device 12 has a pupil-division filter 16. As described above, the imaging device 10 is a multispectral camera that captures light that has been dispersed into multiple wavelength bands λ by the pupil-division filter 16, thereby generating and outputting multiple spectral images 72A-72C.

[0043] As an example, as shown in FIG. 3, the pupil division filter 16 has a frame 18, spectral filters 20A to 20C, and polarizing filters 22A to 22C.

[0044] The frame 18 has openings 24A to 24C. The openings 24A to 24C are formed in a line around the optical axis OA. Hereinafter, when there is no need to distinguish between the openings 24A to 24C, each opening 24A to 24C will be referred to as an "opening 24." The spectral filters 20A to 20C are provided in the openings 24A to 24C, respectively, and are arranged in a line around the optical axis OA. As a result, the center of gravity of each of the spectral filters 20A to 20C is located at a position different from the optical axis OA. The spectral filters 20A to 20C are an example of "multiple spectral filters" according to the technology of the present disclosure.

[0045] Polarizing filters 22A to 22C are provided corresponding to spectral filters 20A to 20C, respectively. Specifically, polarizing filter 22A is provided in opening 24A and overlaps spectral filter 20A. Polarizing filter 22B is provided in opening 24B and overlaps spectral filter 20B. Polarizing filter 22C is provided in opening 24C and overlaps spectral filter 20C.

[0046] Each of the polarizing filters 22A to 22C is an optical filter that transmits light that vibrates in a specific direction. The polarizing filters 22A to 22C have polarization axes with different polarization angles. Specifically, the polarizing filter 22A has a first polarization angle α １ and the polarizing filter 22B has a second polarization angle α ２ and the polarizing filter 22C has a third polarization angle α ３ The polarization axis may be referred to as a transmission axis. For example, the first polarization angle α １ is set to 0°, and the second polarization angle α ２ is set to 45°, and the third polarization angle α ３ is set to 90°.

[0047] Hereinafter, when there is no need to distinguish between the polarizing filters 22A to 22C, each of the polarizing filters 22A to 22C will be referred to as a "polarizing filter 22." １ , the second polarization angle α ２ , and the third polarization angle α ３ When it is not necessary to distinguish between the first polarization angle α１ , the second polarization angle α ２ , and the third polarization angle α ３ are referred to as "polarization angles α."

[0048] 3, the number of the openings 24 is three, corresponding to the number of the wavelength bands λ, but the number of the openings 24 may be greater than the number of the wavelength bands λ (i.e., the number of the spectral filters 20). Furthermore, unused openings 24 of the openings 24 may be blocked by a shielding member (not shown). Furthermore, although the spectral filters 20 have different wavelength bands λ in the example shown in FIG. 3, the spectral filters 20 may include spectral filters 20 having the same wavelength band λ.

[0049] 4, the lens device 12 includes an optical system 26, and the imaging device body 14 includes an image sensor 28. The optical system 26 includes the pupil division filter 16, a first lens 30, and a second lens 32.

[0050] The first lens 30 causes light reflected by the subject 200 to be incident on the pupil division filter 16. The second lens 32 causes the light transmitted through the pupil division filter 16 to form an image on a light receiving surface 34A of a photoelectric conversion element 34 provided in the image sensor 28.

[0051] The pupil division filter 16 is disposed at the pupil position of the optical system 26. The pupil position refers to the diaphragm surface that limits the brightness of the optical system 26. The pupil position here includes nearby positions, and nearby positions refer to the range from the entrance pupil to the exit pupil. The configuration of the pupil division filter 16 is as described using Figure 3. For convenience, Figure 4 shows a state in which a plurality of spectral filters 20 and a plurality of polarizing filters 22 are arranged linearly along a direction perpendicular to the optical axis OA.

[0052] The image sensor 28 includes a photoelectric conversion element 34 and a signal processing circuit 36. The image sensor 28 is, for example, a CMOS image sensor. In the present embodiment, a CMOS image sensor is used as the image sensor 28, but the technology of the present disclosure is not limited thereto. For example, the technology of the present disclosure can be applied even if the image sensor 28 is another type of image sensor, such as a CCD image sensor.

[0053] As an example, Fig. 4 shows a schematic configuration of the photoelectric conversion element 34. Also, as an example, Fig. 5 specifically shows the configuration of a portion of the photoelectric conversion element 34. The photoelectric conversion element 34 has a pixel layer 38, a polarizing filter layer 40, and a spectral filter layer 42. Note that the configuration of the photoelectric conversion element 34 shown in Fig. 5 is just an example, and the technology of the present disclosure is applicable even if the photoelectric conversion element 34 does not have the spectral filter layer 42.

[0054] The pixel layer 38 has a plurality of pixels 44. The plurality of pixels 44 are arranged in a matrix and form the light receiving surface 34A of the photoelectric conversion element 34. Each pixel 44 is a physical pixel having a photodiode (not shown), which photoelectrically converts received light and outputs an electrical signal according to the amount of received light.

[0055] Hereinafter, the pixels 44 provided in the photoelectric conversion element 34 will be referred to as "physical pixels 44" to distinguish them from pixels that form the spectral image. Also, the pixels that form the spectral image 72 will be referred to as "image pixels."

[0056] The photoelectric conversion element 34 outputs the electrical signals output from the plurality of physical pixels 44 as imaging data to the signal processing circuit 36. The signal processing circuit 36 ​​digitizes the analog imaging data input from the photoelectric conversion element 34. The imaging data is image data representing a captured image 70.

[0057] The plurality of physical pixels 44 form a plurality of pixel blocks 46. Each pixel block 46 is formed by two vertical and two horizontal rows, for a total of four physical pixels 44. For convenience, in Fig. 4, the four physical pixels 44 forming each pixel block 46 are shown as being linearly arranged in a direction perpendicular to the optical axis OA, but the four physical pixels 44 are arranged adjacent to each other in the vertical and horizontal directions of the photoelectric conversion element 34 (see Fig. 5).

[0058] The polarizing filter layer 40 has multiple types of polarizers 48A to 48D. Each of the polarizers 48A to 48D is an optical filter that transmits light vibrating in a specific direction. The polarizers 48A to 48D have polarization axes with different polarization angles. Specifically, the polarizer 48A has a first polarization angle β １ and the polarizer 48B has a second polarization angle β ２ and the polarizer 48C has a third polarization angle β ３ and the polarizer 48D has a fourth polarization angle β ４ As an example, the first polarization angle β １ is set to 0°, and the second polarization angle β ２ is set to 45°, and the third polarization angle β ３ is set to 90°, and the fourth polarization angle β ４ is set to 135°.

[0059] Hereinafter, when there is no need to distinguish between the polarizers 48A to 48D, each of the polarizers 48A to 48D will be referred to as a "polarizer 48." １ , second polarization angle β ２ , third polarization angle β ３ , and the fourth polarization angle β ４ When it is not necessary to distinguish between the first polarization angle β １ , second polarization angle β ２ , third polarization angle β ３ , and the fourth polarization angle β ４ are referred to as "polarization angles β."

[0060] The spectral filter layer 42 has a B filter 50A, a G filter 50B, and an R filter 50C. The B filter 50A is a blue-pass filter that transmits the most light in the blue wavelength band among light in a plurality of wavelength bands. The G filter 50B is a green-pass filter that transmits the most light in the green wavelength band among light in a plurality of wavelength bands. The R filter 50C is a red-pass filter that transmits the most light in the red wavelength band among light in a plurality of wavelength bands. The B filter 50A, G filter 50B, and R filter 50C are assigned to each pixel block 46.

[0061] For convenience, Fig. 4 shows the B filter 50A, G filter 50B, and R filter 50C arranged in a line along a direction perpendicular to the optical axis OA, but as an example, as shown in Fig. 5, the B filter 50A, G filter 50B, and R filter 50C are arranged in a matrix in a predetermined pattern arrangement. In the example shown in Fig. 5, the B filter 50A, G filter 50B, and R filter 50C are arranged in a matrix in a Bayer array, which is an example of a predetermined pattern arrangement. Note that the predetermined pattern arrangement may be an RGB stripe array, an R / G checkerboard array, an X-Trans (registered trademark) array, a honeycomb array, or the like, in addition to the Bayer array.

[0062] Hereinafter, when it is not necessary to distinguish between the B filter 50A, the G filter 50B, and the R filter 50C, they will each be referred to as "filters 50."

[0063] 4, the imaging device body 14 includes, in addition to the image sensor 28, a control driver 52, an input / output I / F 54, a computer 56, and a communication device 58. The signal processing circuit 36, the control driver 52, the computer 56, and the communication device 58 are connected to the input / output I / F 54.

[0064] The computer 56 has a processor 60, a storage 62, and a RAM 64. The processor 60 controls the entire imaging device 10. The processor 60 is, for example, an arithmetic processing device including a CPU and a GPU, and the GPU operates under the control of the CPU and is responsible for executing image-related processing. Here, an arithmetic processing device including a CPU and a GPU is given as an example of the processor 60, but this is merely one example, and the processor 60 may be one or more CPUs that integrate a GPU function, or one or more CPUs that do not integrate a GPU function.

[0065] The processor 60, storage 62, and RAM 64 are connected via a bus 66, which is connected to the input / output I / F 54. The storage 62 is a non-transitory storage medium and stores various parameters and programs. For example, the storage 62 is a flash memory (e.g., an EEPROM). However, this is merely an example, and an HDD or the like may also be used as the storage 62 in addition to the flash memory. The RAM 64 temporarily stores various information and is used as a work memory. Examples of the RAM 64 include a DRAM and / or an SRAM.

[0066] The processor 60 reads out a necessary program from the storage 62 and executes the read program on the RAM 64. The processor 60 controls the control driver 52 and the signal processing circuit 36 ​​in accordance with the program executed on the RAM 64. The control driver 52 controls the photoelectric conversion element 34 under the control of the processor 94.

[0067] The communication device 58 is connected to the processor 60 via the input / output I / F 54 and the bus 66. The communication device 58 is also connected to the processing device 90 so as to be able to communicate with it via a wired or wireless connection. The communication device 58 is responsible for exchanging information with the processing device 90. For example, the communication device 58 transmits data to the processing device 90 in response to a request from the processor 60. The communication device 58 also receives data transmitted from the processing device 90 and outputs the received data to the processor 60 via the bus 66.

[0068] 6 , a spectral image generation program 80 is stored in the storage 62. The processor 60 reads the spectral image generation program 80 from the storage 62 and executes the read spectral image generation program 80 on the RAM 64. The processor 60 executes a spectral image generation process for generating a plurality of spectral images 72 in accordance with the spectral image generation program 80 executed on the RAM 64. The spectral image generation process is realized by the processor 60 operating as an output value acquisition unit 82 and an interference removal processing unit 84 in accordance with the spectral image generation program 80.

[0069] 7 , when imaging data output from the image sensor 28 is input to the processor 60, the output value acquisition unit 82 acquires an output value Y of each physical pixel 44 based on the imaging data. The output value Y of each physical pixel 44 corresponds to the luminance value of each image pixel included in the captured image 70 represented by the imaging data.

[0070] Here, the output value Y of each physical pixel 44 is a value including interference (i.e., crosstalk). １ , second wavelength band λ ２ , and the third wavelength band λ ３ Since light of each wavelength band λ is incident, the output value Y is １ The value according to the amount of light in the second wavelength band λ ２ and the third wavelength band λ ３ The value is a mixture of values ​​according to the amount of light.

[0071] To obtain the spectral image 72, the processor 60 needs to perform a process of separating and extracting values ​​corresponding to each wavelength band λ from the output value Y for each physical pixel 44, that is, an interference removal process that removes interference, on the output value Y. Therefore, in this embodiment, to obtain the spectral image 72, the interference removal processing unit 84 performs the interference removal process on the output value Y of each physical pixel 44 acquired by the output value acquisition unit 82.

[0072] Here, the interference removal process will be described. The output value Y of each physical pixel 44 includes, for red, green, and blue, the luminance values ​​for each polarization angle β as components of the output value Y. The output value Y of each physical pixel 44 is expressed by equation (1).

[0073] However, Y β１＿Ｒ is the red output value Y, and the polarization angle is the first polarization angle β １ The luminance value of the component Y β２＿Ｒ is the red output value Y, and the polarization angle is the second polarization angle β ２ The luminance value of the component Y β３＿Ｒ is the red output value Y, and the polarization angle is the third polarization angle β ３ The luminance value of the component Y β４＿Ｒ is the red output value Y, and the polarization angle is the fourth polarization angle β ４ is the luminance value of the component.

[0074] Also, Y β１＿Ｇ is the green output value Y, and the polarization angle is the first polarization angle β １ The luminance value of the component Y β２＿Ｇ is the green output value Y, and the polarization angle is the second polarization angle β ２ The luminance value of the component Y β３＿Ｇ is the green output value Y, and the polarization angle is the third polarization angle β ３ The luminance value of the component Y β４＿Ｇ is the green output value Y, and the polarization angle is the fourth polarization angle β ４ is the luminance value of the component.

[0075] Also, Y β１＿Ｂ is the blue output value Y, and the polarization angle is the first polarization angle β １ The luminance value of the component Y β２＿Ｂ is the blue output value Y, and the polarization angle is the second polarization angle β ２ The luminance value of the component Y β３＿Ｂ is the blue output value Y, and the polarization angle is the third polarization angle β ３ The luminance value of the component Y β４＿Ｂ is the blue output value Y, and the polarization angle is the fourth polarization angle β ４ is the luminance value of the component.

[0076] The pixel value X of each image pixel forming the spectral image 72 is determined by the first polarization angle α １ A first waveband λ having １ The brightness value X of the polarized light (hereinafter referred to as "first wavelength band polarized light") λ１ and the second polarization angle α ２ A second waveband λ having ２ The brightness value X of the polarized light (hereinafter referred to as "second wavelength band polarized light") λ２ and the third polarization angle α ３ A third waveband λ having ３ The brightness value X of the polarized light (hereinafter referred to as the "third wavelength band polarized light") λ３ and as components of the pixel value X. The pixel value X of each image pixel is expressed by equation (2).

[0077] The output value Y of each physical pixel 44 is expressed by equation (3).

[0078] In equation (3), A is an interference matrix. The interference matrix A (not shown) is a matrix that indicates the characteristics of interference. The interference matrix A is determined in advance based on a plurality of known values, such as the spectrum of the incident light, the spectral transmittance of the first lens 30, the spectral transmittance of the second lens 32, the spectral transmittances of the plurality of spectral filters 20, and the spectral sensitivity of the image sensor 28.

[0079] The interference cancellation matrix, which is the generalized inverse matrix of the interference matrix A, is defined as A ＋ In this case, the pixel value X of each image pixel is expressed by equation (4).

[0080] Interference cancellation matrix A ＋ Similarly to the interference matrix A, the interference cancellation matrix A is a matrix defined based on the spectrum of the incident light, the spectral transmittance of the first lens 30, the spectral transmittance of the second lens 32, the spectral transmittances of the plurality of spectral filters 20, the spectral sensitivity of the image sensor 28, etc. ＋ is stored in advance in the storage 62.

[0081] The interference cancellation processor 84 uses the interference cancellation matrix A stored in the storage 62 ＋ and the output value Y of each physical pixel 44 acquired by the output value acquisition unit 82, and＋ and the output value Y of each physical pixel 44, the pixel value X of each image pixel is output according to equation (4).

[0082] Here, as described above, the pixel value X of each image pixel is the luminance value X of the first wavelength band polarized light. λ１ and the brightness value X of the second wavelength band polarized light. λ２ and the brightness value X of the third wavelength band polarized light. λ３ and are included as components of the pixel value X.

[0083] The spectral image 72A of the captured image 70 is a first wavelength band λ １ The brightness value of the light X λ１ (i.e., the brightness values ​​X λ１ The spectral image 72B of the captured image 70 is an image based on the second wavelength band λ ２ The brightness value of the light X λ２ (i.e., the brightness values ​​X λ２ The spectral image 72C of the captured image 70 is an image based on the third wavelength band λ ３ The brightness value of the light X λ３ (i.e., the brightness values ​​X λ３ (Image based on ).

[0084] In this way, the interference removal processing is performed by the interference removal processing unit 84, and the captured image 70 is converted into a first wavelength band polarized light luminance value X λ１ and a spectral image 72A corresponding to the second wavelength band polarized light intensity value X λ２ and a spectral image 72B corresponding to the third wavelength band polarized light intensity value X λ３ That is, the captured image 70 is separated into spectral images 72 for each wavelength band λ of the plurality of spectral filters 20.

[0085] Next, the processing device 90 according to this embodiment will be described in detail.

[0086] As an example, as shown in Fig. 8, the processing device 90 includes a computer 92. The computer 92 includes a processor 94, a storage 96, and a RAM 98. The processor 94, the storage 96, and the RAM 98 are realized by hardware similar to the above-described processor 60, the storage 62, and the RAM 64 (see Fig. 4). The processor 94 is an example of a "processor" according to the technology of the present disclosure.

[0087] A measurement program 100 is stored in the storage 96. The measurement program 100 is an example of a "program" according to the technology of the present disclosure. The processor 94 reads the measurement program 100 from the storage 96 and executes the read measurement program 100 on the RAM 98. The processor 94 executes a measurement process in accordance with the measurement program 100 executed on the RAM 98. The measurement process is realized by the processor 94 operating as an imaging data acquisition unit 102, an image area setting unit 104, an area derivation unit 106, an area determination unit 108, a boundary setting unit 110, a reference image area setting unit 112, an average value derivation unit 114, an evaluation image area setting unit 116, an evaluation value derivation unit 118, and an evaluation result output unit 120 in accordance with the measurement program 100.

[0088] As an example, as shown in Fig. 9 , light is irradiated onto a first surface 202A and a second surface 202B of the subject 200 by a first light source 132A and a second light source 132B. The first surface 202A and the second surface 202B are irradiated with light under different conditions. In the example shown in Fig. 9 , as an example of the different conditions, the light intensity of the first light source 132A located on the first surface 202A side is set to be higher than the light intensity of the second light source 132B located on the second surface 202B side. As a result, the intensity of the light irradiated onto the first surface 202A is higher than the intensity of the light irradiated onto the second surface 202B.

[0089] The imaging device 10 captures an image of the subject 200 in a state in which the first surface 202A and the second surface 202B are irradiated with light from the first light source 132A and the second light source 132B. The imaging device 10 then transmits imaging data obtained by capturing the image of the subject 200 to the processing device 90. The imaging data is image data representing a plurality of spectral images. The imaging data is an example of "first imaging data" and "second imaging data" according to the technology of the present disclosure.

[0090] The imaging data acquisition unit 102 acquires imaging data received by the processing device 90. The image region setting unit 104 selects two spectral images 72 from the multiple spectral images 72 included in the imaging data acquired by the imaging data acquisition unit 102. The two spectral images 72 are selected based on an image selection instruction given to the processing device 90 by, for example, a user or the like.

[0091] In the example shown in FIG. 9, two spectral images 72 are １ and a spectral image 72A corresponding to the second waveband λ ２ 1 and spectral image 72B corresponding to object 200 are selected. Spectral image 72A and spectral image 72B include object image 140 corresponding to object 200. Spectral image 72A is an example of a "first waveband image" according to the technology of the present disclosure. Spectral image 72B is an example of a "first waveband image" according to the technology of the present disclosure.

[0092] Then, the image region setting unit 104 divides the image region of the object image 140 based on the luminance distribution of the image pixels for each selected spectral image 72. In the example shown in Fig. 9, the image region setting unit 104 detects the luminance distribution of the image pixels for the spectral image 72A, and divides the object image 140 into a first image region 142A having a luminance equal to or greater than a predetermined value, and a second image region 142B having a luminance less than the predetermined value.

[0093] 9 , the image area setting unit 104 detects the luminance distribution of image pixels for the spectral image 72B and divides the subject image 140 into a first image area 142A where the luminance is equal to or greater than a predetermined value, and a second image area 142B where the luminance is less than the predetermined value. The predetermined value is set to a luminance corresponding to the median value of the difference between the light intensity of the first light source 132A and the light intensity of the second light source 132B. The first image area 142A is an image area corresponding to the first surface 202A, and the second image area 142B is an image area corresponding to the second surface 202B.

[0094] The size of the first image region 142A and the second image region 142B differs for each spectral image 72 because the spectral filters 20 corresponding to each spectral image 72 are provided in openings 24 formed at positions different from the optical axis OA, causing parallax between the multiple spectral filters 20. When parallax occurs between the multiple spectral filters 20, a positional shift occurs due to the parallax in the optical images formed on the light receiving surface 34A for each wavelength band λ.

[0095] 10 , the area derivation unit 106 derives a first area A1, which is the area of ​​the first image region 142A, based on one of the two selected spectral images 72. The area derivation unit 106 also derives a second area A2, which is the area of ​​the first image region 142A, based on the other of the two selected spectral images 72. In the example shown in FIG. 10 , the area derivation unit 106 derives the first area A1, which is the area of ​​the first image region 142A, based on the spectral image 72A. Similarly, the area derivation unit 106 derives the second area A2, which is the area of ​​the first image region 142A, based on the spectral image 72B.

[0096] The first area A1 is an area derived based on the luminance distribution of the spectral image 72A, and the second area A2 is an area derived based on the luminance distribution of the spectral image 72B. The first area A1 is an example of the "first area" according to the technology of the present disclosure. The second area A2 is an example of the "second area" according to the technology of the present disclosure.

[0097] The area determination unit 108 determines whether the first area A1 and the second area A2 derived by the area derivation unit 106 are different. That is, the area determination unit 108 determines whether the area derivation unit 106 has obtained a calculation result indicating that the first area A1 and the second area A2 are different. The calculation result is an example of the "calculation result" according to the technology of the present disclosure.

[0098] Here, when the first surface 202A and the second surface 202B form an angle, the first area A1 derived based on the spectral image 72A differs from the second area A2 derived based on the spectral image 72B. Therefore, when the first surface 202A and the second surface 202B form an angle, the area derivation unit 106 obtains a calculation result that the first area A1 and the second area A2 differ.

[0099] On the other hand, for example, if the subject 200 is planar and the first surface 202A and the second surface 202B do not form an angle (i.e., if the first surface 202A and the second surface 202B are on the same plane), the first area A1 derived based on the spectral image 72A and the second area A2 derived based on the spectral image 72B will be the same. Therefore, if the first surface 202A and the second surface 202B do not form an angle, the area derivation unit 106 will obtain a calculation result that the first area A1 and the second area A2 are the same.

[0100] In this embodiment, since the first surface 202A and the second surface 202B have an angle, in the example shown in Figure 10, the first area A1 derived based on the spectral image 72A is different from the second area A2 derived based on the spectral image 72B.

[0101] 10 , the first area A1 may be the area of ​​the second image region 142B derived based on the spectral image 72A. Similarly, the second area A2 may be the area of ​​the second image region 142B derived based on the spectral image 72B. The area determination unit 108 may determine whether the first area A1, which is the area of ​​the second image region 142B derived based on the spectral image 72A, is different from the second area A2, which is the area of ​​the second image region 142B derived based on the spectral image 72B.

[0102] As an example, as shown in Figure 11, when the area determination unit 108 (see Figure 10) determines that the first area A1 and the second area A2 are different, the boundary setting unit 110 selects one of the spectral images 72 into which the image area of ​​the subject image 140 is divided by the image area setting unit 104 (see Figure 9).

[0103] One of the spectral images 72 is selected based on, for example, an image selection instruction given to the processing device 90 by a user or the like. In the example shown in FIG. 11 , the spectral image 72A is selected. Note that the spectral image 72B may be selected, and the same processing as that performed on the spectral image 72A may be performed on the spectral image 72B. The boundary setting unit 110 then sets a boundary 144 between the first image region 142A and the second image region 142B for the object image 140 included in one of the selected spectral images 72. The boundary 144 is a line corresponding to the ridge line 204 between the first surface 202A and the second surface 202B. The ridge line 204 is an example of a "boundary" according to the technology of the present disclosure.

[0104] The reference image area setting unit 112 sets the image area of ​​the first image area 142A on the boundary 144 side as the first reference image area 146A, and sets the image area of ​​the second image area 142B on the boundary 144 side as the second reference image area 146B. Then, the reference image area setting unit 112 sets the reference image area 146 including the first reference image area 146A and the second reference image area 146B for the subject image 140.

[0105] The first reference image area 146A is an image area corresponding to the first reference area 206A on the edge line 204 side of the first surface 202A, and the second reference image area 146B is an image area corresponding to the second reference area 206B on the edge line 204 side of the second surface 202B. The first reference area 206A and the second reference area 206B form the reference area 206. The reference image area 146 is an image area corresponding to the reference area 206.

[0106] The reference area 206 is an area including the first reference area 206A and the second reference area 206B, and is an area spanning the first surface 202A and the second surface 202B of the subject 200. The reference area 206 is an area within the first surface 202A and the second surface 202B where there is little difference between the way light emitted from the first light source 132A hits the object and the way light emitted from the second light source 132B hits the object.

[0107] 11 is imaging data representing the reference image area 146, and is imaging data in the reference area 206. The reference image area setting unit 112 sets the reference image area 146, whereby the reference imaging data is acquired from the imaging data (see FIG. 9 ). The reference imaging data is an example of "reference imaging data" according to the technology of the present disclosure.

[0108] As an example, as shown in FIG. 12, the average value derivation unit 114 calculates the average value S(REF) of the luminance of the image pixels included in the reference image region 146 by using the formula (5). ａｖｅ where S(REF) ａ is the average brightness of the image pixels included in the first reference image region 146A, and S(REF) ｂ is the average brightness of the image pixels included in the second reference image region 146B.

[0109] The evaluation image area setting unit 116 sets the image area of ​​the first image area 142A other than the first reference image area 146A as the first evaluation image area 148A for the subject image 140, and sets the image area of ​​the second image area 142B other than the second reference image area 146B as the second evaluation image area 148B.

[0110] As an example, as shown in FIG. 13, the evaluation value derivation unit 118 calculates S(EVE) by using equation (6). ａ Based on this, an evaluation value S(EVE) is calculated for each image pixel included in the first evaluation image area 148A. ａ＋ Derive S(EVE) ａ is the luminance value of each image pixel included in the first evaluation image area 148A.

[0111] Similarly, the evaluation value derivation unit 118 calculates S(EVE) using equation (7). ｂ Based on this, an evaluation value S(EVE) is calculated for each image pixel included in the second evaluation image area 148B. ｂ＋ Derive S(EVE) ｂ is the luminance value of each image pixel included in the second evaluation image region 148B.

[0112] In this way, the evaluation value derivation unit 118 calculates the evaluation value S(EVE) for each image pixel included in the first evaluation image area 148A. ａ＋ By deriving the luminance value S(EVE) for each image pixel included in the first evaluation image area 148A, ａ Similarly, the evaluation value derivation unit 118 calculates an evaluation value S(EVE) for each image pixel included in the second evaluation image region 148B. ｂ＋ By deriving the luminance value S(EVE) for each image pixel included in the second evaluation image region 148B, ｂ is corrected.

[0113] Here, the first evaluation image area 148A is an image area corresponding to the first evaluation area 208A of the subject 200, and has a brightness value S(EVE) ａ corresponds to the spectral reflectance of the first evaluation region 208A. The first evaluation region 208A is the region of the first surface 202A other than the first reference region 206A. Similarly, the second evaluation image region 148B is an image region corresponding to the second evaluation region 208B of the subject 200, and has a luminance value S(EVE) ｂ corresponds to the spectral reflectance of the second evaluation region 208B. The second evaluation region 208B is a region of the second surface 202B other than the second reference region 206B.

[0114] Also, the average value S(REF) ａｖｅ (see FIG. 12 ) corresponds to the average value of the spectral reflectance in the reference region 206. The reference imaging data (see FIG. 11 ) includes data on the average value of the spectral reflectance in the reference region 206, and the imaging data (see FIG. 9 ) includes data on the non-uniformity of the spectral reflectance in the first evaluation region 208A and data on the non-uniformity of the spectral reflectance in the second evaluation region 208B.

[0115] Therefore, as described above, the evaluation value derivation unit 118 calculates the average value S(REF) ａｖｅ Based on this, the brightness value S(EVE) included in the first evaluation image area 148A is calculated. ａ By correcting the average value S(REF), the data relating to the non-uniformity of the spectral reflectance of the first evaluation area 208A is corrected based on the reference imaging data. ａｖｅ Based on this, the brightness value S(EVE) included in the second evaluation image area 148B is calculated. ｂ By correcting the spectral reflectance non-uniformity of the second evaluation area 208B based on the reference imaging data, the data relating to the non-uniformity of the spectral reflectance of the second evaluation area 208B is corrected. Note that the non-uniformity of the spectral reflectance refers to variations in the spectral reflectance.

[0116] The evaluation result output unit 120 outputs the evaluation value S(EVE) derived for each image pixel included in the first evaluation image area 148A. ａ＋ falls within a first predetermined range. The first predetermined range is set to a range within which a user or the like can visually evaluate that there is no unevenness in the spectral reflectance of first surface 202A of subject 200. First evaluation result 126A is an evaluation result regarding the degree of unevenness in the spectral reflectance of first evaluation area 208A.

[0117] Similarly, the evaluation result output unit 120 outputs the evaluation value S(EVE) derived for each image pixel included in the second evaluation image area 148B. ｂ＋ falls within a second predetermined range, and outputs a second evaluation result 126B. The second predetermined range is set to a range within which a user or the like can visually evaluate that there is no unevenness in the spectral reflectance of the second surface 202B of the subject 200. The second evaluation result 126B is an evaluation result regarding the degree of unevenness in the spectral reflectance of the second evaluation area 208B.

[0118] The first evaluation result 126A and the second evaluation result 126B are included in the measurement result 124, and the measurement result 124 including the first evaluation result 126A and the second evaluation result 126B is displayed, for example, on the display device 122 (see Figure 1).

[0119] Next, the operation of the inspection device 130 according to this embodiment will be described. First, the spectral image generation process executed by the imaging device 10 according to this embodiment will be described. Fig. 14 shows an example of the flow of the spectral image generation process according to this embodiment.

[0120] 14 , first, in step ST10, the output value acquisition unit 82 acquires the output value Y of each physical pixel 44 based on the imaging data output from the image sensor 28 (see FIG. 7 ). After the processing of step ST10 is executed, the spectral image generation processing proceeds to step ST12.

[0121] In step ST12, the interference cancellation processor 84 uses the interference cancellation matrix A stored in the storage 62 ＋ and the output value Y of each physical pixel 44 acquired in step ST10 are acquired, and the acquired interference cancellation matrix A ＋ and the output value Y of each physical pixel 44, the pixel value X of each image pixel is output (see FIG. 7). By performing the interference removal process in step ST12, the captured image 70 is converted into a luminance value X of the first wavelength band polarized light. λ１ and a spectral image 72A corresponding to the second wavelength band polarized light intensity value X λ２ and a spectral image 72B corresponding to the third wavelength band polarized light luminance value X λ３ After the processing of step ST12 is executed, the spectral image generation processing ends.

[0122] Next, a description will be given of the measurement process executed by the processing device 90 according to this embodiment. Fig. 15 shows an example of the flow of the measurement process according to this embodiment.

[0123] 15, first, in step ST20, the imaging data acquisition unit 102 acquires imaging data received by the processing device 90 (see FIG. 9). After the processing of step ST20 is executed, the measurement processing proceeds to step ST22.

[0124] In step ST22, the image region setting unit 104 selects two spectral images 72 from the plurality of spectral images 72 included in the imaging data acquired in step ST20. Then, for each selected spectral image 72, the image region setting unit 104 divides the subject image 140 into a first image region 142A and a second image region 142B based on the luminance distribution of the image pixels (see FIG. 9 ). After the processing of step ST22 is executed, the measurement processing proceeds to step ST24.

[0125] In step ST24, the area derivation unit 106 derives a first area A1, which is the area of ​​the first image region 142A, based on one of the two spectral images 72 selected in step ST22. The area derivation unit 106 also derives a second area A2, which is the area of ​​the first image region 142A, based on the other of the two spectral images 72 selected in step ST22 (see FIG. 10 ). After the processing of step ST24 is executed, the measurement processing proceeds to step ST26.

[0126] In step ST26, the area determination unit 108 determines whether the first area A1 and the second area A2 derived in step ST24 are different (see FIG. 10). If the first area A1 and the second area A2 are different, the determination is affirmative, and the measurement process proceeds to step ST28. If the first area A1 and the second area A2 are the same, the determination is negative, and the measurement process ends.

[0127] In step ST28, the boundary setting unit 110 selects one of the two spectral images 72 selected in step ST22. Then, the boundary setting unit 110 sets a boundary 144 between a first image region 142A and a second image region 142B for the object image 140 included in one of the selected spectral images 72 (see FIG. 11 ). After the processing of step ST28 is performed, the measurement processing proceeds to step ST30.

[0128] In step ST30, reference image area setting unit 112 sets the image area of ​​first image area 142A on the boundary 144 side as first reference image area 146A, and sets the image area of ​​second image area 142B on the boundary 144 side as second reference image area 146B. Then, reference image area setting unit 112 sets reference image area 146 including first reference image area 146A and second reference image area 146B for subject image 140 (see FIG. 11 ). After the processing of step ST30 is executed, the measurement processing proceeds to step ST32.

[0129] In step ST32, the average value derivation unit 114 calculates the average value S(REF) of the luminance of the image pixels included in the reference image region 146. ａｖｅ After the process of step ST30 is performed, the measurement process proceeds to step ST34.

[0130] In step ST34, evaluation image area setting unit 116 sets, for subject image 140, the image area of ​​first image area 142A other than first reference image area 146A as first evaluation image area 148A, and sets the image area of ​​second image area 142B other than second reference image area 146B as second evaluation image area 148B (see FIG. 12 ). After the processing of step ST34 is executed, the measurement processing proceeds to step ST36.

[0131] In step ST36, the evaluation value derivation unit 118 calculates an evaluation value S(EVE) for each image pixel included in the first evaluation image area 148A. ａ＋ Similarly, the evaluation value derivation unit 118 derives the evaluation value S(EVE) for each image pixel included in the second evaluation image region 148B using equation (7). ｂ＋ (See FIG. 13.) After the process of step ST36 is executed, the measurement process proceeds to step ST38.

[0132] In step ST38, the evaluation result output unit 120 outputs the evaluation value S(EVE) derived for each image pixel included in the first evaluation area 208A. ａ＋is within the first predetermined range. Similarly, the evaluation result output unit 120 outputs the first evaluation result 126A, which is the result of determining whether the evaluation value S(EVE) derived for each image pixel included in the second evaluation area 208B is within the first predetermined range. ｂ＋ is within the second predetermined range, and outputs a second evaluation result 126B (see FIG. 13 ). After the processing of step ST38 is executed, the measurement processing ends. Note that the processing method described above as the operation of the processing device 90 is an example of the "processing method" according to the technology of the present disclosure.

[0133] Next, the effects of this embodiment will be described.

[0134] As described above, in the processing device 90 according to this embodiment, the processor 94 acquires imaging data obtained by imaging the subject 200 having the first surface 202A and the second surface 202B using the imaging device 10, and acquires reference imaging data for the reference region 206 spanning the first surface 202A and the second surface 202B from the imaging data. The processor 94 then corrects the imaging data based on the reference imaging data. Therefore, measurement accuracy can be improved compared to when the spectral reflectance of the subject 200 is measured by using the imaging data as is.

[0135] Furthermore, the imaging data obtained by imaging the subject 200 with the imaging device 10 is corrected based on the reference imaging data. Therefore, the imaging data obtained each time the subject 200 is imaged with the imaging device can be corrected.

[0136] Correcting the imaging data also includes correcting data related to the non-uniformity of the spectral reflectance of the subject 200. Therefore, the measurement accuracy can be improved compared to when the non-uniformity of the spectral reflectance of the subject 200 is measured by using the imaging data as is.

[0137] Correcting the imaging data also includes deriving an average value of the spectral reflectance in the reference area 206 and correcting, based on the average value, data relating to the non-uniformity of the spectral reflectance in the first evaluation area 208A of the first surface 202A, which is different from the reference area 206. Therefore, measurement accuracy can be improved compared to measuring the non-uniformity of the spectral reflectance in the first evaluation area 208A by using the imaging data as is.

[0138] Correcting the imaging data also includes correcting, based on the average value, data relating to the non-uniformity of the spectral reflectance of the second evaluation area 208B on the second surface 202B, which is different from the reference area 206. Therefore, the measurement accuracy can be improved compared to when the non-uniformity of the spectral reflectance of the second evaluation area 208B is measured by using the imaging data as is.

[0139] The processor 94 also outputs a first evaluation result 126A relating to the degree of unevenness in the spectral reflectance of the first evaluation area 208A obtained by the correction, thereby making it possible to provide the user with the degree of unevenness in the spectral reflectance of the first evaluation area 208A.

[0140] The processor 94 also outputs a second evaluation result 126B relating to the degree of unevenness in the spectral reflectance of the second evaluation area 208B obtained by the correction, thereby making it possible to provide the user with the degree of unevenness in the spectral reflectance of the second evaluation area 208B.

[0141] Furthermore, the centers of gravity of the spectral filters 20 are located at positions different from the optical axis OA. Therefore, by utilizing the parallax occurring among the spectral filters 20 to determine whether the first area A1 and the second area A2 are different, it is possible to determine whether the first surface 202A and the second surface 202B form an angle.

[0142] Furthermore, the processor 94 acquires the reference imaging data when a calculation result indicating that the first surface 202A and the second surface 202B form an angle is obtained based on the imaging data, thereby eliminating the need to execute a process to acquire the reference imaging data even when the first surface 202A and the second surface 202B do not form an angle.

[0143] Furthermore, the calculation result is that the first area A1 derived based on one of the two selected spectral images 72 is different from the second area A2 derived based on the other of the two selected spectral images 72. Therefore, it is possible to determine whether the first surface 202A and the second surface 202B have an angle based on the two selected spectral images 72.

[0144] Furthermore, the first area A1 is an area derived based on the luminance distribution of one of the two selected spectral images 72, and the second area A2 is an area derived based on the luminance distribution of the other of the two selected spectral images 72. Therefore, the first area A1 and the second area A2 can be derived based on the spectral reflectance of the subject 200.

[0145] Furthermore, the reference area 206 is an area that is determined based on the boundary 144 between the first surface 202A and the second surface 202B. Therefore, the reference area 206 can be an area on the first surface 202A or the second surface 202B where there is little difference between the way light emitted from the first light source 132A hits the surface and the way light emitted from the second light source 132B hits the surface.

[0146] Furthermore, the first surface 202A and the second surface 202B are illuminated with light under different conditions. Therefore, for each selected spectral image 72, the object image 140 can be divided into a first image region 142A and a second image region 142B based on the luminance distribution of the image pixels.

[0147] The imaging device 10 is a multispectral camera, and therefore can acquire multiple spectral images 72 in one imaging session.

[0148] Next, a modification of this embodiment will be described.

[0149] In the above embodiment, the imaging data obtained by imaging the subject 200 with the imaging device 10 is corrected based on the reference imaging data. However, the imaging data may be used as first imaging data for obtaining the reference imaging data. Then, second imaging data obtained separately from the first imaging data (i.e., imaging data different from the first imaging data) may be corrected based on the reference imaging data.

[0150] In this way, even if second imaging data is obtained for each of multiple subjects 200 that are the same type of object, for example, the reference imaging data obtained in advance based on the first imaging data can be repeatedly used.

[0151] In the above embodiment, the subject 200 is a triangular prism, but the subject 200 may be any object having a first surface 202A and a second surface 202B. In addition, if the subject 200 is an object having three or more surfaces facing the imaging device 10, the first surface 202A and the second surface 202B may be selected from the multiple surfaces by a user or the like.

[0152] Furthermore, the first surface 202A and the second surface 202B are flat surfaces, but may be curved surfaces. Furthermore, if the first surface 202A and the second surface 202B are curved surfaces, they may be approximated to flat surfaces.

[0153] In addition, in the above embodiment, the processor 60 is exemplified for the imaging device 10, but instead of the processor 60, or together with the processor 60, at least one other CPU, at least one GPU, and / or at least one TPU may be used.

[0154] In addition, in the above embodiment, the processor 94 is exemplified as the processing device 90, but instead of the processor 94, or together with the processor 94, at least one other CPU, at least one GPU, and / or at least one TPU may be used.

[0155] In the above embodiment, the imaging device 10 has been described with reference to an example in which the spectral image generation program 80 is stored in the storage 62. However, the technology of the present disclosure is not limited to this. For example, the spectral image generation program 80 may be stored in a portable, non-transitory, computer-readable storage medium (hereinafter simply referred to as a "non-transitory storage medium") such as an SSD or a USB memory. The spectral image generation program 80 stored in the non-transitory storage medium may be installed in the computer 56 of the imaging device 10.

[0156] Alternatively, the spectral image generation program 80 may be stored in a storage device such as another computer or server device connected to the imaging device 10 via a network, and the spectral image generation program 80 may be downloaded and installed on the computer 56 of the imaging device 10 in response to a request from the imaging device 10.

[0157] Furthermore, it is not necessary to store the entire spectral image generation program 80 in a storage device such as another computer or server device connected to the imaging device 10, or in the storage 62; only a portion of the spectral image generation program 80 may be stored therein.

[0158] In the above embodiment, the processing device 90 has been described with reference to an example in which the measurement program 100 is stored in the storage 96, but the technology of the present disclosure is not limited to this. For example, the measurement program 100 may be stored in a non-transitory storage medium. The measurement program 100 stored in the non-transitory storage medium may be installed in the computer 92 of the processing device 90.

[0159] In addition, the measurement program 100 may be stored in a storage device such as another computer or server device connected to the processing device 90 via a network, and the measurement program 100 may be downloaded in response to a request from the processing device 90 and installed on the computer 92 of the processing device 90.

[0160] Furthermore, it is not necessary to store the entire measurement program 100 in a storage device such as another computer or server device connected to the processing device 90, or in the storage 96; only a portion of the measurement program 100 may be stored therein.

[0161] Furthermore, although the imaging device 10 has a built-in computer 56 , the technology of the present disclosure is not limited to this. For example, the computer 56 may be provided outside the imaging device 10 .

[0162] Furthermore, although the processing device 90 has a built-in computer 92 , the technology of the present disclosure is not limited to this, and for example, the computer 92 may be provided outside the processing device 90 .

[0163] In addition, in the above embodiment, the imaging device 10 is exemplified as a computer 56 including a processor 60, a storage 62, and a RAM 64, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 56. Furthermore, instead of the computer 56, a combination of a hardware configuration and a software configuration may be used.

[0164] In addition, in the above embodiment, the processing device 90 is exemplified as a computer 92 including a processor 94, a storage 96, and a RAM 98, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 92. Furthermore, instead of the computer 92, a combination of a hardware configuration and a software configuration may be used.

[0165] Furthermore, the following various processors can be used as hardware resources for executing the various processes described in the above embodiments. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing various processes by executing software, i.e., a program. Examples of processors include dedicated electronic circuits, such as FPGAs, PLDs, and ASICs, which are processors with a circuit configuration designed specifically for executing specific processes. Each processor has built-in or connected memory, and each processor uses the memory to execute various processes.

[0166] The hardware resources that execute various processes may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resources that execute various processes may be a single processor.

[0167] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute various processes, on a single IC chip, as typified by SoC. In this way, various processes are realized using one or more of the above-mentioned various processors as hardware resources.

[0168] Furthermore, the hardware structure of these various processors can be, more specifically, electronic circuits that combine circuit elements such as semiconductor devices. The above-described gaze detection process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0169] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0170] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0171] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processing device comprising a processor, wherein the processor obtains first imaging data obtained by imaging a subject having a first surface and a second surface with an imaging device, obtains reference imaging data in a reference region spanning the first surface and the second surface from the first imaging data, and corrects second imaging data obtained by imaging with the imaging device based on the reference imaging data.

2. The first imaging data includes the second imaging data, The processing device according to claim 1.

3. The second imaging data is different from the first imaging data, The processing device according to claim 1.

4. Correcting the second imaging data includes correcting data regarding non-uniformity of the spectral reflectance of the subject, The processing device according to claim 1.

5. The imaging device is a spectral imaging device, The processing device according to claim 1.

6. The spectral imaging device includes a plurality of spectral filters, The center-of-gravity positions of the plurality of spectral filters are at positions different from the optical axis, The processing device according to claim 5.

7. When an operation result indicating that the first surface and the second surface have an angle is obtained based on the first imaging data, the processor obtains the reference imaging data, The processing device according to claim 1.

8. The imaging device is a spectral imaging device, The spectral imaging device includes a plurality of spectral filters, The center-of-gravity positions of the plurality of spectral filters are at positions different from the optical axis, The operation result is an operation result in which a first area, which is an area of an image region corresponding to the first surface derived based on a first wavelength band image corresponding to a first spectral filter among the plurality of spectral filters, and a second area, which is an area of an image region corresponding to the first surface derived based on a second wavelength band image corresponding to a second spectral filter among the plurality of spectral filters, are different, The processing device according to claim 7.

9. The first area is an area derived based on the luminance distribution of the first wavelength band image, The second area is an area derived based on the luminance distribution of the second wavelength band image, The processing device according to claim 8.

10. The reference region is a region determined based on a boundary between the first surface and the second surface, The processing device according to claim 1.

11. Correcting the second captured data includes calculating an average value of the spectral reflectance in the reference region, and correcting data regarding non-uniformity of the spectral reflectance in a first evaluation region different from the reference region on the first surface based on the average value. The processing apparatus according to claim 1.

12. Correcting the second captured data includes correcting data regarding non-uniformity of the spectral reflectance in a second evaluation region different from the reference region on the second surface based on the average value. The processing apparatus according to claim 11.

13. The processor outputs a first evaluation result regarding the degree of unevenness of the spectral reflectance in the first evaluation region obtained by the correcting. The processing apparatus according to claim 11.

14. The processor outputs a second evaluation result regarding the degree of unevenness of the spectral reflectance in the second evaluation region obtained by the correcting. The processing apparatus according to claim 12.

15. The first surface and the second surface are irradiated with light under different conditions. The processing apparatus according to claim 1.

16. The processing apparatus according to claim 1, the imaging device, and an inspection apparatus comprising the same.

17. Obtaining first captured data obtained by imaging a subject having a first surface and a second surface with an imaging device, obtaining reference captured data in a reference region straddling the first surface and the second surface from the first captured data, and correcting second captured data obtained by imaging with the imaging device based on the reference captured data. A processing method comprising the above.

18. Obtaining first captured data obtained by imaging a subject having a first surface and a second surface with an imaging device, obtaining reference captured data in a reference region straddling the first surface and the second surface from the first captured data, and correcting second captured data obtained by imaging with the imaging device based on the reference captured data. A program for causing a computer to execute a process including the above.