Imaging method and program

Abstract

Provided are an image-capturing method and program which can obtain a wavelength image with high wave reproducibility maintained. The image-capturing method captures subjects with a multi-spectrum camera that includes a processor, wherein the processor executes: a data acquisition step (step S10) for acquiring first spectrum data of a first subject, second spectrum data of a second subject, and third spectrum data of a third subject; a wavelength selection step (step S11) for selecting a plurality of wavelengths from wavelength bands of the acquired first to third spectrum data; and an image-capturing step (step S12) for selecting a plurality of wavelengths by obtaining at least two calculation amounts that are determined, in the wavelength selection step (step S11), as the difference or ratio between feature amounts of two pieces among the first spectrum data, the second spectrum data, or the third spectrum data, and capturing subjects including at least one among the first subject, the second subject, or the third subject in the plurality of wavelengths.

Description

Technical Field

[0001] The present invention relates to an imaging method and a program, and particularly to an imaging method and a program for a multispectral camera.

Background Art

[0002] Conventionally, images for different wavelengths have been obtained using a multispectral camera and applied to various applications.

[0003] For example, Patent Document 1 describes a technique for estimating the color of a subject using the spectral sensitivity characteristics of a camera measured in advance.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

[0005] One embodiment according to the technology of the present disclosure provides an imaging method and a program capable of obtaining a wavelength image maintaining high wavelength reproducibility.

Means for Solving the Problems

[0006] One aspect of the present invention is an imaging method for imaging a subject using a multispectral camera equipped with a processor, wherein the processor performs a data acquisition step of acquiring first spectral data of a first subject, second spectral data of a second subject, and third spectral data of a third subject; a wavelength selection step of selecting a plurality of wavelengths from the wavelength range of the acquired first to third spectral data; and an imaging step of selecting a plurality of wavelengths based on at least two of the factors, using the difference or ratio of feature quantities of two of the first spectral data, second spectral data, and third spectral data as factors, and imaging a subject including at least one of the first subject, second subject, and third subject at the plurality of wavelengths.

[0007] Preferably, the wavelength ranges of the first to third spectral data are wavelength ranges where at least the wavelength range of the first spectral data and the wavelength range of the second spectral data overlap.

[0008] Preferably, the factors include a first factor which is the difference or ratio of the feature quantities of the first spectral data and the third spectral data, and a second factor which is the difference or ratio of the feature quantities of the second spectral data and the third spectral data, and multiple wavelengths are selected based on the first factor and the second factor.

[0009] Preferably, one of the multiple wavelengths is the wavelength at which at least one of the first and second factors is minimized.

[0010] Preferably, the process further includes a step of measuring the intensity ratio of luminances of multiple wavelengths within multiple regions of the image data of the third subject using the multiple wavelengths selected in the wavelength selection step, and a correction step of correcting at least the image data of the third subject based on the intensity ratio.

[0011] Preferably, the correction step includes a first correction step of correcting the intensity ratios of multiple regions based on a first intensity ratio, which is one of the measured intensity ratios.

[0012] Preferably, the correction step includes a second correction step that corrects the difference in the measured intensity ratio to reduce it.

[0013] Preferably, the correction step includes a first correction step of correcting the intensity ratios of multiple regions based on a first intensity ratio, which is one of the measured intensity ratios, and a second correction step of correcting the difference between the measured intensity ratios.

[0014] Preferably, the third subject has a constant intensity ratio of the luminances of multiple wavelengths selected in the wavelength selection process across multiple regions.

[0015] Preferably, in the wavelength selection step, a first wavelength and a second wavelength are selected such that the reflectance α of the first subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following relationship (1), and the reflectance γ of the second subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following relationship (2), where (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first wavelength. |β-α|÷(β+α)≦0.15···(1) |β-γ|÷(β+γ)≦0.15···(2)

[0016] Preferably, in the wavelength selection step, a first wavelength and a second wavelength are selected such that the reflectance α of the first subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following relationship (3), and the reflectance γ of the second subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following relationship (4), where (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first wavelength. |β-α|÷(β+α)≦0.05···(3) |β-γ|÷(β+γ)≦0.05···(4)

[0017] Preferably, in the wavelength selection step, a first wavelength, a second wavelength, and a third wavelength are selected such that the reflectance α of the first subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (5), and the reflectance γ of the second subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (6), where (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first and third wavelengths. |β-α|÷(β+α)≦0.15···(5) |β-γ|÷(β+γ)≦0.15···(6)

[0018] Preferably, in the wavelength selection step, a first wavelength, a second wavelength, and a third wavelength are selected such that the reflectance α of the first subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (7), and the reflectance γ of the second subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (8), where (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first and third wavelengths. |β-α|÷(β+α)≦0.05···(7) |β-γ|÷(β+γ)≦0.05···(8)

[0019] Preferably, the system includes a memory that stores a plurality of replaceable subjects that can be replaced by a third subject, and a fourth spectral data for each of the plurality of replaceable subjects. The processor performs a notification step of notifying that one of the plurality of replaceable subjects is the recommended third subject based on the first spectral data, the second spectral data, and the fourth spectral data.

[0020] Preferably, the processor performs a sensitivity correction step to correct the sensitivity of the image sensor of the multispectral camera based on the correction step.

[0021] Preferably, a third subject variable display step is further performed, in which the third subject is displayed in a variable manner.

[0022] Preferably, an irradiation process of shadowless illumination is further performed.

[0023] Another aspect of the present invention, a program, is a program for causing a multi-spectral camera including a processor to execute an imaging method for imaging a subject, the program causing the processor to perform a data acquisition step of acquiring first spectral data of a first subject, second spectral data of a second subject, and third spectral data of a third subject, a wavelength selection step of selecting a plurality of wavelengths from the wavelength ranges of the acquired first to third spectral data, and in the wavelength selection step, selecting a plurality of wavelengths based on at least two or more factors, the factors being the difference or ratio of the feature amounts of two spectral data among the first spectral data, the second spectral data, and the third spectral data, and an imaging step of imaging a subject including at least one of the first subject, the second subject, and the third subject at the plurality of wavelengths.

Brief Description of the Drawings

[0024] [Figure 1] FIG. 1 is a diagram for explaining a first example in which a wavelength image becomes inaccurate due to a change in the surrounding environment. [Figure 2] FIG. 2 is a diagram for explaining a first example in which a wavelength image becomes inaccurate due to a change in the surrounding environment. [Figure 3] FIG. 3 is a diagram for explaining a second example in which a wavelength image becomes inaccurate due to a change in the surrounding environment. [Figure 4] FIG. 4 is a diagram for explaining a second example in which a wavelength image becomes inaccurate due to a change in the surrounding environment. [Figure 5] FIG. 5 is a flowchart showing an imaging method. [Figure 6] FIG. 6 is a schematic diagram showing a subject imaged by a multi-spectral camera. [Figure 7] FIG. 7 is a diagram showing spectral data of a background, a first subject, and a second subject. [Figure 8] FIG. 8 is a diagram showing spectral data of a background, a first subject, and a second subject. [Figure 9]Figure 9 shows the case where equations (5), (6), and condition 1 are satisfied. [Figure 10] Figure 10 illustrates the imaging process using a multispectral camera. [Figure 11] Figure 11 is a flowchart showing the imaging method. [Figure 12] Figure 12 illustrates the imaging of the background during the intensity ratio measurement process. [Figure 13] Figure 13 illustrates the intensity ratio of the background brightness measured in the intensity ratio measurement process and the first correction coefficient that is calculated. [Figure 14] Figure 14 illustrates the intensity ratio of the background brightness measured in the intensity ratio measurement process and the first correction coefficient that is calculated. [Figure 15] Figure 15 is a flowchart showing the imaging method. [Figure 16] Figure 16 illustrates the intensity ratio of the background brightness measured in the intensity ratio measurement process, as well as the first and second correction coefficients calculated therein. [Figure 17] Figure 17 illustrates the intensity ratio of the background brightness measured in the intensity ratio measurement process, as well as the first and second correction coefficients calculated therein. [Figure 18] Figure 18 illustrates an example of a recommended background. [Figure 19] Figure 19 illustrates an example of a recommended background. [Figure 20] Figure 20 is a schematic diagram showing an example of a multispectral camera. [Figure 21] Figure 21 is a schematic diagram showing an example of a filter unit included in a multispectral camera. [Modes for carrying out the invention]

[0025] Preferred embodiments of the imaging method and program according to the present invention will be described below with reference to the attached drawings.

[0026] First, I will explain why the wavelength image may become inaccurate (wavelength reproducibility may decrease).

[0027] Conventionally, multispectral cameras have been used to acquire images corresponding to each wavelength (wavelength images). For example, a multispectral camera using polarizing pixels (e.g., 0°, 45°, 90°) (see Figure 20) can obtain wavelength images for each wavelength by assigning a different wavelength to each polarizing pixel. In the process of generating wavelength images, interference rejection is generally performed. Interference rejection is the process of removing signals related to other wavelengths that interfered during imaging, and it involves removing the interfering wavelengths based on a pre-measured mixing ratio (inverse matrix operation). By performing this interference rejection appropriately, accurate wavelength images can be obtained in which interference from signals of other wavelengths is suppressed.

[0028] Generally, the mixing ratio is measured before actual imaging, such as during the manufacturing of a multispectral camera, and is often stored and used in the camera's built-in memory. Therefore, there are cases where the pre-measured mixing ratio differs from the mixing ratio used during actual imaging. When the pre-measured mixing ratio differs from the mixing ratio used during actual imaging, interference rejection may not be performed effectively, and the resulting wavelength image may become inaccurate (wavelength reproducibility decreases). One factor that causes the pre-measured mixing ratio to differ from the mixing ratio used during actual imaging is a change in the surrounding environment. Specifically, if the surrounding environment when the mixing ratio is measured differs from the surrounding environment when the image is actually taken, flare changes and the mixing ratio changes. The following explains the effect of changes in the surrounding environment on wavelength images using specific examples.

[0029] Figures 1 and 2 illustrate the first example of how changes in the surrounding environment can lead to inaccurate wavelength images.

[0030] Images 501, 503, and 505 are images of a white plate with uniform reflection characteristics captured by a multispectral camera at various wavelengths. Image 501 is a U-wavelength image of wavelength U, image 503 is a V-wavelength image of wavelength V, and image 505 is a W-wavelength image of wavelength W. Image 507 is a pseudo-color image obtained by replacing, for example, wavelength U with blue (B: BLUE), wavelength V with green (G: GREEN), and wavelength W with red (R: RED). Figure 2 shows the intensity ratios of luminance for wavelengths U, V, and W in regions (A) to (I) of image 507, as shown in the corresponding graphs (A) to (I).

[0031] Ideally, when a white plate with uniform reflective properties is imaged using the multispectral camera 100, the intensity ratio of luminance in regions (A) to (I) should be uniform. Furthermore, ideally, when a white plate with uniform reflective properties is imaged using the multispectral camera 100, the intensity ratio of luminance between image 501 (U-wavelength image), image 503 (V-wavelength image), and image 505 (W-wavelength image) should also be equal. However, because the surrounding environment when images 501, 503, and 505 were actually imaged differs from the surrounding environment when the mixing ratio was measured, as shown in the figure, the intensity ratios of luminance shown by images 501, 503, and 505 differ in regions (A) to (I). Also, the intensity ratios of luminance between images 501, 503, and 505 are not equal (for example, in region (E), the difference in intensity ratios between images 501, 503, and 505 is approximately 2%).

[0032] Thus, because the surrounding environment differs between the time the mixing ratio is measured and the time of actual imaging, flare and other phenomena may change, making it impossible to obtain accurate wavelength images (while maintaining wavelength reproducibility).

[0033] Figures 3 and 4 illustrate a second example where the wavelength image becomes inaccurate due to changes in the surrounding environment. In this example, a black subject 519 is imaged against a white plate with uniform reflective properties as the background.

[0034] Image 511 is a U-wavelength image of wavelength U, Image 513 is a V-wavelength image of wavelength V, and Image 515 is a W-wavelength image of wavelength W. Image 517 is a pseudo-color image generated under the same conditions as described in Figure 1. Figure 4 shows the intensity ratios of luminance at wavelengths U, V, and W in regions (A) to (I) of Image 517, represented by corresponding graphs (A) to (I).

[0035] As shown in the figure, the luminance intensity ratios shown by images 511, 513, and 515 differ in regions (A) to (I) due to differences in the surrounding environment when images 511, 513, and 515 were captured and when the mixing ratio was measured. Furthermore, the luminance intensity ratios between images 511, 513, and 515 are not equal (for example, in region (E), the difference in luminance intensity ratios between images 511, 513, and 515 is approximately 6%).

[0036] Thus, because the surrounding environment differs between the time the mixing ratio is obtained and the time of actual imaging, flare and other factors may change, making it impossible to obtain accurate wavelength images (with maintained wavelength reproducibility). Furthermore, in cases where there is a large difference in reflectivity between the background and the subject, such as with black subjects 519, the deviation in the intensity ratio of brightness between wavelengths becomes even larger, reducing wavelength reproducibility.

[0037] As shown in the example above, simply performing interference rejection when the mixing ratio has changed will result in obtaining inaccurate wavelength images.

[0038] Therefore, in the embodiments disclosed below, by suppressing changes in flare, etc., or by correcting the wavelength image to match the flare, etc. that occurs in the initial stages of imaging, an accurate wavelength image can be obtained even if the surrounding environment changes (wavelength reproducibility can be maintained).

[0039] <First Embodiment> Figure 5 is a flowchart of the imaging method of the present invention. Each step of the imaging method described below is performed by the processor 142 of the multispectral camera 100 (Figure 20). The processor 142 performs each step by executing a dedicated program for the imaging method stored in the memory 144.

[0040] In the data acquisition step (step S10), the processor 142 acquires spectral data of background 2, first subject 3, and second subject 4. Next, in the wavelength selection step (step S11), the processor 142 selects multiple wavelengths from the wavelength range (wavelength band) of the spectral data of background 2, first subject 3, and second subject 4. In this example, the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) are selected. Next, in the imaging step (step S12), the processor 142 images a subject that includes at least one of background 2, first subject 3, and second subject 4 using the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3).

[0041] The following provides a detailed explanation of each step.

[0042] [Data acquisition process] In the data acquisition process, the processor 142 acquires spectral data of the subjects (background 2, first subject 3, and second subject 4).

[0043] Figure 6 is a schematic diagram showing the subjects (first subject, second subject, and background (third subject)) captured by the multispectral camera 100.

[0044] The subject consists of a first subject 3, a second subject 4, and a background 2. The multispectral camera 100 images the subject, which includes at least one of the background 2, the first subject 3, and the second subject 4, and performs sensing to identify the first subject 3 and the second subject 4 located on the background 2. Accordingly, the processor 142 acquires spectral data regarding the reflectance of the background 2, the first subject 3, and the second subject 4.

[0045] Figures 7 and 8 show the spectral data for background 2, first subject 3, and second subject 4.

[0046] In Figure 7, the reflectances corresponding to the wavelengths of background 2, first subject 3, and second subject 4 are displayed as spectral data. Spectral data SD1 (first spectral data) corresponds to first subject 3, spectral data SD2 (second spectral data) corresponds to second subject 4, and spectral data SD3 (third spectral data) corresponds to background 2. The wavelength range of spectral data SD1, spectral data SD2, and spectral data SD3 acquired by processor 142 is the overlapping wavelength range of spectral data SD1 and spectral data SD2. In other words, in the case shown in Figure 7, the wavelength range of spectral data SD1 and spectral data SD2 is 400nm to 1000nm, and spectral data SD3 is in the wavelength range of 400nm to 1000nm, so the wavelength ranges of spectral data SD1 to SD3 overlap at 400nm to 1000nm. Therefore, processor 142 acquires spectral data SD1 to SD3 in the wavelength range of 400nm to 1000nm.

[0047] In Figure 8, similar to Figure 7, the reflectances corresponding to the wavelengths of background 2, first subject 3, and second subject 4 are displayed as spectral data. Spectral data SD1 corresponds to first subject 3, spectral data SD2 corresponds to second subject 4, and spectral data SD3 corresponds to background 2.

[0048] In the case shown in Figure 8, the wavelength ranges of spectral data SD1 and spectral data SD2 overlap in the 550nm to 1000nm range. Therefore, the wavelength range of spectral data SD3 is also sufficiently within the 550nm to 1000nm range, and processor 142 acquires spectral data SD1 to SD3 in the 550nm to 1000nm wavelength range.

[0049] The processor 142 acquires spectral data SD1 to SD3 in various forms. For example, spectral data SD1 to SD3 may be acquired by a hyperspectral camera when it images the background 2, the first subject 3, and the second subject 4, and input to the multispectral camera 100. Alternatively, spectral data SD1 to SD3 may be selected from a database containing various spectral data, input to the multispectral camera 100, and acquired by the processor 142. Furthermore, spectral data SD1 to SD3 may be pre-stored in the memory 144 of the multispectral camera 100, and the processor 142 may acquire them from there.

[0050] [Wavelength selection process] In wavelength acquisition and selection, the processor 142 selects the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) based on the acquired spectral data SD1, spectral data SD2, and spectral data SD3. The processor 142 uses the difference or ratio of the feature quantities of two of the spectral data SD1, spectral data SD2, and spectral data SD3 as factors, and selects multiple wavelengths by obtaining at least two such factors. The following explanation will focus on the case where the difference in the feature quantities (reflectance) of the spectral data is used as a factor.

[0051] In conventional techniques, in the spectral data SD1, spectral data SD2, and spectral data SD3 explained in Figure 7, it is common to use wavelengths with a small difference (reference wavelength) and wavelengths with a large difference between at least the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4. Therefore, conventionally, for example, 420 nm has been selected as at least the second wavelength λ(2) (reference wavelength), and 500 nm has been selected as the first wavelength λ(1).

[0052] However, in this embodiment, the processor 142 selects the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) based on the relationship between the difference in reflectance between spectral data SD3 and spectral data SD1 (first factor) and the difference in reflectance between spectral data SD3 and spectral data SD2 (second factor). For example, the processor 142 selects 850 nm as the second wavelength λ(2) (reference wavelength), where the first and second factors are minimized. The processor 142 may also select a wavelength as the reference wavelength where at least one of the first and second factors is minimized. Alternatively, the processor 142 selects 925 nm as the first wavelength λ(1), where the first and second factors are greater than in the case of the second wavelength λ(2), and where the first subject 3 and the second subject 4 can be distinguished. For example, the first wavelength λ(1) may be selected as the wavelength where the sum of the first and second factors is maximized within a predetermined range (15% or 5%) described below. Furthermore, the processor 142 selects the third wavelength λ(3) as 875 nm, which is between the first wavelength λ(1) and the second wavelength λ(2).

[0053] Furthermore, the processor 142 can select a first wavelength λ(1) to a third wavelength λ(3) in which the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2) are within a 15% range.

[0054] Specifically, the processor 142 selects the first wavelength λ(1) to the third wavelength λ(3) such that the following equation is satisfied. |β-α|÷(β+α)≦0.15···(5) |β-γ|÷(β+γ)≦0.15···(6)

[0055] In the above equation, α is input to the reflectances α1 to α3 of the first subject 3 at the first wavelength λ(1) to the third wavelength λ(3), β is input to the reflectances β1 to β3 of the background 2 at the first wavelength λ(1) to the third wavelength λ(3), and γ is input to the reflectances γ1 to γ3 of the second subject 4 at the first wavelength λ(1) to the third wavelength λ(3). In this case, (β-α) and (β-γ) at the second wavelength are selected from the first wavelength λ(1) to the third wavelength λ(3) that are smaller than the corresponding values ​​for the first wavelength λ(1) and the third wavelength λ(3) (Condition 1).

[0056] Figure 9 shows the case where equations (5), (6), and condition 1 are satisfied. Figure 9 also shows spectral data SD1 to SD3, and the first wavelength λ(1) to the third wavelength λ(3).

[0057] As shown in Figure 9, the first wavelength λ(1) to λ(3) is selected such that the differences M1, N1, M2 (not shown because it is 0), N2 (not shown because it is 0), M3, and N3 are within 15%. Furthermore, the first wavelength λ(1) to third wavelength λ(3) is selected such that the differences M2 and N2 are smaller than the differences M1, N1, M3, and N3.

[0058] Furthermore, more preferably, the processor 142 can select a first wavelength λ(1) to a third wavelength λ(3) in which the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2) are within the range of 5%.

[0059] Specifically, the processor 142 selects the first wavelength λ(1) to the third wavelength λ(3) such that the following equation is satisfied. |β-α|÷(β+α)≦0.05···(7) |β-γ|÷(β+γ)≦0.05···(8)

[0060] In the above equation, α is input to the reflectances α1 to α3 of the first subject 3 at the first wavelength λ(1) to the third wavelength λ(3), β is input to the reflectances β1 to β3 of the background 2 at the first wavelength λ(1) to the third wavelength λ(3), and γ is input to the reflectances γ1 to γ3 of the second subject 4 at the first wavelength λ(1) to the third wavelength λ(3). In this case, (β-α) and (β-γ) at the second wavelength are selected from the first wavelength λ(1) to the third wavelength λ(3) that are smaller than the corresponding values ​​for the first wavelength λ(1) and the third wavelength λ(3) (Condition 1).

[0061] As explained above, the processor 142 selects the first wavelength λ(1) to the third wavelength λ(3) based on factors derived from spectral data SD1 to SD3. This ensures that the difference from the reflectance of background 2 remains within a certain range, thereby suppressing the effects of changes in the surrounding environment.

[0062] In the above explanation, we described the case in which the processor 142 automatically selects the first wavelength λ(1) to the third wavelength λ(3) from the acquired spectral data SD1 to spectral data SD3. In the wavelength selection process, the processor 142 may select the first wavelength λ(1) to the third wavelength λ(3) in other ways. For example, the processor 142 may display the acquired spectral data SD1 to spectral data SD3 on a display unit provided on the back of the multispectral camera 100, the user may select the first wavelength λ(1) to the third wavelength λ(3) from the display, and the processor 142 may select the first wavelength λ(1) to the third wavelength λ(3) based on the user's instructions input via the operation unit (not shown) of the multispectral camera 100.

[0063] [Imaging Process] In the imaging process, the processor 142 images a subject that includes at least one of the first subject 3, the second subject 4, and the background 2 (third subject) at a first wavelength λ(1), a second wavelength λ(2), and a third wavelength λ(3).

[0064] Figure 10 illustrates the imaging process using the multispectral camera 100. In Figure 10, the background 2 is a conveyor belt, and a specific example is shown in which the first subject 3 and the second subject 4 are placed on it and continuously transported. The multispectral camera 100 images the first subject 3 and the second subject 4 as they are continuously transported.

[0065] The processor 142 causes the multispectral camera 100 to capture images of the scene with the first subject 3 and the second subject 4 located on the background 2. Depending on the situation, the multispectral camera 100 may capture images of the background 2 only, the first subject 3 only on the background 2, or the second subject 4 only on the background 2. The multispectral camera 100 is configured to acquire first-wavelength images to third-wavelength images corresponding to the first wavelength λ(1) to third wavelength λ(3) selected in the wavelength selection step. The illumination device 10 illuminates the background 2, the first subject 3, and the second subject 4 in conjunction with the imaging of the multispectral camera 100 or continuously (illumination step). The illumination device 10 uses light source a, light source b, or a light source that is a mixture of light source a and light source b. Furthermore, since the shadows of the first subject 3 and the second subject 4 do not have a good effect on the acquisition of accurate first-wavelength images to third-wavelength images, it is preferable that the illumination device 10 is a shadowless illumination device. The multispectral camera 100 performs interference rejection processing on the acquired first to third wavelength images as needed.

[0066] The multispectral camera 100 performs sensing to identify the first subject 3 and the second subject 4 (detecting their presence or absence, etc.) based on the captured first to third wavelength images.

[0067] As explained above, according to the imaging method of this embodiment, even if flare and the like change depending on the surrounding environment during imaging, the first wavelength λ(1) to the third wavelength(3) are selected in which the reflectance factors of the first subject 3, the second subject 4, and the background 2 are within a predetermined range. Therefore, the influence of changes in the surrounding environment is suppressed, and wavelength images with high wavelength reproducibility can be acquired.

[0068] <Second Embodiment> Next, a second embodiment of the present invention will be described. In this embodiment, in addition to the imaging method of the first embodiment described above, an intensity ratio measurement step and a correction step (first correction step) are included.

[0069] Figure 11 is a flowchart of the imaging method according to this embodiment. The imaging method described below is performed by the processor 142 of the multispectral camera 100. The processor 142 performs each step by executing a dedicated program for the imaging method stored in the memory 144.

[0070] In the data acquisition step (step S20), the processor 142 acquires spectral data of background 2, first subject 3, and second subject 4. Next, in the wavelength selection step (step S21), the processor 142 selects the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) from the wavelength range of the spectral data of background 2, first subject 3, and second subject 4. Next, in the intensity ratio measurement step (step S22), the processor 142 images background 2 using the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3), acquires wavelength images (image data) of background 2 at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3), and measures the intensity ratio of the brightness from the first wavelength λ(1) to the third wavelength λ(3) within multiple regions. Next, in the imaging process (step S23), the processor 142 images a subject including at least one of the background 2, the first subject 3, and the second subject 4 using the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3). Then, in the first correction process (step S24), the processor 142 corrects the intensity ratios of multiple regions based on the first intensity ratio, which is one of the measured intensity ratios.

[0071] Next, the intensity ratio measurement process and the first correction process, which are characteristic parts of this embodiment, will be described. Note that the other processes in this embodiment (data acquisition process, wavelength selection process, and imaging process) are the same as in the first embodiment, so their description will be omitted.

[0072] [Intensity ratio measurement process] In the intensity ratio measurement process, the processor 142 measures the intensity ratio of the brightness of background 2 using the first wavelength λ(1) to the third wavelength λ(3). Specifically, the processor 142 uses the multispectral camera 100 to image only background 2 using the first wavelength λ(1) to the third wavelength λ(3), and acquires first wavelength images to third wavelength images.

[0073] Figure 12 illustrates the imaging of background 2 during the intensity ratio measurement process.

[0074] In the intensity ratio measurement process, the background 2 is imaged by the multispectral camera 100, and first-wavelength, second-wavelength, and third-wavelength images of the background 2 are acquired. The background 2 imaged here is the same as the background 2 imaged in the imaging process. For example, a white plate with uniform reflectivity at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) is used for the background 2. Alternatively, for example, a chart in which the intensity ratio at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) is constant in multiple regions is used for the background 2. The illumination device 10 is the illumination device used in the imaging process. It is preferable that the same light source is used in both the intensity ratio measurement process and the imaging process.

[0075] Figures 13 and 14 illustrate the intensity ratio of the luminance of background 2 measured in the intensity ratio measurement process and the first correction coefficient calculated. Figure 13 shows the measurement results from light source a, and Figure 14 shows the measurement results from light source b.

[0076] The pseudo-color image 101 (Figure 13) is generated based on the first to third wavelength images. Specifically, the pseudo-color image 101 is generated by superimposing the intensity ratios of the brightness of the first wavelength image as red (R:RED), the intensity ratio of the brightness of the second wavelength image as blue (B:BLUE), and the intensity ratio of the brightness of the third wavelength image as green (G:GREEN). Furthermore, in reference numeral 105, the intensity ratios of the brightness at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) of regions (A) to (I) of the pseudo-color image 101 are shown correspondingly in graphs (A) to (I). Note that the intensity ratios of the brightness shown in graphs (A) to (I) represent the average or representative value of the intensity ratios of the brightness of each region from region (A) to (I).

[0077] As shown in the pseudo-color image 101 and graphs (A) to (I) indicated by reference numeral 105, variations occur in the intensity ratio of luminance at each wavelength in the first to third wavelength images due to the influence of changes in the surrounding environment. Therefore, the processor 142 obtains a first correction coefficient to equalize the intensity ratio of luminance at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) in regions (A) to (I). For example, the processor 142 obtains a first correction coefficient to match the intensity ratio (first intensity ratio) of luminance in the central region (E) of the pseudo-color image 101 with the intensity ratios of regions (A) to (D) and regions (F) to (I). The pseudo-color image 103 is obtained by applying the first correction coefficient to the pseudo-color image 101. The pseudo-color image 103 has a uniform luminance intensity ratio, as shown in graphs (A) to (I) indicated by reference numeral 107. That is, the first correction coefficient makes the luminance intensity ratios of regions (A) to (I) the same.

[0078] The pseudo-color image 109 (Figure 14) is generated based on the first to third wavelength images, similar to the pseudo-color image 101. In addition, reference numeral 113 indicates the intensity ratios of the luminance at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) in regions (A) to (I) of the pseudo-color image 109, as shown in graphs (A) to (I).

[0079] As shown in the pseudo-color image 109 and graphs (A) to (I) indicated by reference numeral 113, variations in the intensity ratio of luminance at each wavelength occur in the first to third wavelength images due to the influence of changes in the surrounding environment. Therefore, the processor 142 obtains a first correction coefficient that matches the intensity ratio of regions (A) to (D) and regions (F) to (I) to the intensity ratio of luminance in the central region (E) of the pseudo-color image 109 (first intensity ratio). Applying the first correction coefficient to the pseudo-color image 109 yields the pseudo-color image 111. As shown in the graphs (A) to (I) indicated by reference numeral 115, the pseudo-color image 111 has a uniform luminance intensity ratio. That is, the first correction coefficient makes the intensity ratio of luminance in regions (A) to (I) uniform.

[0080] As explained above, in the intensity ratio measurement process, the intensity ratio of the brightness of the background 2 at wavelengths 1 to 3 is measured, and a first correction coefficient is calculated based on the measurement results. In the above explanation, an example was described in which the image is divided into 3x3 9 regions and the first correction coefficient is obtained for each region, but this embodiment is not limited to this. For example, the first correction coefficient may be obtained for each pixel of the image, or the first correction coefficient may be obtained for a larger or smaller region.

[0081] [Correction process] In the correction step (first correction step), the processor 142 applies the first correction coefficient obtained in the intensity ratio measurement step to the first wavelength image, second wavelength image, and third wavelength image captured in the imaging step to correct the intensity ratio of the brightness of regions (A) to (I). While it is preferable to apply the first correction coefficient to the entire first to third wavelength image, it may also be applied, for example, to the portion of the first to third wavelength image where background 2 is visible (image data of background 2). The first, second, and third wavelength images corrected with the first correction coefficient are less affected by changes in the surrounding environment. Therefore, first to third wavelength images with maintained wavelength reproducibility can be obtained, enabling accurate sensing.

[0082] <Third Embodiment> Next, a third embodiment of the present invention will be described. In this embodiment, in addition to the imaging method of the second embodiment described above, a correction step (second correction step) is included.

[0083] Figure 15 is a flowchart showing the imaging method of this embodiment. The imaging method described below is performed by the processor 142 of the multispectral camera 100. The processor 142 performs each step by executing a dedicated program for the imaging method stored in the memory 144.

[0084] In the data acquisition step (step S30), the processor 142 acquires spectral data of background 2, first subject 3, and second subject 4. Next, in the wavelength selection step (step S31), the processor 142 selects the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) from the wavelength range of the spectral data of background 2, first subject 3, and second subject 4. Next, in the intensity ratio measurement step (step S32), the processor 142 images background 2 using the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3), acquires wavelength images (image data) of background 2 at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3), and measures the intensity ratio of luminance at multiple wavelengths within multiple regions. Next, in the imaging step (step S33), the processor 142 images a subject including at least one of the background 2, the first subject 3, and the second subject 4 using the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3). Next, in the first correction step (step S34), the processor 142 corrects the intensity ratios of multiple regions based on the first intensity ratio, which is one of the intensity ratios measured in the intensity ratio measurement step. Next, in the second correction step (step S35), the processor 142 performs a correction to reduce the difference in the intensity ratios measured in the intensity ratio measurement step.

[0085] Next, the intensity ratio measurement process and the second correction process, which are characteristic parts of this embodiment, will be described. Note that the data acquisition process, wavelength selection process, imaging process, and first correction process are the same as in the first and second embodiments, so their explanation will be omitted here.

[0086] [Intensity ratio measurement process] In the intensity ratio measurement process of this embodiment, a second correction coefficient is calculated in addition to the first correction coefficient. The following description will focus on the case where the first and second correction coefficients are calculated separately, but the description is not limited to this. For example, a correction coefficient may be calculated by combining the first and second correction coefficients.

[0087] Figures 16 and 17 illustrate the intensity ratio of the luminance of background 2 measured in the intensity ratio measurement process, and the first and second correction coefficients calculated thereafter. In the following explanation, parts already explained in Figures 13 and 14 are denoted by the same reference numerals and their explanations are omitted.

[0088] The second correction factor is a correction factor that reduces the difference in the intensity ratio of luminance at each wavelength in region (A) to region (I). For example, the second correction factor is a correction factor that makes the luminance intensity between the first wavelength λ(1) and the third wavelength λ(3) the same in region (A) to region (I).

[0089] In the pseudo-color image 103 (Figure 16), the intensity ratio of luminance in regions (A) to (I) is uniform due to the first correction coefficient. However, as shown in the graphs (A) to (I) of reference numeral 107, the intensity ratio of luminance in the first wavelength λ(1) to the third wavelength λ(3) is not uniform. Specifically, in the graph (E) of reference numeral 107, the intensity ratios are 0.33, 0.32, and 0.35. By applying the second correction coefficient, this non-uniform intensity ratio is corrected to become uniform at 0.33, 0.33, and 0.33. The second correction coefficient is calculated similarly for the other regions. By applying the second correction coefficient to the pseudo-color image 103, the pseudo-color image 151 is obtained. In the pseudo-color image 151, the intensity ratios of luminance in regions (A) to (I) are 0.33, 0.32, and 0.35, as shown in reference numeral 153. The pseudo-color image 151 will now show the original white color of background 2.

[0090] As shown in graphs (A) to (I) of reference numeral 115, the intensity ratio of the luminance from the first wavelength λ(1) to the third wavelength λ(3) is not uniform in the pseudo-color image 111 (Figure 17). Specifically, in graph (E) of reference numeral 115, the intensity ratios are 0.34, 0.34, and 0.32, but by applying the second correction coefficient, this is corrected to 0.33, 0.33, and 0.33. By applying the second correction coefficient to the pseudo-color image 111, the pseudo-color image 155 is obtained. In regions (A) to (I) of the pseudo-color image 155, the intensity ratios of the luminance are 0.33, 0.32, and 0.35, as shown in reference numeral 157. The pseudo-color image 155 now shows the original white color of background 2.

[0091] As explained above, in the intensity ratio measurement process, the intensity ratio of the luminance of the background 2 at the first wavelength λ(1) to the third wavelength λ(3) is measured, and the first and second correction coefficients are calculated based on the measurement results. In the above explanation, an example was described in which the image is divided into 3x3 9 regions and the second correction coefficient is obtained for each region, but this embodiment is not limited to this. For example, the second correction coefficient may be obtained for each pixel of the image, or the second correction coefficient may be obtained for a larger or smaller region.

[0092] [Correction process (second correction process)] In the correction step (second correction step), the processor 142 uses the second correction coefficient obtained in the intensity ratio measurement step to apply the second correction coefficient to the first wavelength image, second wavelength image, and third wavelength image captured in the imaging step, thereby correcting the intensity ratio of regions (A) to (I). While it is preferable to apply the second correction coefficient to the entire first to third wavelength image, it may also be applied, for example, to the portion of the first to third wavelength image where background 2 is visible (background 2 image data). By applying the second correction coefficient to the first to third wavelength image acquired in the imaging step and performing the correction, stable sensing can be achieved with the influence of imaging conditions suppressed. The processor 142 may also correct the sensitivity of the image sensor 130 (Figure 20) based on the first and / or second correction coefficients (sensitivity correction step). In areas where the first or second correction coefficient is large, the sensing result (brightness of the wavelength image) may be output brighter or darker than it should be. Therefore, the processor 142 can perform highly accurate sensing by correcting the sensitivity of the image sensor 130 (Figure 20) based on the first correction coefficient and / or the second correction coefficient.

[0093] <Another Embodiment 1> In the embodiments described above, an example was given in which three different first wavelengths λ(1), second wavelength λ(2), and third wavelength λ(3) are selected in the wavelength selection step. However, the number of wavelengths selected in the wavelength selection step is not limited to three, and any number of wavelengths may be selected. For example, in the wavelength selection step, two wavelengths may be selected, and the first wavelength λ(1) and second wavelength λ(2) described above may be selected.

[0094] For example, the processor 142 can select a first wavelength λ(1) and a second wavelength λ(2) such that the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2) are within a 15% range.

[0095] Specifically, the processor 142 selects a first wavelength λ(1) and a second wavelength λ(2) that satisfy the following equation. |β-α|÷(β+α)≦0.15···(1) |β-γ|÷(β+γ)≦0.15···(2)

[0096] In the above equation, α is input to the reflectances α1 and α2 of the first subject 3 at the first wavelength λ(1) and the second wavelength λ(2), respectively; β is input to the reflectances β1 and β2 of the background 2 at the first wavelength λ(1) and the second wavelength λ(2), respectively; and γ is input to the reflectances γ1 and γ2 of the second subject 4 at the first wavelength λ(1) and the second wavelength λ(2), respectively. In this case, (β-α) and (β-γ) at the second wavelength are selected to be smaller than the corresponding values ​​for the first wavelength λ(1) and the second wavelength λ(2) (Condition 2).

[0097] Furthermore, more preferably, the processor 142 can select a first wavelength λ(1) and a second wavelength λ(2) such that the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2) are within the range of 5%.

[0098] Specifically, the processor 142 selects a first wavelength λ(1) and a second wavelength λ(2) that satisfy the following equation. |β-α|÷(β+α)≦0.05···(3) |β-γ|÷(β+γ)≦0.05···(4)

[0099] In the above equation, α is input to the reflectances α1 and α2 of the first subject 3 at the first wavelength λ(1) and the second wavelength λ(2), respectively; β is input to the reflectances β1 and β2 of the background 2 at the first wavelength λ(1) and the second wavelength λ(2), respectively; and γ is input to the reflectances γ1 and γ2 of the second subject 4 at the first wavelength λ(1) and the second wavelength λ(2), respectively. In this case, (β-α) and (β-γ) at the second wavelength λ(2) are selected to be smaller than the corresponding values ​​at the first wavelength λ(1) (Condition 2).

[0100] As explained above, in this embodiment, two wavelengths (first wavelength λ(1) and second wavelength λ(2)) are selected in the wavelength selection step. In this way, any number of wavelengths are selected and imaging is performed, so the user can obtain a desired number of wavelength images.

[0101] <Another Embodiment 2> In the embodiment described above, background 2 was a predetermined white board. However, in this embodiment, the processor 142 notifies the optimal background based on the reflectance (spectral data) of the first subject 3 and the second subject 4 (notification step).

[0102] The memory 144 of the multispectral camera 100 in this embodiment stores multiple replacement subjects (multiple backgrounds) that can replace the background 2, and the spectral data of each of the multiple replacement subjects. The processor 142 then notifies the user of one of the multiple replacement subjects (background) as the recommended background based on the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4. The processor 142 can also notify the user of the replacement subject on a display unit (not shown) provided on the back of the multispectral camera 100. The multispectral camera 100 may also be equipped with a background display device 150 that displays the background 2 variably (see Figure 20). The background display device 150 is configured, for example, as a liquid crystal display and can display a desired color as the background under the control of the processor 142 (third subject variable display step). In this way, by changing the background color with the background display device 150, it is possible to change the background color for each location when simultaneously detecting subjects with completely different spectral reflectances, or to easily change the background color when you want to use the same multispectral camera 100 for different purposes (for example, detecting other subjects).

[0103] Figures 18 and 19 illustrate examples of recommended backgrounds.

[0104] Figure 18 shows the spectral data SD4 (fourth spectral data) of the recommended background. Based on the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4, the background with spectral data SD4 is selected and reported as the recommended subject. Spectral data SD4 has a small difference in reflectance between spectral data SD1 and spectral data SD2 in the wavelength range of 400 nm to 1000 nm. By using a background with such spectral data SD4, multiple wavelengths can be selected for imaging with the multispectral camera 100 over a wide wavelength range.

[0105] Figure 19 shows the spectral data SD5 of the recommended background. Based on the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4, the background with spectral data SD5 is selected and reported as the recommended subject. Since spectral data SD5 intersects with spectral data SD1 and spectral data SD2 at multiple points, it has points where the difference in reflectance between spectral data SD1 and spectral data SD2 is small. By using a background with such spectral data SD5, imaging can be performed at multiple points (wavelength ranges).

[0106] <Other> Next, we will explain the multispectral camera 100 used in the imaging method described above.

[0107] [Multispectral camera] Figure 20 is a schematic diagram showing an example of a multispectral camera 100 used in the imaging method of the present invention. Figure 21 is a schematic diagram showing an example of a filter unit 120 included in the multispectral camera 100.

[0108] The multispectral camera 100 shown in Figure 20 consists of an imaging optical system 110 including lenses 110A, 110B and a filter unit 120, an image sensor (image sensor) 130, and a signal processing unit 140. A background display device 150 may also be connected to the multispectral camera 100. The background display device 150 is connected to the signal processing unit 140 and controlled by a processor 142. The bandpass filter unit 124 included in the filter unit 120 (Figure 21) consists of a first bandpass filter (first wavelength selector) 124A, a second bandpass filter (second wavelength selector) 124B, and a third bandpass filter (third wavelength selector) 124C, which transmit light in wavelength ranges centered on the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3), respectively, suitable for distinguishing between the first subject 3 and the second subject 4. The filter unit 120 has four pupil regions (first pupil region to fourth pupil region), and the fourth pupil region, which is not used, is shielded by shielding member B (see Figure 21).

[0109] The filter unit 120 is composed of a polarizing filter unit 122 and a bandpass filter unit 124, and is preferably positioned at or near the pupil position of the imaging optical system 110.

[0110] The polarizing filter unit 122 consists of a first polarizing filter 122A, a second polarizing filter 122B, and a third polarizing filter 122C, which linearly polarize the light transmitted through the first pupil region, the second pupil region, and the third pupil region of the imaging optical system 110, respectively. For example, the first polarizing filter 122A is set to a polarization direction of 0°, the second polarizing filter 122B is set to a polarization direction of 90°, and the third polarizing filter 122C is set to a polarization direction of 45°.

[0111] The bandpass filter unit 124 consists of a first bandpass filter 124A, a second bandpass filter 124B, and a third bandpass filter 124C, which select the wavelength ranges of light transmitted through the first pupil region, second pupil region, and third pupil region of the imaging optical system 110, respectively. Therefore, light transmitted through the first pupil region of the imaging optical system 110 is linearly polarized by the first polarizing filter 122A, and only light in the wavelength range including the first wavelength λ(1) is transmitted by the first bandpass filter 124A. On the other hand, light transmitted through the second pupil region of the imaging optical system 110 is linearly polarized by the second polarizing filter 122B (linearly polarized in a direction 90° different from the first polarizing filter 122A), and only light in the wavelength range including the second wavelength λ(2) is transmitted by the second bandpass filter 124B. Furthermore, the light passing through the third pupil region of the imaging optical system 110 is linearly polarized by the third polarizing filter 122C, and only light in the wavelength range including the third wavelength λ(3) is transmitted by the third bandpass filter 124C.

[0112] The image sensor 130 is configured such that a first polarizing filter, a second polarizing filter, and a third polarizing filter with polarization directions of 0°, 45°, and 90° are regularly arranged on a plurality of pixels made up of photoelectric conversion elements arranged in a two-dimensional manner.

[0113] Furthermore, the polarization direction of the first polarizing filter 122A and the first polarizing filter of the image sensor 130 are the same, the polarization direction of the second polarizing filter 122B and the second polarizing filter of the image sensor 130 are the same, and the polarization direction of the third polarizing filter 122C and the third polarizing filter of the image sensor 130 are the same.

[0114] The signal processing unit 140 reads pixel signals from pixels on the image sensor 130 where the first polarizing filter is located to acquire a first wavelength image with a narrow bandwidth selected by the first bandpass filter 124A, reads pixel signals from pixels on the image sensor 130 where the second polarizing filter is located to acquire a second wavelength image with a narrow bandwidth selected by the second bandpass filter 124B, and reads pixel signals from pixels on the image sensor 130 where the third polarizing filter is located to acquire a third wavelength image with a narrow bandwidth selected by the third bandpass filter 124C.

[0115] The first, second, and third wavelength images acquired by the signal processing unit 140 are suitable for separating the first subject 3 and the second subject 4. For example, by combining the first, second, and third wavelength images, a composite image with an expanded dynamic range and enhanced sensing performance can be created. The signal processing unit 140 also performs interference removal processing on the first, second, and third wavelength images as needed.

[0116] In the above embodiment, the hardware structure of the processing unit that performs various processes is a variety of processors as shown below. These various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as a processing unit; a Programmable Logic Device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing; and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which has a circuit configuration specifically designed to perform a particular process.

[0117] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as client and server computers. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0118] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit composed of circuit elements such as semiconductor devices.

[0119] Each of the above-described configurations and functions can be appropriately implemented using any hardware, software, or a combination thereof. For example, the present invention can also be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable recording medium (non-temporary recording medium) that records such a program, or a computer on which such a program can be installed.

[0120] Although examples of the present invention have been described above, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention. [Explanation of symbols]

[0121] 2:Background 3: First subject 4: Second subject 10: Lighting equipment 100: Multispectral camera 110: Imaging Optics 110A: Lens 110B: Lens 120: Filter Unit 122: Polarizing filter unit 122A: First polarizing filter 122B: Second polarizing filter 122C: Third polarizing filter 124: Bandpass filter unit 124A: First bandpass filter 124B: Second bandpass filter 124C: Third bandpass filter 130: Image sensor 140: Signal Processing Unit 142: Processor 144: Memory 150: Background display device

Claims

1. An imaging method for imaging a subject using a multispectral camera equipped with a processor, The aforementioned processor, A data acquisition process that acquires the first spectral data of the first subject, the second spectral data of the second subject, and the third spectral data of the third subject. A wavelength selection step of selecting multiple wavelengths from the wavelength range of the acquired first to third spectral data, In the wavelength selection step, the difference or ratio of feature quantities of two of the first, second, and third spectral data sets is used as a factor, and multiple wavelengths are selected based on at least two of these factors. The imaging step is performed to image a subject including at least one of the first subject, second subject, and third subject at the plurality of wavelengths, The imaging method comprises a first factor which is the difference or ratio of the feature quantities of the first spectral data and the third spectral data, and a second factor which is the difference or ratio of the feature quantities of the second spectral data and the third spectral data, and a plurality of wavelengths are selected based on the first factor and the second factor.

2. The wavelength ranges of the first to third spectral data are at least the wavelength ranges where the wavelength ranges of the first spectral data and the wavelength ranges of the second spectral data overlap. The imaging method according to claim 1.

3. The imaging method according to claim 1, wherein one of the plurality of wavelengths is the wavelength at which at least one of the first factor and the second factor is minimized.

4. The imaging method according to any one of claims 1 to 3, further comprising: a step of measuring the intensity ratio of the luminance of a plurality of wavelengths within a plurality of regions of image data of the third subject using the plurality of wavelengths selected in the wavelength selection step; and a correction step of correcting at least the image data of the third subject based on the intensity ratio.

5. The imaging method according to claim 4, wherein the correction step includes a first correction step of correcting the intensity ratios of the plurality of regions based on a first intensity ratio which is one of the measured intensity ratios.

6. The imaging method according to claim 4, wherein the correction step includes a second correction step of correcting to reduce the difference in the measured intensity ratio.

7. The imaging method according to claim 4, wherein the correction step includes a first correction step of correcting the intensity ratios of the plurality of regions based on a first intensity ratio which is one of the measured intensity ratios, and a second correction step of correcting to reduce the difference between the measured intensity ratios.

8. The imaging method according to claim 1, wherein the third subject has a constant intensity ratio of the luminances of multiple wavelengths selected in the wavelength selection step across multiple regions.

9. The imaging method according to claim 1, wherein a first wavelength and a second wavelength are selected in the wavelength selection step, the reflectance α of the first subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following equation (1), the reflectance γ of the second subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following equation (2), and (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first wavelength. ｜β－α｜÷（β＋α）≦０．１５・・・（１） |β-γ|÷(β+γ)≦0.15...(2)

10. The imaging method according to claim 1, wherein a first wavelength and a second wavelength are selected in the wavelength selection step, the reflectance α of the first subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following relationship (3), the reflectance γ of the second subject and the reflectance β of the third subject at the first and second wavelengths satisfy the following relationship (4), and (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first wavelength. ｜β－α｜÷（β＋α）≦０．０５・・・（３） |β-γ|÷(β+γ)≦0.05...(4)

11. The imaging method according to claim 1, wherein a first wavelength, a second wavelength, and a third wavelength are selected in the wavelength selection step, and the reflectance α of the first subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (5), and the reflectance γ of the second subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (6), and (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first and third wavelengths. ｜β－α｜÷（β＋α）≦０．１５・・・（５） |β-γ|÷(β+γ)≦0.15...(6)

12. The imaging method according to claim 1, wherein a first wavelength, a second wavelength, and a third wavelength are selected in the wavelength selection step, and the reflectance α of the first subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (7), and the reflectance γ of the second subject and the reflectance β of the third subject at the first, second, and third wavelengths satisfy the following relationship (8), and (β-α) and (β-γ) at the second wavelength are smaller than (β-α) and (β-γ) at the first and third wavelengths. ｜β－α｜÷（β＋α）≦０．０５・・・（７） |β-γ|÷(β+γ)≦0.05...(8)

13. The system includes a memory that stores a plurality of replaceable subjects that can be replaced with the third subject, and the fourth spectral data of each of the plurality of replaceable subjects. The aforementioned processor, The imaging method according to claim 1, which includes a notification step of notifying that one of the multiple replacement subjects is a recommended third subject based on the first spectral data, the second spectral data, and the fourth spectral data.

14. The imaging method according to claim 4, wherein the processor performs a sensitivity correction step to correct the sensitivity of the image sensor of the multispectral camera based on the correction step.

15. The imaging method according to claim 1, further comprising a third subject variable display step of displaying the third subject in a variable manner.

16. The imaging method according to claim 1, further comprising a process of irradiating with shadowless illumination.

17. A program that causes a multispectral camera equipped with a processor to execute an imaging method for imaging a subject, The aforementioned processor, A data acquisition process that acquires the first spectral data of the first subject, the second spectral data of the second subject, and the third spectral data of the third subject. A wavelength selection step of selecting multiple wavelengths from the wavelength range of the acquired first to third spectral data, In the wavelength selection step, the difference or ratio of feature quantities of two of the first, second, and third spectral data sets is used as a factor, and multiple wavelengths are selected based on at least two of these factors. The imaging process involves imaging a subject that includes at least one of the first subject, second subject, and third subject at the aforementioned plurality of wavelengths. The factors include a first factor which is the difference or ratio of the feature quantities of the first spectral data and the third spectral data, and a second factor which is the difference or ratio of the feature quantities of the second spectral data and the third spectral data, and a plurality of wavelengths are selected based on the first factor and the second factor. program.

18. A non-temporary and computer-readable recording medium on which the program described in claim 17 is recorded.

Images