Imaging method and recording medium

Abstract

The present application provides a kind of to be able to obtain the wavelength image of maintaining high wavelength reproducibility imaging method and recording medium.In the imaging method for photographing object by multispectral camera with processor, processor carries out: data acquisition process (step S10), obtains the first object first spectral data, the second object second spectral data and the third object third spectral data;Wavelength selection process (step S11), select multiple wavelengths from the wavelength region of the first to third spectral data obtained, in wavelength selection process (step S11), the difference or ratio of the characteristic quantity of two spectral data in the first spectral data, the second spectral data, the third spectral data is determined as the operation quantity, and at least two or more operation quantities are calculated to select multiple wavelengths;And imaging process (step S12), at least one of the object including the first object, the second object, the third object is photographed with multiple wavelengths.

Description

Technical Field

[0001] This invention relates to a camera method and program, and more particularly to a camera method and program for a multispectral camera. Background Technology

[0002] Previously, multispectral cameras were used to acquire images of each different wavelength, and these images were then applied to various purposes.

[0003] For example, Patent Document 1 describes a technique for estimating the color of a subject using pre-measured spectral sensitivity characteristics of a camera.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2006-30014 Summary of the Invention

[0007] One embodiment of the present invention provides a method and procedure for obtaining wavelength images that maintain high wavelength reproducibility.

[0008] means for solving technical problems

[0009] One embodiment of the imaging method of the present invention is an imaging method for capturing images of a subject using a multispectral camera equipped with a processor, wherein...

[0010] The processor performs the following steps: a data acquisition step, acquiring first spectral data of a first subject, second spectral data of a second subject, and third spectral data of a third subject; a wavelength selection step, selecting multiple wavelengths from the wavelength range of the acquired first to third spectral data, wherein the difference or ratio of characteristic quantities of two spectral data among the first, second, and third spectral data is used as a factor, and multiple wavelengths are selected based on at least two of the factors; and an imaging step, capturing images of at least one of the first, second, and third subjects at multiple wavelengths.

[0011] Preferably, the wavelength regions of the first to third spectral data are wavelength regions that overlap with at least the wavelength regions of the first spectral data and the wavelength regions of the second spectral data.

[0012] The preferred factors include a first factor which is the difference or ratio of the characteristic quantities of the first spectral data and the third spectral data, and a second factor which is the difference or ratio of the characteristic quantities of the second spectral data and the third spectral data, and multiple wavelengths are selected based on the first factor and the second factor.

[0013] Preferably, one of the multiple wavelengths is the minimum wavelength, which is at least one of the first factor and the second factor.

[0014] The preferred imaging method further has the following characteristics:

[0015] A process of measuring the intensity ratio of the brightness of multiple wavelengths selected through a wavelength selection process in multiple regions of image data of a third subject; and

[0016] The correction process involves correcting the image data of at least the third subject based on the intensity ratio.

[0017] The preferred calibration process includes a first calibration process, which calibrates the intensity ratio of multiple regions based on one of the measured intensity ratios, namely a first intensity ratio.

[0018] The preferred calibration process includes a second calibration process, which performs a correction to reduce the difference in the measured intensity ratio.

[0019] The preferred calibration process includes:

[0020] The first calibration process involves calibrating the strength ratios of multiple regions based on one of the measured strength ratios, namely the first strength ratio; and

[0021] The second calibration process involves correcting for the difference in the measured strength ratio.

[0022] Preferably, the intensity ratio of the brightness of multiple wavelengths selected by the wavelength selection process in multiple regions of the third subject is constant.

[0023] Preferably, in the wavelength selection process, the first wavelength and the second wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (1). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (2). (β-α) and (β-γ) under the second wavelength are smaller than (β-α) and (β-γ) under the first wavelength.

[0024] |β-α|÷(β+α)≤0.15···(1)

[0025] |β-γ|÷(β+γ)≤0.15···(2)

[0026] Preferably, in the wavelength selection process, the first wavelength and the second wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (3). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (4). (β-α) and (β-γ) under the second wavelength are smaller than (β-α) and (β-γ) under the first wavelength.

[0027] |β-α|÷(β+α)≤0.05···(3)

[0028] |β-γ|÷(β+γ)≤0.05···(4)

[0029] Preferably, in the wavelength selection process, the first wavelength, the second wavelength and the third wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength, the second wavelength and the third wavelength satisfy the following relationship (5). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength, the second wavelength and the third wavelength satisfy the following relationship (6). (β-α) and (β-γ) under the second wavelength are smaller than (β-α) and (β-γ) under the first wavelength and the third wavelength.

[0030] |β-α|÷(β+α)≤0.15···(5)

[0031] |β-γ|÷(β+γ)≤0.15···(6)

[0032] Preferably, in the wavelength selection process, the first wavelength, the second wavelength and the third wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength, the second wavelength and the third wavelength satisfy the following relationship (7). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength, the second wavelength and the third wavelength satisfy the following relationship (8). (β-α) and (β-γ) under the second wavelength are smaller than (β-α) and (β-γ) under the first wavelength and the third wavelength.

[0033] |β-α|÷(β+α)≤0.05···(7)

[0034] |β-γ|÷(β+γ)≤0.05···(8)

[0035] The preferred multispectral camera has a memory that stores multiple replacement subjects that can be replaced with the third subject and the fourth spectral data of each of the multiple replacement subjects. The processor performs a notification process, which, based on the first spectral data, the second spectral data and the fourth spectral data, notifies one of the multiple replacement subjects as the recommended third subject.

[0036] The preferred processor performs a sensitivity calibration process to calibrate the sensitivity of the imaging element of the multispectral camera according to the calibration process.

[0037] The preferred imaging method further includes a third subject variable display process for variablely displaying a third subject.

[0038] The preferred imaging method is used to further perform the shadowless lighting process.

[0039] As another embodiment of the present invention, the program is a program for causing a multispectral camera equipped with a processor to perform an imaging method for capturing a subject. The program causes the processor to perform: a data acquisition step, acquiring first spectral data of a first subject, second spectral data of a second subject, and third spectral data of a third subject; a wavelength selection step, selecting multiple wavelengths from the wavelength regions of the acquired first to third spectral data, wherein in the wavelength selection step, the difference or ratio of characteristic quantities of two spectral data of the first, second, and third spectral data is used as a factor, and multiple wavelengths are selected according to at least two of the factors; and an imaging step, capturing a subject including at least one of the first, second, and third subjects at multiple wavelengths. Attached Figure Description

[0040] Figure 1 This is the first example illustrating how a wavelength image can become incorrect due to changes in the surrounding environment.

[0041] Figure 2 This is the first example illustrating how a wavelength image can become incorrect due to changes in the surrounding environment.

[0042] Figure 3 This is the second example illustrating how wavelength images can become incorrect due to changes in the surrounding environment.

[0043] Figure 4 This is the second example illustrating how wavelength images can become incorrect due to changes in the surrounding environment.

[0044] Figure 5 This is a flowchart illustrating the camera recording method.

[0045] Figure 6 This is a schematic diagram showing a subject captured by a multispectral camera.

[0046] Figure 7 It is a graph representing the spectral data of the background, the first subject, and the second subject.

[0047] Figure 8 It is a graph representing the spectral data of the background, the first subject, and the second subject.

[0048] Figure 9 This is a diagram showing the cases where equations (5), (6), and condition 1 are satisfied.

[0049] Figure 10 This diagram illustrates the imaging process based on a multispectral camera.

[0050] Figure 11 This is a flowchart illustrating the camera recording method.

[0051] Figure 12 This is a photograph taken in the background to illustrate the strength ratio measurement process.

[0052] Figure 13 This is a graph illustrating the intensity ratio of the background brightness measured in the intensity ratio measurement process and the calculated first correction factor.

[0053] Figure 14 This is a graph illustrating the intensity ratio of the background brightness measured in the intensity ratio measurement process and the calculated first correction factor.

[0054] Figure 15 This is a flowchart illustrating the camera recording method.

[0055] Figure 16 This is a graph illustrating the intensity ratio of the background brightness measured in the intensity ratio measurement process, as well as the calculated first and second correction coefficients.

[0056] Figure 17 This is a graph illustrating the intensity ratio of the background brightness measured in the intensity ratio measurement process, as well as the calculated first and second correction coefficients.

[0057] Figure 18 This is an image illustrating an example of a recommended background.

[0058] Figure 19 This is an image illustrating an example of a recommended background.

[0059] Figure 20 This is a schematic diagram illustrating an example of a multispectral camera.

[0060] Figure 21 This is a schematic diagram illustrating an example of a filter unit in a multispectral camera. Detailed Implementation

[0061] Hereinafter, preferred embodiments of the imaging method and program involved in the present invention will be described with reference to the accompanying drawings.

[0062] First, the following explains the situation where the wavelength image becomes incorrect (wavelength reproducibility is reduced).

[0063] Previously, multispectral cameras were used to acquire images corresponding to each wavelength (wavelength images). For example, multispectral cameras utilizing polarization pixels (e.g., 0°, 45°, 90°) (see reference). Figure 20 This allows for the generation of wavelength images by assigning different wavelengths to each polarization pixel. In the process of generating wavelength images, interference removal is typically performed. Interference removal is the process of removing signals related to other wavelengths that interfere with the image during shooting; it removes these other wavelengths of interference based on a pre-measured mixing ratio (inverse matrix operation). Then, by performing this interference removal appropriately, a correct wavelength image that suppresses interference from other wavelengths can be obtained.

[0064] The mixing ratio is typically measured before actual shooting, such as during the manufacturing of a multispectral camera, and is usually stored in the camera's built-in memory. Therefore, it is possible that the pre-measured mixing ratio differs from the actual mixing ratio during shooting. When this difference occurs, interference removal cannot be performed effectively, and sometimes the resulting wavelength image becomes incorrect (wavelength reproducibility is reduced). One reason for this difference is changes in the surrounding environment. Specifically, because the surrounding environment at the time of pre-measurement differs from that at the time of actual shooting, the light spot changes, and thus the mixing ratio changes. The following examples illustrate the impact of environmental changes on wavelength images.

[0065] Figure 1 and Figure 2 This is the first example illustrating how a wavelength image can become incorrect due to changes in the surrounding environment.

[0066] Images 501, 503, and 505 are images of a white plate with uniform reflectivity obtained by a multispectral camera at various wavelengths. Image 501 is a U-wavelength image of wavelength U, image 503 is a V-wavelength image of wavelength V, and image 505 is a W-wavelength image of wavelength W. Furthermore, image 507 is a simulated color image, for example, when wavelength U is replaced with blue (B: BLUE), wavelength V with green (G: GREEN), and wavelength W with red (R: RED). Figure 2 In the figure, the intensity ratio of the brightness of wavelengths U, V and W of regions (A) to (I) of image 507 is shown in the corresponding charts (A) to (I).

[0067] Typically, when a white plate with uniform reflectivity is photographed using the multispectral camera 100, the intensity ratio of brightness in regions (A) to (I) becomes uniform. Furthermore, when a white plate with uniform reflectivity is photographed using the multispectral camera 100, the intensity ratio of brightness among images 501 (U-wavelength image), 503 (V-wavelength image), and 505 (W-wavelength image) also typically becomes the same. However, because the surrounding environment when actually photographing images 501, 503, and 505 differs from the surrounding environment when measuring the mixing ratio, as illustrated, the intensity ratio of brightness shown in each image of images 501, 503, and 505 in regions (A) to (I) is different. Moreover, the intensity ratio of brightness among images 501, 503, and 505 is also different (for example, in region (E), the deviation of the intensity ratio among images 501, 503, and 505 is approximately 2%).

[0068] Thus, when measuring the mixing ratio and during actual shooting, different surrounding environments cause changes in light spots, etc., and sometimes it is impossible to obtain the correct wavelength image (maintaining wavelength reproducibility).

[0069] Figure 3 and Figure 4 This is the second example illustrating how a wavelength image can become incorrect due to changes in the surrounding environment. In this example, a black subject 519 is photographed against a white background with uniform reflective properties.

[0070] Image 511 is a U-wavelength image of wavelength U, image 513 is a V-wavelength image of wavelength V, and image 515 is a W-wavelength image of wavelength W. Furthermore, image 517 is... Figure 1 The simulated color image generated under the same conditions as described in the text. Figure 4 In the figure, the intensity ratios of the brightness of regions (A) to (I) of image 517 at wavelengths U, V and W are shown in the corresponding charts (A) to (I).

[0071] As shown in the figure, the ambient environment when images 511, 513, and 515 were captured differs from the ambient environment when the blending ratio was measured. Therefore, the intensity ratio of brightness shown in each image of images 511, 513, and 515 differs in regions (A) to (I). Furthermore, the intensity ratios of brightness among images 511, 513, and 515 are also different (for example, in region (E), the deviation in the intensity ratio of brightness among images 511, 513, and 515 is approximately 6%).

[0072] Thus, during the acquisition of the mixing ratio and in actual shooting, different surrounding environments cause changes in light spots, which sometimes makes it impossible to obtain the correct wavelength image (to maintain wavelength reproducibility). Furthermore, in cases where the difference in reflectivity between the background and the subject is large, such as with a black subject 519, the deviation in the intensity ratio of brightness between wavelengths becomes larger, resulting in reduced wavelength reproducibility.

[0073] As in the example above, even if the mixing ratio changes, simply performing interference removal will result in an incorrect wavelength image.

[0074] Therefore, in the embodiments disclosed below, by suppressing changes such as light spots or compensating for light spots that occur during the initial stage of shooting, the wavelength image is corrected so that even if the surrounding environment changes, the correct wavelength image can be obtained (and wavelength reproducibility can be maintained).

[0075] <First Implementation>

[0076] Figure 5This is a flowchart illustrating the imaging method of the present invention. Each step of the imaging method described below is performed by a multispectral camera 100 (…). Figure 20 The processor 142 implements the imaging method. Furthermore, the processor 142 executes each step by executing a dedicated program for the imaging method stored in the memory 144.

[0077] In the data acquisition step (step S10), the processor 142 acquires the spectral data of the background 2, the first subject 3, and the second subject 4. Next, in the wavelength selection step (step S11), the processor 142 selects multiple wavelengths from the wavelength regions (wavelength bands) of the spectral data of the background 2, the first subject 3, and the second subject 4. In this example, the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) are selected. Next, in the imaging step (step S12), the processor 142 captures images of at least one of the subjects, including the background 2, the first subject 3, and the second subject 4, using the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3).

[0078] The following is a detailed explanation of each process.

[0079] [Data Acquisition Process]

[0080] In the data acquisition process, the processor 142 acquires the spectral data of the subjects (background 2, first subject 3 and second subject 4).

[0081] Figure 6 This is a schematic diagram showing the subjects (first subject, second subject and background (third subject)) captured by the multispectral camera 100.

[0082] The subject is composed of a first subject 3, a second subject 4, and a background 2. A multispectral camera 100 captures images of at least one of the subjects, including the background 2, the first subject 3, and the second subject 4, and senses the first subject 3 and the second subject 4 located on the background 2. Therefore, the processor 142 acquires spectral data related to the reflectance of the background 2, the first subject 3, and the second subject 4.

[0083] Figure 7 and Figure 8 This is a graph representing the spectral data of background 2, first subject 3, and second subject 4.

[0084] Figure 7In the image, the reflectance corresponding to each wavelength of the background 2, the first subject 3, and the second subject 4 is displayed as spectral data. Spectral data SD1 (first spectral data) corresponds to the first subject 3, spectral data SD2 (second spectral data) corresponds to the second subject 4, and spectral data SD3 (third spectral data) corresponds to the background 2. The wavelength regions of the spectral data SD1, SD2, and SD3 acquired by the processor 142 are the overlapping wavelength regions of spectral data SD1 and SD2. That is, in... Figure 7 In the illustrated case, the wavelength regions of spectral data SD1 and SD2 are 400 nm to 1000 nm, and the wavelength region of spectral data SD3 is also 400 nm to 1000 nm. The wavelength regions of spectral data SD1 to SD3 overlap in the 400 nm to 1000 nm range. Therefore, processor 142 acquires spectral data SD1 to SD3 in the wavelength region of 400 nm to 1000 nm.

[0085] Figure 8 In, with Figure 7 Similarly, the reflectance corresponding to each wavelength of the background 2, the first subject 3 and the second subject 4 is displayed as spectral data, with spectral data SD1 corresponding to the first subject 3, spectral data SD2 corresponding to the second subject 4 and spectral data SD3 corresponding to the background 2.

[0086] exist Figure 8 In the case shown, the wavelength regions of spectral data SD1 and spectral data SD2 overlap in the range of 550 nm to 1000 nm. Therefore, the wavelength region of spectral data SD3 is also sufficiently within the range of 550 nm to 1000 nm, and the processor 142 acquires spectral data SD1 to SD3 in the wavelength region of 550 nm to 1000 nm.

[0087] The processor 142 acquires spectral data SD1 to SD3 through various methods. For example, spectral data SD1 to SD3 can be acquired by photographing the background 2, the first subject 3, and the second subject 4 using a hyperspectral camera, and then input into the multispectral camera 100. Furthermore, spectral data SD1 to SD3 can be selected from a database storing various spectral data, input into the multispectral camera 100, and acquired by the processor 142. Additionally, spectral data SD1 to SD3 can be pre-stored in the memory 144 of the multispectral camera 100 and acquired by the processor 142.

[0088] [Wavelength selection process]

[0089] In the wavelength selection process, processor 142 selects a first wavelength λ(1), a second wavelength λ(2), and a third wavelength λ(3) based on the acquired spectral data SD1, SD2, and SD3. Processor 142 uses the difference or ratio of characteristic quantities of two spectral data points from SD1, SD2, and SD3 as factors to calculate at least two factors to select multiple wavelengths. The following description will explain the case where the difference in characteristic quantities (reflectance) of the spectral data is used as a factor.

[0090] In previous technologies, it was usually used in Figure 7 The wavelengths (reference wavelengths) with the smallest difference between the spectral data SD1, spectral data SD2, and spectral data SD3 of the first subject 3 and the second subject 4, as described in the text, are included. Therefore, conventionally, for example, at least 420 nm is selected as the second wavelength λ(2) (reference wavelength), and 500 nm is selected as the first wavelength λ(1).

[0091] However, in this embodiment, the processor 142 selects a first wavelength λ(1), a second wavelength λ(2), and a third wavelength λ(3) based on the relationship between the difference in reflectance between spectral data SD3 and spectral data SD1 (first factor) and the difference in reflectance between spectral data SD3 and spectral data SD2 (second factor). For example, the processor 142 selects 850nm, where the first and second factors are the smallest, as the second wavelength λ(2) (reference wavelength). Alternatively, the processor 142 can select at least one of the first and second factors as the minimum wavelength as the reference wavelength. Furthermore, the processor 142 selects 925nm, where the first and second factors are larger than the second wavelength λ(2) and the first subject 3 and the second subject 4 can be identified, as the first wavelength λ(1). For example, the first wavelength λ(1) can be selected where the sum of the first and second factors is within the range specified below (15% or 5%). Furthermore, the processor 142 selects 875nm between the first wavelength λ(1) and the second wavelength λ(2) as the third wavelength λ(3).

[0092] Furthermore, the processor 142 can select the first wavelength λ(1) to the third wavelength λ(3) within a range of 15% for the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2).

[0093] Specifically, the processor 142 selects wavelengths λ(1) to λ(3) that satisfy the following equation.

[0094] |β-α|÷(β+α)≤0.15···(5)

[0095] |β-γ|÷(β+γ)≤0.15···(6)

[0096] Furthermore, in the above formula, α is input to each of the reflectances α1 to α3 of the first subject 3 under the first wavelength λ(1) to the third wavelength λ(3), β is input to each of the reflectances β1 to β3 of the background 2 under the first wavelength λ(1) to the third wavelength λ(3), and γ is input to each of the reflectances γ1 to γ3 of the second subject 4 under the first wavelength λ(1) to the third wavelength λ(3). And, at this time, the values ​​of (β-α) and (β-γ) under the second wavelength λ(2) are selected to be smaller than the corresponding values ​​of the first wavelength λ(1) and the third wavelength λ(3) (condition 1).

[0097] Figure 9 This is a diagram showing the cases where equations (5), (6), and condition 1 are satisfied. Additionally, Figure 9 The spectral data SD1~SD3 and the first wavelength λ(1)~the third wavelength λ(3) are shown.

[0098] like Figure 9 As shown, wavelengths λ(1) to λ(3) of the first wavelength are selected to be within 15% of the differences M1, N1, M2 (which is 0 and therefore not shown), N2 (which is 0 and therefore not shown), M3, and N3. Furthermore, wavelengths λ(1) to λ(3) of the first wavelength are selected to be smaller than the differences M1, N1, M3, and N3.

[0099] Furthermore, the processor 142 is further preferred to select the first wavelength λ(1) to the third wavelength λ(3) within a range of 5% for the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2).

[0100] Specifically, the processor 142 selects wavelengths λ(1) to λ(3) that satisfy the following equation.

[0101] |β-α|÷(β+α)≤0.05···(7)

[0102] |β-γ|÷(β+γ)≤0.05···(8)

[0103] Furthermore, in the above formula, α is input to each of the reflectances α1 to α3 of the first subject 3 under the first wavelength λ(1) to the third wavelength λ(3), β is input to each of the reflectances β1 to β3 of the background 2 under the first wavelength λ(1) to the third wavelength λ(3), and γ is input to each of the reflectances γ1 to γ3 of the second subject 4 under the first wavelength λ(1) to the third wavelength λ(3). And, at this time, the values ​​of (β-α) and (β-γ) under the second wavelength λ(2) are selected to be smaller than the corresponding values ​​of the first wavelength λ(1) and the third wavelength λ(3) (condition 1).

[0104] As explained above, the processor 142 selects wavelengths λ(1) to λ(3) based on factors based on spectral data SD1 to spectral data SD3. Therefore, since the difference in reflectivity with the background 2 remains within a constant range, the influence of changes in the surrounding environment can be suppressed.

[0105] Furthermore, the above description describes the case where the first wavelength λ(1) to the third wavelength λ(3) is automatically selected from the spectral data SD1 to SD3 acquired by the processor 142. In the wavelength selection process, the processor 142 can select the first wavelength λ(1) to the third wavelength λ(3) in other ways. For example, the processor 142 displays the acquired spectral data SD1 to SD3 on a display unit located on the back of the multispectral camera 100, and the user selects the first wavelength λ(1) to the third wavelength λ(3) from this display. The processor 142 can select the first wavelength λ(1) to the third wavelength λ(3) based on the user's instruction input via the operation unit (not shown) of the multispectral camera 100.

[0106] [Camera recording process]

[0107] During the camera operation, the processor 142 captures a subject including at least one of the following: a first subject 3, a second subject 4, and a background 2 (the third subject) at a first wavelength λ (1), a second wavelength λ (2), and a third wavelength λ (3).

[0108] Figure 10 This diagram illustrates the imaging process based on the multispectral camera 100. Figure 10 In the image, background 2 is a conveyor belt, and a specific example is shown where a first subject 3 and a second subject 4 are placed on it and continuously transported. A multispectral camera 100 captures images of the continuously transported first subject 3 and second subject 4.

[0109] The processor 142 enables the multispectral camera 100 to capture a scene of the first subject 3 and the second subject 4 located on the background 2. The multispectral camera 100 captures a scene of only the background 2, a scene of only the first subject 3 on the background 2, or a scene of only the second subject 4 on the background 2, depending on the situation. In addition, the multispectral camera 100 is set to acquire first wavelength images to third wavelength images corresponding to the first wavelength λ(1) to the third wavelength λ(3) selected by the wavelength selection process. The illumination device 10 illuminates the background 2, the first subject 3, and the second subject 4 in conjunction with the imaging of the multispectral camera 100 (illumination process). In addition, the illumination device 10 uses a light source a, a light source b, or a combination of light sources a and b. Furthermore, since the shadows of the first subject 3 and the second subject 4 do not have a good impact on acquiring the correct first wavelength images to third wavelength images, the illumination device 10 is preferably a shadowless illumination device. The multispectral camera 100 performs interference removal processing on the acquired first to third wavelength images as needed.

[0110] The multispectral camera 100 performs sensing (such as detecting the presence or absence of the first subject 3 and the second subject 4) based on the captured images of the first to third wavelengths.

[0111] As explained above, according to the imaging method of this embodiment, even if the light spot or the like changes depending on the surrounding environment in which the shooting is performed, since the reflectivity factors of the first subject 3, the second subject 4, and the background 2 are selected within the specified range of the first wavelength λ(1) to the third wavelength λ(3), it is possible to acquire a wavelength image that suppresses the influence of changes in the surrounding environment and maintains high wavelength reproducibility.

[0112] <Second Implementation Method>

[0113] Next, the second embodiment of the present invention will be described. In this embodiment, in addition to the imaging method of the first embodiment described above, an intensity ratio measurement step and a calibration step (first calibration step) are also included.

[0114] Figure 11 This is a flowchart illustrating the imaging method of this embodiment. The imaging method described below is implemented by the processor 142 of the multispectral camera 100. Furthermore, the processor 142 implements each step by executing a dedicated program for the imaging method stored in the memory 144.

[0115] In the data acquisition process (step S20), the processor 142 acquires the spectral data of the background 2, the first subject 3, and the second subject 4. Next, in the wavelength selection process (step S21), the processor 142 selects the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) from the wavelength regions of the spectral data of the background 2, the first subject 3, and the second subject 4. Next, in the intensity ratio measurement process (step S22), the processor 142 photographs the background 2 with the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3), and acquires the wavelength images (image data) of the background 2 with the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3), and measures the intensity ratio of the brightness of the first wavelength λ(1) to the third wavelength λ(3) in multiple regions. Next, in the imaging process (step S23), the processor 142 captures images of at least one of the following subjects: background 2, first subject 3, and second subject 4, using a first wavelength λ (1), a second wavelength λ (2), and a third wavelength λ (3). Then, in the first correction process (step S24), the processor 142 corrects the intensity ratio of multiple regions based on a first intensity ratio, one of the measured intensity ratios.

[0116] Next, the intensity ratio measurement process and the first calibration process, which are characteristic parts of this embodiment, will be described. Furthermore, the other processes in this embodiment (data acquisition process, wavelength selection process, and imaging process) are the same as in the first embodiment, and therefore their descriptions are omitted.

[0117] [Strength ratio measurement process]

[0118] In the intensity ratio measurement process, the processor 142 measures the intensity ratio of the brightness of the background 2 at wavelengths 1 to 3. Specifically, the processor 142 captures images of the background 2 using wavelengths 1 to 3 and a multispectral camera 100, acquiring images of wavelengths 1 to 3.

[0119] Figure 12 This is a background image taken to illustrate the strength ratio measurement process.

[0120] In the intensity ratio measurement process, the background 2 is photographed by the multispectral camera 100, and images of the background 2 at the first wavelength, the second wavelength, and the third wavelength are acquired. Furthermore, the background 2 photographed here is the same as the background 2 photographed in the imaging process. For example, a white plate with uniform reflectivity at the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) is used in the background 2. Also, for example, the intensity ratio at the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) is shown to be constant in multiple regions. Furthermore, the illumination device 10 is the same illumination device used in the imaging process. It is preferable to use the same light source in both the intensity ratio measurement process and the imaging process.

[0121] Figure 13 and Figure 14 This is a graph illustrating the intensity ratio of the brightness of background 2 measured in the intensity ratio measurement process and the calculated first correction coefficient. Figure 13 The measurement results obtained by light source a are shown in the figure. Figure 14 The measurement results obtained by light source b are shown in the figure.

[0122] Simulated color image 101 ( Figure 13 The image is generated based on images of wavelengths 1 to 3. Specifically, a simulated color image 101 is generated by overlaying the brightness intensity ratio of the first wavelength image with red (R: RED), the brightness intensity ratio of the second wavelength image with blue (B: BLUE), and the brightness intensity ratio of the third wavelength image with green (G: GREEN). Furthermore, in reference numeral 105, the brightness intensity ratios of regions (A) to (I) of the simulated color image 101 at the first wavelength λ (1), the second wavelength λ (2), and the third wavelength λ (3) are correspondingly shown in graphs (A) to (I). In addition, regarding the brightness intensity ratios shown in graphs (A) to (I), the average or representative value of the brightness intensity ratio of each region of regions (A) to (I) is shown.

[0123] As shown in Figures (A) to (I) of the analog color image 101 and reference numeral 105, the intensity ratio of brightness at each wavelength in the first to third wavelength images deviates due to changes in the surrounding environment. Therefore, processor 142 obtains a first correction coefficient that makes the intensity ratios of brightness at the first wavelength λ(1), second wavelength λ(2), and third wavelength λ(3) in regions (A) to (I) consistent. For example, processor 142 obtains a first correction coefficient that aligns the intensity ratios of regions (A) to (D) and regions (F) to (I) with the intensity ratio of brightness in region (E) at the center of analog color image 101 (the first intensity ratio). By applying the first correction coefficient to analog color image 101, analog color image 103 is obtained. As shown in Figures (A) to (I) of reference numeral 107, analog color image 103 has a uniform intensity ratio of brightness. That is, by using the first correction factor, the intensity ratio of brightness in regions (A) to (I) becomes the same.

[0124] Similar to the analog color image 101, an analog color image 109 is generated based on the first wavelength image to the third wavelength image. Figure 14 Furthermore, in symbol 113, the intensity ratios of brightness at the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3) of regions (A) to (I) of the simulated color image 109 are shown in graphs (A) to (I).

[0125] As shown in the diagrams (A) to (I) in the analog color image 109 and reference numeral 113, the intensity ratio of brightness at each wavelength in the first to third wavelength images deviates due to changes in the surrounding environment. Therefore, the processor 142 obtains a first correction coefficient that aligns the intensity ratio of regions (A) to (D) and regions (F) to (I) with the intensity ratio of brightness in region (E) at the center of the analog color image 109 (the first intensity ratio). Applying the first correction coefficient to the analog color image 109 yields the analog color image 111. As shown in the diagrams (A) to (I) in reference numeral 115, the analog color image 111 achieves a uniform intensity ratio of brightness. That is, the intensity ratio of brightness in regions (A) to (I) becomes uniform through the first correction coefficient.

[0126] As explained above, in the intensity ratio measurement process, the intensity ratio of the brightness of the background 2 at the first wavelength λ(1) to the third wavelength λ(3) is measured, and the first correction coefficient is calculated based on the measurement result. Furthermore, while the above description illustrates an example of dividing the image into nine 3×3 regions and obtaining the first correction coefficient for each region, this embodiment is not limited to this. For example, the first correction coefficient can be obtained for each pixel of the image, or it can be obtained in a relatively large or small region.

[0127] [Calibration Process]

[0128] In the calibration process (first calibration process), the processor 142 applies a first calibration coefficient obtained through the intensity ratio measurement process to the first wavelength image, second wavelength image, and third wavelength image captured by the imaging process, thereby correcting the intensity ratio of the brightness of regions (A) to (I). Furthermore, the first calibration coefficient is preferably applied to the entire first wavelength image to third wavelength image, but it can also be applied, for example, to the portion of background 2 (image data of background 2) that reflects the first wavelength image to third wavelength image. The first wavelength image, second wavelength image, and third wavelength image corrected by the first calibration coefficient suppress the influence of changes in the surrounding environment. Therefore, the first wavelength image to third wavelength image that maintains wavelength reproducibility can be obtained, and accurate sensing is possible.

[0129] <Third Implementation Method>

[0130] Next, a third embodiment of the present invention will be described. In this embodiment, in addition to the imaging method of the second embodiment described above, a correction step (second correction step) is also included.

[0131] Figure 15This is a flowchart illustrating the imaging method of this embodiment. The imaging method described below is implemented by the processor 142 of the multispectral camera 100. Furthermore, the processor 142 implements each step by executing a dedicated program for the imaging method stored in the memory 144.

[0132] In the data acquisition process (step S30), the processor 142 acquires the spectral data of the background 2, the first subject 3, and the second subject 4. Next, in the wavelength selection process (step S31), the processor 142 selects a first wavelength λ(1), a second wavelength λ(2), and a third wavelength λ(3) from the wavelength regions of the spectral data of the background 2, the first subject 3, and the second subject 4. Next, in the intensity ratio measurement process (step S32), the processor 142 photographs the background 2 with the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3), and acquires wavelength images (image data) of the background 2 with the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3), and measures the intensity ratio of the brightness of multiple wavelengths in multiple regions. Next, in the imaging process (step S33), the processor 142 captures images of at least one of the following: background 2, first subject 3, and second subject 4, using a first wavelength λ(1), a second wavelength λ(2), and a third wavelength λ(3). Next, in the first correction process (step S34), the processor 142 corrects the intensity ratios of multiple regions based on a first intensity ratio, one of the intensity ratios measured in the intensity ratio measurement process. Next, in the second correction process (step S35), the processor 142 performs a correction to reduce the difference in intensity ratios measured in the intensity ratio measurement process.

[0133] Next, the intensity ratio measurement process and the second calibration process, which are characteristic parts of this embodiment, will be described. Furthermore, the data acquisition process, wavelength selection process, imaging process, and first calibration process are the same as in the first and second embodiments, and therefore will not be described here.

[0134] [Strength ratio measurement process]

[0135] In the strength ratio measurement process of this embodiment, a second correction factor is calculated in addition to the first correction factor. Furthermore, the following description explains the case where the first and second correction factors are calculated separately, but it is not limited to this. For example, a correction factor combining the first and second correction factors may also be calculated.

[0136] Figure 16 and Figure 17 This is a graph illustrating the intensity ratio of the background 2's brightness measured during the intensity ratio measurement process, and the calculated first and second correction coefficients. Additionally, in the following explanation, regarding... Figure 13 and Figure 14 The parts that have already been described are marked with the same symbol and the description is omitted.

[0137] The second correction factor is a correction factor that reduces the difference in the intensity ratio of the brightness of each wavelength in regions (A) to (I). For example, the second correction factor is a correction factor that makes the brightness intensity of the first wavelength λ(1) to the third wavelength λ(3) in regions (A) to (I) the same.

[0138] Simulated color image 103 ( Figure 16 In the analog color image 103, the intensity ratio of brightness in regions (A) to (I) becomes uniform through the first correction coefficient. However, as shown in the graphs (A) to (I) of symbol 107, the intensity ratio of brightness in the first wavelength λ(1) to the third wavelength λ(3) is not uniform. Specifically, in the graph (E) of symbol 107, the intensity ratios are 0.33, 0.32, and 0.35. By applying the second correction coefficient, this non-uniform intensity ratio is corrected to a uniform value of 0.33, 0.33, and 0.33. In addition, the second correction coefficient is calculated in the same way for other regions. By applying the second correction coefficient to the analog color image 103, an analog color image 151 is obtained. As shown in symbol 153, the intensity ratio of brightness in regions (A) to (I) of the analog color image 151 is 0.33, 0.33, and 0.33. The analog color image 151 displays the original white of the background 2.

[0139] As shown in the diagrams (A) to (I) of symbol 115, the simulated color image 111 ( Figure 17 The intensity ratios of the brightness of the first wavelength λ(1) to the third wavelength λ(3) are not uniform. Specifically, in the graph (E) of symbol 115, the intensity ratios are 0.34, 0.34, and 0.32, but are corrected to 0.33, 0.33, and 0.33 by applying a second correction factor. By applying the second correction factor to the analog color image 111, an analog color image 155 is obtained. As shown by symbol 157, the intensity ratios of the brightness of regions (A) to (I) of the analog color image 155 are 0.33, 0.33, and 0.33. The analog color image 155 displays the original white of the background 2.

[0140] As explained above, in the intensity ratio measurement process, the intensity ratio of the brightness of the background 2 at the first wavelength λ(1) to the third wavelength λ(3) is measured, and a first correction coefficient and a second correction coefficient are calculated based on the measurement results. Furthermore, while the above description illustrates an example of dividing the image into nine 3×3 regions and obtaining the second correction coefficient for each region, this embodiment is not limited to this. For example, the second correction coefficient can be obtained for each pixel of the image, or it can be obtained in a relatively large or small region.

[0141] [Correction process (Second correction process)]

[0142] In the calibration process (second calibration process), the processor 142 uses the second calibration coefficient obtained through the intensity ratio measurement process to apply the second calibration coefficient to the first wavelength image, the second wavelength image, and the third wavelength image captured through the imaging process, thereby correcting the intensity ratio of regions (A) to (I). Furthermore, the second calibration coefficient is preferably applied to the entire first wavelength image to the third wavelength image, but it can also be applied, for example, to the portion of background 2 (image data of background 2) that reflects the first wavelength image to the third wavelength image. By applying the second calibration coefficient to the first wavelength image to the third wavelength image acquired through the imaging process, stable sensing can be achieved by suppressing the influence of imaging conditions, etc. Additionally, the processor 142 can calibrate the image sensor 130 based on the first calibration coefficient and / or the second calibration coefficient. Figure 20 The sensitivity of the image sensor 130 is adjusted (sensitivity calibration process). At locations where the first or second calibration coefficient is large, the sensing result (brightness of the wavelength image) will be output as brighter or darker than before. Therefore, the processor 142 can calibrate the image sensor 130 based on the first and / or second calibration coefficients. Figure 20 It has high sensitivity and can perform high-precision sensing.

[0143] <Other Implementation Method 1>

[0144] The above embodiments illustrate an example of selecting three different first wavelengths λ(1), second wavelengths λ(2), and third wavelengths λ(3) in the wavelength selection process. However, the number of wavelengths selected by the wavelength selection process is not limited to three wavelengths; multiple wavelengths can be selected. For example, in the wavelength selection process, two wavelengths can be selected, or the aforementioned first wavelength λ(1) and second wavelength λ(2) can be selected.

[0145] For example, the processor 142 can select a first wavelength λ(1) and a second wavelength λ(2) within a range of 15% for the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2).

[0146] Specifically, the processor 142 selects a first wavelength λ(1) and a second wavelength λ(2) that satisfy the following equations.

[0147] |β-α|÷(β+α)≤0.15···(1)

[0148] |β-γ|÷(β+γ)≤0.15···(2)

[0149] Furthermore, in the above formula, α is input to each of the reflectances α1 and α2 of the first subject 3 under the first wavelength λ(1) and the second wavelength λ(2), β is input to each of the reflectances β1 and β2 of the background 2 under the first wavelength λ(1) and the second wavelength λ(2), and γ is input to each of the reflectances γ1 and γ2 of the second subject 4 under the first wavelength λ(1) and the second wavelength λ(2). And, at this time, the first wavelength λ(1) and the second wavelength λ(2) are selected to be smaller than the corresponding values ​​of the first wavelength λ(1) (condition 2).

[0150] Furthermore, the processor 142 is further preferred to select the first wavelength λ(1) and the second wavelength λ(2) within a range of 5% for the first factor (the difference in reflectance between spectral data SD3 and spectral data SD1) and the second factor (the difference in reflectance between spectral data SD3 and spectral data SD2).

[0151] Specifically, the processor 142 selects a first wavelength λ(1) and a second wavelength λ(2) that satisfy the following equations.

[0152] |β-α|÷(β+α)≤0.05···(3)

[0153] |β-γ|÷(β+γ)≤0.05···(4)

[0154] Furthermore, in the above formula, α is input to each of the reflectances α1 and α2 of the first subject 3 under the first wavelength λ(1) and the second wavelength λ(2), β is input to each of the reflectances β1 and β2 of the background 2 under the first wavelength λ(1) and the second wavelength λ(2), and γ is input to each of the reflectances γ1 and γ2 of the second subject 4 under the first wavelength λ(1) and the second wavelength λ(2). And, at this time, the first wavelength λ(1) and the second wavelength λ(2) are selected to be smaller than the corresponding values ​​of the first wavelength λ(1) (condition 2).

[0155] As explained above, in this embodiment, two wavelengths (the first wavelength λ(1) and the second wavelength λ(2)) are selected in the wavelength selection process. In this way, by selecting multiple arbitrary numbers of wavelengths and taking pictures, it is possible to obtain the number of wavelength images desired by the user.

[0156] <Other Implementation Method 2>

[0157] In the above embodiment, the background 2 is a predetermined white board. However, in this embodiment, the processor 142 notifies the optimal background (notification process) based on the reflectance (spectral data) of the first subject 3 and the second subject 4.

[0158] The memory 144 of the multispectral camera 100 in this embodiment stores a plurality of replacement subjects (a plurality of backgrounds) that can be replaced with the background 2, and spectral data of each of the plurality of replacement subjects. Then, the processor 142 notifies one of the plurality of replacement subjects (backgrounds) as the recommended background based on the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4. In addition, the processor 142 can notify the replacement subject on a display unit (not shown) provided on the back of the multispectral camera 100. Furthermore, the multispectral camera 100 may include a background display device 150 for variably displaying the background 2 (see reference). Figure 20 The background display device 150 is, for example, a liquid crystal display, and can display the desired color as the background under the control of the processor 142 (third subject variable display process). Thus, in cases where the background display device 150 changes the background color to simultaneously detect subjects with completely different spectral reflectances, the background color can be changed for each position, or in cases where the multispectral camera 100 with the same settings is to be used for different purposes (e.g., detection of other subjects), the background color can be easily changed.

[0159] Figure 18 and Figure 19 This is an image illustrating an example of a recommended background.

[0160] Figure 18 The recommended background spectral data SD4 (fourth spectral data) is shown. Based on the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4, a background with spectral data SD4 is selected and notified as the recommended subject. In the wavelength region of 400nm to 1000nm, the difference in reflectance between spectral data SD1 and spectral data SD2 is small. By using a background with this spectral data SD4, multiple wavelengths for shooting using the multispectral camera 100 can be selected over a wide wavelength range.

[0161] Figure 19 The recommended background spectral data SD5 is shown. Based on the spectral data SD1 of the first subject 3 and the spectral data SD2 of the second subject 4, a background with spectral data SD5 is selected and notified as the recommended subject. Since spectral data SD5 intersects with spectral data SD1 and spectral data SD2 at multiple locations, it has areas where the difference in reflectance between spectral data SD1 and spectral data SD2 is smaller. By using a background with this spectral data SD5, it is possible to take pictures at multiple locations (wavelength regions).

[0162] <Other>

[0163] Next, the multispectral camera 100 used in the above-described imaging method will be described.

[0164] [Multispectral Camera]

[0165] Figure 20 This is a schematic diagram illustrating an example of a multispectral camera 100 used in the imaging method of the present invention. Furthermore, Figure 21 This is a schematic diagram illustrating an example of the filter unit 120 provided by the multispectral camera 100.

[0166] Figure 20 The multispectral camera 100 shown comprises a photographic optical system 110 including lenses 110A and 110B and a filter unit 120, an image sensor (imaging element) 130, and a signal processing unit 140. Furthermore, a background display device 150 can be connected to the multispectral camera 100. The background display device 150 is connected to the signal processing unit 140 and controlled by a processor 142. The filter unit 120 (… Figure 21 The bandpass filter unit 124 included in the filter unit is composed of a first bandpass filter (first wavelength selection element) 124A, a second bandpass filter (second wavelength selection element) 124B, and a third bandpass filter (third wavelength selection element) 124C, which are light transmission wavelength regions. The wavelength regions are those where the first wavelength λ(1), the second wavelength λ(2), and the third wavelength λ(3), which are suitable for identifying the first subject 3 and the second subject 4, are respectively used as the center wavelengths. In addition, the filter unit 120 has four pupil regions (first pupil region to fourth pupil region), and the unused fourth pupil region is blocked by the blocking member B (see reference). Figure 21 ).

[0167] The filter unit 120 is preferably composed of a polarization filter unit 122 and a bandpass filter unit 124, and is disposed at or near the pupil position of the photographic optical system 110.

[0168] The polarization filter unit 122 is composed of a first polarization filter 122A, a second polarization filter 122B, and a third polarization filter 122C that respectively linearly polarize the light in the first pupil region, the second pupil region, and the third pupil region of the transmission imaging optical system 110. For example, the polarization direction of the first polarization filter 122A is set to 0°, the polarization direction of the second polarization filter 122B is set to 90°, and the polarization direction of the third polarization filter 122C is set to 45°.

[0169] The bandpass filter unit 124 is composed of a first bandpass filter 124A, a second bandpass filter 124B, and a third bandpass filter 124C, which respectively select the wavelength regions of light in the first pupil region, the second pupil region, and the third pupil region of the transmission imaging optical system 110. Therefore, the light in the first pupil region of the transmission imaging optical system 110 is linearly polarized by the first polarization filter 122A, and only light containing the wavelength region of the first wavelength λ(1) is transmitted through the first bandpass filter 124A. On the other hand, the light in the second pupil region of the transmission imaging optical system 110 is linearly polarized by the second polarization filter 122B (linearly polarized in a direction 90° different from the first polarization filter 122A), and only light containing the wavelength region of the second wavelength λ(2) is transmitted through the second bandpass filter 124B. Furthermore, the light in the third pupil region of the transmission imaging optical system 110 is linearly polarized by the third polarization filter 122C, and only the light in the wavelength region containing the third wavelength λ(3) is transmitted through the third bandpass filter 124C.

[0170] The image sensor 130 is configured such that a first polarization filter, a second polarization filter, and a third polarization filter with polarization directions of 0°, 45°, and 90° are regularly arranged among a plurality of pixels composed of photoelectric conversion elements arranged in a two-dimensional shape.

[0171] In addition, the first polarization filter 122A has the same polarization direction as the first polarization filter of the image sensor 130, the second polarization filter 122B has the same polarization direction as the second polarization filter of the image sensor 130, and the third polarization filter 122C has the same polarization direction as the third polarization filter of the image sensor 130.

[0172] The signal processing unit 140 acquires a first wavelength image with a narrow long frequency band selected by the first bandpass filter 124A by reading pixel signals from the pixels configured with the first polarization filter of the image sensor 130, acquires a second wavelength image with a narrow frequency band selected by the second bandpass filter 124B by reading pixel signals from the pixels configured with the second polarization filter of the image sensor 130, and acquires a third wavelength image with a narrow frequency band selected by the third bandpass filter 124C by reading pixel signals from the pixels configured with the third polarization filter of the image sensor 130.

[0173] The first wavelength image, the second wavelength image, and the third wavelength image acquired by the signal processing unit 140 become images suitable for separating the first subject 3 and the second subject 4. For example, by synthesizing the first wavelength image, the second wavelength image, and the third wavelength image, a composite image with a magnified dynamic range and enhanced sensing performance can be produced. In addition, the signal processing unit 140 performs interference removal processing on the first wavelength image, the second wavelength image, and the third wavelength image as needed.

[0174] In the above embodiments, the hardware structure of the processing unit that performs various processes is as shown below. These processors include general-purpose processors that execute software (programs) and function as various processing units, such as CPUs (Central Processing Units), FPGAs (Field Programmable Gate Arrays), and programmable logic devices (PLDs), as well as dedicated electrical circuits with circuit structures specifically designed for performing specific processes.

[0175] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of CPU and FPGA). Furthermore, a single processor can also constitute multiple processing units. As examples of multiple processing units composed of one processor, firstly, in computers such as client or server computers, a single processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Secondly, in systems-on-a-chip (SoC), a processor is used to implement the functionality of the entire system containing multiple processing units using a single IC (Integrated Circuit) chip. Thus, various processing units are constructed as hardware structures using one or more of the aforementioned processors.

[0176] Moreover, more specifically, the hardware architecture of these various processors is an electrical circuit composed of semiconductor elements and other circuitry.

[0177] The aforementioned structures and functions can be appropriately implemented using any hardware, software, or a combination of both. For example, the present invention can also be applied to programs that cause a computer to perform the aforementioned processing steps (processing sequence), computer-readable recording media (non-transitory recording media) containing such programs, or computers capable of installing such programs.

[0178] The examples of the present invention have been described above, but the present invention is not limited to the above embodiments. It is self-evident that various modifications can be made without departing from the spirit of the present invention.

[0179] Symbol Explanation

[0180] 2-Background, 3-First subject, 4-Second subject, 10-Illumination device, 100-Multispectral camera, 110-Photographic optical system, 110A-Lens, 110B-Lens, 120-Filter unit, 122-Polarization filter unit, 122A-First polarization filter, 122B-Second polarization filter, 122C-Third polarization filter, 124-Bandpass filter unit, 124A-First bandpass filter, 124B-Second bandpass filter, 124C-Third bandpass filter, 130-Image sensor, 140-Signal processing unit, 142-Processor, 144-Memory, 150-Background display device.

Claims

1. A photographic method, comprising photographing a subject using a multispectral camera equipped with a processor, wherein, The processor performs: The data acquisition process involves acquiring the first spectral data of the first subject, the second spectral data of the second subject, and the third spectral data of the third subject. In the wavelength selection process, multiple wavelengths are selected from the wavelength regions of the acquired first to third spectral data. In the wavelength selection process, the ratio of the characteristic quantities of two spectral data in the first, second, and third spectral data is used as a factor, and multiple wavelengths are selected based on at least two of the factors. and The camera recording process involves capturing images of a subject, including at least one of the first subject, the second subject, and the third subject, at the multiple wavelengths. The factors include a first factor that is the ratio of the feature quantity of the first spectral data to the feature quantity of the third spectral data, and a second factor that is the ratio of the feature quantity of the second spectral data to the feature quantity of the third spectral data, and multiple wavelengths are selected based on the first factor and the second factor.

2. The imaging method according to claim 1, wherein, The wavelength regions of the first to third spectral data are wavelength regions that overlap with at least the wavelength region of the first spectral data and the wavelength region of the second spectral data.

3. The imaging method according to claim 1, wherein, One of the plurality of wavelengths is the minimum wavelength, which is at least one of the first factor and the second factor.

4. The imaging method according to any one of claims 1 to 3, further comprising: A process of measuring the intensity ratio of the brightness of multiple wavelengths selected through the wavelength selection process in multiple regions of the image data of the third subject; and The correction process for at least correcting the image data of the third subject based on the intensity ratio.

5. The imaging method according to claim 4, wherein, The calibration process includes a first calibration process, which calibrates the intensity ratio of the plurality of regions based on one of the measured intensity ratios, namely a first intensity ratio.

6. The imaging method according to claim 4, wherein, The calibration process includes a second calibration process, which corrects for the difference in the measured intensity ratio.

7. The imaging method according to claim 4, wherein, The calibration process includes: The first calibration step involves calibrating the intensity ratios of the plurality of regions based on one of the measured intensity ratios, namely, the first intensity ratio; and The second calibration step involves correcting for the difference in the measured intensity ratio.

8. The imaging method according to any one of claims 1 to 3, wherein, The intensity ratio of the brightness of the multiple wavelengths selected by the wavelength selection process in multiple regions of the third subject is constant.

9. The imaging method according to any one of claims 1 to 3, wherein, In the wavelength selection process, a first wavelength and a second wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (1). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (2). The (β-α) and (β-γ) under the second wavelength are smaller than the (β-α) and (β-γ) under the first wavelength. |β-α|÷（β+α）≤0.15···（1）， |β-γ|÷(β+γ)≤0.15···(2).

10. The imaging method according to any one of claims 1 to 3, wherein, In the wavelength selection process, a first wavelength and a second wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (3). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength and the second wavelength satisfy the following relationship (4). The (β-α) and (β-γ) under the second wavelength are smaller than the (β-α) and (β-γ) under the first wavelength. |β-α|÷（β+α）≤0.05···（3）， |β-γ|÷(β+γ)≤0.05···(4).

11. The imaging method according to any one of claims 1 to 3, wherein, In the wavelength selection process, a first wavelength, a second wavelength, and a third wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength, the second wavelength, and the third wavelength satisfy the following relationship (5). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength, the second wavelength, and the third wavelength satisfy the following relationship (6). The (β-α) and (β-γ) under the second wavelength are smaller than the (β-α) and (β-γ) under the first wavelength and the third wavelength. |β-α|÷（β+α）≤0.15···（5）， |β-γ|÷(β+γ)≤0.15···(6).

12. The imaging method according to any one of claims 1 to 3, wherein, In the wavelength selection process, a first wavelength, a second wavelength, and a third wavelength are selected. The reflectance α of the first subject and the reflectance β of the third subject under the first wavelength, the second wavelength, and the third wavelength satisfy the following relationship (7). The reflectance γ of the second subject and the reflectance β of the third subject under the first wavelength, the second wavelength, and the third wavelength satisfy the following relationship (8). The (β-α) and (β-γ) under the second wavelength are smaller than the (β-α) and (β-γ) under the first wavelength and the third wavelength. |β-α|÷（β+α）≤0.05···（7）， |β-γ|÷(β+γ)≤0.05···(8).

13. The imaging method according to any one of claims 1 to 3, wherein, The multispectral camera has the following features: The memory stores a plurality of substitute subjects that can be replaced by the third subject, and the fourth spectral data of each of the plurality of substitute subjects. The processor performs: In the notification process, based on the first spectral data, the second spectral data, and the fourth spectral data, one of the plurality of replacement subjects is notified as the recommended third subject.

14. The imaging method according to claim 4, wherein, The processor performs a sensitivity calibration process to correct the sensitivity of the imaging element of the multispectral camera according to the calibration process.

15. The imaging method according to any one of claims 1 to 3, further comprising a third subject variable display step of variablely displaying the third subject.

16. The imaging method according to any one of claims 1 to 3, further comprising an illumination step of shadowless lighting.

17. A recording medium that is non-transitory and computer-readable, and which contains a program that causes a computer to perform the imaging method according to any one of claims 1 to 16.

Images