Spectral characteristic estimating device and spectral characteristic estimating method

The spectral characteristics estimation device and method leverage deep learning models to estimate illumination characteristics from non-polarized spectral images, addressing the need for reference objects and lighting fluctuations, achieving high accuracy and ease of use.

WO2026028400A1PCT designated stage Publication Date: 2026-02-05NT T INC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/027573
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2026-02-05

AI Technical Summary

Technical Problem

Conventional spectral imaging methods require capturing a reference object like a white board under the same lighting conditions to estimate spectral characteristics, which is cumbersome and difficult to handle temporal and spatial fluctuations in ambient light, especially when polarization information is not available.

Method used

A spectral characteristics estimation device and method that utilize deep learning models trained on both polarized and non-polarized spectral images to estimate illumination characteristics without needing a reference object, using multimodal contrastive learning to improve estimation accuracy.

Benefits of technology

Enables accurate estimation of spectral characteristics from non-polarized spectral images, reducing the need for separate reference object capture and enhancing handling of lighting variations, thus simplifying the process and improving estimation performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024027573_05022026_PF_FP_ABST
    Figure JP2024027573_05022026_PF_FP_ABST
Patent Text Reader

Abstract

This spectral characteristic estimating device comprises: a spectral image acquiring unit that acquires one spectral image; and a spectral characteristic estimating unit that, on the basis of the one spectral image acquired by the spectral image acquiring unit and a deep model generated on the basis of a plurality of polarized spectral images and a plurality of spectral images, estimates spectral characteristics relating to the one spectral image without using a reference object having a known spectral reflection characteristic for estimating illumination spectral characteristics.
Need to check novelty before this filing date? Find Prior Art

Description

Spectral characteristic estimation device and spectral characteristic estimation method

[0001] The present invention relates to a spectral characteristics estimation device and a spectral characteristics estimation method.

[0002] Spectral imaging, which scans a sample surface in two dimensions and collects spectra for each pixel to store data relating the position on the sample to the spectrum, uses the spectral reflectance characteristics of the target object to perform analyses that are difficult to perform using RGB (Red, Green, Blue) information. Spectral reflectance characteristics indicate the reflectance at each wavelength. Spectral imaging cannot directly obtain the target's spectral reflectance characteristics; instead, it measures the product of the spectral reflectance characteristics of the illumination (ambient light source). Therefore, the spectral characteristics of the illumination are acquired separately, and the acquired data is corrected to obtain the spectral reflectance characteristics of the target object with the effects of the illumination removed.

[0003] The term "lighting" (also referred to as "ambient light" or "light source") refers to a light source that illuminates the subject, regardless of whether it is intentionally installed, and includes not only artificial lighting such as LEDs (Light Emitting Diodes), fluorescent lights, and incandescent bulbs, but also natural lighting such as sunlight (and a mixture of multiple types). When the type of lighting is sunlight, its spectral characteristics vary greatly depending on the weather, time of day, and other factors, making accurate analysis difficult without correction.

[0004] Generally, as shown in FIG. 6A, in order to obtain the spectral characteristics of illumination, a white board 100a is prepared as a reference object with known spectral reflectance characteristics, and the white board 100a is photographed simultaneously with the subject 200a under the same lighting conditions using a spectral camera 300a, and the spectral characteristics of the area of ​​the white board 100a are regarded as the spectral characteristics of the illumination.

[0005] Generally, as shown in FIG. 6B, in order to obtain the spectral characteristics of the illumination, a white board 100b is prepared as a reference object with known spectral reflectance characteristics, and the white board 100b is photographed separately from the subject 200b under the same illumination conditions using a spectral camera 300b, and the spectral characteristics of the area of ​​the white board 100b are regarded as the spectral characteristics of the illumination.

[0006] Patent Document 1 discloses a spectral characteristic correction device that requires polarization information in addition to the acquired spectral characteristics and is equipped with a correction unit that performs correction by estimating the spectral characteristics of the illumination information from the specular reflection component separated by a reflection separation unit using the polarization information.

[0007] Furthermore, Non-Patent Document 1 discloses a technique for estimating illumination information by deep learning using supervised learning, without using an explicit reference object such as a whiteboard.

[0008] Furthermore, Non-Patent Document 2 discloses a technology for performing illumination correction using polarization information without using an explicit reference object such as a white board, based on the fact that the use of polarization information makes it possible to separate specular reflection components from diffuse reflection components.

[0009] WO 2024 / 127584

[0010] Yuqi Li, Qiang Fu, and Wolfgang Heidrich. Multispectral illumination estimation using deep unrolling network. In Proceedings of the IEEE / CVF international conference on computer vision, pp. 2672-2681, 2021. Taishi Ono, Yuhi Kondo, Legong Sun, Teppei Kurita, and Yusuke Moriuchi. Degree-of-linear-polarization-based color constancy. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 19740-19749, 2022.

[0011] In image processing using conventional hyperspectral cameras, the spectral characteristics of lighting are acquired using a white board to eliminate the effects of lighting before processing, but the white board must be photographed separately under the same lighting environment as the target subject, making the processing cumbersome. Furthermore, there is also the issue of it being difficult to deal with temporal and spatial fluctuations in the ambient light.

[0012] The present invention aims to provide a technology that can estimate the spectral characteristics of lighting even from a spectral image that does not contain polarization information, without the need to capture a reference object for estimating the spectral characteristics of lighting, such as a whiteboard.

[0013] One aspect of the present invention is a spectral characteristics estimation device including: a spectral image acquisition unit that acquires one spectral image; a deep model generated based on a plurality of polarized spectral images and a plurality of spectral images; and a spectral characteristics estimation unit that estimates the spectral characteristics of the one spectral image based on the one spectral image acquired by the spectral image acquisition unit without using a reference object whose spectral reflectance characteristics are known for estimating the spectral characteristics of illumination.

[0014] Another aspect of the present invention is a spectral characteristics estimation method including: a spectral image acquisition process for acquiring one spectral image; and a spectral characteristics estimation process for estimating the spectral characteristics of the one spectral image based on a plurality of polarization spectral images and a plurality of spectral images, without using a reference object whose spectral reflectance characteristics are known, based on the one spectral image acquired in the spectral image acquisition process.

[0015] According to the present invention, there is no need to capture an image of a reference object for estimating the spectral characteristics of illumination, such as a white board, and the spectral characteristics can be estimated even from a spectral image that does not contain polarization information.

[0016] Fig. 1 is a schematic configuration diagram of a spectral characteristic estimation device according to an embodiment of the present invention. Fig. 2 is a schematic configuration diagram of a deep model generation unit of the spectral characteristic estimation device according to an embodiment of the present invention. Fig. 3 is a flowchart showing processing of the spectral characteristic estimation device according to an embodiment of the present invention. Fig. 4 is a flowchart showing processing of the deep model generation unit of the spectral characteristic estimation device according to an embodiment of the present invention. Fig. 5 is a diagram explaining the effect when using the spectral characteristic estimation device according to an embodiment of the present invention. Fig. 6 is a diagram showing an example of a conventional spectral characteristic estimation device. Fig. 7 is a diagram showing another example of a conventional spectral characteristic estimation device.

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0018] 1 is a schematic diagram of a spectral characteristics estimation device 1 according to an embodiment of the present invention. The spectral characteristics estimation device 1 is, for example, a personal computer (PC). The spectral characteristics estimation device 1 includes a spectral characteristics estimation device control unit 11, a spectral characteristics estimation device storage unit 12, a spectral image acquisition unit 13, a spectral characteristics estimation unit 14, and a spectral characteristics output unit 15.

[0019] The spectral characteristics estimation device control unit 11 includes a CPU (Central Processing Unit) and controls each unit of the spectral characteristics estimation device 1. The spectral characteristics estimation device storage unit 12 includes semiconductor memories such as RAM (Random Access Memory) and ROM (Read Only Memory). The spectral characteristics estimation device storage unit 12 is connected to the spectral characteristics estimation unit 14. The spectral characteristics estimation device storage unit 12 stores various data necessary for the operation of the spectral characteristics estimation device 1, and the data is read and written under the control of the spectral characteristics estimation device control unit 11.

[0020] The spectral characteristics estimation device storage unit 12 includes a spectral characteristics estimation model storage unit 121. Before the spectral characteristics estimation device 1 starts processing, the spectral characteristics estimation model storage unit 121 acquires and stores a first illumination spectral characteristics estimation model M1 (described later).

[0021] The spectral image acquisition unit 13 is connected to the spectral characteristic estimation unit 14. The spectral image acquisition unit 13 is, for example, a hyperspectral camera that can capture hyperspectral images that visualize differences in physical properties that are difficult to evaluate with the human eye and phenomena that are invisible to the human eye by finely spectrally dividing light. Instead of acquiring spectral images, the spectral image acquisition unit 13 included in the spectral characteristic estimation device 1 may receive real-world spectral images captured by a spectral imaging system external to the spectral characteristic estimation device 1 via a network such as wireless communication. The spectral image acquisition unit 13 captures a spectral image of the subject 200b that does not include polarization information, and outputs the spectral image to the spectral characteristic estimation unit 14. The spectral image is three-dimensional data that has a wavelength axis in addition to the two-dimensional spatial axis of the image, and is expressed as I(x, y, λ).

[0022] The spectral characteristics estimation unit 14 is connected to the spectral characteristics estimation device storage unit 12, the spectral image acquisition unit 13, and the spectral characteristics output unit 15. The spectral characteristics estimation unit 14 estimates polarization information corresponding to the spectral image that is output from the spectral image acquisition unit 13 and does not contain polarization information, using the first illumination spectral characteristics estimation model M1 stored in the spectral characteristics estimation model storage unit 121, without using a reference object (e.g., the white board 100a in FIG. 6A or the white board 100b in FIG. 6B) whose spectral reflectance characteristics are known for estimating the spectral characteristics of the illumination, and outputs the polarization information to the spectral characteristics output unit 15.

[0023] The spectral characteristic output unit 15 is a display or the like and is connected to the spectral characteristic estimation unit 14. The spectral characteristic output unit 15 displays, on the display, the spectral images acquired by the spectral image acquisition unit 13 that do not contain polarization information and the spectral images acquired by the spectral image acquisition unit 13 that contain polarization information (also referred to as spectral characteristics) based on the polarization information estimated by the spectral characteristic estimation unit 14. Note that the spectral characteristic output unit 15 may transmit the spectral images that contain polarization information to another device instead of displaying them on the display.

[0024] 2 is a schematic configuration diagram of the deep model generation unit 16 of the spectral characteristics estimation device 1 according to an embodiment of the present invention. The deep model generation unit 16 shown in FIG. 2 may be built into the spectral characteristics estimation device 1 ( FIG. 1 ), or the function of the deep model generation unit 16 may be provided in an external device other than the spectral characteristics estimation device 1, and information on the first illumination spectral characteristics estimation model M1 generated by the external device may be stored in the spectral characteristics estimation model storage unit 121 of the spectral characteristics estimation device 1.

[0025] The deep model generation unit 16 includes a training data acquisition unit 161, a first illumination spectral characteristic estimation model generation unit 162, a second illumination spectral characteristic estimation model generation unit 163, a first error loss function calculation unit 164, a second error loss function calculation unit 165, a contrast learning loss calculation unit 166, and a loss function addition unit 167.

[0026] The training data acquisition unit 161 is connected to the first illumination spectral characteristics estimation model generation unit 162, the second illumination spectral characteristics estimation model generation unit 163, and the first error loss function calculation unit 164. The training data acquisition unit 161 acquires training data. The training data is a set of a spectral image I(x, y, λ) that does not contain polarization information, a polarization spectral image I(x, y, λ, s) that contains polarization information, and a correct illumination spectral characteristic y.

[0027] Here, we will explain the case where the polarization spectrum image I(x, y, λ, s) is four-dimensional data that has a polarization axis in addition to the two-dimensional spatial axis and wavelength axis of the image. However, for convenience in handling with each deep learning library, the wavelength axis and polarization axis can be integrated into a channel axis, and the entire data can be three-dimensional.

[0028] The training data acquisition unit 161 outputs data of the spectral image I(x, y, λ) to the first illumination spectral characteristic estimation model generation unit 162. The training data acquisition unit 161 outputs data of the polarized spectral image I(x, y, λ, s) to the second illumination spectral characteristic estimation model generation unit 163. The training data acquisition unit 161 outputs data of the correct illumination spectral characteristic y to the first error loss function calculation unit 164. The training data acquisition unit 161 outputs data of the correct illumination spectral characteristic y to the second error loss function calculation unit 165.

[0029] In the present embodiment, a case will be described in which the learning data acquisition unit 161 acquires N sets of learning data (N is an integer equal to or greater than 2). However, since the estimation accuracy by the first illumination spectral characteristics estimation model M1 tends to increase as the number N increases, it is preferable to make N as large as possible.

[0030] The learning data acquiring unit 161 may acquire the learning data in any manner. For example, the learning data acquiring unit 161 may acquire N sets of learning data from an external device, or may acquire N sets of learning data using a data augmentation technique such as that disclosed in Non-Patent Document 1.

[0031] The first illumination spectral characteristics estimation model generation unit 162 is connected to the training data acquisition unit 161 and the first error loss function calculation unit 164. The first illumination spectral characteristics estimation model generation unit 162 generates and stores a first illumination spectral characteristics estimation model M1. The first illumination spectral characteristics estimation model M1 converts the spectral image I(x, y, λ) output from the training data acquisition unit 161 into an embedded representation of an M-dimensional vector, and then calculates an estimated illumination spectral characteristic y A to the first error loss function calculation unit 164. Furthermore, the first illumination spectral characteristic estimation model M1 outputs data a, which is expressed by the following formula (1) regarding the embedded representation, to the contrast training loss calculation unit 166, based on the spectral image I(x, y, λ) output from the training data acquisition unit 161.

[0032]

[0033] The second illumination spectral characteristics estimation model generation unit 163 is connected to the training data acquisition unit 161 and the second error loss function calculation unit 165. The second illumination spectral characteristics estimation model generation unit 163 generates and stores a second illumination spectral characteristics estimation model M2. The second illumination spectral characteristics estimation model M2 converts the spectral image I(x, y, λ, s) output from the training data acquisition unit 161 into an embedded representation of an M-dimensional vector, and then calculates an estimated illumination spectral characteristic y Bto the second error loss function calculation unit 165. Furthermore, the second illumination spectral characteristic estimation model M2 outputs data b, which is expressed by the following equation (2) regarding the embedded representation, to the contrast learning loss calculation unit 166, based on the spectral image I(x, y, λ, s) output from the learning data acquisition unit 161.

[0034]

[0035] The first illumination spectral characteristics estimation model M1 and the second illumination spectral characteristics estimation model M2 are deep learning models having the same structure. The first illumination spectral characteristics estimation model M1 and the second illumination spectral characteristics estimation model M2 may have any network structure.

[0036] The first error loss function calculation unit 164 is connected to the training data acquisition unit 161, the first illumination spectral characteristics estimation model generation unit 162, and the loss function addition unit 167. The second error loss function calculation unit 165 is connected to the training data acquisition unit 161, the second illumination spectral characteristics estimation model generation unit 163, and the loss function addition unit 167.

[0037] The first error loss function calculation unit 164 and the second error loss function calculation unit 165 calculate the estimated illumination spectral characteristic y A , y B The estimated illumination spectral spectra of the first illumination spectral characteristic estimation model M1 and the second illumination spectral characteristic estimation model M2 of the i-th training data are expressed as y A,i , y B,i The correct illumination spectral characteristic is y i In this case, the error loss function Le taking the estimation error into consideration is expressed as the following equation (3).

[0038]

[0039] The first error loss function calculation unit 164 and the second error loss function calculation unit 165 output the error loss function Le obtained by the above (3) to the loss function addition unit 167.

[0040] The embedded representation of the M-dimensional vector of the first illumination spectral characteristics estimation model M1 and the second illumination spectral characteristics estimation model M2 of the i-th training data is expressed as in the following formula (4).

[0041]

[0042] In this case, the contrastive learning loss function Lc that takes the contrastive learning into consideration is expressed as in the following equation (5).

[0043]

[0044] In the above equation (5), τ is a constant.

[0045] The contrast learning loss calculation unit 166 is connected to the first illumination spectral characteristics estimation model generation unit 162, the second illumination spectral characteristics estimation model generation unit 163, and the loss function addition unit 167. The contrast learning loss calculation unit 166 receives the embedded representation as input, and outputs the contrast learning loss function Lc to the loss function addition unit 167 based on the above formula (5).

[0046] The loss function adder 167 is connected to the first error loss function calculator 164, the second error loss function calculator 165, and the contrast learning loss calculator 166. The loss function adder 167 calculates the sum of the error loss function Le output from the first error loss function calculator 164 and the second error loss function calculator 165, and the contrast learning loss function Lc output from the contrast learning loss calculator 166.

[0047] For example, the first illumination spectral characteristics estimation model generation unit 162 corrects the first illumination spectral characteristics estimation model M1 so as to reduce the sum of the error loss function Le and the contrast-based learning loss function Lc calculated by the loss function addition unit 167. This can improve the estimation accuracy of the first illumination spectral characteristics estimation model M1.

[0048] Furthermore, for example, the second illumination spectral characteristics estimation model generation unit 163 corrects the second illumination spectral characteristics estimation model M2 so as to reduce the sum of the error loss function Le and the contrast-based learning loss function Lc calculated by the loss function addition unit 167. This can improve the estimation accuracy of the second illumination spectral characteristics estimation model M2.

[0049] 3 is a flowchart showing the processing performed by the spectral characteristics estimation device 1 according to an embodiment of the present invention. First, the spectral characteristics estimation model storage unit 121 of the spectral characteristics estimation device 1 acquires and stores the first illumination spectral characteristics estimation model M1 generated by the first illumination spectral characteristics estimation model generation unit 162 of the deep model generation unit 16 ( FIG. 2 ) (step S11).

[0050] Next, the spectral image acquisition unit 13 of the spectral characteristic estimation device 1 determines whether or not it has acquired a spectral image that does not contain polarization information (step S12). If the spectral image acquisition unit 13 has not acquired a spectral image that does not contain polarization information, the result of step S12 is "NO," and the process of step S12 is performed again after a predetermined time (e.g., one second) has elapsed.

[0051] On the other hand, if the spectral image acquisition unit 13 acquires one spectral image that does not include polarization information, the determination in step S12 is “YES,” and the spectral characteristic estimation unit 14 of the spectral characteristic estimation device 1 estimates the polarization information of the spectral image acquired in step S12 using the first illumination spectral characteristic estimation model M1 stored in step S11 (step S13), without using a reference object whose spectral reflectance characteristics are known (e.g., the white board 100a in FIG. 6A or the white board 100b in FIG. 6B) for estimating the spectral characteristics of the illumination.

[0052] Thereafter, the spectral characteristic output unit 15 of the spectral characteristic estimation device 1 displays (i.e., outputs) on a display or the like information combining one spectral image not containing polarization information acquired by the spectral image acquisition unit 13 in step S12 with the polarization information corresponding to that spectral image estimated in step S13 (i.e., one spectral image containing polarization information) (step S14). After that, after a predetermined time (e.g., one second) has elapsed, the process of step S12 is performed again.

[0053] 4 is a flowchart showing the processing of the deep model generation unit 16 of the spectral characteristic estimation apparatus 1 according to the embodiment of the present invention. First, the training data acquisition unit 161 of the deep model generation unit 16 acquires, as training data from an external device or the like, sets of a spectral image I(x, y, λ) that does not contain polarization information, a polarization spectral image I(x, y, λ, s) that contains polarization information, and a ground truth illumination spectral characteristic y (step S21).

[0054] Next, the first illumination spectral characteristic estimation model generation unit 162 of the deep model generation unit 16 generates an estimated illumination spectral characteristic y A and also generates data a expressed by the above-mentioned equation (1) (step S22).

[0055] Next, the second illumination spectral characteristic estimation model generation unit 163 of the deep model generation unit 16 calculates the estimated illumination spectral characteristic y B and also generates data b expressed by the above-mentioned equation (2) (step S23).

[0056] Next, the first error loss function calculation unit 164 and the second error loss function calculation unit 165 of the deep model generation unit 16 calculate the correct illumination spectral characteristic y obtained in step S21 and the estimated illumination spectral characteristic y generated in step S22. A and the estimated illumination spectral characteristic y B Using the above, an error loss function Le expressed by equation (3) is generated (step S24).

[0057] Next, the contrastive learning loss calculation unit 166 of the deep model generation unit 16 calculates the contrastive learning loss function Lc expressed by the above-mentioned equation (5) based on the data a generated in step S22 and the data b generated in step S23 (step S25).

[0058] Next, the loss function adder 167 of the deep model generation unit 16 calculates the sum of the error loss function Le generated in step S24 and the contrast learning loss function Lc generated in step S25 (step S26).

[0059] Next, the first illumination spectral characteristics estimation model generation unit 162 and the second illumination spectral characteristics estimation model generation unit 163 of the deep model generation unit 16 modify the first illumination spectral characteristics estimation model M1 and the second illumination spectral characteristics estimation model M2 so that the sum of the error loss function Le and the contrast learning loss function Lc calculated in step S26 becomes small (step S27).

[0060] Conventionally, when adding polarization information to spectral information that does not include polarization information, it was necessary to image the white board 100a and the subject 200a together using a spectral camera 300a, as shown in FIG. 6A, or to image the white board 100b and the subject 200b simultaneously using a spectral camera 300b, as shown in FIG. 6B.

[0061] However, according to this embodiment, there is no need to capture a reference object for estimating the spectral characteristics of illumination, such as a white board, and it is possible to estimate the spectral characteristics even from a spectral image that does not contain polarization information. In other words, according to this embodiment, as shown in Fig. 5, there is no need to use a white board to obtain polarization information, and by capturing an image of subject 200c with spectral camera 300c to obtain a spectral image that does not contain polarization information, it is possible to generate a spectral image that contains polarization information.

[0062] In other words, in this embodiment, in view of the difficulty of obtaining polarization information, polarization information is used only during learning as shown in the flowchart of FIG. 4, and is not used when estimating the illumination spectral characteristics, thereby achieving both ease of use and high estimation performance.

[0063] Furthermore, according to this embodiment, multimodal contrastive learning is used to train the model so that when only a spectral image is input, and when a polarization spectral image containing polarization information is input, similar embeddings are generated. This multimodal contrastive learning can be expected to achieve higher performance than a model trained only with spectral images by canceling out errors contained in the ground truth data and by transferring knowledge cross-modally.

[0064] According to this embodiment, in spectral imaging, it is possible to estimate the spectral characteristics of lighting with high accuracy using deep learning without placing a reference object such as a white board. This reduces the effort required for photography and post-processing. This is also effective in cases where it is difficult to place a white board.

[0065] At least some of the functions of the components of the spectral characteristic estimation apparatus 1 according to the above-described embodiment may be implemented by a computer. In this case, a program for implementing the functions may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. The term "computer system" as used herein includes hardware such as an operating system (OS) and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, read-only memories (ROMs), and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" may also include media that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, or media that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client. The program may be a program for implementing some of the above-described functions, or may be a program that can realize the above-described functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA.

[0066] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0067] The present invention can be applied to a spectral characteristics estimation device and a spectral characteristics estimation method that do not require capturing an image of a reference object for estimating the spectral characteristics of lighting, such as a whiteboard, and that require estimating the spectral characteristics even from a spectral image that does not contain polarization information.

[0068] REFERENCE SIGNS LIST 1...Spectral characteristics estimation device, 11...Spectral characteristics estimation device control unit, 12...Spectral characteristics estimation device storage unit, 13...Spectral image acquisition unit, 14...Spectral characteristics estimation unit, 15...Spectral characteristics output unit, 16...Deep model generation unit, 121...Spectral characteristics estimation model storage unit, 161...Learning data acquisition unit, 162...First illumination spectral characteristics estimation model generation unit, 163...Second illumination spectral characteristics estimation model generation unit, 164...First error loss function calculation unit, 165...Second error loss function calculation unit, 166...Contrast learning loss calculation unit, 167...Loss function addition unit

Claims

1. A spectral characteristics estimation device comprising: a spectral image acquisition unit that acquires one spectral image; a deep model generated based on multiple polarized spectral images and multiple spectral images; and a spectral characteristics estimation unit that estimates the spectral characteristics of the one spectral image based on the one spectral image acquired by the spectral image acquisition unit, without using a reference object whose spectral reflectance characteristics are known for estimating the spectral characteristics of lighting.

2. The spectral characteristic estimation device according to claim 1, further comprising a deep model generation unit that generates the deep model based on the plurality of polarization spectral images and the plurality of spectral images.

3. The spectral characteristic estimation device according to claim 2, wherein the deep model generation unit modifies the deep model using a loss function generated based on the deep model.

4. A spectral characteristics estimation method comprising: a spectral image acquisition process for acquiring one spectral image; and a spectral characteristics estimation process for estimating the spectral characteristics of the one spectral image based on a deep model generated based on a plurality of polarized spectral images and a plurality of spectral images, and the one spectral image acquired in the spectral image acquisition process, without using a reference object whose spectral reflectance characteristics are known for estimating the spectral characteristics of the illumination.

Citation Information

Patent Citations

  • Hyperspectral full-polarization image compression and reconstruction method for optimizing sparse base through machine learning

    CN111426383A

  • Polarization spectrum and image reconstruction joint optimization design method and chip integration method

    CN118275354A

  • Weather measuring apparatus

    JP2012026927A

  • Signal processing device, signal processing method, and program

    JP2023111625A

  • Systems and methods for synthesizing data for training statistical models across different imaging modalities, including polarization images

    JP2023511747A