A method and system for optimizing spectral response function based on the full-parameter imaging model of digital cameras

Abstract

The present invention relates to a method and system for optimizing the spectral response function based on the full-parameter imaging model of a digital camera, including: collecting the existing publicly available spectral sensitivity database; taking pictures to obtain the response values in the original format of training samples under different imaging scenarios and recording the imaging parameters of the camera; synchronously measuring the spectral power distribution of the light source in the imaging scenario; calculating the principal components of the spectral sensitivity database by channel using principal component analysis, and using the pseudoinverse algorithm to calculate the initial spectral response function; then, based on the full-parameter imaging model of the digital camera, predicting the raw response values of the training sample set under the same imaging conditions, and respectively normalizing the response values in the original format of the training samples and the response values of the predicted training sample set in the RGB three channels, calculating the color difference between the two, using it as the loss function, and iteratively optimizing to obtain the final camera spectral response function. The method of the present invention can significantly improve the prediction accuracy of the camera response value and support various practical applications based on digital cameras.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer digital image processing, and particularly relates to a method and system for optimizing a spectral response function based on a full-parameter imaging model of a digital camera. Background Art

[0002] In the fields of computer vision and color science, the estimation of the spectral response function of a digital camera is a key technology, which is widely used in multispectral imaging, illumination estimation, color correction, and color characterization of digital cameras. Since consumer digital cameras are not designed for high-precision vision tasks, manufacturers often do not disclose their spectral response function data. Therefore, researchers have explored various methods for obtaining the spectral response function of a camera, which are mainly divided into two categories: direct measurement methods and indirect estimation methods.

[0003] The direct measurement method calculates the spectral response function of a digital camera by using the digital camera to capture monochromatic light images of different wavelengths generated by a monochromator, and then based on the responses of the camera at different wavelengths. Although this method can provide accurate results, its operation process is complex, requiring expensive equipment and specific measurement environments, which limits its wide application in practical applications.

[0004] In view of the limitations of the direct measurement method, researchers have proposed indirect estimation methods to estimate the spectral response function of a camera. However, estimating the spectral response function of a camera faces multiple challenges, including:

[0005] 1. The ill-posedness of the problem caused by the finite dimensionality of the spectral reflectance makes it difficult to accurately estimate the spectral response function from limited measurement data.

[0006] 2. The influence of noise during the camera shooting process on the estimation algorithm, which may reduce the accuracy of the estimation results.

[0007] 3. The limitation of the estimation algorithm by the curve characteristics of the camera spectral response function, and different camera models may require different estimation strategies.

[0008] 4. There is a large color difference between the estimated spectral response curve and the response value obtained by the actual shooting of the camera in practical applications, which limits its application in tasks such as color correction and characterization.

[0009] Currently, most estimation methods mainly focus on solving the similarity between the curve and the true curve, but ignore the main role of the spectral response function in practical applications. This deviation leads to a large color difference problem between the predicted camera response value and the response value obtained by the actual shooting of the camera, thus causing accuracy errors in subsequent multispectral imaging and color correction applications. Summary of the Invention

[0010] The object of the present invention is to solve the problems described in the background art and propose a method for optimizing the spectral response function based on the full-parameter imaging model of a digital camera.

[0011] In view of the problems existing in the above-mentioned existing research, the present invention proposes a method for solving the problems. First, collect the existing public spectral response function database, measure the spectra of the training samples, take pictures to obtain the raw format response values of the training samples under different imaging scenarios and record the camera imaging parameters, synchronously measure the spectral power distribution of the light source in the imaging scenario, select the data in the 400 to 700 nanometer band in the spectral response function database, and normalize the spectral response function in the RGB three channels. Use principal component analysis (PCA) to calculate the principal components of the spectral response function database in each channel, and use the pseudoinverse algorithm to calculate the initial spectral response function. Then, based on the full-parameter imaging model of the digital camera, predict the raw response values of the training sample set under the same imaging conditions, and normalize the raw format response values of the training samples and the predicted response values of the training sample set in the RGB three channels respectively, calculate the CIEDE2000 color difference between the two, and use it as the loss function. Use the sequential quadratic programming method to iteratively optimize to obtain the final camera spectral response function. The technical solution of the present invention is a method for optimizing the spectral response function based on the full-parameter imaging model of a digital camera, which specifically includes the following steps:

[0012] Step 1: Collect the existing public spectral response function database and normalize the spectral response function in the RGB three channels;

[0013] Step 2: Measure the spectral reflectance of the training samples;

[0014] Step 3: Obtain the raw format digital response values of the training samples under different imaging scenarios and record the camera imaging parameters;

[0015] Step 4: Measure the spectral power distribution of the light source during the shooting process;

[0016] Step 5: Combine the spectral reflectance of the training samples and the spectral power distribution of the light source, use principal component analysis to calculate the principal components of the spectral response function database in each channel, and use the pseudoinverse algorithm to calculate the initial spectral response function;

[0017] Step 6: Based on the initial digital camera imaging model, combine the camera imaging parameters and the initial spectral response function, and use the curve fitting model to obtain the non-linear parameters in the full-parameter imaging model of the digital camera;

[0018] Step 7: Based on the full-parameter imaging model of the digital camera, combine the initial camera spectral response function, and predict the raw response values of the training sample set under any same imaging conditions;

[0019] Step 8: Normalize the response values of the original format of the test samples and the response values of the predicted training sample set separately for the three RGB channels;

[0020] Step 9: Calculate the color difference between the normalized response values of the original format and the predicted response values, and use it as the loss function to iteratively optimize to obtain the final camera spectral response function.

[0021] Furthermore, the specific implementation of Step 5 is as follows;

[0022] Decompose the camera spectral response function into s using the principal component analysis method k，n = σ k * E k , where k = R, G, B, and the coefficient σ k = [σ k，1 , σ k，2 is a 1×2 vector, and E k = [e k，1 , e k，2 T is a 2×31 eigenvector matrix; Based on this, the initial digital camera imaging model is expressed as:

[0023]

[0024] where is the predicted response of the camera, g k σ k E k is the camera spectral response function before normalization, l is the spectral power distribution of the scene light source, and R is the spectral reflectance of the training samples;

[0025] By solving k = R, G, B, where the superscript ‘+’ represents the pseudoinverse, to restore the camera spectral response function, then the initial spectral response function S k is expressed as:

[0026]

[0027] where r is the digital response value of the original format of the training samples obtained by the camera shooting.

[0028] Furthermore, the specific implementation of Step 6 is as follows;

[0029] When the initial camera spectral response function is known, the preliminary predicted camera response can be obtained:

[0030]

[0031] where S k is the initial spectral response function, l is the spectral power distribution of the scene light source, and R is the spectral reflectance of the training samples;​

[0032] The initial value of the non - linear parameter can be obtained through a curve fitting model:

[0033]

[0034] where r is the digital response value of the original format of the training sample obtained by camera shooting, and f S is the sensitivity function, and c1, c2, and β are parameters characterizing the non - equivalence between the camera exposure time and ISO.

[0035] Furthermore, the specific implementation method of step 7 is as follows;

[0036] First, the radiation spectral distribution l(λ) of the light source irradiates on the object surface. After absorption and reflection by the object, a radiation spectral image is formed, as shown in Equation (6):

[0037] L scene (x, y, λ) = ∫r(x, y, λ)l(λ) (6)

[0038] In the formula, λ represents the wavelength, r(x, y, λ) is the spectral reflectance of the pixel (x, y) in the image, l(λ) represents the relative spectral irradiance of the light source, and L scene (x, y, λ) represents the radiation spectrum at each pixel; subsequently, the radiation spectral image enters the sensor after being focused by the camera lens. Before entering the sensor, an irradiance image is formed, as shown in Equation (7):

[0039]

[0040] In the formula, f / # is the effective aperture number of the lens, m is the lens magnification, T(λ) is the transmittance of the lens, L(x, y, λ) represents the radiation spectrum at each pixel, and I(x, y, λ) is the irradiance at each pixel; further, the irradiance image enters the camera sensor, and after photoelectric conversion and analog - to - digital conversion processing, a Bayer - pattern mosaic image is obtained, and the camera raw response value image of the scene is obtained through an interpolation algorithm. The process is shown in Equations (8) to (12):

[0041] Q(x, y) = ∫ Ω I image (x, y, λ)S k (λ)dλ (8)

[0042] In the formula, Ω is the spectral wavelength range, S k (λ) is the spectral response function of the camera, Q(x, y) is the number of photoelectric conversions per unit time at the pixel (x, y), and within a given exposure time τ, the total photoelectric conversion amount of the camera sensor is defined as:

[0043] φ(x, y) = f s ∫ τ Q(x, y)dt (9)

[0044] where f s is the sensitivity function and τ is the exposure time; when the spectral irradiance and the quantum efficiency of the sensor remain unchanged during one shot, φ(x) can be further simplified to the form shown in Equation (10):

[0045] φ(x, y) = f s τQ(x, y) (10)

[0046] According to Equations (8) and (10), the raw response value p(x, y) at any pixel point (x, y) of the image can be expressed as:

[0047]

[0048] In addition, considering the non-linear relationship between the irradiance of the camera sensor and the camera readout response, the model uses three coefficients c1, c2, and β to characterize the non-equivalence between the exposure time and ISO, and the expression of the raw response value p(x, y) at any pixel point can be obtained finally:

[0049]

[0050] Furthermore, the channel-wise normalization method in step 8 is as shown in Equation (13):

[0051]

[0052] where R, G, and B are the R channel, G channel, and B channel of the sample respectively, represents the response value vector after channel-wise normalization of a sample, and p k represents the response value before normalization.

[0053] Furthermore, the specific implementation of step 9 is as follows;

[0054] First, convert the response value after channel-wise normalization in step 8 from the RGB color space to the XYZ color space, then to the Lab color space, and then calculate its CIEDE2000 color difference ΔE 00 as the loss function:

[0055]

[0056] where p represents the camera predicted response and the raw format response of taking pictures for each sample respectively, and μ is an adjustable parameter, which is the maximum color difference in the sample to set the regularization constraint of the loss function.

[0057] Further, in step 2, a spectrophotometer is used to measure the spectral reflectance of the training samples;

[0058] In step 3, a digital camera is used to capture the original format digital response values of the training samples under different imaging scenarios;

[0059] In step 4, a color illuminometer is used to synchronously measure the spectral power distribution of the light source during the shooting process.

[0060] Further, in step 3, the camera imaging parameters include the exposure time and the ISO sensitivity.

[0061] Further, in step 9, the sequential quadratic programming method is used to iteratively optimize to obtain the final camera spectral response function.

[0062] The present invention also provides a spectral response function optimization system based on a digital camera full-parameter imaging model, including:

[0063] One or more processors;

[0064] A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement a spectral response function optimization method based on a digital camera full-parameter imaging model as described in the above solution.

[0065] Aiming at the problem that there is a large color difference between the spectral response function curve obtained by the existing camera spectral response estimation method and the response value obtained by the actual shooting of the camera in practical applications, based on the digital camera full-parameter imaging model, the raw response values of the training sample set under the same imaging conditions are predicted, and the original format response values of the training samples and the response values of the predicted training sample set are normalized separately for the RGB three channels, the CIEDE2000 color difference between the two is calculated, which is used as the loss function, and the sequential quadratic programming method is used to iteratively optimize to obtain the final camera spectral response function. The method of the present invention can significantly improve the prediction accuracy of the camera response value and support various practical applications based on digital cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of an embodiment of the present invention.

[0067] Figure 2 It is a distribution diagram of the RGB three channels of the camera spectral response, where (a) is the initially estimated camera spectral response and (b) is the optimized camera spectral response. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0068] When the technical solution of the present invention is specifically implemented, those skilled in the art can use computer software technology to run it.

[0069] As Figure 1 shown, the present invention provides a multispectral reconstruction method based on camera raw response value prediction, specifically including the following steps:

[0070] Step 1: Collect the existing public spectral response function database, select the data in the wavelength range of 400 to 700 nanometers, and normalize the spectral response function in three RGB channels;

[0071] The method for normalizing the spectral response function in three RGB channels in Step 1 is as follows:

[0072]

[0073] where s k，n is the normalized spectral response function, s k，n =[s k (400nm), s k (410nm), …, s k (700nm)], S k is the spectral response function before normalization, S k =g k *s k，n , g k >0, k = R, G, B, g k is the constant for the R, G, and B channels. R, G, and B represent the R channel, G channel, and B channel respectively. In this method, the existing spectral response function is normalized channel by channel, and the peaks of all RGB channels are normalized to 1.

[0074] Step 2: Measure the spectral reflectance R of the training samples using a spectrophotometer;

[0075] Step 3: Use a digital camera to capture the raw format digital response values of the training samples under different imaging scenarios, and record the camera imaging parameters (such as exposure time, ISO, etc.);

[0076] Step 4: Use a color illuminometer to synchronously measure the spectral power distribution l of the light source during the shooting process;

[0077] Step 5: Combine the spectral reflectance of the training samples and the spectral power distribution of the light source, calculate the principal components of the spectral response function database channel by channel using principal component analysis (PCA), and use the pseudo-inverse algorithm to calculate the initial spectral response function;

[0078] The method for calculating the principal components of the spectral response function database channel by channel using principal component analysis (PCA) and calculating the initial spectral response function using the pseudo-inverse algorithm in Step 5 is as follows:

[0079] Decompose the camera spectral response function into s k，n =σk *E k where \(k = R, G, B\), and the coefficient \(\sigma\) k =\([\sigma k，1 , \sigma k，2 \) is a \(1\times2\) vector, and \(E k = [e k，1 , e k，2 \) T is a \(2\times31\) eigenvector matrix. Based on this, the initial digital camera imaging model can be expressed as:

[0080]

[0081] where is the predicted response of the camera, \(g k \sigma k E k is the camera spectral response function before normalization, \(l\) is the spectral power distribution of the scene light source, and \(R\) is the spectral reflectance of the training sample.

[0082] By solving for \(k = R, G, B\), where the superscript ‘+’ represents the pseudoinverse, the camera spectral response function can be restored, and the initial spectral response function \(S k is expressed as:

[0083]

[0084] where \(r\) is the digital response value in the original format of the training sample obtained by the camera shooting;

[0085] Step 6: Based on the initial digital camera imaging model, combined with the camera imaging parameters and the initial spectral response function, use the curve fitting model to obtain the non - linear parameters in the full - parameter imaging model of the digital camera;

[0086] The method for obtaining the non - linear parameters in the full - parameter imaging model of the digital camera using the curve fitting model based on the initial digital camera imaging model in Step 6 is as follows:

[0087] When the initial camera spectral response function is known, the preliminary predicted camera response can be obtained:

[0088]

[0089] The initial values of the non - linear parameters can be obtained through the curve fitting model:

[0090]

[0091] where \(r\) is the digital response value in the original format of the training sample obtained by the camera shooting, and \(f Sis the sensitivity function, usually obtained by dividing the ISO value by 100. c1, c2, and β are parameters that characterize the non-equivalence between the camera exposure time and ISO.

[0092] Step 7: Based on the full-parameter imaging model of the digital camera, combined with the initial camera spectral response function, camera imaging parameters, and non-linear parameters, predict the raw response values of the training sample set under any same imaging conditions.

[0093] The method for predicting the raw response values of the training sample set under any imaging conditions based on the full-parameter imaging model of the digital camera in Step 7 is as follows:

[0094] First, the radiation spectral distribution l(λ) of the light source irradiates the object surface. After absorption and reflection by the object, a radiation spectral image is formed, as shown in Equation (6):

[0095] L scene (x, y, λ) = ∫r(x, y, λ)l(λ) (6)

[0096] In the formula, λ represents the wavelength, r(x, y, λ) is the spectral reflectance of the pixel (x, y) in the image, l(λ) represents the relative spectral irradiance of the light source, and L scene (x, y, λ) represents the radiation spectrum at each pixel. Subsequently, the radiation spectral image enters the sensor after being focused by the camera lens and forms an irradiance image before entering the sensor, as shown in Equation (7):

[0097]

[0098] In the formula, f / # is the effective aperture number of the lens, m is the lens magnification, T(λ) is the transmittance of the lens, L(x, y, λ) represents the radiation spectrum at each pixel, and I(x, y, λ) is the irradiance at each pixel. Further, the irradiance image enters the camera sensor and, after photoelectric conversion and analog-to-digital conversion processing, obtains a mosaic image in Bayer pattern, and the camera raw response value image of the scene is obtained through an interpolation algorithm. The process is shown in Equations (8) to (12):

[0099] Q(x, y) = ∫ Ω I image (x, y, λ)S k (λ)dλ (8)

[0100] In the formula, Ω is the spectral wavelength range, and the value range in this study is 400 - 700 nm. S k (λ) is the spectral response function of the camera, Q(x, y) is the number of photoelectric conversions per unit time at the pixel (x, y), and within the given exposure time τ, the total photoelectric conversion amount of the camera sensor is defined as:

[0101] φ(x, y) = f s ∫ τ Q(x, y)dt (9)

[0102] Where f s is the sensitivity function, usually obtained by dividing the ISO value by 100, and τ is the exposure time. When the spectral irradiance and the quantum efficiency of the sensor remain unchanged during one shot, φ(x) can be further simplified to the form shown in Equation (10):

[0103] φ(x, y) = f s τQ(x, y) (10)

[0104] According to Equations (8) and (10), the raw response value p(x, y) at any pixel point (x, y) of the image can be expressed as:

[0105]

[0106] In addition, considering the non-linear relationship between the irradiance of the camera sensor and the camera readout response, the model uses three coefficients c1, c2, and β to characterize the non-equivalence between the exposure time and ISO, and thus the expression of the raw response value p(x, y) at any pixel point can be obtained finally.

[0107]

[0108] Step 8: Perform normalization processing on the raw format response values of the test samples obtained in Step 3 and the response values of the predicted training sample set obtained in Step 7 in the RGB three channels respectively;

[0109] The channel-wise normalization method in Step 8 is as shown in Equation (13):

[0110]

[0111] Where R, G, and B are the R channel, G channel, and B channel of the sample respectively, represents the response value vector of a sample after channel-wise normalization, and p k represents the response value before normalization.

[0112] Step 9: Calculate the CIEDE2000 color difference between the two, that is, the color difference between the raw format response value and the predicted response value after normalization, and use it as the loss function. Use the sequential quadratic programming method to iteratively optimize to obtain the final camera spectral response function.

[0113] First, convert the response values after channel-wise normalization in Step 8 from the RGB color space to the XYZ color space, then to the Lab color space, and then calculate their CIEDE2000 color difference (ΔE 00)As the loss function:

[0114]

[0115] Among them, p represents the camera predicted response and the original raw format response of each sample respectively. Since we selected ColorChecker SG140 as the training sample, the total number of samples N is 140. μ is an adjustable parameter, which is the maximum color difference in the samples to set the regularization constraint of the loss function. This method is empirically set to 0.1.

[0116] Our goal is to find the camera spectral response function S k (λ) and the non - linear parameters. Given that the full - parameter imaging model of digital cameras is in the form of a power function, it is difficult to directly calculate its gradient during the optimization process. Therefore, we choose the sequential quadratic programming method to solve this constrained non - linear optimization problem.

[0117] The following is illustrated with a specific example, and the specific description of the embodiments of the present invention is provided as follows:

[0118] The embodiments use a Nikon D7200 digital camera, an EVERFINE SPIC - 300AW spectral color illuminometer, and an X - rite i1 - Pro3 spectrophotometer as experimental instruments, and conduct experiments in the laboratory environment and the outdoor open environment respectively. Specifically, when implementing, the ColorChecker SG140 (CCSG) color card is used as the training sample, and the ColorChecker 24 (CC) color card is used as the test sample.

[0119] In the embodiments, in the laboratory environment, based on a closed - type uniform lighting light box, the Nikon D7200 digital camera is used to take digital images of the CCSG and CC color cards, and the raw response values of the color cards are extracted. During the shooting process, the color card is placed at the center of the shooting field of view, where the plane of the color card is about 1 meter away from the plane of the camera sensor, and the camera focal length is 35 mm. The CC color card is photographed under 5 different combinations of exposure and ISO (as shown in Table 1), and there is no over - exposure problem in the color card images under each group of exposure combination conditions to ensure the effectiveness of the experimental data.

[0120] Table 1 Five groups of exposure settings for experiments in the laboratory environment

[0121]

[0122] In an outdoor open environment, the CCSG and CC color cards were also photographed using a Nikon D7200 digital camera, and the raw response value data of the color card was extracted. In addition, in the open environment experiment, five different combinations of exposure and ISO were also set, as shown in Table 2. Among them, the color card image under the first group of exposure and ISO combinations was slightly overexposed, and there was no overexposure problem in the color card images under the remaining groups of exposure and ISO combinations. During the shooting process, the color card was placed at the center of the shooting field of view, where the plane of the color card was about 3 meters away from the plane of the camera sensor, and the camera focal length was 22 mm. The remaining experimental conditions and the method for predicting the raw response value of the color card were the same as those in the laboratory environment.

[0123] Table 2 Five groups of exposure settings for the open environment experiment

[0124]

[0125] In the experiment, the relative spectral power distribution of the light source was measured using an EVERFINE SPIC-300AW spectral color illuminometer. The spectral reflectance of the color card was obtained using an X-rite i1-Pro3 spectrophotometer. Given the spectral reflectance of the training samples, the spectral response function of the Nikon D7200, the spectral power distribution of the light source, and the imaging parameters, the raw response data of the color card under the corresponding imaging conditions can be predicted using the response value prediction model, as shown in the following formula:

[0126]

[0127] In the formula, f s is the sensitivity function, usually obtained by dividing the ISO value by 100, τ is the exposure time, Ω is the spectral wavelength range, which is taken as 400 - 700 nm in this study, f / # is the effective aperture number of the lens, m is the lens magnification factor, represents the radiation spectrum at each pixel, and S k (λ) is the spectral response function of the camera.

[0128] Normalization processing was performed on the raw format response values of the test samples and the response values of the predicted training sample set respectively. The specific normalization method is shown in the following formula:

[0129]

[0130] In the formula, R, G, and B are the R channel, G channel, and B channel of the sample respectively, represents the response value vector of a sample after channel-wise normalization.

[0131] After obtaining the response values of the normalized predicted training sample set, the spectral reflectance of the training sample set, and the response values of the test samples in the original format, convert them from the RGB color space to the XYZ color space, then to the Lab color space, and then calculate the CIEDE2000 color difference between the two, which is used as the loss function. Use the sequential quadratic programming method to iteratively optimize to obtain the final camera spectral response function.

[0132]

[0133] Among them, p represents the camera predicted response and the original format response of each sample during photographing, respectively. Since we selected ColorChecker SG140 as the training sample, the total number of samples N is 140. μ is an adjustable parameter, which is the maximum color difference in the samples to set the regularization constraint of the loss function, and this method is empirically set to 0.1.

[0134] In the embodiment, the CIEDE2000 color difference (ΔE 00 ) is used as the evaluation index for predicting the camera response value, and the performance of the method of the present invention is evaluated by calculating the color difference between the true original format digital response value obtained by shooting and the predicted raw response value.

[0135] Table 3 Color differences between the true original format digital response values and the predicted raw response values obtained by shooting under five different imaging conditions in the laboratory environment before optimizing the camera spectral response

[0136]

[0137] Table 4 Color differences between the true original format digital response values and the predicted raw response values obtained by shooting under five different imaging conditions in the laboratory environment after optimizing the camera spectral response

[0138]

[0139] Table 5 Color differences between the true original format digital response values and the predicted raw response values obtained by shooting under five different imaging conditions in the open environment before optimizing the camera spectral response

[0140]

[0141]

[0142] Table 6 Color differences between the true original format digital response values and the predicted raw response values obtained by shooting under five different imaging conditions in the open environment after optimizing the camera spectral response

[0143]

[0144] It can be intuitively seen from the numerical values of the evaluation indicators in the table that the method of the present invention can significantly improve the prediction accuracy of the full-parameter imaging model of digital cameras in both laboratory environments and open environments, providing support for subsequent applications.

[0145] On the other hand, the embodiment of the present invention also provides a spectral response function optimization system based on a full-parameter imaging model of a digital camera, including:

[0146] One or more processors;

[0147] A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a spectral response function optimization method based on a full-parameter imaging model of a digital camera as described in the above solution.

[0148] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the described specific embodiments or use similar methods for substitution, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for optimizing the spectral response function based on the full-parameter imaging model of a digital camera, characterized in that It includes the following steps: Step 1: Collect the existing public spectral response database and normalize the spectral response for the RGB three channels; Step 2: Measure the spectral reflectance of the training samples; Step 3: Obtain the original format digital response values of the training samples under different imaging scenarios and record the camera imaging parameters; Step 4: Measure the spectral power distribution of the light source during the shooting process; Step 5: Combine the spectral reflectance of the training samples and the spectral power distribution of the light source, calculate the principal components of the spectral response database for each channel using principal component analysis, and use the pseudo-inverse algorithm to calculate the initial spectral response function; Step 6: Based on the initial digital camera imaging model, combine the camera imaging parameters and the initial spectral response function, and use the curve fitting model to obtain the non-linear parameters in the full-parameter imaging model of the digital camera; Step 7: Based on the full-parameter imaging model of the digital camera, combine the initial camera spectral response function, and predict the raw response values of the training sample set under any same imaging conditions; Step 8: Normalize the original format response values of the test samples and the predicted response values of the training sample set for the RGB three channels respectively; Step 9: Calculate the color difference between the normalized original format response values and the predicted response values, use it as the loss function, and iteratively optimize to obtain the final camera spectral response function.

2. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, characterized in that: The specific implementation method of Step 5 is as follows; The camera spectral response is decomposed into s using the principal component analysis method k,n = σ k * E k , where k = R, G, B, and the coefficient σ k = [σ k,1 , σ k,2 is a 1×2 vector, and E k = [e k,1 , e k,2 T is a 2×31 eigenvector matrix; based on this, the initial digital camera imaging model is expressed as:​ where is the predicted response of the camera, g k σ k E k is the camera spectral response function before normalization, l is the spectral power distribution of the scene light source, and R is the spectral reflectance of the training sample; By solving where the superscript ‘+’ represents the pseudoinverse to recover the camera spectral response function, the initial spectral response function S k is expressed as: Among them, r is the original format digital response value of the training sample obtained by the camera shooting.

3. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, wherein: The specific implementation method of Step 6 is as follows; When the initial camera spectral response function is known, the preliminary predicted camera response can be obtained: Among them, S k is the initial spectral response function, l is the spectral power distribution of the scene light source, and R is the spectral reflectance of the training sample; The initial value of the non-linear parameter can be obtained through the curve fitting model: where r is the digital response value of the original format of the training sample obtained by camera shooting, f S is the sensitivity function, and c1, c2, and β are parameters characterizing the non-equivalence between the camera exposure time and ISO.

4. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, wherein: The specific implementation method of Step 7 is as follows; First, the radiation spectral distribution l(λ) of the light source irradiates on the object surface, and after the absorption and reflection of the object, a radiation spectral image is formed, as shown in Equation (6): L scene (x,y,λ) = ∫r(x,y,λ)l(λ) (6) where λ represents the wavelength, r(x, y, λ) is the spectral reflectance of the pixel at (x, y) in the image, l(λ) represents the relative spectral irradiance of the light source, and L scene (x, y, λ) represents the radiation spectrum at each pixel; subsequently, the radiation spectrum image is focused by the camera lens and then enters the sensor, forming an irradiance image before entering the sensor, as shown in Equation (7): Where f / # is the effective f-number of the lens, m is the magnification of the lens, and T(λ) is the transmittance of the lens. represents the radiation spectrum at each pixel, and I image (x, y, λ) is the irradiance at each pixel; further, the irradiance image enters the camera sensor, undergoes photoelectric conversion and analog-to-digital conversion processing to obtain a Bayer pattern mosaic image, and the camera raw response value image of the scene is obtained through an interpolation algorithm. The process is shown in Equations (8) to (12) as follows: Q(x,y) = ∫ Ω I image (x,y,λ)S k (λ)dλ (8) where Ω is the spectral wavelength range, S k (λ) is the spectral response function of the camera, Q(x, y) is the number of photoelectric conversions per unit time at the pixel (x, y), and within the given exposure time τ, the total photoelectric conversion amount of the camera sensor is defined as: φ(x,y) = f s ∫ τ Q(x,y)dt (9) where f s is the sensitivity function and τ is the exposure time; when the spectral irradiance and the quantum efficiency of the sensor remain unchanged during one shot, φ(x, y) can be further simplified to the form shown in Equation (10): φ(x,y) = f s τQ(x,y) (10) According to Equation (8) and Equation (10), the raw response value p(x, y) at any pixel point (x, y) of the image can be expressed as: In addition, considering the non-linear relationship between the irradiance of the camera sensor and the camera readout response, the model uses three coefficients c1, c2, and β to characterize the non-equivalence between the exposure time and ISO, and the expression of the final raw response value p(x, y) at any pixel point can be obtained:

5. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, characterized in that: The channel normalization method in Step 8 is as shown in Equation (13): Wherein, R, G, and B are the R channel, G channel, and B channel of the sample respectively, represents the response value vector after channel normalization of a sample, and p k represents the response value before normalization.

6. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, wherein: The specific implementation method of Step 9 is as follows; First, convert the response values after channel normalization in step 8 from the RGB color space to the XYZ color space, then to the Lab color space, and finally calculate the CIEDE2000 color difference ΔE 00 As the loss function: Among them, p represents the camera prediction response and the original capture format response of each sample respectively, and μ is an adjustable parameter, which is the maximum color difference in the sample to set the regularization constraint of the loss function.

7. A method for optimizing the spectral response function based on the full-parameter imaging model of a digital camera according to claim 1, characterized in that: In Step 2, a spectrophotometer is used to measure the spectral reflectance of the training samples; In Step 3, a digital camera is used to shoot and obtain the original format digital response values of the training samples under different imaging scenarios; In Step 4, a color illuminometer is used to synchronously measure the spectral power distribution of the light source during the shooting process.

8. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, characterized in that: The camera imaging parameters in Step 3 include the exposure time and the sensitivity ISO.

9. The spectral response function optimization method based on the full-parameter imaging model of a digital camera according to claim 1, wherein: In Step 9, the sequential quadratic programming method is used to iteratively optimize to obtain the final camera spectral response function.

10. A spectral response function optimization system based on the full-parameter imaging model of a digital camera, characterized in that, It includes: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement a method for optimizing a spectral response function based on a full-parameter imaging model of a digital camera as claimed in any one of claims 1 to 9.

Images