Camera response curve calculation method and device

Abstract

The embodiment of the present invention discloses a camera response curve calculation method and device, the method includes data acquisition, feature vector extraction, image acquisition and preprocessing, and response curve calculation and verification. The device includes a data acquisition module, a feature vector extraction module, an image acquisition and preprocessing module, and a response curve calculation and verification module. The technical solution provided by the embodiment of the present invention can generate a dynamic OTP according to the relative color temperature by calibrating the spectral response curve of the camera through the production line, realize the mapping between different mobile phones, and improve the color consistency of the terminal to be calibrated; and through spectral calibration, different terminals to be calibrated can obtain more accurate mapping relationships under different color temperatures, which can effectively reduce the differences between different terminals to be calibrated (such as mobile phones) to achieve more accurate white balance.

Description

[Technical field]

[0001] The present invention relates to the field of computer imaging technology, and in particular to a camera response curve calculation method and device. [Background technology]

[0002] In recent years, mobile phones and other terminal devices have not only become more lightweight in appearance design, but also have continuously improved color quantization accuracy. With the use of higher-pixel cameras, the response differences between different camera modules are amplified, which will cause the pictures taken by different mobile phones to be more prone to color inconsistencies, affecting the color effect. This phenomenon is called the color consistency problem between modules.

[0003] like Figure 1 As shown in the figure, different manufacturers use different color filters when manufacturing CMOS (Complementary Metal Oxide Semiconductor), and the photoelectric responsivity of CMOS in different bands is different, which leads to certain differences in the spectral response curve of the camera, and ultimately causes different color effects for imaging in the same environment. This situation seriously affects the customer experience. Therefore, it is necessary to propose a response curve calibration method based on spectral calibration to improve the color consistency effect of the module in various subjective and objective scenarios.

[0004] In the Chinese patent application number CN2017103042620, the calibration of the camera's spectral response curve is divided into two stages: training and use. The spectral response curve of the three channels of RGB (Red Green Blue) is solved by a dictionary learning method to obtain a sparse dictionary. Then, a hyperspectral imaging image fusion platform is used to take high-resolution images using the camera to be calibrated, and then a low-resolution image of the same scene is taken using the hyperspectral camera. Based on the sparse dictionary of the two images and three channels, a sparse encoding and decoding method is used to solve the camera's spectral response curve. However, in this patent, a large number of training data sets need to be collected. The accuracy and rationality of the training set determine the accuracy of the sparse dictionary, which ultimately affects the accuracy of the response curve; and it is difficult to ensure accuracy when only one image is used to calculate the camera's spectral response curve. [Summary of the invention]

[0005] In view of this, an embodiment of the present invention provides a method for solving the above-mentioned technical problems existing in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a camera response curve calculation method, the method comprising the following steps: data acquisition, using a light source device to collect spectral response data of multiple terminals to be calibrated, and establishing a sample database based on the spectral response data; feature vector extraction, extracting the feature vector of each terminal to be calibrated based on the sample database; image acquisition and preprocessing, the terminal to be calibrated takes n photos of n spectra emitted by the light source device in sequence, obtains n images and captures corresponding original image data, and preprocesses the n images; calculation and verification of the response curve, calculating the response curve of the terminal to be calibrated based on the original image data of the first n-1 images collected, the corresponding spectral response data and the feature vector, and using the response curve and the spectral response data of the nth image to verify the accuracy of the calibration.

[0007] Through the solution provided by this embodiment, the response curve of the terminal to be calibrated is calculated by utilizing the collected spectral response data of the terminal to be calibrated, the extracted feature vector of the terminal to be calibrated, and the captured original image data of the image taken by the terminal to be calibrated, and the accuracy of the calibration is verified accordingly, thereby improving the color consistency effect of the terminal to be calibrated in various subjective and objective scenes. In addition, the embodiment of the present invention calculates and verifies the response curve based on multiple images obtained by multiple photographs, which can improve the calibration accuracy of the response curve.

[0008] In a preferred embodiment, in the data collection step, the terminal to be calibrated takes a photo of the single pulse light source emitted by the light source device to read the spectral response data of the terminal to be calibrated.

[0009] Through the scheme provided in this embodiment, the spectral response data of the terminal to be calibrated in a certain band and the RGB three-channel response data of the terminal to be calibrated for different bands at a certain point can be obtained, and the feature vector of the terminal to be calibrated can be obtained based on the RGB three-channel response data.

[0010] In a preferred embodiment, in the feature vector extraction step, the terminal to be calibrated is divided into 3×3 blocks, and the feature vectors and corresponding weights of the RGB three channels of each block are respectively obtained.

[0011] Through the solution provided in this embodiment, under different bands, the response data of the RGB three channels of the light emitted by the light source device at different positions in the 3×3 blocks are obtained by the terminal to be calibrated, so as to establish a more detailed sample database of spectral response data, thereby obtaining a more accurate feature vector and its corresponding weight value.

[0012] In a preferred embodiment, in the image acquisition and preprocessing steps, the image is divided into 3×3 blocks, the grayscale values ​​of the RGB three channels of the central area of ​​each block are calculated, and correction and calibration are performed by subtracting fixed parameters from the grayscale values.

[0013] Through the solution provided in this embodiment, by obtaining the grayscale values ​​of the RGB three channels of the central area of ​​the 3×3 block of the image used to determine the characteristic vector of the terminal to be calibrated, the image is grayscale corrected, so that the corrected image can be used for calculation and verification of the response curve.

[0014] In a preferred embodiment, the grayscale response of the RGB three channels of the original image data of the image includes using the mean or median of the central area.

[0015] Through the solution provided in this embodiment, it is possible to realize dynamic calibration of OTP (One Time Programmable) between different terminal devices, more accurately map golden devices under different color temperatures, optimize the white balance effect of the terminal to be calibrated, compensate for the color response error of the device, improve the color consistency effect between different devices, improve the color restoration accuracy, and enhance the user experience.

[0016] In a preferred embodiment, in the image acquisition and preprocessing steps, the illumination of each light source device is in the range of 300 to 500 lux.

[0017] Through the solution provided in this embodiment, the environment of the terminal to be calibrated during calibration is close to the actual use environment, so that the calibration accuracy of the terminal to be calibrated is higher and more suitable for providing reference for actual use.

[0018] In a preferred embodiment, in the steps of calculating and verifying the response curve, the RGB three-channel data of the nth image is calculated using the response curve and the spectral response data of the nth image, and the error between the RGB three-channel data of the nth image and the original image data of the nth image is calculated. If the error is within a preset error accuracy range, the calibration is determined to be accurate; if the error is outside the preset error accuracy range, the calibration is determined to have failed.

[0019] Through the scheme provided by this embodiment, the first n-1 images among the acquired n images are used to calculate the response curve of the terminal to be calibrated, that is, the calculation step of the response curve is executed, and the nth image is used to verify whether the calculated response curve has achieved the desired accuracy, that is, the verification step of the response curve is executed. The error between the calculated response curve and the original image data of the nth image is used as the criterion for determining whether the calibration is successful or not. It can be seen that the larger n is, the higher the accuracy of the verification, and the closer it can be to the response level of the terminal to be calibrated in the actual use environment.

[0020] In a preferred embodiment, in the sample database, the wavelength band interval of the spectrum emitted by the light source device is one of 2 nanometers, 3 nanometers, and 4 nanometers.

[0021] Through the solution provided by this embodiment, different band intervals are selected according to different actual needs so that the light source device emits a corresponding spectrum. For example, if a band interval of 2 nanometers is selected, the color difference of each band in the emitted spectrum is small, and the image captured by the terminal to be calibrated covers more bands; if a band interval of 3 nanometers is selected, the color difference of each band in the emitted spectrum is moderate, and the image captured by the terminal to be calibrated covers moderate bands; if a band interval of 4 nanometers is selected, the color difference of each band in the emitted spectrum is large, and the image captured by the terminal to be calibrated covers fewer bands.

[0022] In a preferred embodiment, the feature vector is obtained by using a dimensionality reduction method, which includes a principal component analysis method or a singular value decomposition method.

[0023] Through the solution provided by this embodiment, the singular value decomposition method is used to obtain the eigenvector, and the principal component analysis method is used to determine the weight of the eigenvector.

[0024] In a preferred embodiment, the number of the feature vectors is two, three or four.

[0025] Through the solution provided in this embodiment, the number of feature vectors required is determined according to the actual type of the terminal to be calibrated and the actual requirements of the calibration. A smaller number can improve the calculation and verification speed, while a larger number can improve the calculation and verification accuracy.

[0026] In a preferred embodiment, n=5.

[0027] Through the solution provided in this embodiment, the light source device sets 5 environments with different color temperatures, and the terminal to be calibrated takes 5 shots to obtain corresponding images respectively, which is sufficient to meet most calibration requirements and takes into account both calibration efficiency and calibration accuracy.

[0028] In a second aspect, an embodiment of the present invention provides a camera response curve calculation device, the device comprising: a data acquisition module, used to use a light source device to collect spectral response data of multiple terminals to be calibrated to a single-channel spectral light source, and establish a sample database based on the spectral response data; a feature vector extraction module, used to extract the feature vector of each terminal to be calibrated based on the sample database; an image acquisition and preprocessing module, used for the terminal to be calibrated to take n photos of n spectra emitted by the light source device in sequence, obtain n images and capture corresponding original image data, and preprocess the n images; a response curve calculation and verification module, used to calculate the response curve of the terminal to be calibrated based on the original image data of the first n-1 images collected, the corresponding spectral response data and the feature vector, and use the response curve and the spectral response data of the nth image to verify the accuracy of the calibration.

[0029] Through the solution provided by this embodiment, the response curve of the terminal to be calibrated is calculated by utilizing the collected spectral response data of the terminal to be calibrated, the extracted feature vector of the terminal to be calibrated, and the captured original image data of the image taken by the terminal to be calibrated, and the accuracy of the calibration is verified accordingly, thereby improving the color consistency effect of the terminal to be calibrated in various subjective and objective scenes. In addition, the embodiment of the present invention calculates and verifies the response curve based on multiple images obtained by multiple photographs, which can improve the calibration accuracy of the response curve.

[0030] In a preferred embodiment, in the data acquisition module, the terminal to be calibrated takes a photo of the single pulse light source emitted by the light source device to read the spectral response data of the terminal to be calibrated.

[0031] Through the scheme provided in this embodiment, the spectral response data of the terminal to be calibrated in a certain band and the RGB three-channel response data of the terminal to be calibrated for different bands at a certain point can be obtained, and the feature vector of the terminal to be calibrated can be obtained based on the RGB three-channel response data.

[0032] In a preferred implementation, in the feature vector extraction module, the terminal to be calibrated is divided into 3×3 blocks, and the feature vectors of the RGB three channels and the corresponding weights of each block are respectively obtained.

[0033] Through the solution provided in this embodiment, under different bands, the response data of the RGB three channels of the light emitted by the light source device at different positions in the 3×3 blocks are obtained by the terminal to be calibrated, so as to establish a more detailed sample database of spectral response data, thereby obtaining a more accurate feature vector and its corresponding weight value.

[0034] In a preferred embodiment, in the image acquisition and preprocessing module, the image is divided into 3×3 blocks, the grayscale values ​​of the RGB three channels of the central area of ​​each block are calculated, and correction and calibration are performed by subtracting fixed parameters from the grayscale values.

[0035] Through the solution provided in this embodiment, by obtaining the grayscale values ​​of the RGB three channels of the central area of ​​the 3×3 block of the image used to determine the characteristic vector of the terminal to be calibrated, the image is grayscale corrected, so that the corrected image can be used for calculation and verification of the response curve.

[0036] In a preferred embodiment, the grayscale response of the RGB three channels of the original image data of the image includes using the mean or median of the central area.

[0037] Through the solution provided in this embodiment, it is possible to realize dynamic calibration of OTP between different terminal devices, more accurately map golden devices under different color temperatures, optimize the white balance effect of the terminal to be calibrated, compensate for the color response error of the device, improve the color consistency effect between different devices, improve color restoration accuracy, and enhance user experience.

[0038] In a preferred embodiment, in the image acquisition and preprocessing module, the illumination of each light source device is in the range of 300 to 500 lux.

[0039] Through the solution provided in this embodiment, the environment of the terminal to be calibrated during calibration is close to the actual use environment, so that the calibration accuracy of the terminal to be calibrated is higher and more suitable for providing reference for actual use.

[0040] In a preferred embodiment, in the calculation and verification module of the response curve, the RGB three-channel data of the nth image is calculated using the response curve and the spectral response data of the nth image, and the error between the RGB three-channel data of the nth image and the original image data of the nth image is calculated. If the error is within a preset error accuracy range, the calibration is determined to be accurate; if the error is outside the preset error accuracy range, the calibration is determined to have failed.

[0041] Through the scheme provided by this embodiment, the first n-1 images among the acquired n images are used to calculate the response curve of the terminal to be calibrated, that is, the calculation step of the response curve is executed, and the nth image is used to verify whether the calculated response curve has achieved the desired accuracy, that is, the verification step of the response curve is executed. The error between the calculated response curve and the original image data of the nth image is used as the criterion for determining whether the calibration is successful or not. It can be seen that the larger n is, the higher the accuracy of the verification, and the closer it can be to the response level of the terminal to be calibrated in the actual use environment.

[0042] In a preferred embodiment, in the sample database, the wavelength band interval of the spectrum emitted by the light source device is one of 2 nanometers, 3 nanometers, and 4 nanometers.

[0043] Through the solution provided by this embodiment, different band intervals are selected according to different actual needs so that the light source device emits a corresponding spectrum. For example, if a band interval of 2 nanometers is selected, the color difference of each band in the emitted spectrum is small, and the image captured by the terminal to be calibrated covers more bands; if a band interval of 3 nanometers is selected, the color difference of each band in the emitted spectrum is moderate, and the image captured by the terminal to be calibrated covers moderate bands; if a band interval of 4 nanometers is selected, the color difference of each band in the emitted spectrum is large, and the image captured by the terminal to be calibrated covers fewer bands.

[0044] In a preferred embodiment, the feature vector is obtained by using a dimensionality reduction method, which includes a principal component analysis method or a singular value decomposition method.

[0045] Through the solution provided by this embodiment, the singular value decomposition method is used to obtain the eigenvector, and the principal component analysis method is used to determine the weight of the eigenvector.

[0046] In a preferred embodiment, the number of the feature vectors is two, three or four.

[0047] Through the solution provided in this embodiment, the number of feature vectors required is determined according to the actual type of the terminal to be calibrated and the actual requirements of the calibration. A smaller number can improve the calculation and verification speed, while a larger number can improve the calculation and verification accuracy.

[0048] In a preferred embodiment, n=5.

[0049] Through the solution provided in this embodiment, the light source device sets 5 environments with different color temperatures, and the terminal to be calibrated takes 5 shots to obtain corresponding images respectively, which is sufficient to meet most calibration requirements and takes into account both calibration efficiency and calibration accuracy.

[0050] Compared with the prior art, this technical solution has at least the following beneficial effects:

[0051] The camera response curve calculation method and device disclosed in the embodiment of the present invention can achieve a more accurate mapping relationship between different terminals to be calibrated at different color temperatures through spectral calibration; and can effectively reduce the differences between different terminals to be calibrated (such as mobile phones) to achieve more accurate white balance.

Brief Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 is a schematic diagram of spectral response curves of different mobile phones in the prior art;

[0054] Figure 2 is a schematic diagram of the steps of the camera response curve calculation method provided in Example 1 of the present invention;

[0055] Figure 3 is a flowchart of the camera response curve calculation method provided by Embodiment 1 of the present invention in a specific application;

[0056] Figure 4a It is the spectral response data of the terminal to be calibrated in a certain band in the camera response curve calculation method provided in Example 1 of the present invention;

[0057] Figure 4b It is the spectral response data of the three channels of RGB when the terminal to be calibrated is located at a certain position in the camera response curve calculation method provided in Example 1 of the present invention;

[0058] Figure 5 It is a sample database of spectral response data collected by different terminals to be calibrated in the camera response curve calculation method provided in Embodiment 1 of the present invention;

[0059] Figure 6a is a schematic diagram of five different spectral data emitted by a light source device in the camera response curve calculation method provided in Example 1 of the present invention;

[0060] Figure 6b is a schematic diagram of spectral response curves of central areas of different terminals to be calibrated in the camera response curve calculation method provided in Example 1 of the present invention;

[0061] Figure 7 It is a module schematic diagram of the camera response curve calculation device provided in Embodiment 2 of the present invention. [Specific implementation method]

[0062] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0063] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] like Figures 1 to 7 As shown, Figure 1 is a schematic diagram of spectral response curves of different mobile phones in the prior art; Figure 2 is a schematic diagram of the steps of the camera response curve calculation method provided in Example 1 of the present invention; Figure 3 is a flowchart of the camera response curve calculation method provided by Embodiment 1 of the present invention in a specific application; Figure 4a It is the spectral response data of the terminal to be calibrated in a certain band in the camera response curve calculation method provided in Example 1 of the present invention; Figure 4b It is the spectral response data of the three channels of RGB when the terminal to be calibrated is located at a certain position in the camera response curve calculation method provided in Example 1 of the present invention; Figure 5 It is a sample database of spectral response data collected by different terminals to be calibrated in the camera response curve calculation method provided in Embodiment 1 of the present invention; Figure 6a is a schematic diagram of five different spectral data emitted by a light source device in the camera response curve calculation method provided in Example 1 of the present invention; Figure 6b is a schematic diagram of spectral response curves of central areas of different terminals to be calibrated in the camera response curve calculation method provided in Example 1 of the present invention; Figure 7 It is a module schematic diagram of the camera response curve calculation device provided in Embodiment 2 of the present invention.

[0065] Example 1

[0066] like Figure 2 As shown, embodiment 1 of the present invention discloses a method for calculating a camera response curve, which includes the following steps: data acquisition, using a light source device to collect spectral response data of multiple terminals to be calibrated, the light source device can be a monochromator device, and a sample database is established according to the spectral response data; feature vector extraction, extracting the feature vector of each terminal to be calibrated according to the sample database; image acquisition and preprocessing, the terminal to be calibrated takes n photos of n spectra emitted by the light source device in sequence, obtains n images and captures corresponding original image data, and preprocesses the n images; calculation and verification of the response curve, calculating the response curve of the terminal to be calibrated according to the original image data of the first n-1 images collected, the corresponding spectral response data and the feature vector, and using the response curve and the spectral response data of the nth image to verify the accuracy of the calibration.

[0067] The camera response curve calculation method of the present embodiment 1 calculates the response curve of the terminal to be calibrated by using the collected spectral response data of the terminal to be calibrated, the extracted feature vector of the terminal to be calibrated, and the captured original image data of the image taken by the terminal to be calibrated, and verifies the accuracy of the calibration accordingly, thereby improving the color consistency effect of the terminal to be calibrated in various subjective and objective scenes, and the embodiment of the present invention calculates and verifies the response curve based on multiple images obtained by multiple photos, which can improve the calibration accuracy of the response curve.

[0068] In the camera response curve calculation method of the present embodiment 1, in the step of data collection, the terminal to be calibrated takes a picture of the single pulse light source emitted by the light source device to read the spectral response data of the terminal to be calibrated.

[0069] The camera response curve calculation method of this embodiment 1 can obtain the spectral response data of the terminal to be calibrated in a certain band and the RGB three-channel response data of the terminal to be calibrated for different bands at a certain point, and obtain the feature vector of the terminal to be calibrated based on the RGB three-channel response data.

[0070] In the camera response curve calculation method of the first embodiment, in the feature vector extraction step, the terminal to be calibrated is divided into 3×3 blocks, and the feature vectors and corresponding weights of the RGB three channels of each block are respectively obtained.

[0071] The camera response curve calculation method of this embodiment 1 uses the response data of the RGB three channels of the terminal to be calibrated to the light emitted by the light source device at different positions in 3×3 blocks in different bands to establish a more detailed sample database of spectral response data, so as to derive more accurate feature vectors and their corresponding weight values.

[0072] In the camera response curve calculation method of the present embodiment 1, in the image acquisition and preprocessing steps, the image is divided into 3×3 blocks, the grayscale values ​​of the RGB three channels in the central area of ​​each block are obtained, and correction and calibration are performed by subtracting fixed parameters from the grayscale values.

[0073] The camera response curve calculation method of this embodiment 1 obtains the grayscale values ​​of the RGB three channels of the central area of ​​the 3×3 block of the image used to determine the feature vector of the terminal to be calibrated, and performs grayscale correction on the image, so that the corrected image can be used for calculation and verification of the response curve.

[0074] In the camera response curve calculation method of the present embodiment 1, the grayscale response of the RGB three channels of the original image data of the image includes using the mean value or the median value of the central area.

[0075] The camera response curve calculation method of this embodiment 1 can realize the dynamic calibration of OTP between different terminal devices, more accurately map golden devices under different color temperatures, optimize the white balance effect of the terminal to be calibrated, compensate for the color response error of the device, improve the color consistency effect between different devices, improve the color restoration accuracy, and enhance the user experience.

[0076] In the camera response curve calculation method of the present embodiment 1, in the steps of image acquisition and preprocessing, the illumination of each light source device is in the range of 300 to 500 lux.

[0077] In the camera response curve calculation method of the first embodiment, the environment of the terminal to be calibrated during calibration is close to the actual use environment, so that the calibration accuracy of the terminal to be calibrated is higher and more suitable for providing reference for actual use.

[0078] In the camera response curve calculation method of the present embodiment 1, in the steps of calculating and verifying the response curve, the RGB three-channel data of the n-th image is calculated using the response curve and the spectral response data of the n-th image, and the error between the RGB three-channel data of the n-th image and the original image data of the n-th image is calculated. If the error is within a preset error accuracy range, the calibration is determined to be accurate; if the error is outside the preset error accuracy range, the calibration is determined to have failed.

[0079] The camera response curve calculation method of this embodiment 1 uses the first n-1 images of the acquired n images to calculate the response curve of the terminal to be calibrated, that is, executes the response curve calculation step, and the nth image is used to verify whether the calculated response curve has achieved the desired accuracy, that is, executes the response curve verification step, and uses the error between the calculated response curve and the original image data of the nth image as the criterion for determining whether the calibration is successful or not. It can be seen that the larger n is, the higher the accuracy of the verification, and the closer it can be to the response level of the terminal to be calibrated in the actual use environment.

[0080] In the camera response curve calculation method of the first embodiment, in the sample database, the wavelength band interval of the spectrum emitted by the light source device is one of 2 nanometers, 3 nanometers, and 4 nanometers.

[0081] The camera response curve calculation method of this embodiment 1 selects different band intervals to make the light source device emit a corresponding spectrum according to different actual needs. For example, if a band interval of 2 nanometers is selected, the color difference of each band in the emitted spectrum is small, and the image captured by the terminal to be calibrated covers more bands; if a band interval of 3 nanometers is selected, the color difference of each band in the emitted spectrum is moderate, and the image captured by the terminal to be calibrated covers moderate bands; if a band interval of 4 nanometers is selected, the color difference of each band in the emitted spectrum is large, and the image captured by the terminal to be calibrated covers fewer bands.

[0082] In the camera response curve calculation method of the first embodiment, the feature vector is obtained by using a dimensionality reduction method, which includes a principal component analysis (PCA) method or a singular value decomposition (SVD) method.

[0083] The camera response curve calculation method of the first embodiment uses a singular value decomposition method to obtain a eigenvector, and uses a principal component analysis method to determine the weight of the eigenvector.

[0084] In the camera response curve calculation method of the first embodiment, the number of eigenvectors is two, three or four.

[0085] The camera response curve calculation method of this embodiment 1 determines the number of feature vectors required based on the actual type of the terminal to be calibrated and the actual requirements of the calibration. A smaller number can improve the calculation and verification speed, and a larger number can improve the calculation and verification accuracy.

[0086] In the camera response curve calculation method of the present embodiment 1, the number of spectra emitted by the light source device, the number of photographs taken by the terminal to be calibrated, and the number of images n acquired can be determined according to actual calibration requirements. Preferably, n=5.

[0087] In the camera response curve calculation method of the present embodiment 1, the light source device sets 5 environments with different color temperatures, and the terminal to be calibrated takes 5 shots to obtain corresponding images respectively, which is sufficient to meet most calibration requirements and takes into account both calibration efficiency and calibration accuracy.

[0088] like Figure 3 As shown, the specific application process of the camera response curve calculation method of this embodiment 1 is as follows:

[0089] Phase 1: Data collection and feature vector extraction

[0090] (1) Data collection

[0091] Select a sample of the mobile phone to be calibrated, the monochromator emits a single pulse light source, the sample takes a photo of the light source, and reads the response data of the mobile phone, such as Figure 4a and Figure 4b As shown; Change the mobile phone, continue to collect, and form Figure 5 The sample database shown.

[0092] (2) Feature vector extraction

[0093] According to the sample database, the response of the camera module of the mobile phone is divided into 3×3 blocks, and the PCA algorithm is used to obtain the feature vectors and corresponding weights of the three RGB channels of each block.

[0094] Phase 2: Module Calibration

[0095] (1) Image acquisition and preprocessing

[0096] The light source equipment sets five environments with different color temperatures. The illumination of each light source is in the range of 300-550 lux. The mobile phone to be calibrated takes an image each time and captures the corresponding original image data; the image is divided into 3×3 blocks to obtain the grayscale values ​​of the RGB three channels in the central area of ​​each block, and BLC (Black Level Correction) calibration is performed by subtracting fixed parameters.

[0097] (2) Calculation of response curve and verification

[0098] The sensor response curve of the camera module of the mobile phone to be calibrated is calculated based on the data of the first four original images collected and the light source data and spectral feature vector of the corresponding images, such as Figure 6a As shown, the first four light source data are: the first lighting channel: 455nm, 560nm, 620nm, 650nm, the second lighting channel: 425nm, 480nm, 560nm, 630nm, the third lighting channel: 500nm, 590nm, 670nm, the fourth lighting channel: 440nm, 525nm, 580nm, 650nm, as shown in FIG. Figure 6b As shown in the figure, in the central area of ​​different mobile phones to be calibrated, the spectral response curves are obviously different due to the different lighting channels; the RGB three-channel data of the fifth photo of the mobile phone to be calibrated is calculated using the response curve and the spectral response data of the fifth photo, and the error with the actual fifth original image data is calculated. If the error accuracy range is met, the calibration is considered accurate, otherwise the calibration fails.

[0099] The camera response curve calculation method of embodiment 1 of the present invention selects a small number of terminals to be calibrated as samples, uses a monochromator to measure the spectral response data of the samples, and generates a sample database; uses PCA to analyze the sample database to obtain the spectral response feature vector of the RGB three-channel; the light source device emits five LED light source combinations with different illumination and color temperature, and the terminal to be calibrated (such as a mobile phone) takes a photo of the light source device; the spectral response curve of the terminal to be calibrated is obtained based on the spectral response data of the first four images, the original image data and the feature vector; and the accuracy of the response curve is verified using the obtained spectral response curve, the fifth spectral response data and the image.

[0100] Example 2

[0101] like Figure 7As shown, embodiment 2 of the present invention discloses a camera response curve calculation device, which includes: a data acquisition module, which is used to use a light source device to collect spectral response data of multiple terminals to be calibrated to a single-channel spectral light source, and establish a sample database according to the spectral response data; a feature vector extraction module, which is used to extract the feature vector of each terminal to be calibrated according to the sample database; an image acquisition and preprocessing module, which is used for the terminal to be calibrated to take n photos of n spectra emitted by the light source device in sequence, obtain n images and capture corresponding original image data, and preprocess the n images; a response curve calculation and verification module, which is used to calculate the response curve of the terminal to be calibrated according to the original image data of the first n-1 images collected, the corresponding spectral response data and the feature vector, and use the response curve and the spectral response data of the nth image to verify the accuracy of the calibration.

[0102] The camera response curve calculation device of the second embodiment uses the collected spectral response data of the terminal to be calibrated, the extracted feature vector of the terminal to be calibrated, and the captured original image data of the image taken by the terminal to be calibrated to calculate the response curve of the terminal to be calibrated and verify the accuracy of the calibration accordingly, thereby improving the color consistency effect of the terminal to be calibrated in various subjective and objective scenes. In addition, the embodiment of the present invention calculates and verifies the response curve based on multiple images obtained by multiple photos, which can improve the calibration accuracy of the response curve.

[0103] In the camera response curve calculation device of the second embodiment, in the data acquisition module, the terminal to be calibrated takes a photo of the single pulse light source emitted by the light source device to read the spectral response data of the terminal to be calibrated.

[0104] The camera response curve calculation device of this embodiment 2 can obtain the spectral response data of the terminal to be calibrated in a certain band and the RGB three-channel response data of the terminal to be calibrated for different bands at a certain point, and obtain the feature vector of the terminal to be calibrated based on the RGB three-channel response data.

[0105] In the camera response curve calculation device of the second embodiment, in the feature vector extraction module, the terminal to be calibrated is divided into 3×3 blocks, and the feature vectors and corresponding weights of the RGB three channels of each block are respectively obtained.

[0106] The camera response curve calculation device of the second embodiment uses the response data of the RGB three channels of the terminal to be calibrated at different positions in 3×3 blocks to the light emitted by the light source device in different bands to establish a more detailed sample database of spectral response data, so as to derive more accurate feature vectors and their corresponding weight values.

[0107] In the camera response curve calculation device of the present embodiment 2, in the image acquisition and preprocessing module, the image is divided into 3×3 blocks, the grayscale values ​​of the RGB three channels of the central area of ​​each block are obtained, and correction and calibration are performed by subtracting fixed parameters from the grayscale values.

[0108] The camera response curve calculation device of the second embodiment obtains the grayscale values ​​of the RGB three channels of the central area of ​​the 3×3 block of the image used to determine the feature vector of the terminal to be calibrated, and performs grayscale correction on the image, so that the corrected image can be used for calculation and verification of the response curve.

[0109] In the camera response curve calculation device of the second embodiment, the grayscale response of the RGB three channels of the original image data of the image includes using the mean value or median value of the central area.

[0110] The camera response curve calculation device of this embodiment 2 can realize dynamic calibration of OTP between different terminal devices, more accurately map golden devices under different color temperatures, optimize the white balance effect of the terminal to be calibrated, compensate for the color response error of the device, improve the color consistency effect between different devices, improve the color reproduction accuracy of the device, and enhance the user experience.

[0111] In the camera response curve calculation device of the second embodiment, in the image acquisition and preprocessing module, the illumination of each light source device is in the range of 300 to 500 lux.

[0112] In the camera response curve calculation device of the second embodiment, the environment of the terminal to be calibrated during calibration is close to the actual use environment, so that the calibration accuracy of the terminal to be calibrated is higher and more suitable for providing reference for actual use.

[0113] In the camera response curve calculation device of the present embodiment 2, in the calculation and verification module of the response curve, the RGB three-channel data of the n-th image is calculated using the response curve and the spectral response data of the n-th image, and the error between the RGB three-channel data of the n-th image and the original image data of the n-th image is calculated. If the error is within the preset error accuracy range, the calibration is determined to be accurate; if the error is outside the preset error accuracy range, the calibration is determined to have failed.

[0114] The camera response curve calculation device of the second embodiment uses the first n-1 images among the acquired n images to calculate the response curve of the terminal to be calibrated, that is, executes the response curve calculation step, and the nth image is used to verify whether the calculated response curve has achieved the desired accuracy, that is, executes the response curve verification step. The error between the calculated response curve and the original image data of the nth image is used as the criterion for determining whether the calibration is successful or not. It can be seen that the larger n is, the higher the accuracy of the verification is, and the closer it can be to the response level of the terminal to be calibrated in the actual use environment.

[0115] In the camera response curve calculation device of the second embodiment, in the sample database, the wavelength band interval of the spectrum emitted by the light source device is one of 2 nanometers, 3 nanometers, and 4 nanometers.

[0116] The camera response curve calculation device of the second embodiment selects different band intervals according to different actual needs to make the light source device emit a corresponding spectrum. For example, if a band interval of 2 nanometers is selected, the color difference of each band in the emitted spectrum is small, and the image captured by the terminal to be calibrated covers more bands; if a band interval of 3 nanometers is selected, the color difference of each band in the emitted spectrum is moderate, and the image captured by the terminal to be calibrated covers moderate bands; if a band interval of 4 nanometers is selected, the color difference of each band in the emitted spectrum is large, and the image captured by the terminal to be calibrated covers fewer bands.

[0117] In the camera response curve calculation device of the second embodiment, the feature vector is obtained by using a dimensionality reduction method, and the dimensionality reduction method includes a principal component analysis method or a singular value decomposition method.

[0118] The camera response curve calculation device of the second embodiment uses the singular value decomposition method to obtain the eigenvector, and uses the principal component analysis method to determine the weight of the eigenvector.

[0119] In the camera response curve calculation device of the second embodiment, the number of eigenvectors is two, three or four.

[0120] The camera response curve calculation device of this embodiment 2 determines the number of feature vectors required according to the actual type of the terminal to be calibrated and the actual requirements of the calibration. A smaller number can improve the calculation and verification speed, and a larger number can improve the calculation and verification accuracy.

[0121] In the camera response curve calculation device of the second embodiment, the number of spectra emitted by the light source device, the number of photographs taken by the terminal to be calibrated, and the number of images n acquired can be determined according to actual calibration requirements, preferably n=5.

[0122] In the camera response curve calculation device of the second embodiment, the light source device sets five environments with different color temperatures, and the terminal to be calibrated takes five shots to obtain corresponding images respectively, which is sufficient to meet most calibration requirements and takes into account both calibration efficiency and calibration accuracy.

[0123] The camera response curve calculation method and device disclosed in the embodiment of the present invention can generate a dynamic OTP according to the relative color temperature by calibrating the spectral response curve of the camera on the production line, realize the mapping of different color temperatures between different mobile phones, and improve the color consistency of the terminal to be calibrated; and through spectral calibration, it is possible to achieve a more accurate mapping relationship between different terminals to be calibrated at different color temperatures, which can effectively reduce the differences between different terminals to be calibrated (such as mobile phones) and achieve a more accurate white balance.

[0124] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A camera response curve calculation method, characterized in that: The method comprises the following steps: Data collection, using a light source device to collect spectral response data of multiple terminals to be calibrated, and establishing a sample database based on the spectral response data; Feature vector extraction, extracting a feature vector of each terminal to be calibrated according to the sample database; Image acquisition and preprocessing: the terminal to be calibrated takes n photos of n spectra emitted by the light source device in sequence, obtains n images and captures corresponding original image data, and preprocesses the n images; Calculation and verification of the response curve: Calculate the response curve of the terminal to be calibrated based on the original image data of the first n-1 images collected, the corresponding spectral response data and the characteristic vector, and use the response curve and the spectral response data of the nth image to verify the accuracy of the calibration.

2. The camera response curve calculation method according to claim 1, characterized in that: In the data collection step, the terminal to be calibrated takes a photo of the single pulse light source emitted by the light source device to read the spectral response data of the terminal to be calibrated.

3. The camera response curve calculation method according to claim 1, characterized in that: In the feature vector extraction step, the spectral response data of the terminal to be calibrated is divided into 3×3 blocks, and the feature vectors and corresponding weights of the RGB three channels of each block are respectively obtained.

4. The camera response curve calculation method according to claim 1, characterized in that: In the image acquisition and preprocessing step, the image is divided into 3×3 blocks, the grayscale values ​​of the RGB three channels in the central area of ​​each block are obtained, and correction and calibration are performed by subtracting fixed parameters from the grayscale values.

5. The camera response curve calculation method according to claim 4, characterized in that: The grayscale response of the RGB three channels of the original image data of the image includes using the mean or median of the central area.

6. The camera response curve calculation method according to claim 1, characterized in that: In the image acquisition and preprocessing steps, the illumination of each light source device is in the range of 300 to 500 lux.

7. The camera response curve calculation method according to claim 1, characterized in that: In the steps of calculating and verifying the response curve, the RGB three-channel data of the nth image is calculated using the response curve and the spectral response data of the nth image, and the error between the RGB three-channel data of the nth image and the original image data of the nth image is calculated. If the error is within a preset error accuracy range, the calibration is determined to be accurate; if the error is outside the preset error accuracy range, the calibration is determined to have failed.

8. The camera response curve calculation method according to claim 1, characterized in that: In the sample database, the wavelength band interval of the spectrum emitted by the light source device is one of 2 nanometers, 3 nanometers, and 4 nanometers.

9. The camera response curve calculation method according to claim 1, characterized in that: The eigenvector is obtained by using a dimensionality reduction method, which includes a principal component analysis method or a singular value decomposition method.

10. The camera response curve calculation method according to claim 1, characterized in that: The number of the feature vectors is two, three or four.

11. The camera response curve calculation method according to any one of claims 1 to 10, characterized in that: n＝5。 12. A camera response curve calculation device, characterized in that: The device comprises: A data acquisition module, used to collect spectral response data of multiple terminals to be calibrated to a single-channel spectral light source, and establish a sample database based on the spectral response data; A feature vector extraction module, used to extract a feature vector of each terminal to be calibrated according to the sample database; An image acquisition and preprocessing module is used for the terminal to be calibrated to take n photos of n spectra emitted by the light source device in sequence, obtain n images and capture corresponding original image data, and preprocess the n images; The response curve calculation and verification module is used to calculate the response curve of the terminal to be calibrated based on the original image data of the first n-1 images collected, the corresponding spectral response data and the characteristic vector, and use the response curve and the spectral response data of the nth image to verify the accuracy of the calibration.

13. The camera response curve calculation device according to claim 12, characterized in that: In the data acquisition module, the terminal to be calibrated takes a photo of the single pulse light source emitted by the light source device to read the spectral response data of the terminal to be calibrated.

14. The camera response curve calculation device according to claim 12, characterized in that: In the feature vector extraction module, the spectral response data of the terminal to be calibrated is divided into 3×3 blocks, and the feature vectors and corresponding weights of the RGB three channels of each block are respectively obtained.

15. The camera response curve calculation device according to claim 12, characterized in that: In the image acquisition and preprocessing module, the image is divided into 3×3 blocks, the grayscale values ​​of the RGB three channels in the central area of ​​each block are obtained, and correction and calibration are performed by subtracting fixed parameters from the grayscale values.

16. The camera response curve calculation device according to claim 15, characterized in that: The grayscale response of the RGB three channels of the original image data of the image includes using the mean or median of the central area.

17. The camera response curve calculation device according to claim 12, characterized in that: In the image acquisition and preprocessing module, the illumination of each light source device is in the range of 300 to 500 lux.

18. The camera response curve calculation device according to claim 12, characterized in that: In the calculation and verification module of the response curve, the RGB three-channel data of the nth image is calculated using the response curve and the spectral response data of the nth image, and the error between the RGB three-channel data of the nth image and the original image data of the nth image is calculated. If the error is within a preset error accuracy range, the calibration is determined to be accurate; if the error is outside the preset error accuracy range, the calibration is determined to have failed.

19. The camera response curve calculation device according to claim 12, characterized in that: In the sample database, the wavelength band interval of the spectrum emitted by the light source device is one of 2 nanometers, 3 nanometers, and 4 nanometers.

20. The camera response curve calculation device according to claim 12, characterized in that: The eigenvector is obtained by using a dimensionality reduction method, which includes a principal component analysis method or a singular value decomposition method.

21. The camera response curve calculation device according to claim 12, characterized in that: The number of the feature vectors is two, three or four.

22. The camera response curve calculation device according to any one of claims 12 to 21, characterized in that: n＝5。

Images