Image photometric calibration method, device and computer-readable storage medium

Abstract

The present invention discloses an image photometric calibration method, device and computer-readable storage medium, and relates to the field of image processing technology. The image photometric calibration method comprises: obtaining the grayscale value and corresponding reference information of pixels in an image captured by an imaging device to be calibrated; constructing an inverse function of the response function of the imaging device based on a generalized exponential gamma model, and constructing a corresponding loss function; substituting the obtained pixel grayscale value and corresponding reference information into the loss function for solution, so as to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then determining the inverse function of the response function of the imaging device; using the inverse function of the response function to map the grayscale value of the image to be corrected captured by the imaging device, so as to obtain a corrected image. The present invention can obtain better estimation accuracy, or use fewer image samples while achieving the same estimation accuracy. Thereby, the robustness of image photometric calibration is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image photometric calibration method, device and computer-readable storage medium. Background Art

[0002] Imaging devices, such as photographic films, CCD / CMOS arrays, and nonlinear effects in the post-processing process of imaging devices will lead to a nonlinear mapping relationship between the pixel brightness value (Pixel Intensity) of the digital image and the actual light intensity (Light irradiance). Photometric calibration, also known as radiometric calibration, refers to the process of obtaining the nonlinear response function (Response Function) of the imaging device to light intensity through experimental means and corresponding algorithms, thereby restoring linear radiometric information. By obtaining real radiometric information, we can improve the performance of various pattern recognition or computer vision algorithms, such as face recognition, target detection, stereo reconstruction, etc.

[0003] In the related art, a simple gamma curve is used as an approximation of the camera response function (CRF), which is not accurate and cannot be expanded. The non-parametric model used for photometric calibration requires 256 points (parameters) to represent the CRF curve, and the redundancy is too high. Other models also have problems such as parameter redundancy, numerical instability, out-of-bounds values, and training sample dependence. In some specific scenarios, such as when the number of image blocks that can be used for photometric calibration experiments is small, or the constraints generated by the image blocks are weak, these methods will encounter robustness problems, that is, the optimization process is unstable and the solution accuracy is reduced. Summary of the invention

[0004] A technical problem to be solved by the embodiments of the present invention is: how to improve the robustness of image photometric calibration.

[0005] According to a first aspect of some embodiments of the present invention, there is provided an image photometric calibration method, comprising: obtaining grayscale values ​​and corresponding reference information of pixels in an image taken by an imaging device to be calibrated; constructing an inverse function of a response function of the imaging device based on a generalized exponential gamma model, and constructing a corresponding loss function; substituting the obtained grayscale values ​​of the pixels and the corresponding reference information into the loss function to solve the loss function, so as to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then determining the inverse function of the response function of the imaging device; and using the inverse function of the response function to map the grayscale values ​​of the image to be corrected taken by the imaging device, so as to obtain a corrected image.

[0006] In some embodiments, obtaining the grayscale value of a pixel in an image captured by the imaging device to be calibrated and the corresponding reference information includes: obtaining the grayscale value of each color block in the image captured by the imaging device to be calibrated of a standard color card; obtaining the spectral reflectance of each color block of the standard color card as reference information.

[0007] In some embodiments, obtaining the grayscale values ​​of pixels in an image taken by the imaging device to be calibrated and the corresponding reference information includes: obtaining the grayscale values ​​of pixels in each image after the imaging device to be calibrated uses multiple exposure rates to take multiple shots of the same scene; for each image, using the grayscale values ​​of pixels in other images and the exposure rates used during shooting as reference information for the corresponding pixels in the image.

[0008] In some embodiments, an inverse function of a response function of an imaging device is constructed based on a second-order generalized exponential gamma model, wherein the input of the inverse function of the response function is a normalized grayscale value variable of pixels in an image, and the output is a corrected grayscale value variable.

[0009] In some embodiments, the second-order generalized exponential gamma model f a (x) is expressed as:

[0010]

[0011] The vector α represents the parameters a, b and c of the model. α (x) satisfies monotonicity, b≤e, e is a natural constant, exp[·] is an exponential function with the natural constant e as the base, and the value range of x is between 0 and 1.

[0012] In some embodiments, the second-order generalized exponential gamma model f a (x) is equivalent to:

[0013]

[0014] or

[0015]

[0016] Among them, the vector α represents the parameter a of the model, and

[0017] In some embodiments, the response function is any one of the following: a second-order generalized exponential gamma model; an inverse function of the second-order generalized exponential gamma model; a function obtained by compounding multiple second-order generalized exponential gamma models; an inverse function of the result of compounding multiple second-order generalized exponential gamma models.

[0018] In some embodiments, compounding the plurality of second-order generalized exponential gamma models includes additive compounding, multiplicative compounding, and functional compounding.

[0019] In some embodiments, the loss function σ(α) is:

[0020]

[0021] Where F(·) represents the response function, F -1 (·) represents the inverse function of the response function, and α represents the -1 (·) is the set of parameters of all second-order generalized exponential gamma models, each element in the vector u is the grayscale value of each color block in the captured image, and each element in the vector v is the spectral reflectance of each color block of the pre-acquired standard color card.

[0022] In some embodiments, the loss function σ(α) is:

[0023]

[0024] or

[0025]

[0026] Where F(·) represents the response function, F -1 (·) represents the inverse function of the response function; α represents the components of F(·) and F -1 The set of parameters of all second-order generalized exponential gamma models in (·); p represents the pixel identifier, k represents the image identifier, and I p,k+1 represents the gray value of the pth pixel in the kth image; E k It represents the ratio of the exposure used for the k-th image and the k+1-th image.

[0027] According to a second aspect of some embodiments of the present invention, there is provided an image photometric calibration device, comprising: an image information acquisition module, configured to acquire the grayscale values ​​of pixels in an image taken by an imaging device to be calibrated and corresponding reference information; a model construction module, constructing an inverse function of the response function of the imaging device based on a generalized exponential gamma model, and constructing a corresponding loss function; a parameter determination module, configured to substitute the acquired grayscale values ​​of the pixels and the corresponding reference information into the loss function for solution, so as to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then determine the inverse function of the response function of the imaging device; an image correction module, configured to map the grayscale values ​​of the image to be corrected taken by the imaging device using the inverse function of the response function, so as to obtain a corrected image.

[0028] According to a third aspect of some embodiments of the present invention, there is provided an image photometric calibration device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the aforementioned image photometric calibration methods based on instructions stored in the memory.

[0029] According to a fourth aspect of some embodiments of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements any one of the aforementioned image photometric calibration methods.

[0030] Some embodiments of the above invention have the following advantages or beneficial effects: The method of the present invention ensures the continuity, boundedness and monotonicity of the tone mapping curve, and does not cause abnormal results in the optimization solution process. Therefore, under the condition of having the same number of samples, the present invention can obtain better estimation accuracy, or use fewer image samples under the premise of achieving the same estimation accuracy. Thereby, the robustness of image photometric calibration is improved.

[0031] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 A schematic flow chart of an image photometric calibration method according to some embodiments of the present invention is shown.

[0034] Figures 2A to 2D The figure shows the estimation errors of different algorithms under the premise of different numbers of pixel pairs.

[0035] Figure 3 The result of fitting the model using the image data in the database is shown exemplarily.

[0036] Figure 4 A schematic structural diagram of an image photometric calibration device according to some embodiments of the present invention is shown.

[0037] Figure 5 A schematic structural diagram of an image photometric calibration device according to some other embodiments of the present invention is shown.

[0038] Figure 6A schematic structural diagram of an image photometric calibration device according to some further embodiments of the present invention is shown. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0041] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0042] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0043] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0044] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0045] Figure 1 FIG. 2 is a flow chart showing a method for image photometric calibration according to some embodiments of the present invention. Figure 1 As shown, the image photometric calibration method of this embodiment includes steps S102 to S108.

[0046] In step S102, the grayscale values ​​of pixels in an image captured by an imaging device to be calibrated and corresponding reference information are obtained.

[0047] The image captured by the imaging device can be a grayscale image or an RGB image. For an RGB image, the grayscale value of each pixel refers to the grayscale value of the pixel in each color channel image.

[0048] In step S104, an inverse function of the response function of the imaging device is constructed based on the generalized exponential gamma model, and a corresponding loss function is constructed.

[0049] In some embodiments, an inverse function of a response function of an imaging device is constructed based on a second-order generalized exponential gamma model, wherein the input of the inverse function of the response function is a normalized grayscale value variable of pixels in an image, and the output is a corrected grayscale value variable.

[0050] In step S106, the obtained grayscale value of the pixel and the corresponding reference information are substituted into the loss function to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then the inverse function of the response function of the imaging device is determined.

[0051] In step S108, the inverse function of the response function is used to map the grayscale value of the image to be corrected captured by the imaging device to obtain a corrected image.

[0052] The method of the above embodiment ensures the continuity, boundedness and monotonicity of the tone mapping curve, and will not cause abnormal results in the optimization process. Therefore, under the condition of having the same number of samples, the present invention can obtain better estimation accuracy, or use fewer image samples under the premise of achieving the same estimation accuracy. Thereby, the robustness of image photometric calibration is improved.

[0053] The generalized exponential gamma model adopted by the present invention is exemplarily introduced in more detail below.

[0054] In some embodiments, an inverse function of the response function is constructed based on a second-order generalized exponential gamma model. The second-order generalized exponential gamma model is shown in formula (1).

[0055]

[0056] Where b≤e, e is a natural constant, exp[·] is an exponential function with the natural constant e as the base, and the value range of x is between 0 and 1. The specific analysis of the constraints of formula (1) is as follows.

[0057] In order to make f(x) satisfy the non-monotonicity requirement f′(x)≥0, for formula (1), there is a constraint condition Since 2cx+b represents a straight line, it must be monotonic (monotonically increasing or monotonically decreasing), so we only need to ensure that its two ends (x=0, x=1) are less than or equal to e, that is, there is a constraint as shown in formula (2).

[0058]

[0059] The second-order generalized exponential gamma model defined in the embodiment of the present invention has the following properties: 1) passing through a fixed point: f(0)=0, f(1)=1; 2) boundedness: 0≤f(x)≤1; 3) smoothness: any order derivative of f(x) is continuous; 4) monotonicity: f′(x)≥0.

[0060] In some embodiments, a slack variable method may be used, where b = e-p1, p1 and p2 are terms greater than or equal to 0. Thus, the constraint of formula (2) can be satisfied.

[0061] In some embodiments, variable substitution may be used to express the constraint so as to implicitly embed the constraint into the formula, for example, using the constraint shown in formula (3) or (4), where: and as a parameter.

[0062]

[0063]

[0064] Therefore, the second-order generalized exponential gamma model f(x) can also be expressed as formula (5) or (6). α represents the parameter a of the model, and

[0065]

[0066]

[0067] In some embodiments, the constraint range can be further narrowed based on experience, and formulas (3) and (4) can be adjusted without affecting the constraint conditions of formula (2). Or order Where m1 and m2 are non-negative constants. For example, b and c can be expressed as When the values ​​of m1 and m2 are both 0, the constraints shown in formulas (3) and (4) are obtained. Thus, the second-order generalized exponential gamma model can be determined more flexibly as needed.

[0068] The second-order generalized exponential gamma model defined in the embodiment of the present invention has the following advantages: 1) few parameters and compact representation: other methods require 4 to 6 parameters to represent, while the embodiment of the present invention only requires three parameters to represent a mapping curve; 2) numerical stability: only legal mapping curves are represented, so in the optimization and solution process, no matter how the parameters are taken, no abnormal results will occur; 3) implicit inclusion of monotonicity: the methods in the related art require additional monotonicity constraints, while the model proposed in the embodiment of the present invention itself implicitly includes monotonicity requirements, so there will be no non-monotonic abnormal results, and because monotonicity limits the solution space of the problem, its speed in the optimization and solution process can be further improved.

[0069] Based on the second-order generalized exponential gamma model proposed in the embodiment of the present invention, the response function can have multiple representations. In some embodiments, the response function is any one of the following: a second-order generalized exponential gamma model; an inverse function of a second-order generalized exponential gamma model; a function obtained by compounding multiple second-order generalized exponential gamma models; an inverse function of the result of compounding multiple second-order generalized exponential gamma models.

[0070] In some embodiments, compounding multiple second-order generalized exponential gamma models includes additive compounding, multiplicative compounding, and functional compounding. mix (x) represents the result of compounding multiple second-order generalized exponential gamma models. The exemplary method can refer to formulas (7) to (9), which correspond to additive compounding, multiplicative compounding and functional compounding respectively.

[0071] f mix (x) = ∑ i λ i f i (x) / ∑ i λ i (7)

[0072]

[0073] f mix (x)=f9(…f2(f1(x))) (9)

[0074] In formulas (7) to (9), λ i is any non-negative number, f i (x) is any second-order exponential generalized gamma function or its inverse function, and can also be a composite function composed of multiple generalized exponential gamma models or its inverse function.

[0075] Therefore, the second-order generalized exponential gamma model proposed in the embodiment of the present invention is scalable, and thus the response function can be defined more flexibly to meet the needs of more application scenarios.

[0076] Two embodiments of obtaining photometric calibration values ​​are described below in combination with two application scenarios.

[0077] In some embodiments, the grayscale value of each color block in the image captured by the imaging device to be calibrated on the standard color card is obtained; and the spectral reflectance of each color block of the standard color card is obtained as reference information.

[0078] This method can utilize the spectral reflectance of known color blocks, or reflectance information. In some embodiments, an imaging device is first used to capture an image of a standard color card, and the grayscale values ​​of all grayscale blocks in the image are extracted. i For example, 6 grayscale blocks can be extracted from each image of the 24-color standard color card, i.e., i = 1, ..., 6. Then, the second-order generalized exponential gamma model is used to calculate the inverse response function F -1 (·) is used to model the loss function. An example of the loss function σ(α) can be found in formula (10).

[0079]

[0080] In formula (10), F -1 (·) represents the inverse function of the response function; α represents the -1 (·) is the set of parameters of all second-order generalized exponential gamma models; NCC(·) represents normalization processing; v is a vector, and each element in the vector v is the spectral reflectance of each color block of the pre-acquired standard color card, for example, the vector corresponding to the relative reflectance value of the gray color block of the 24-color standard color card is generally v = [90.0, 59.1, 36.2, 19.8, 9.0, 3.1]; each element in the vector u is the grayscale value of each color block in the captured image, for example, vector u = [I1, I2, I3, I4, I5, I6]. When F(·) matches the true response curve, the loss function has a minimum value of -1.

[0081] Since the number of color blocks in the standard color card is often very limited, and in order to avoid the additional errors caused by the post-processing process of the digital camera, color space conversion (color transformation), and tone mapping (gamut mapping), only a few areas with low color saturation, such as grayscale blocks, are often selected for response curve estimation. At this time, the number of samples of color blocks that can be used to solve the constraint equation is very limited (for example, a 24-color standard color card has only 6 grayscale blocks), which may result in too few samples to meet the requirement of stable solution of the response curve, or the response curve obtained by solution has large errors and other adverse consequences. However, the method proposed in the present invention can obtain an accurate response curve when the number of color block samples is limited, so the robustness of photometric calibration can be improved.

[0082] In some embodiments, the grayscale value of the pixel in each image is obtained after the imaging device to be calibrated uses multiple exposure rates to shoot the same scene multiple times; for each image, the product of the grayscale value of the pixel in other images and the exposure rate used during shooting is used as reference information for the corresponding pixel in the image.

[0083] For example, an imaging device can be used to apply different exposure rates E to a standard color chart. k Shoot multiple times to obtain K+1 images. For example, the automatic exposure bracketing (AEB) function of the camera can be used.

[0084] For the same color block in the Kth and K+1th images, a pixel value pair can be constructed p,k ,I p,k+1 >. Taking the same 24-color standard color card as an example, 6 pixel pairs can be extracted from each image, that is, p=1,...,6. Of course, other scenes can also be photographed, and the present invention does not limit the photographed content.

[0085] In this way, formula (11) or (12) can be used as the loss function.

[0086] σ(α)=∑ p,k ||F -1 (I p,k )-E k F -1 (I p,k+1 )|| 2 (11)

[0087] σ(α)=∑ p,k ||I p,k -F[E k F -1 (I p,k+1 )]|| 2 (12)

[0088] In formulas (11) and (12), F(·) represents the response function, F -1 (·) represents the inverse function of the response function; α represents the components of F(·) and F -1 The set of parameters of all second-order generalized exponential gamma models in (·); p represents the pixel identifier, k represents the image identifier, and I p,k+1 represents the gray value of the pth pixel in the kth image; E k It represents the ratio of the exposure used for the k-th image and the k+1-th image.

[0089] ​In some embodiments, when solving the loss function of each of the above embodiments, a nonlinear optimization algorithm, such as Levenberg-Marquardt, is used to optimize the loss function to obtain the inverse function F of the response curve of the imaging device. -1 (x). Then, for the image I(x, y) captured by the imaging device, F -1 (I(x,y)) is mapped to obtain the corrected radiometric image.

[0090] Figures 2A to 2D The estimation errors of different algorithms are shown when using different numbers of pixel pairs. Figures 2A to 2D It represents the use of 1 pair, 2 pairs, 3 pairs, and 6 pairs of pixels respectively. The ordinate represents the normalized error, and the abscissa represents the algorithm. The Debvec algorithm uses a non-parametric model, λ represents the parameter value used to control the smoothness; the Mits algorithm uses a restricted polynomial curve model, and the number after Mits represents the model dimension; Grossberg uses the EMoR model, and the number after Grossberg represents the model dimension; 2nd-MEGGCM represents the method based on the generalized exponential gamma model used in the present invention, A and B correspond to the loss function used in formula (11), and F in strategy A -1 (·) represents the inverse function of the response function of the imaging device directly represented by the second-order generalized exponential gamma model. F in strategy B -1 (·) represents the inverse function obtained numerically after the response function of the imaging device is represented by the second-order generalized exponential gamma model. Strategy C corresponds to the loss function used in formula (12), which is also the inverse function of the response function of the imaging device directly represented by the second-order generalized exponential gamma model.

[0091] The smaller the value of the ordinate, the more accurate the algorithm's estimation is; the narrower the ordinate error distribution is, the stronger the robustness is. From the test results in the figure, it can be seen that the solution used in the present invention has achieved good results in terms of accuracy and robustness.

[0092] Figure 3 The results of fitting the model using the image data in the database are shown in an exemplary manner. Curves 1 to 3 represent the model parameters obtained by fitting several typical response function curves in the DoRF database using the second-order generalized exponential gamma model, and curves 4 to 6 represent the model parameters obtained by fitting the inverse functions of the above response function curves using the second-order generalized exponential gamma model. It can be seen that in both cases, the generalized exponential gamma model proposed in the present invention can fit the above response curves well. The second-order generalized exponential gamma model in the model adopts In the case of , the parameters corresponding to each curve are shown in Table 1.

[0093] Table 1

[0094] curve a b c 1 -0.8826 -1.1378 0.9064 2 0.7677 -5.4869 2.297 3 -0.5130 -5.0073 3.8628 4 0.6273 1.0419 -0.4027 5 -0.2014 1.2452 0.7365 6 0.3812 1.1866 0.5469

[0095] Reference below Figure 4 An embodiment of the image photometric calibration device of the present invention is described.

[0096] Figure 4 FIG. 4 is a schematic diagram showing the structure of an image photometric calibration device according to some embodiments of the present invention. Figure 4 As shown, the image photometric calibration device 40 of this embodiment includes: an image information acquisition module 410, configured to acquire the grayscale values ​​of pixels in the image taken by the imaging device to be calibrated and the corresponding reference information; a model construction module 420, which constructs the inverse function of the response function of the imaging device based on the generalized exponential gamma model, and constructs the corresponding loss function; a parameter determination module 430, configured to substitute the acquired pixel grayscale values ​​and the corresponding reference information into the loss function for solution, so as to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then determine the inverse function of the response function of the imaging device; an image correction module 440, configured to map the grayscale values ​​of the image to be corrected taken by the imaging device using the inverse function of the response function, so as to obtain a corrected image.

[0097] In some embodiments, the image information acquisition module 410 is further configured to acquire the gray value of each color block in the image captured by the imaging device to be calibrated on the standard color card; and acquire the spectral reflectance of each color block of the standard color card as reference information.

[0098] In some embodiments, the image information acquisition module 410 is further configured to obtain the grayscale value of the pixel in each image after the imaging device to be calibrated uses multiple exposure rates to shoot the same scene multiple times; for each image, the grayscale value of the pixel in other images and the exposure rate used during shooting are used as reference information for the corresponding pixel in the image.

[0099] In some embodiments, an inverse function of a response function of an imaging device is constructed based on a second-order generalized exponential gamma model, wherein the input of the inverse function of the response function is a normalized grayscale value variable of pixels in an image, and the output is a corrected grayscale value variable.

[0100] In some embodiments, the second-order generalized exponential gamma model f α (x) is expressed as:

[0101]

[0102] Among them, the vector α represents the parameters a, b and c of the model, b≤e, e is a natural constant, exp[·] is an exponential function with the natural constant e as the base, and the value range of x is between 0 and 1.

[0103] In some embodiments, the second-order generalized exponential gamma model f a (x) is equivalent to:

[0104]

[0105] or

[0106]

[0107] Among them, the vector α represents the parameter a of the model, and

[0108] In some embodiments, the response function is any one of the following: a second-order generalized exponential gamma model; an inverse function of the second-order generalized exponential gamma model; a function obtained by compounding multiple second-order generalized exponential gamma models; an inverse function of the result of compounding multiple second-order generalized exponential gamma models.

[0109] In some embodiments, compounding the plurality of second-order generalized exponential gamma models includes additive compounding, multiplicative compounding, and functional compounding.

[0110] In some embodiments, the loss function σ(α) is:

[0111]

[0112] Where F(·) represents the response function, F -1 (·) represents the inverse function of the response function, and the vector α represents the components of F -1 (·) is the set of parameters of the inverse functions of all response functions. Each element in the vector u is the grayscale value of each color block in the captured image. Each element in the vector v is the spectral reflectance of each color block of the pre-acquired standard color card.

[0113] In some embodiments, the loss function σ(α) is:

[0114]

[0115] or

[0116]

[0117] Where F(·) represents the response function, F -1 (·) represents the inverse function of the response function; α represents the components of F(·) and F -1The set of parameters of the inverse functions of all response functions in (·); p represents the pixel identifier, k represents the image identifier, I p,k+1 represents the gray value of the pth pixel in the kth image; E k It represents the ratio of the exposure used for the k-th image and the k+1-th image.

[0118] Figure 5 FIG. 2 shows a schematic diagram of the structure of an image photometric calibration device according to some other embodiments of the present invention. Figure 5 As shown, the image photometric calibration device 50 of this embodiment includes: a memory 510 and a processor 520 coupled to the memory 510 , and the processor 520 is configured to execute the image photometric calibration method in any one of the aforementioned embodiments based on the instructions stored in the memory 510 .

[0119] The memory 510 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0120] Figure 6 FIG. 4 shows a schematic diagram of the structure of an image photometric calibration device according to some other embodiments of the present invention. Figure 6 As shown, the image photometric calibration device 60 of this embodiment includes: a memory 610 and a processor 620, and may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650 and the memory 610 and the processor 620 may be connected, for example, via a bus 660. Among them, the input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, and a touch screen. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0121] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements any of the aforementioned image photometric calibration methods when executed by a processor.

[0122] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for image photometric calibration, comprising: Obtaining the grayscale value of the pixel in the image captured by the imaging device to be calibrated and the corresponding reference information; Constructing the inverse function of the response function of the imaging device based on the generalized exponential gamma model and constructing the corresponding loss function; Substituting the obtained pixel grayscale value and the corresponding reference information into the loss function to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then determining the inverse function of the response function of the imaging device; The inverse function of the response function is used to map the grayscale value of the image to be corrected captured by the imaging device to obtain a corrected image.

2. The image photometric calibration method according to claim 1, wherein: The step of obtaining the grayscale values ​​of pixels in the image captured by the imaging device to be calibrated and the corresponding reference information includes: Obtaining the grayscale value of each color block in the image taken by the imaging device to be calibrated on the standard color card; The spectral reflectance of each color block of the standard color card is obtained as reference information.

3. The image photometric calibration method according to claim 1, wherein: The step of obtaining the grayscale values ​​of pixels in the image captured by the imaging device to be calibrated and the corresponding reference information includes: Obtaining the grayscale value of pixels in each image after the imaging device to be calibrated uses multiple exposure rates to shoot the same scene multiple times; For each image, the grayscale value of the pixel in other images and the exposure rate used when shooting are used as reference information of the corresponding pixel in the image.

4. The image photometric calibration method according to any one of claims 1 to 3, wherein: An inverse function of a response function of an imaging device is constructed based on a second-order generalized exponential gamma model, wherein the input of the inverse function of the response function is a normalized grayscale value variable of a pixel in an image, and the output is a corrected grayscale value variable.

5. The image photometric calibration method according to claim 4, wherein: The second-order generalized exponential gamma model It is expressed as: Among them, the vector Represents the parameters of the model , , The natural constant The exponential function with base , The value range is between 0 and 1.

6. The image photometric calibration method according to claim 5, wherein: The second-order generalized exponential gamma model The equivalent expression is: or Among them, the vector Represents the parameters of the model .

7. The image photometric calibration method according to claim 5, wherein: The response function is any one of the following: The second-order generalized exponential gamma model; the inverse function of the second-order generalized exponential gamma model; the function obtained by compounding multiple second-order generalized exponential gamma models; the inverse function of the result of compounding multiple second-order generalized exponential gamma models.

8. The image photometric calibration method according to claim 7, wherein: The compounding of multiple second-order generalized exponential gamma models includes additive compounding, multiplicative compounding and functional compounding.

9. The image photometric calibration method according to claim 2, wherein: The loss function for: in, represents the response function, represents the inverse function of the response function, Representation composition The set of parameters of all second-order generalized exponential gamma models in , vector Each element in is the gray value of each color block in the captured image, and the vector Each element in is the spectral reflectance of each color block of the standard color card obtained in advance.

10. The image photometric calibration method according to claim 3, wherein: The loss function for: or in, represents the response function, represents the inverse function of the response function; Representation composition , is the set of parameters of all second-order generalized exponential gamma models in ; p represents the pixel identifier, k represents the image identifier, Represents the gray value of the p-th pixel in the k-th image; It represents the ratio of the exposure used for the k-th image and the k+1-th image.

11. An image photometric calibration device, comprising: An image information acquisition module, configured to acquire grayscale values ​​of pixels in an image captured by an imaging device to be calibrated and corresponding reference information; A model building module, which builds an inverse function of the response function of the imaging device based on the generalized exponential gamma model and builds a corresponding loss function; A parameter determination module is configured to substitute the obtained pixel grayscale value and corresponding reference information into the loss function to obtain the parameters of the generalized exponential gamma model when the loss function is minimized, and then determine the inverse function of the response function of the imaging device; The image correction module is configured to map the grayscale value of the image to be corrected taken by the imaging device using the inverse function of the response function to obtain a corrected image.

12. An image photometric calibration device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the image photometric calibration method according to any one of claims 1 to 10 based on instructions stored in the memory.

13. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the image photometric calibration method according to any one of claims 1 to 10 is implemented.

Images