A two-stage enhanced camera response calibration method

Abstract

The application discloses a two-stage enhanced camera response calibration method, which comprises the following steps: fixed pattern noise filtering on a to-be-calibrated image; collecting response images of an enhanced camera under continuous linear enhanced input light intensity conditions; calculating the change relationship between the difference between the actual output response of a pixel and the standard response of the pixel under different input light intensity conditions and the input light intensity based on the response images under different input light intensity conditions; performing primary response error compensation on the to-be-calibrated image after noise filtering based on the change relationship between the difference between the actual output response of a pixel and the standard response of the pixel under different input light intensity conditions and the input light intensity; and performing secondary response error compensation on the to-be-calibrated image after the primary response error compensation based on a pre-trained machine learning algorithm to obtain the final calibration result corresponding to the to-be-calibrated image. The application can significantly improve the sensitivity, accuracy and linearity of the response of an enhanced imaging system to an input signal.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of radiation measurement and diagnosis, and particularly relates to a two-stage enhanced camera response calibration method. BACKGROUND

[0002] In recent years, the enhanced imaging sensor technology has developed rapidly, and the related camera systems are widely used in low-light night vision, medical radiation, industrial non-destructive testing, ultrafast X-ray imaging and other fields. Common enhanced imaging sensors include EMCCD (Electron-Multiplying CCD), ICCD, ICMOS, SCMOS (Scientific CMOS) and the like. Compared with traditional charge coupled devices (CCD) or complementary metal oxide semiconductor (CMOS) cameras, the enhanced imaging sensor adopts more optical devices in the optical design, such as optical fiber light cone, image intensifier and the like, so that the system is more complex and the calibration is more difficult; at the same time, since the enhanced imaging sensor is often used for measurement and diagnosis of radiation physical quantities, the sensitivity, accuracy and linearity of the response capability of the related camera are also required to be high.

[0003] Some related schemes have been proposed at home and abroad around the measurement and diagnosis of the enhanced camera. For example, the patent application with the application number CN201611270441.9 proposes a system and method for imaging measurement of transient temperature field, and the main problem of this method is that it does not model and analyze the response calibration problem of the imaging sensor. For another example, the patent application with the application number CN201511021016.1 proposes a method for self-adaptive gain adjustment of the image intensifier according to the image gray scale, which can significantly improve the adaptability of the imaging system to complex environments. However, the deficiency of this method is that the imaging system adopts a global calculation strategy for the gain adjustment of the micro channel plate (MCP), and the related adjustment method is not suitable for the system application in the measurement and diagnosis level.

[0004] From the above analysis, it can be seen that in the field of enhanced imaging sensor response calibration, the related technology still has the following obvious deficiencies: on the one hand, the current enhanced imaging sensor response calibration research is relatively less, resulting in relatively low sensitivity, accuracy and linearity of the output performance of the enhanced imaging sensor. On the other hand, in the field of enhanced imaging sensor calibration, most of the existing calibration methods belong to a kind of overall calibration method, that is, the method calculates the entire image / image block as input, and the calculation of the single pixel calibration result will refer to the size of the pixel response result around the pixel, so the calibration result has poor interpretability, high uncertainty, and is not suitable for measurement and diagnosis application occasions. SUMMARY

[0005] The present application provides a two-stage enhanced camera response calibration method to solve the technical problems of poor interpretability, high uncertainty and unsuitability for measurement and diagnosis application occasions of the existing calibration results.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] A two-stage enhanced camera response calibration method, comprising:

[0008] Fixed pattern noise filtering of the to-be-calibrated image;

[0009] Acquiring a response image of the enhanced camera under the condition of continuous linear enhanced input light intensity;

[0010] Based on the response images under different input light intensity conditions, the change relationship of the difference value between the pixel actual output response and the pixel standard response under different input light intensity conditions with the input light intensity is calculated;

[0011] Based on the change relationship of the difference value between the pixel actual output response and the pixel standard response under different input light intensity conditions with the input light intensity, the to-be-calibrated image after noise filtering is subjected to first-order response error compensation;

[0012] Based on the pre-trained machine learning algorithm, the to-be-calibrated image after first-order response error compensation is subjected to second-order response error compensation to obtain the final calibration result corresponding to the to-be-calibrated image.

[0013] Further, the fixed pattern noise filtering of the to-be-calibrated image comprises:

[0014] Acquiring a plurality of response images of the enhanced camera under the condition of no input light intensity;

[0015] For the acquired plurality of response images under the condition of no input light intensity, the mean value of the pixel output intensity under the condition of no input light intensity is calculated;

[0016] determine a fixed pattern noise estimation value based on the mean of the pixel output intensity under the condition of no input light intensity;

[0017] subtract the fixed pattern noise estimation value at the corresponding pixel position from the pixel output intensity at each pixel position in the image to be calibrated to achieve fixed pattern noise filtering of the image to be calibrated.

[0018] Further, the determination of the fixed pattern noise estimation value based on the mean of the pixel output intensity under the condition of no input light intensity comprises:

[0019] using a preset modeling algorithm, performing weighted calculation on the calculated mean of the pixel output intensity under the condition of no input light intensity according to different input light intensities to obtain the fixed pattern noise estimation value under different input light intensities;

[0020] When the fixed pattern noise estimation value at the corresponding pixel position is subtracted from the pixel output intensity at each pixel position in the image to be calibrated to achieve fixed pattern noise filtering of the image to be calibrated, the fixed pattern noise estimation value to be subtracted is the fixed pattern noise estimation value corresponding to the input light intensity when the image to be calibrated is collected.

[0021] Further, when collecting the response image of the enhanced camera under the condition of continuous linear enhanced input light intensity, under a single input light intensity condition, multiple response images need to be repeatedly collected.

[0022] Further, the calculation of the change relationship of the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions based on the response images under different input light intensity conditions comprises:

[0023] for the response images under different input light intensity conditions, the mean of the pixel output intensity of multiple response images under each input light intensity condition is calculated in sequence;

[0024] using the least square method to calculate a fitting straight line of the mean of the pixel output intensity corresponding to different input light intensity conditions, and using the obtained fitting straight line to represent the relationship between the pixel standard response and the input light intensity;

[0025] the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions is calculated;

[0026] using a preset fitting algorithm to perform fitting calculation on the obtained difference to obtain the change relationship of the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions with the input light intensity.

[0027] Further, before the mean of the pixel output intensity of multiple response images under each input light intensity condition is calculated in sequence for the response images under different input light intensity conditions, the two-stage enhanced camera response calibration method further comprises:

[0028] The pixel response data collected under the same input light intensity condition is clustered by using a preset clustering algorithm to obtain a pixel response cluster center;

[0029] The distance between each pixel response measurement result and the corresponding cluster center is calculated;

[0030] If the distance between the pixel response measurement result and the corresponding cluster center is greater than a preset threshold, the corresponding pixel response measurement result is removed, and the abnormal response in the response image is removed.

[0031] Further, the preset clustering algorithm is a K-means clustering algorithm.

[0032] Further, based on the pre-trained machine learning algorithm, a second-level response error compensation is performed on the to-be-calibrated image after the first-level response error compensation to obtain the final calibration result corresponding to the to-be-calibrated image, including:

[0033] The preset machine learning algorithm is trained; wherein the input of the machine learning algorithm includes the performance and process parameters of a single separated optical device in the enhanced camera, and the performance and manufacturing process parameters between two adjacent optical devices; the supervised data of the machine learning algorithm is the second-level response error compensation;

[0034] The performance and process parameters of a single separated optical device in the enhanced camera corresponding to the to-be-calibrated image, and the performance and manufacturing process parameters between two adjacent optical devices are input into the trained machine learning algorithm, and the output result of the machine learning algorithm is used to perform the second-level response error compensation on the to-be-calibrated image.

[0035] Further, the performance and process parameters of the single separated optical device include modulation transfer function, spectral response capability, integral sensitivity level, brightness gain, equivalent background illumination, and signal-to-noise ratio.

[0036] Further, the performance and manufacturing process parameters between two adjacent optical devices include coupling efficiency, coupling signal-to-noise ratio, coupling resolution, coupling transmittance, coupling uniformity, coupling gain, and coupling linearity.

[0037] The technical solution provided by the present application has at least the following beneficial effects:

[0038] 1、The present application can significantly improve the response sensitivity of the enhanced camera. Sensitivity refers to the degree of response of the enhanced imaging sensor to the change of the unit to-be-measured light intensity. Obviously, by using the calibration method of the present application, the response ability of the related imaging sensor to the weak light intensity signal change can be significantly improved.

[0039] 2、The present application can significantly improve the response accuracy of the enhanced camera. The accuracy refers to the degree of conformity of the average value of multiple measurements under certain experimental conditions with the true value, which is used to represent the size of the error. Obviously, the camera response error can be further reduced by the calibration method of the present application, thereby improving the accuracy of measurement and diagnosis.

[0040] 3、The present application can significantly improve the response linearity of the ICCD, ICMOS and other enhanced cameras. The linearity refers to the degree of linearity of the camera output response result under the linear enhancement light source input condition of the enhanced imaging sensor. Obviously, by the calibration method of the present application, the ideal linear response straight line of the camera is estimated, and the camera response is corrected to the ideal response straight line, so the linearity of the camera response can be significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is the execution flow chart of the two-stage enhanced camera response calibration method provided by the embodiment of the present application;

[0043] Figure 2 is the standard response fitting result graph of the enhanced camera provided by the embodiment of the present application;

[0044] Figure 3 is the first-order error compensation polynomial fitting result graph of the enhanced camera provided by the embodiment of the present application;

[0045] Figure 4 is the first-order and second-order error compensation result graph of the enhanced camera provided by the embodiment of the present application;

[0046] Figure 5 is the result graph of the enhanced camera before and after the whole image response compensation provided by the embodiment of the present application; wherein (a) is the image before correction and the image three-dimensional projection graph, and (b) is the image after correction and the image three-dimensional projection graph. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0048] The present embodiment provides a two-stage enhanced camera response calibration method, which can be realized by an electronic device, and the execution flow of the method is as shown in Figure 1 .

[0049] Specifically, the calibration method of the embodiment includes the following steps:

[0050] S1, fixed pattern noise filtering is performed on the image to be calibrated;

[0051] It should be noted that the fixed pattern noise is a "built-in" noise mode of the enhanced camera, which is manifested as a series of parallel and perpendicular to the output image edge strips.

[0052] To eliminate the fixed pattern noise, the embodiment takes the following steps:

[0053] S11, a plurality of response images of the enhanced camera under the condition of no input light intensity are collected;

[0054] S12, the mean value of the pixel output intensity at the corresponding position of the collected image is calculated, and the formula is as follows:

[0055]

[0056] Where FPN(i,j) is the mean value of the pixel output intensity, N is the number of observations (the number of images collected under the condition of fixed input light intensity), and in the actual experiment, it is recommended that N is greater than 30 times; I_0 t (i,j) is the gray output response of the enhanced camera at the (i,j) pixel position under the condition of no input light intensity (the value is 0-255);

[0057] S13, the calculated mean value of the pixel output intensity is regarded as the estimated value of the fixed pattern noise, and in the actual output, the fixed pattern noise is eliminated by using the difference value calculation at the corresponding position, and the formula is as follows:

[0058]

[0059] Where, is the fixed pattern noise reduction result at the (i,j) pixel position, and I(i,j) is the actual response output result of the enhanced camera at the (i,j) pixel position.

[0060] Further, to improve the accuracy of the fixed pattern noise estimation, modeling methods such as multiplicative factor and neural network can be used, and according to different camera input light intensity, the fixed pattern noise estimated under the condition of no light intensity in the previous step is weighted calculated, as shown in formula (3). On the basis of the above calculation, the intensity of the fixed pattern noise estimated by the base is adjusted according to the change of the light intensity, and then the difference calculation similar to formula (2) is performed, as shown in formula (4), so that the filtering effect of the fixed pattern noise under different light intensity input conditions can be effectively improved. In the estimation of the weighted weight calculation, when the multiplicative factor is used to calculate the weight c kDuring the calculation, the relationship between different multiplicative factors and the image quality evaluation index of the fixed pattern noise reduced image can be established. The image quality evaluation index can be selected as the variance of the image pixels, noise estimation result, etc. The smaller the variance and noise estimation result, the better the noise reduction effect. By repeatedly adjusting the multiplicative factor, the corresponding image quality evaluation index calculation result is obtained. Finally, the fitting result of the minimum value is selected as the final determined multiplicative factor. When the neural network is used to predict the weight c k During the calculation, the BP network, deep learning network, etc. can be selected to predict the weight according to the noise estimation result or abstract features of the original image.

[0061] FPN'(i,j) = c k × FPN(i,j) (3)

[0062]

[0063] wherein, c k is the adjustment parameter obtained by using the multiplicative factor, neural network, etc. under the condition that the input brightness of the camera is k; FPN'(i,j) is the correction value of the fixed pattern noise under the condition that the input light intensity is k brightness.

[0064] S2, collect the response image of the enhanced camera under the condition of continuous linear enhancement of the input light intensity; wherein, when collecting the image, under the condition of a single input light intensity, a plurality of response images need to be repeatedly collected;

[0065] S3, based on the response images under different input light intensity conditions, the relationship between the difference between the actual output response of the pixel and the standard response of the pixel and the input light intensity under different input light intensity conditions is calculated;

[0066] It should be noted that, in order to maximize the preservation of original imaging information, the embodiment proposes a first-order error compensation method for the nonlinearity of the camera pixel response and the non-uniformity of the whole field response based on polynomial fitting. The steps of the first-order error compensation method are as follows:

[0067] S31, for the response images under different input light intensity conditions, the average pixel output intensity of each pixel position of a plurality of response images under each input light intensity condition is calculated in sequence; by repeatedly testing and calculating the response average, the average response result of the enhanced camera with the linear light intensity can be obtained, and the formula is as follows:

[0068]

[0069] wherein, I_S k (i,j) is the average actual pixel output intensity of the pixel (i,j) position of the enhanced camera when the input brightness is k, M is the number of observations, and Sk,M (i,j) is the actual output intensity of the pixel at position (i,j) when the environmental input brightness is k.

[0070] S32, a fitting straight line of the mean value of the pixel output intensity corresponding to different input light intensity conditions is calculated by using the least square method, and the obtained fitting straight line is taken as the ideal linear response of the enhanced camera, and the formula is as follows:

[0071] I_S k ′(i,j)=a(i,j)E_B k +b(i,j) (6)

[0072] wherein I_S k ′(i,j) is the standard response estimation value of the pixel (i,j) of the enhanced camera at position (i,j) when the input brightness is k, a(i,j), b(i,j) are the linear fitting parameters, E_B k is the kth input light intensity brightness.

[0073] S33, the difference value between the actual output response and the standard response under different input light intensity conditions is calculated; a plurality of difference value calculation results can be obtained by continuous calculation, and the formula is as follows:

[0074] D k (i,j)=I_S k ′(i,j)-S k (i,j) (7)

[0075] wherein D k (i,j) is the first-order compensation value of the pixel intensity at position (i,j) when the environmental input brightness is k;

[0076] S34, the obtained difference value is calculated by using a preset fitting algorithm, and the change relationship between the difference value between the actual output response and the standard response of the pixel under different input light intensity conditions and the input light intensity is obtained. The fitting algorithm can adopt neural network, exponential, linear, logarithmic, polynomial, quadratic function, high-order function and the like parameter fitting algorithm; specifically, the polynomial fitting is adopted for the change rule calculation of the difference value result in the embodiment, that is, the polynomial fitting mode is adopted to calculate the change relationship formula of the difference value with the input environmental light intensity which is satisfied by the difference value calculated in the previous step; the fitting result can be taken as the first-order compensation estimation value.

[0077]

[0078] wherein C0(i,j), C1(i,j), C2(i,j), …, C M (i,j) are polynomial coefficients, and N is the polynomial order, and preferably, N can be taken as 3.

[0079] Of course, it can be understood that the present embodiment does not limit the specific fitting algorithm.

[0080] In addition, in the above calculation process, due to the difference between the pixel response capabilities in the imaging system, some pixels may have abnormal response when collected in different light intensity environments. Therefore, the present embodiment proposes a calculation strategy for removing abnormal response by using K-means method, which is used to refine the data obtained in S2. The related calculation steps are as follows: 1) first, the pixel response data collected under the same light intensity condition is calculated by using K-means clustering method, preferably, only one cluster center is selected in the K-means clustering calculation in the present embodiment; obviously, the calculation result of the pixel response cluster center can be obtained by calculation; 2) calculate the distance between each pixel response measurement result and the corresponding cluster center; preferably, the distance calculation can use the calculation method of Euclidean distance; 3) judge the distance result, if the distance result is greater than a certain threshold, the pixel response measurement result is removed, that is, it is considered that the reliability of the pixel response measurement result is too low, and it cannot participate in the subsequent calculation of the standard response. Preferably, the distance threshold can be taken as 5.

[0081] S4, based on the change relationship between the difference between the actual output response of the pixel under different input light intensity conditions and the pixel standard response and the input light intensity, the noise filtered to-be-calibrated image is subjected to a first response error compensation;

[0082] S5, based on the trained machine learning algorithm, the to-be-calibrated image after the first response error compensation is subjected to a second response error compensation, and the final calibration result corresponding to the to-be-calibrated image is obtained.

[0083] It should be noted that, in order to further improve the accuracy of the enhanced camera response calibration, the embodiment proposes a secondary compensation of pixel response error by using a machine learning method, that is, a secondary compensation of the nonlinearity of the camera pixel response and the non-uniformity of the integral response. As shown in formula (9), the pixel response compensation calculation model considering the secondary error compensation proposed in the embodiment is shown in formula (9). In order to reflect that the pixel response error is derived from the optical device in the imaging system, the machine learning tool is used to estimate the secondary response compensation error of the camera in the embodiment: the input of the machine learning tool can include the related performance of a single separated optical device such as modulation transfer function (MTF), spectral response capability, integral sensitivity level, brightness gain, equivalent background illumination, signal-to-noise ratio, and manufacturing process parameters; or it can include the performance and manufacturing process parameters between two adjacent optical devices such as coupling efficiency, coupling signal-to-noise ratio, coupling resolution, coupling transmittance, coupling uniformity, coupling gain, and coupling linearity. The supervised data of the machine learning tool is the secondary error compensation amount of the camera. Preferably, the machine learning error compensation model can be selected as a support vector regression (SVR) model, which has the advantages of less training data and high training accuracy. Obviously, after the secondary error compensation, the estimation process of the compensation result reflects the performance and manufacturing process characteristics of the imaging sensor itself, and therefore it is more representative and can better explain and compensate the causes of the pixel response error.

[0084] I_O k ′=I_O k (i,j)+D k (i,j)+D_SVR k (i,j) (9)

[0085] wherein I_O′ k (i,j) is the final error compensation result; I_O k (i,j) is the actual output of the enhanced camera; D k (i,j) is the primary error compensation amount of the enhanced camera; D_SVR k (i,j) is the secondary error compensation amount of the enhanced camera.

[0086] Further, the embodiment proposes a calculation method of an improved SVR, i.e. an Improved Adaptive Genetic Algorithm-Support Vector Regression (IAGA-SVR) model. Here, an Improved Adaptive Genetic Algorithm (IAGA) is used to optimize and seek the optimal solution of the SVR calculation process, i.e. the IAGA is used to find the optimal parameter value of the SVR algorithm to construct a regression model. The Genetic Algorithm (GA) is a kind of optimization algorithm derived by imitating the biological evolution mechanism. Through imitating the operations of the chromosomes in the process of species reproduction and evolution, such as cross, mutation and selection, the problem solving space is abstracted into the genes carried by the chromosomes by using coding, and the chromosome gene composition is updated in the process of population evolution until the highest fitness of the offspring individual is obtained, which is also the optimal solution of the problem.

[0087] The core of the IAGA algorithm proposed in the embodiment is to update the cross rate and mutation rate by the evolution degree of the population particle, so that the population particle can still maintain high population diversity after one iteration after another, thereby achieving more extensive global search and detailed local search. This means that the IAGA algorithm can more likely escape from the predicament of early convergence, so that the final optimization result often has better results. The algorithm steps of the improved adaptive genetic algorithm IAGA are as follows:

[0088] 1) Initialize the population, generate enough initial sample population, and perform substantive chromosome coding according to the problem to be solved.

[0089] 2) Perform the selection process.

[0090] 3) Adaptive cross operation. According to the evolution of the particle population, the adaptive cross probability P c is generated according to formula (10) to determine whether to perform cross operation. This step can expand the sample size of the entire population and improve the diversity of the entire biological population, providing sufficient sample size for the subsequent adaptive mutation operation.

[0091]

[0092] Wherein, f is the fitness value; fmean represents the overall fitness average of the current population particles; f Max is the highest fitness value of the current population particles; P cmax is the maximum value of the cross rate P c .

[0093] 4) Adaptive mutation operation. According to the evolution of the current generation of the particle population, the adaptive mutation probability P m is generated according to formula (11) and formula (12), and it is judged whether the mutation operation is performed. If the mutation operation is performed, the simplest single-point mutation is adopted, so as to ensure that the population diversity is improved and the complexity of the IAGA algorithm is reduced.

[0094]

[0095]

[0096] wherein P mmax is the maximum value of the mutation rate P m ; NIC is a nonlinear correction coefficient; t is the iteration number; and T is the maximum iteration number.

[0097] 5) The fitness value of the current individual is recalculated according to the constituent genes carried by the newly generated offspring individual, and the previous global optimal extreme value is replaced according to the order of the high and low.

[0098] 6) Algorithm stop condition judgment. If the current output of the algorithm meets the use requirement, the algorithm is immediately stopped, and the best feasible solution of the problem is output; otherwise, the cycle is started again from 2).

[0099] To sum up, the embodiment provides a two-stage enhanced camera response calibration method, which can significantly improve the response capability of the related imaging sensor to weak light intensity signal changes, further reduce the camera response error, and further improve the accuracy of measurement and diagnosis, and can significantly improve the linearity of the camera response.

[0100] In addition, it should be noted that the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer usable storage media containing computer usable program code.

[0101] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a method for implementing the flows and / or blocks in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0102] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks. The computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks. The computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or multiple flows and / or blocks.

[0103] It is also important to note that while the above describes example embodiments, actually these teachings can be carried out in other ways than those specifically set forth herein without departing from the essence of the present teachings. For example, software can be implemented in a high level procedural language or object oriented programming methodology. Also, functions can be carried out in peripherals or other devices by way of embedded systems. Alternatively, one or more functions of the present teachings can be implemented by way of one or more computer programs operating on a computing device. Furthermore, the teachings could be implemented by way of other technologies as they become available.

[0104] Finally, it should be understood that the above description is intended for illustrating and not for limiting the application. While the present application has been described with reference to the preferred embodiments, it is understood that the application is not limited to the preferred embodiments. Changes can be made by those having ordinary skill in the art and equally fall within the spirit of the application. Therefore, the scope of the application should be determined not with reference to the above description but should instead be determined with reference to the appended claims along with their full scope of equivalents.

Claims

1. A two-stage enhanced camera response calibration method, characterized in that, The method comprises the following steps: fixed pattern noise filtering is performed on the to-be-calibrated image; a response image of the enhanced camera under a continuous linear enhanced input light intensity condition is collected; a change relationship between a difference between a pixel actual output response and a pixel standard response under different input light intensity conditions and the input light intensity is calculated based on the response images under different input light intensity conditions; first-level response error compensation is performed on the to-be-calibrated image after the noise filtering based on the change relationship between the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions and the input light intensity; second-level response error compensation is performed on the to-be-calibrated image after the first-level response error compensation based on a pre-trained machine learning algorithm, and a final calibration result corresponding to the to-be-calibrated image is obtained; when the response image of the enhanced camera under the continuous linear enhanced input light intensity condition is collected, multiple response images under a single input light intensity condition need to be repeatedly collected; the change relationship between the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions and the input light intensity is calculated based on the response images under different input light intensity conditions, which comprises the following steps: the pixel output intensity mean of the multiple response images under each input light intensity condition is calculated in sequence for the response images under different input light intensity conditions; a fitting straight line of the pixel output intensity mean corresponding to different input light intensity conditions is calculated by using the least square method, and the fitting straight line is used to represent the relationship between the pixel standard response and the input light intensity; the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions is calculated; the difference between the pixel actual output response and the pixel standard response under different input light intensity conditions and the input light intensity is calculated by using a preset fitting algorithm, and the change relationship is obtained; the second-level response error compensation is performed on the to-be-calibrated image after the first-level response error compensation based on a pre-trained machine learning algorithm, and a final calibration result corresponding to the to-be-calibrated image is obtained, which comprises the following steps: the preset machine learning algorithm is trained; wherein the input of the machine learning algorithm comprises the performance and process parameters of a single separated optical device in the enhanced camera and the performance and manufacturing process parameters between two adjacent optical devices; the supervision data of the machine learning algorithm is the second-level response error compensation; the performance and process parameters of a single separated optical device in the enhanced camera corresponding to the to-be-calibrated image and the performance and manufacturing process parameters between two adjacent optical devices are input into the trained machine learning algorithm, and the output result of the machine learning algorithm is used to perform the second-level response error compensation on the to-be-calibrated image.

2. The two-stage enhanced camera response calibration method of claim 1, wherein, The fixed pattern noise filtering performed on the to-be-calibrated image comprises the following steps: multiple response images of the enhanced camera under a no-input light intensity condition are collected; the pixel output intensity mean under the no-input light intensity condition is calculated for the collected multiple response images under the no-input light intensity condition; a fixed pattern noise estimation value is determined based on the pixel output intensity mean under the no-input light intensity condition; the pixel output intensity of each pixel position in the to-be-calibrated image is subtracted by the fixed pattern noise estimation value of the corresponding pixel position, so that the fixed pattern noise filtering of the to-be-calibrated image is realized.

3. The two-stage enhanced camera response calibration method of claim 2, wherein, Determine a fixed pattern noise estimation value based on the average pixel output intensity under the condition of no input light intensity, comprising: Using a preset modeling algorithm, weight the average pixel output intensity under the condition of no input light intensity according to different input light intensities to obtain the fixed pattern noise estimation value under different input light intensities. When the pixel output intensity at each pixel position in the image to be calibrated is subtracted by the fixed pattern noise estimation value at the corresponding pixel position to realize the fixed pattern noise filtering of the image to be calibrated, the subtracted fixed pattern noise estimation value is the fixed pattern noise estimation value corresponding to the input light intensity when the image to be calibrated is collected.

4. The two-stage enhanced camera response calibration method of claim 1, wherein, Before the average pixel output intensity of a plurality of response images under each input light intensity condition is calculated in sequence for the response images under different input light intensity conditions, the two-stage enhanced camera response calibration method further comprises: Using a preset clustering algorithm to cluster the pixel response data collected under the same input light intensity condition to obtain a pixel response clustering center; Calculate the distance between each pixel response measurement result and the corresponding clustering center; If the distance between the pixel response measurement result and the corresponding clustering center is greater than a preset threshold, the corresponding pixel response measurement result is removed to realize the removal of the abnormal response in the response image.

5. The two-stage enhanced camera response calibration method of claim 4, wherein, The preset clustering algorithm is a K-means clustering algorithm.

6. The two-stage enhanced camera response calibration method of claim 1, wherein, The performance and process parameters of the single separated optical device include modulation transfer function, spectral response capability, integral sensitivity level, brightness gain, equivalent background illumination, and signal-to-noise ratio.

7. The two-stage enhanced camera response calibration method of claim 6, wherein, The performance and process parameters between the two adjacent optical devices include coupling efficiency, coupling signal-to-noise ratio, coupling resolution, coupling transmittance, coupling uniformity, coupling gain, and coupling linearity.

Images