Non-uniformity correction method based on unsupervised machine learning

Abstract

The invention discloses a non-uniformity correction method based on unsupervised machine learning. The non-uniformity correction method comprises the steps that system working points are calibrated at different blackbody temperatures; establishing a unified calibration database; building an unsupervised learning network based on clustering and carrying out regression training; and deploying trained network parameters, and performing energy domain non-uniformity correction on an actual scene image. On the basis of a deduced infrared physical model, a unified database including black body gray values, integral time and attenuation slices is established, and under the condition that no black body theoretical radiation quantity label exists, an unsupervised learning method based on clustering is utilized, the average prediction energy domain value of the same black body temperature is used as a learning target, and the black body temperature is predicted. Correlation among calibration data of different working points is fully mined through network training, so that various coefficients of the calibration model are obtained; according to the invention, uniform and stable output is still maintained after the integration time and the attenuation slice are adjusted.

Description

Technical Field

[0001] The invention belongs to the field of infrared image processing, and in particular relates to a non-uniformity correction method based on unsupervised machine learning. Background Art

[0002] Infrared thermal imaging technology is widely used in night vision detection, autonomous driving, medical research, and power fault diagnosis. Compared with visible light imaging systems, images captured by infrared systems usually have significantly more noise, which is caused by theoretical limitations, detector manufacturing technology, and the inherent characteristics of infrared imaging. A key factor affecting the quality of infrared image processing is the non-uniformity of the detector pixel response, which is expressed as fixed pattern noise (FPN). Fixed pattern noise usually manifests itself in the image as noise characteristics such as stripes, block patterns, and gentle light and dark changes, which has a great impact on the quality of infrared imaging.

[0003] Commonly used correction methods based on blackbody calibration include one-point correction, two-point correction and multi-point correction algorithms. In addition, in order to broaden the detection range of the system, it is usually achieved by adjusting the integration time and adding attenuation films. When the system integration time or the transmittance of the attenuation film changes, the grayscale of the image will also change when the system integration time or the transmittance of the attenuation film changes, which is not conducive to the back-end image processing. At present, there are few studies on calibration-based non-uniformity correction algorithms that consider the combined influence of integration time and attenuation film. At the same time, adjusting the system integration time and adding attenuation films will introduce additional non-uniformity. A more effective way to solve the above problems is to convert the grayscale domain correction into energy domain correction, but the energy domain-based non-uniformity correction method usually requires precise knowledge of the blackbody temperature, emissivity and working band to calculate the theoretical radiation.

[0004] Conventional grayscale-based non-uniformity correction methods have the following disadvantages:

[0005] 1. Fixed attenuation sheet and integration time gear: Limited by factors such as storage capacity and calibration workload, the system can only store a limited number of attenuation sheet and integration time combination working points. It is impossible to dynamically select the best measurement working point during use, making it difficult to give full play to the performance of the infrared radiation measurement system;

[0006] 2. Poor correction effect: The parameter calculation of the conventional method is grouped and performed independently according to the attenuation sheet and integration time. The correlation between the data of different calibration working points is ignored. The calibration accuracy is easily affected by the random error of the system and reduced, resulting in poor non-uniformity correction effect when the working point is switched;

[0007] 3. When the system integration time or the transmittance of the attenuation film changes, the grayscale of the image will also change, which is not conducive to back-end image processing.

[0008] However, conventional non-uniformity correction methods based on energy domain have the following disadvantages:

[0009] The non-uniformity correction method based on the energy domain usually requires accurate knowledge of the black body temperature, emissivity and working band to calculate the theoretical radiation. The deviation of the black body temperature, emissivity and working band from the true value will affect the calibration accuracy and thus the correction effect. Summary of the invention

[0010] In order to solve the above technical problems, the present invention provides a non-uniformity correction method based on unsupervised machine learning. Based on the derived infrared physical model, a unified database including infrared image grayscale, integration time and attenuation sheet information is established. The clustering-based unsupervised learning method is adopted, and the average value of the predicted radiance at the same blackbody temperature is used as the learning target to learn the correction coefficient of the model. The present invention does not need to know the actual blackbody temperature, emissivity and working band of the calibration point. The trained correction network can maintain uniform and stable image output after adjusting the integration time and attenuation sheet, which is beneficial to the post-processing of the infrared system.

[0011] Based on the derived infrared physical model, the present invention establishes a unified database including blackbody grayscale value, integration time and attenuation film. In the absence of blackbody theoretical radiation label, the present invention uses a clustering-based unsupervised learning method, takes the average predicted energy domain value of the same blackbody temperature as the learning target, and fully mines the correlation between calibration data of different working points through network training, so as to obtain the coefficients of the correction model.

[0012] In order to achieve the above object, the technical solution adopted by the present invention is:

[0013] A non-uniformity correction method based on unsupervised machine learning comprises the following steps:

[0014] Step 1: Calibrate the system operating point at different blackbody temperatures;

[0015] Step 2: Establish a unified calibration database;

[0016] Step 3: Build a clustering-based unsupervised learning network and perform regression training;

[0017] Step 4: Deploy the trained network parameters to perform energy domain non-uniformity correction on the actual scene image.

[0018] Furthermore, the step 1 comprises:

[0019] Calibration is performed at multiple attenuation plates, integration times, and blackbody temperature points in the system to obtain multiple frames of raw data of image grayscale response as the attenuation plates, integration time, and blackbody temperature change.

[0020] Furthermore, the step 2 comprises:

[0021] Preprocess the multiple frames of raw data in step 1, including multi-frame average denoising and pixel averaging or replacement around blind pixels.

[0022] Furthermore, the step 3 comprises:

[0023] Establish a forward neural network for regression and set various parameters of the network, including: input layer, regression layer, output layer, loss function, error reverse transfer function, activation function, learning rate, and number of iterations;

[0024] The preprocessed normalized grayscale response data, normalized detector integration time and attenuation plate enable signal obtained in step 2 are used as the input of the regression neural network; the average value of the predicted energy domain value at the same blackbody temperature is used as the expected value of the regression neural network.

[0025] Furthermore, the average value of the predicted energy domain at the same blackbody temperature is shown in equation (1):

[0026] (1)

[0027] in, is the average value of the predicted energy domain under the same blackbody temperature, [i,j] is the pixel coordinate, W is the number of pixels in the detector width, H is the number of pixels in the detector height, and M is the number of calibration points under a certain blackbody temperature;

[0028] Establish a unified mathematical regression equation (2):

[0029] (2)

[0030] in, is the normalized mean prediction value term, is the normalized pixel gray value, is the normalized integration time, is the overall gain regression coefficient of the system, is the regression coefficient of the system detector bias term, is the optical system stray radiation bias regression coefficient, is the kth attenuator radiation bias regression coefficient, is the regression coefficient of the transmittance term of the kth attenuator.

[0031] Furthermore, the relative mean square error loss function and its partial derivative are established according to equation (5):

[0032] (5)

[0033] in, is the loss function value, is the relative error reverse transfer function of the loss function, which is the partial differential of the loss function with respect to the radiation of the pixel at coordinate [i, j]. is the predicted regression value corresponding to the nth calibrated blackbody image at the pixel coordinate [i, j], is the average value of the predicted energy domain under the same blackbody temperature, N is the number of radiation calibration points, W is the number of pixels in the detector width, and H is the number of pixels in the detector height. The reciprocal of the coefficient in the denominator of equation (2) is regarded as an overall term. The error reverse transfer function is shown in equation (6):

[0034] (6)

[0035] in, is the overall gain of the system at the pixel with coordinate [i,j], Input selection signal for the kth attenuator, is the gray value of the pixel at the coordinate [i,j] pixel, is the transmittance of the kth attenuator, is the integration time, is the system detector bias at the pixel with coordinate [i,j], is the partial derivative of the mean square error loss function, is the stray radiation of the optical system at the pixel with coordinate [i, j], W is the number of pixels in the width of the detector, H is the number of pixels in the height of the detector, is the radiation of the kth attenuation sheet, yes right The partial differential of yes right The partial differential of yes right The partial differential of is the kth attenuator right The partial differential of is the kth attenuator right The partial differential of

[0036] The iterative changes of each coefficient are shown in equation (7):

[0037] (7)

[0038] in, is the overall system gain, is the transmittance of the kth attenuator, is the system detector bias, is the stray radiation of the optical system, is the radiation of the kth attenuation sheet, yes right The partial differential of yes right The partial differential of yes right The partial differential of is the kth attenuator right The partial differential of is the kth attenuator right The partial differential of , , , , are the learning rates of each coefficient respectively.

[0039] Furthermore, the training methods include:

[0040] Import the calibration data of the first attenuator in the training library into the learning network model, and only iterate the learning coefficients , , ; When the value of the loss function changes slowly, that is, less than 10 -4 , add the second decayed data to the learning network model; at the same time, , , The learning rate is reduced to 10 -6 , only iteratively learn the coefficients , , , ; When the value of the loss function changes slowly, the recovery coefficient , , The learning rate is ; finally, the network is trained to obtain globally optimized non-uniformity correction network parameters that meet the accuracy requirements.

[0041] Furthermore, the hardware platform for training the network includes a central processing unit, a graphics processing unit, an advanced reduced instruction set computer architecture or an application-specific integrated circuit.

[0042] Furthermore, the step 4 comprises:

[0043] The trained parameters are deployed in the non-uniformity correction network, and real infrared scene images are taken at different attenuation film positions and different integration times. The normalized original grayscale response, integration time and attenuation film information are used as network input to infer the average predicted value of the pixel energy domain.

[0044] The beneficial effects of the present invention compared with the prior art are:

[0045] (1) It is not necessary to know the black body temperature, emissivity and working band to calculate the theoretical radiation. The calibration accuracy will not be affected by the deviation of the black body temperature, emissivity and band from the actual value, thereby affecting the correction effect;

[0046] (2) The system working point is continuously adjustable: The trained correction network has good generalization within the system working range and can perform non-uniformity correction on actual infrared images without calibrated working point settings within the working range. This overcomes the shortcomings of the previous attenuation plate and integration time gear being fixed and the large adjustment span, making the adjustment of the working point parameters more delicate and flexible, and improving the infrared system's ability to correct non-uniformity for targets with a large dynamic range.

[0047] (3) The corrected image will not experience grayscale jumps due to changes in the integration time or attenuation film. It can maintain a uniform and stable image output when the working point changes, which is beneficial to the post-processing of the infrared system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of unsupervised machine learning model;

[0049] Figure 2 Training curve for loss function;

[0050] Figure 3 It is the fitting result and relative error diagram of the working point;

[0051] Figure 4 The corrected images and their grayscale histograms of the two-point correction algorithm at different integration times; among them, (a) is the corrected image of the two-point correction algorithm when the integration time is 1920us, (b) is the corrected image of the two-point correction algorithm when the integration time is 2240us, (c) is the corrected image of the two-point correction algorithm when the integration time is 2560us, (d) is the corresponding histogram of the corrected image of the two-point correction algorithm when the integration time is 1920us, (e) is the corresponding histogram of the corrected image of the two-point correction algorithm when the integration time is 2240us, and (f) is the corresponding histogram of the corrected image of the two-point correction algorithm when the integration time is 2560us;

[0052] Figure 5The corrected images and their histograms of the algorithm proposed in the present invention at different integration times; wherein, (a) is the corrected image of the algorithm proposed in the present invention when the integration time is 1920us, (b) is the corrected image of the algorithm proposed in the present invention when the integration time is 2240us, (c) is the corrected image of the algorithm proposed in the present invention when the integration time is 2560us, (d) is the corresponding histogram of the corrected image of the algorithm proposed in the present invention when the integration time is 1920us, (e) is the corresponding histogram of the corrected image of the algorithm proposed in the present invention when the integration time is 2240us, and (f) is the corresponding histogram of the corrected image of the algorithm proposed in the present invention when the integration time is 2560us;

[0053] Figure 6 The corrected images and their grayscale histograms of the two-point correction algorithm at different attenuation plate positions; (a) is the corrected image of the two-point correction algorithm when the attenuation plate transmittance is 1, (b) is the corrected image of the two-point correction algorithm when the attenuation plate transmittance is 0.88, (c) is the corresponding histogram of the corrected image of the two-point correction algorithm when the attenuation plate transmittance is 1, and (d) is the corresponding histogram of the corrected image of the two-point correction algorithm when the attenuation plate transmittance is 0.88;

[0054] Figure 7 The figures are the corrected images and histograms of the algorithm proposed in the present invention at different attenuation sheet positions; wherein (a) is the corrected image of the algorithm proposed in the present invention when the attenuation sheet transmittance is 1, (b) is the corrected image of the algorithm proposed in the present invention when the attenuation sheet transmittance is 0.88, (c) is the corresponding histogram of the corrected image of the algorithm proposed in the present invention when the attenuation sheet transmittance is 1, and (d) is the corresponding histogram of the corrected image of the algorithm proposed in the present invention when the attenuation sheet transmittance is 0.88;

[0055] Figure 8 The present invention provides images corrected by the proposed method at the calibrated working point and the uncalibrated working point and their corresponding histograms; wherein, (a) is the image corrected by the algorithm proposed by the present invention when the integration time is 1472us, (b) is the image corrected by the algorithm proposed by the present invention when the integration time is 1920us, (c) is the image corrected by the algorithm proposed by the present invention when the integration time is 2368us, (d) is the corresponding histogram of the image corrected by the algorithm proposed by the present invention when the integration time is 1472us, (e) is the corresponding histogram of the image corrected by the algorithm proposed by the present invention when the integration time is 1920us, and (f) is the corresponding histogram of the image corrected by the algorithm proposed by the present invention when the integration time is 2368us. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0057] like Figure 1 As shown, a non-uniformity correction method based on unsupervised machine learning in an embodiment of the present invention includes the following steps:

[0058] Step 1, calibrating the working point of the medium-wave infrared system at different blackbody temperatures, including: calibrating at multiple attenuation plates, integration times, and blackbody temperature points of the system, and obtaining multiple frames of raw data of image grayscale response changing with attenuation plates and integration times;

[0059] Step 2: Establish a unified calibration database, including: preprocessing multiple frames of raw data at each working point by averaging and denoising, averaging or replacing pixels around blind pixels, etc.; using the preprocessed normalized grayscale response data, normalized detector integration time, and attenuation plate enable signal as inputs to the recurrent neural network; and calculating the average value of the predicted energy domain value at the same blackbody temperature, including:

[0060] The average predicted value is calculated according to equation (1) based on the initial values ​​of the system parameters:

[0061] (1)

[0062] in, is the average value of the predicted energy domain under the same blackbody temperature, [i,j] is the pixel coordinate, W is the number of pixels in the detector width, H is the number of pixels in the detector height, M is the number of calibration points under a certain blackbody temperature, is the average predicted value, and its expression is shown in equation (2):

[0063] (2)

[0064] in, is the pixel gray value, is the integration time, is the overall system gain, is the system detector bias, is the stray radiation of the optical system, is the radiation of the kth attenuation sheet, is the transmittance of the kth attenuator.

[0065] The average value of the predicted energy domain value at the same blackbody temperature is normalized and used as the expected value of the regression neural network;

[0066] Step 3: Build a clustering-based unsupervised learning network and perform regression training, including:

[0067] Step 3.1, establish a forward neural network for regression, and set various parameters of the neural network, including: input layer, regression layer, output layer, loss function, error reverse transfer function, activation function, learning rate, number of iterations, etc.;

[0068] Step 3.2: The pre-processed grayscale response data, detector integration time and attenuation sheet information obtained in step 2 are used as the input of the recurrent neural network. To facilitate the training of the network and speed up the training, the input is normalized according to equation (3):

[0069] (3)

[0070] in, is the grayscale response value of the pixel with coordinates [i, j], and for The minimum and maximum values ​​of is the normalized grayscale response value, is the detector integration time, and for The minimum and maximum values ​​of is the normalized integration time, Input the selection signal for the kth attenuator: the current attenuator is 1, and the rest are zero; X ​​is the input data matrix of the recurrent neural network.

[0071] Step 3.3, taking the average value of the predicted energy domain value at the same blackbody temperature obtained in step 2 as the expected value of the regression neural network;

[0072] Step 3.4: Establish the regression equation and its error reverse transfer function, including:

[0073] The mathematical regression equation was established according to equation (2);

[0074] According to equation (5), the relative mean square error loss function and its partial derivative are established:

[0075] (5)

[0076] in, is the loss function value, is the partial derivative of the loss function, is the predicted regression value corresponding to the nth calibrated blackbody image at the pixel coordinate [i, j], is the average value of the predicted energy domain under the same blackbody temperature, N is the number of radiation calibration points, W is the number of pixels in the detector width, and H is the number of pixels in the detector height.

[0077] The error back transfer function is the partial differential of the regression equation with respect to each regression parameter, as shown in equation (6):

[0078] (6)

[0079] in, is the overall gain of the system at the pixel with coordinate [i,j], Input selection signal for the kth attenuator, is the gray value of the pixel at the coordinate [i,j] pixel, is the transmittance of the kth attenuator, is the integration time, is the system detector bias at the pixel with coordinate [i,j], is the partial derivative of the mean square error loss function, is the stray radiation of the optical system at the pixel with coordinate [i, j], W is the number of pixels in the width of the detector, H is the number of pixels in the height of the detector, is the radiation of the kth attenuation sheet, yes right The partial differential of yes right The partial differential of yes right The partial differential of is the kth attenuator right The partial differential of is the kth attenuator right The partial differential of . Then the change of each parameter in each iteration is shown by equation (7):

[0080] (7)

[0081] in, yes right The partial differential of yes right The partial differential of yes right The partial differential of is the kth attenuator right The partial differential of is the kth attenuator right The partial differential of . , , , , are the learning rates of each coefficient respectively.

[0082] Step 3.5: Set the nonlinear activation function, including:

[0083] The role of the nonlinear activation function is to introduce nonlinear factors and improve the regression ability of the network. Since equation (2) already has a certain nonlinear regression ability, the nonlinear activation function is not necessary. It can be selected based on the detector response characteristics and the regression effect of equation (2). In order to maintain the continuity of the regression network and avoid singularities, the activation function should select a monotonic, continuous nonlinear function. Nonlinear activation functions include but are not limited to equations (8) and (9):

[0084] (8)

[0085] (9)

[0086] Among them, sig represents the output signal and x represents the input signal.

[0087] Step 3.6: Train the network to obtain network parameters that meet the accuracy requirements, including:

[0088] During training, data of the entire image size is input together, and each pixel parameter is learned in parallel during the training process to shorten the training time. Training is stopped after the expected accuracy is achieved. The hardware platform for training the network includes but is not limited to the central processing unit (CPU), graphics processing unit (GPU), advanced reduced instruction set computer architecture (ARM), and application-specific integrated circuit (ASIC).

[0089] Step 4: Deploy network parameters and capture actual scene images, including:

[0090] The trained parameters are deployed in the non-uniformity correction network. The normalized original grayscale response, integration time and attenuation sheet information are used as inputs. Equation (10) is used to infer the average predicted value of the pixel energy domain.

[0091] (10)

[0092] in, is the predicted value in the energy domain, For the trained regression network, is the sum of equation (2) By regression network Perform inference calculations;

[0093] Example:

[0094] A non-uniformity correction method based on unsupervised machine learning in an embodiment of the present invention includes:

[0095] Step 1: Calibrate at multiple attenuation plates, integration times, and blackbody temperature points in the system to obtain multiple frames of raw data of image grayscale response as the attenuation plates, integration times, and blackbody temperatures change.

[0096] Step 2: Perform average denoising and blind pixel replacement preprocessing on multiple frames of raw data at the calibration working point.

[0097] Step 3: Figure 2 , Figure 3 As shown, build an unsupervised learning network based on clustering and perform regression training:

[0098] Step 3.1: Build a feedforward neural network for regression. The initial value of the parameter learning network is set as follows:

[0099]

[0100] in, is the number of pixels of the detector width, is the number of pixels at the detector height, is the input matrix dimension, is the network learning rate, is the momentum parameter, is the output matrix dimension.

[0101] Step 3.2, the pre-processed grayscale response data, detector integration time and attenuation sheet information obtained in step 2 are used as the input of the recurrent neural network. To facilitate the training of the network and speed up the training, the input is normalized according to equation (3);

[0102] The non-uniformity correction image display effect of this embodiment is as follows:

[0103] (1) The corrected image and its grayscale histogram of the two-point correction algorithm at different integration times are shown in Figure 4 As shown, the corrected image and its histogram of the algorithm proposed in the present invention at different integration times are shown in Figure 5 As shown, the results show that compared with the two-point correction method, the correction image obtained by the method of the present invention is more stable when the integration time is changed. Figure 4 (a) is the image corrected by the two-point correction algorithm when the integration time is 1920us. Figure 4(b) is the image corrected by the two-point correction algorithm when the integration time is 2240us. Figure 4 (c) is the image corrected by the two-point correction algorithm when the integration time is 2560us. Figure 4 (d) is the histogram corresponding to the image corrected by the two-point correction algorithm when the integration time is 1920us. Figure 4 (e) is the histogram corresponding to the image corrected by the two-point correction algorithm when the integration time is 2240us. Figure 4 (f) is the histogram corresponding to the image corrected by the two-point correction algorithm when the integration time is 2560us; Figure 5 (a) is the image corrected by the algorithm proposed by the present invention when the integration time is 1920us. Figure 5 (b) is the image corrected by the algorithm proposed by the present invention when the integration time is 2240us. Figure 5 (c) is the image corrected by the algorithm proposed by the present invention when the integration time is 2560us. Figure 5 (d) is the histogram corresponding to the image corrected by the algorithm proposed by the present invention when the integration time is 1920us. Figure 5 (e) is the histogram corresponding to the image corrected by the algorithm proposed by the present invention when the integration time is 2240us. Figure 5 (f) is the histogram corresponding to the image corrected by the algorithm proposed in the present invention when the integration time is 2560us.

[0104] (2) The corrected image and its grayscale histogram of the two-point correction algorithm under different attenuation plate transmittance are shown in Figure 6 As shown in the figure, the corrected image and histogram of the algorithm proposed in the present invention under different attenuation sheet transmittance are shown in Figure 7 As shown, the results show that compared with the two-point correction method, the correction image obtained by the present invention when changing the transmittance of the attenuation plate is more stable. Figure 6 (a) is the image corrected by the two-point correction algorithm when the transmittance of the attenuation plate is 1. Figure 6 (b) is the image corrected by the two-point correction algorithm when the transmittance of the attenuation plate is 0.88. Figure 6 (c) is the histogram corresponding to the image corrected by the two-point correction algorithm when the transmittance of the attenuation plate is 1. Figure 6 (d) is the histogram corresponding to the image corrected by the two-point correction algorithm when the transmittance of the attenuation plate is 0.88; Figure 7 (a) is the image corrected by the algorithm proposed by the present invention when the transmittance of the attenuation plate is 1. Figure 7 (b) is the image corrected by the algorithm proposed by the present invention when the transmittance of the attenuation plate is 0.88. Figure 7 (c) is the corresponding histogram of the image corrected by the algorithm proposed in the present invention when the transmittance of the attenuation plate is 1. Figure 7 (d) is the corresponding histogram of the image corrected by the algorithm proposed in the present invention when the transmittance of the attenuation plate is 0.88.

[0105] (3) The images corrected by the proposed method at the calibrated working point and the uncalibrated working point and their corresponding histograms are shown in Figure 2. Figure 8 As shown, 1920us is the calibration point, 1472us and 2368us are uncalibrated points. The results show that both the calibrated working point and the uncalibrated working point of the method proposed in the present invention can maintain good correction and display effects. Figure 8 (a) is the image corrected by the algorithm proposed by the present invention when the integration time is 1472us. Figure 8 (b) is the image corrected by the algorithm proposed by the present invention when the integration time is 1920us. Figure 8 (c) is the image corrected by the algorithm proposed by the present invention when the integration time is 2368us. Figure 8 (d) is the histogram corresponding to the image corrected by the algorithm proposed by the present invention when the integration time is 1472us. Figure 8 (e) is the histogram corresponding to the image corrected by the algorithm proposed by the present invention when the integration time is 1920us. Figure 8 (f) is the histogram corresponding to the image corrected by the algorithm proposed in the present invention when the integration time is 2368us.

[0106] A large number of specific details are described in the description provided herein. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0107] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is primarily selected for readability and instructional purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention.

Claims

1. A non-uniformity correction method based on unsupervised machine learning, characterized in that: The steps include: Step 1: Calibrate the system operating point at different blackbody temperatures; Step 2: Establish a unified calibration database; Step 3: Build a clustering-based unsupervised learning network and perform regression training; Step 4: Deploy the trained network parameters to perform energy domain non-uniformity correction on the actual scene image.

2. The non-uniformity correction method based on unsupervised machine learning as claimed in claim 1, characterized in that: The step 1 comprises: Calibration is performed at multiple attenuation plates, integration times, and blackbody temperature points in the system to obtain multiple frames of raw data of image grayscale response as the attenuation plates, integration time, and blackbody temperature change.

3. The non-uniformity correction method based on unsupervised machine learning as claimed in claim 2, characterized in that: The step 2 comprises: Preprocess the multiple frames of raw data in step 1, including multi-frame average denoising and pixel averaging or replacement around blind pixels.

4. The non-uniformity correction method based on unsupervised machine learning as claimed in claim 2, characterized in that: The step 3 comprises: Establish a forward neural network for regression and set various parameters of the network, including: input layer, regression layer, output layer, loss function, error reverse transfer function, activation function, learning rate, and number of iterations; The preprocessed normalized grayscale response data, normalized detector integration time and attenuation plate enable signal obtained in step 2 are used as the input of the regression neural network; the average value of the predicted energy domain value at the same blackbody temperature is used as the expected value of the regression neural network.

5. The non-uniformity correction method based on unsupervised machine learning as claimed in claim 4, characterized in that: The average value of the predicted energy domain at the same blackbody temperature is given by equation (1): （1） in, is the average value of the predicted energy domain under the same blackbody temperature, [i,j] is the pixel coordinate, W is the number of pixels in the detector width, H is the number of pixels in the detector height, and M is the number of calibration points under a certain blackbody temperature; Establish a unified mathematical regression equation (2): （2） in, is the normalized mean prediction value term, is the normalized pixel gray value, is the normalized integration time, is the overall gain regression coefficient of the system, is the regression coefficient of the system detector bias term, is the optical system stray radiation bias regression coefficient, is the kth attenuator radiation bias regression coefficient, is the regression coefficient of the transmittance term of the kth attenuator.

6. The non-uniformity correction method based on unsupervised machine learning according to claim 5, characterized in that: According to equation (5), the relative mean square error loss function and its partial derivative are established: （5） in, is the loss function value, is the relative error reverse transfer function of the loss function, which is the partial differential of the loss function with respect to the radiation of the pixel at coordinate [i, j]. is the predicted regression value corresponding to the nth calibrated blackbody image at the pixel coordinate [i, j], is the average value of the predicted energy domain under the same blackbody temperature, N is the number of radiation calibration points, W is the number of pixels in the detector width, and H is the number of pixels in the detector height. The reciprocal of the coefficient in the denominator of equation (2) is regarded as an overall term. The error reverse transfer function is shown in equation (6): （6） in, is the overall gain of the system at the pixel with coordinate [i,j], Input selection signal for the kth attenuator, is the gray value of the pixel at the coordinate [i,j] pixel, is the transmittance of the kth attenuator, is the integration time, is the system detector bias at the pixel with coordinate [i,j], is the partial derivative of the mean square error loss function, is the stray radiation of the optical system at the pixel with coordinate [i, j], W is the number of pixels in the width of the detector, H is the number of pixels in the height of the detector, is the radiation of the kth attenuation sheet, yes right The partial differential of yes right The partial differential of yes right The partial differential of is the kth attenuator right The partial differential of is the kth attenuator right The partial differential of The iterative changes of each coefficient are shown in equation (7): （7） in, is the overall system gain, is the transmittance of the kth attenuator, is the system detector bias, is the stray radiation of the optical system, is the radiation of the kth attenuation sheet, yes right The partial differential of yes right The partial differential of yes right The partial differential of is the kth attenuator right The partial differential of is the kth attenuator right The partial differential of , , , , are the learning rates of each coefficient respectively.

7. The non-uniformity correction method based on unsupervised machine learning according to claim 6, characterized in that: Training methods include: Import the calibration data of the first attenuator in the training library into the learning network model, and only iterate the learning coefficients , , ; When the value of the loss function changes slowly, that is, less than 10 -4 , add the second decayed data to the learning network model; at the same time, , , The learning rate is reduced to 10 -6 , only iteratively learn the coefficients , , , ; When the value of the loss function changes slowly, the recovery coefficient , , The learning rate is ; finally, the network is trained to obtain globally optimized non-uniformity correction network parameters that meet the accuracy requirements.

8. The non-uniformity correction method based on unsupervised machine learning as claimed in claim 1, characterized in that: The hardware platforms for training networks include central processing units, graphics processing units, advanced reduced instruction set computer architectures, or application-specific integrated circuits.

9. The non-uniformity correction method based on unsupervised machine learning according to claim 1, characterized in that: The step 4 comprises: The trained parameters are deployed in the non-uniformity correction network, and real infrared scene images are taken at different attenuation film positions and different integration times. The normalized original grayscale response, integration time and attenuation film information are used as network input to infer the average predicted value of the pixel energy domain.

Images