Processing method for dark current modeling and outlier detection based on gaussian mixture model

Abstract

The present application relates to the field of image processing. The processing method of dark current modeling and outlier detection based on Gaussian mixture model simulates the dark field condition of actual observation environment in the laboratory, and continuously shoots a group of photos at different temperatures according to the exposure time interval of the camera working in the actual observation environment; in each group of photos shot at different temperatures, the gray scale of each exposure time pixel is taken as a vector, and the characteristic vector of the pixel is modeled; the characteristic vectors of all pixels in the photos are clustered by using the Gaussian mixture model, and the Akaike information quantity BIC and the Bayesian information quantity AIC under the clustering are calculated. The present application compares the non-target part of the actually shot picture with the same part of the image shot in the simulated environment, obtains the temperature of the actual observation environment, and further obtains the dark current of the camera shot in the actual observation environment.

Description

Technical Field

[0001] This invention relates to the field of image processing. Background Technology

[0002] In image processing, the data acquired by an image sensor equipped with operational amplifiers and A / D circuits is typically processed. Due to the presence of dark current, images do not appear perfectly black even under complete darkness. The output level of the image sensor in a completely dark environment is called dark current. Dark current affects color reproduction, dynamic range, and sharpness of the image and needs to be corrected and eliminated.

[0003] The magnitude of dark current is linearly related to exposure time and exponentially related to temperature. Dark current doubles approximately every 6°C increase in temperature. However, in imaging equipment located in environments beyond human control, such as space, the deep sea, or polar regions, where replacement or repair is not possible in time, and where effective temperature control is lacking, we cannot accurately measure the temperature, making it impossible to estimate the magnitude of dark current. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to obtain the temperature and dark current of an imaging device in an environment where there is no intervention, in view of the shortcomings and inconveniences of the existing technology.

[0005] The technical solution adopted in this invention is a processing method for dark current modeling and anomaly detection based on Gaussian mixture models, which is carried out in the following steps.

[0006] S1. Under dark field conditions simulating the actual observation environment in the laboratory, take a series of photos at different temperatures according to the exposure time interval when the camera is working in the actual observation environment.

[0007] S2. In each group of photos taken at different temperatures, the grayscale of each pixel at each exposure time is extracted as a vector and used as the feature vector of that pixel for modeling.

[0008] S3. Cluster the feature vectors of all pixels in the photo using a Gaussian mixture model, and calculate the Akaike Information Content (BIC) and Bayesian Information Content (AIC) under this cluster.

[0009] The specific formula for AIC is:

[0010] AIC = 2k - 2*1n(L)

[0011] Where k is the number of parameters, L is the likelihood function, and ln(L) is the maximum likelihood function value; the specific formula for BIC is:

[0012] BIC = k * ln(n) - 2 * ln(L)

[0013] Where k is the number of parameters, n is the number of samples, L is the likelihood function, and kln(n) is the penalty term; S4, increase the number of clusters and calculate the Akaike Information Content (BIC) and Bayesian Information Content (AIC) under each cluster, and use the Elbow method to obtain the optimal number of clusters;

[0014] The Elbow method works as follows: as the number of clusters increases, the average distortion decreases, the number of samples in each cluster decreases, and the samples are closer to the centroid. The node with the largest decrease in distortion as the number of clusters increases is the optimal number of clusters. Specifically, the Elbow method is implemented by observing the BIC and AIC curves and finding the node with the largest decrease; the value corresponding to this node is the optimal number of clusters.

[0015] The specific formula is as follows:

[0016]

[0017] Where pi represents the point of the i-th cluster, ci represents the cluster center of the i-th cluster, m represents the total number of clusters, and distance represents the sum of squared distances from the sample to the cluster center.

[0018] The exposure time-dark current relationship is fitted to all pixels within the same cluster using the least squares method to calculate the MSE. The specific formula is as follows:

[0019]

[0020] Where yi is the fitted value of the exposure time-dark current relationship, y is the feature vector of the pixel, and m is the number of elements in the pixel feature vector;

[0021] S5. Under the clustering condition with the optimal number of clusters, fit the exposure time-dark current curve for all pixels in each cluster, calculate its mean and variance of MSE, and obtain the formula for the simulated and fitted exposure time-dark current curve and outliers.

[0022] S6. When taking pictures in the actual observation environment, for the part of the image without the target, a small matrix is ​​extracted, its MSE is calculated, and it is compared with the MSE of the matrix at the same position in the simulated image. The simulated image with the MSE closest to that in the actual observation environment is found, and the time of the image in the simulated environment is calculated. The time is approximately equal to the temperature, which is equivalent to obtaining the temperature of the actual observation environment, and thus obtaining the dark current of the camera taken in the actual observation environment.

[0023] In S1, when simulating the dark field conditions of the actual observation environment in the laboratory, the temperature difference between the simulated environment and the actual observation environment is less than 1 degree Celsius. The number of photos in each group is no less than 30, and the exposure time interval is no less than 4 seconds, such as 4-6 seconds. In S2, modeling the feature vector of this pixel means arranging the feature vectors of photos taken at different temperatures into a two-dimensional array in the order of row-to-column, left-to-right, and top-to-bottom. Each array element consists of temperature and grayscale value.

[0024] In S4, increasing the number of clusters means using an enumeration method to increase the number of clusters by 5 each time. In S4, using the Elbow method to determine the optimal number of clusters involves calculating the Akaike Information Value (BIC) and Bayesian Information Value (AIC) for each cluster after each increase in the number of clusters. Then, plot the AIC and BIC curves on a Cartesian coordinate system, with the horizontal axis representing the number of clusters and the vertical axis representing the BIC or AIC coefficients. The node with the largest change in distortion of the BIC and AIC curves corresponds to the optimal number of clusters.

[0025] In S5, exposure time-dark current curves are fitted to all pixels within each cluster, and their mean and variance of MSE are calculated. The formula for the simulated exposure time-dark current curve and outliers are derived. Outliers are defined as points where the x-axis represents time and the y-axis represents grayscale value. The least squares method is used to fit the exposure time-grayscale value relationship curve, and the fitting formula is derived. The mean and variance of MSE are calculated. Points with variances significantly greater than the mean are defined as points where the variance is more than three times the mean variance.

[0026] In S6, when taking pictures in the actual observation environment, a small matrix is ​​extracted from the non-target part of the captured image, its MSE is calculated, and it is compared with the MSE of the matrix at the same position in the simulated image. The simulated image with the closest MSE to the actual observation environment is found, and the temperature of the actual observation environment is obtained. This means that the time of this simulated image is calculated, and the time is approximated by the temperature, which is used as the temperature of the actual observation environment.

[0027] The processing method for dark current modeling and anomaly detection based on Gaussian mixture model includes: a data acquisition module: collecting the required data and transmitting it to the modeling module;

[0028] Modeling module: Models the collected data and uses Gaussian mixture model to perform clustering, calculating AIC and BIC;

[0029] Data processing module: For all pixels within the same cluster, the exposure time-dark current relationship curve is fitted using the least squares method. The mean and variance of the MSE for each cluster are calculated, and the dark current fitting formula for each cluster is derived. Points with a variance significantly greater than the mean are outliers. The criterion here is that the variance is greater than three times the mean. The beneficial effects of this invention are: This invention obtains the temperature of the actual observation environment by comparing the targetless part of the actual captured image with the same part of the image captured under simulated conditions, and thus obtains the dark current of the camera captured under the actual observation environment. This solves the problem of obtaining the temperature and dark current of imaging equipment in environments where humans cannot intervene. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0031] Figure 1 This is a schematic diagram of the process of the present invention;

[0032] Figure 2 This is a graph showing the AIC and BIC curves when the optimal number of clusters was determined in this invention.

[0033] Figure 3 This is an exposure time-dark current curve for clustering in this invention. Detailed Implementation

[0034] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0035] See Figure 1 This invention provides a method for dark current modeling and anomaly detection based on a Gaussian mixture model, comprising:

[0036] The specific steps of the data acquisition module are as follows:

[0037] In the laboratory, 36 time-series images were captured under dark conditions with an exposure time of 4 seconds.

[0038] The specific steps of the dark current modeling module are as follows:

[0039] Import this set of images as a NumPy array;

[0040] The first image is subtracted from the last 35 images in sequence because each pixel has a horizontal error. The first image at startup has a background reading. After subtracting it, the deviation is considered to be eliminated. Dark current modeling is performed by combining the time of each pixel with the feature vector of dark current. Here, the pixel size is 60*60, and the shape of the three-dimensional array is 35*60*60.

[0041] The feature vectors of the pixels are arranged into a two-dimensional array in order from left to right and top to bottom, with each row representing the feature vector of one pixel. The array shape is 3600*35.

[0042] Clustering the two-dimensional array yields a 60x60 mask matrix, where elements label the cell's category. The number of clusters is increased sequentially, and the AIC and BIC are calculated. The Elbow method is used to determine the optimal number of clusters; this is achieved by observing the AIC and BIC curves and identifying the node with the largest decrease in value. The value corresponding to this node represents the optimal number of clusters. Figure 2 As shown, both AIC and BIC show sudden drops at positions 3 and 8, but the drop is the largest at position 3. Therefore, we believe that the optimal number of clusters should be 3.

[0043] The specific steps of the data processing module are as follows:

[0044] Using the least squares method, exposure time-dark current relationship curves are fitted to all pixels within the same cluster, such as... Figure 3 As shown, the dark current fitting function is obtained, and the fitted y value is obtained;

[0045] To calculate the MSE, subtract the fitted y-value from the feature vector of each pixel, square the result, and then sum them up to obtain the MSE value.

[0046] Calculate the mean and variance of MSE;

[0047] Outliers are identified by using the mean and variance. The criterion is that the variance is significantly greater than the mean. Here, a variance greater than 3 times the mean is considered an outlier and is marked as Nan in the MASK matrix. If there are outliers, the clustering process is repeated to calculate the MSE and identify outliers again until there are no outliers.

[0048] In practical applications, a small matrix of the non-target portion of an image captured by a camera is compared with the matrix of the same position in an image taken in the laboratory. Taking 35 images as an example, the MSE of the non-target portion of the 35 images and the image captured by the camera is calculated. Then, the MSE of the 35 images is compared to the matrix of the cropped portion of the image captured by the camera. Since we know the time of each of the 35 images, we can approximate the temperature and finally obtain the dark current of the camera in the current state.

[0049] The above are merely preferred embodiments of the present invention. Those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments described above. Any obvious improvements, substitutions or changes made by those skilled in the art based on the present invention are within the scope of protection of the present invention.

Claims

1. A processing method for dark current modeling and outlier detection based on Gaussian mixture model, characterized in that: The steps are as follows S1, in the laboratory, simulate the dark field conditions of the actual observation environment, take a group of photos continuously at different temperatures according to the exposure time interval when the camera works in the actual observation environment; S2, in each group of photos taken at different temperatures, the gray scale of each exposure time pixel is taken out as a vector, which is modeled as a feature vector of the pixel; S3, the feature vectors of all pixels in the photo are clustered using Gaussian mixture model, and the Akaike information amount BIC and Bayesian information amount AIC under the clustering are calculated; S4, increase the number of clusters and calculate the Akaike information amount BIC and Bayesian information amount AIC under each cluster, and use the Elbow method to obtain the best cluster number; S5, under the condition of the best cluster number of clustering, the exposure time-dark current curve of all pixels in each cluster is fitted, the MSE mean and variance are calculated, and the simulated fitting exposure time-dark current curve formula and abnormal points are obtained; the exposure time-dark current curve of all pixels in each cluster is fitted, the MSE mean and variance are calculated, and the simulated fitting exposure time-dark current curve formula and abnormal points are obtained, which means that in the rectangular coordinate system, the horizontal coordinate is time and the vertical coordinate is gray scale value, the least square method is used to fit the exposure time-gray scale value relationship curve, the fitting formula is obtained, the MSE mean and variance are calculated, and the points with significantly larger variance than the mean are taken as abnormal points; the points with significantly larger variance than the mean are points with variance three times larger than the mean; S6, when taking pictures in the actual observation environment, a small matrix is cut from the non-target part of the photographed image, and its MSE is calculated and compared with the MSE of the matrix at the same position of the simulated image, the simulated image with the most similar MSE to the actual observation environment is found, and the time of the image in the simulated environment is calculated, and then the dark current of the camera in the actual observation environment is obtained.

2. The processing method of dark current modeling and outlier detection based on Gaussian mixture model according to claim 1, characterized in that: In S1, when simulating the dark field conditions of the actual observation environment in the laboratory, the temperature difference between the simulated environment in the laboratory and the actual observation environment is less than 1 degree Celsius, and the number of each group of photos is not less than 30.

3. The processing method of dark current modeling and outlier detection based on Gaussian mixture model according to claim 1, wherein: In S2, the modeling of the feature vector of the pixel means that the feature vectors of the photos taken at different temperatures are arranged in the order of row first, column second, left to right, and top to bottom to form a two-dimensional array, and each array element is composed of temperature and gray scale value.

4. The processing method of dark current modeling and outlier detection based on Gaussian mixture model according to claim 1, characterized in that: In S4, increasing the number of clusters means that the enumeration method is used, and the number of clusters is increased by 5 each time.

5. The processing method of dark current modeling and outlier detection based on Gaussian mixture model according to claim 4, characterized in that: In S4, the best cluster number is obtained by using the Elbow method, which means that after increasing the number of clusters each time, the Akaike information amount BIC and Bayesian information amount AIC under each cluster are calculated, and in the rectangular coordinate system, the horizontal coordinate axis represents the number of clusters, and the vertical coordinate axis represents the BIC or AIC coefficient, and the AIC and BIC curves are drawn; find the node with the largest change in the distortion degree of the BIC and AIC curves, and the corresponding cluster number is the best cluster number. In S4, the best cluster number is obtained by using the Elbow method, which means that after increasing the number of clusters each time, the Akaike information amount BIC and Bayesian information amount AIC under each cluster are calculated, and in the rectangular coordinate system, the horizontal coordinate axis represents the number of clusters, and the vertical coordinate axis represents the BIC or AIC coefficient, and the AIC and BIC curves are drawn; find the node with the largest change in the distortion degree of the BIC and AIC curves, and the corresponding cluster number is the best cluster number.

6. The processing method of dark current modeling and outlier detection based on Gaussian mixture model according to claim 1, wherein: In S6, when taking a picture in the actual observation environment, a small matrix is cut from the part without target in the taken picture, and its MSE is calculated and compared with the MSE of the matrix in the same position of the simulated picture, to find the simulated picture with the most similar MSE to that in the actual observation environment, and the temperature of the actual observation environment is obtained, that is, the time of the simulated picture in the simulated environment is calculated, and the time is approximately the temperature as the temperature of the actual observation environment.

Images