All-sky imager aerosol optical parameter retrieval algorithm based on machine learning

Abstract

The application discloses a machine learning-based all-sky imager aerosol optical parameter inversion algorithm and belongs to the meteorological observation field.The application uses an iterative threshold segmentation method, optimizes a threshold automatically, first constructs a clear sky extraction quantitative standard, and solves the problems of artificial dependence and poor consistency.The application simulates an equal zenith angle scanning mode, performs multi-dimensional feature extraction from a sky area picture, obtains scattering information of aerosols which reflects multi-angle, multi-position information related to an aerosol scattering phase function and multi-angle information of total brightness, and provides key support for SSA inversion.The application sets up a regression model with multi-dimensional features as input and AOD and SSA as output, sets spatial difference and time difference matching samples, realizes synchronous output, makes the sample standard clear, and greatly improves efficiency and precision.The application is applied to the fields of atmospheric remote sensing, climate model research and air quality assessment.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of meteorological observation, and particularly relates to a full-sky imager aerosol optical parameter inversion algorithm based on machine learning. BACKGROUND

[0002] As a common atmospheric component, aerosol plays an important role in the earth's radiation budget, cloud and precipitation development, ecological system and air quality. However, estimating its contribution to the radiation forcing is considered to be the main source of uncertainty in quantifying current climate change, so accurately monitoring the spatial distribution and temporal dynamics of aerosol is very important for understanding the earth's climate system. Aerosol optical depth (AOD) is a dimensionless index that measures the scattering and absorption effects of aerosols in the atmosphere column from the instrument (earth's surface) to the top of the atmosphere on solar radiation. In simple terms, it represents the degree to which light is weakened as it passes through the atmosphere due to the presence of aerosols. Therefore, aerosol optical depth can be used to assess the climate impact and air pollution of aerosols, and in recent years, people have also invested a lot of effort in measuring aerosol optical depth at different scales. Due to the variation of aerosol distribution, chemical composition and radiation characteristics with location and time, detailed monitoring of the temporal and spatial variation of aerosol is needed. At present, many aerosol observations rely on satellites or air-based platforms, which can achieve long-term and continuous monitoring in the time dimension, but the spatial resolution is usually low. Due to factors such as geographical location, instrument cost, maintenance difficulty, etc., the scheme of setting up more atmospheric measurement stations is not practically feasible.

[0003] However, relying solely on aerosol optical depth cannot fully describe the radiation effects of aerosols, and the effect of aerosols on solar radiation also depends on parameters such as single scattering albedo (SSA). Single scattering albedo is defined as the ratio of scattering coefficient to extinction coefficient, and directly reflects whether aerosol is dominated by scattering (such as sea salt) or absorption (such as black carbon), which is a key parameter that determines the sign and intensity of aerosol direct radiation forcing, and its uncertainty contribution to aerosol radiation forcing is more than 30%. Therefore, more accurate observation of SSA is a necessary prerequisite for quantifying the radiation effects of aerosols.

[0004] Total Sky Imager (TSI) is a kind of instrument for automatic cloud cover monitoring, aiming to replace the traditional manual observation. The TSI, which is represented by the Total Sky Imager developed by Yankee Company in the United States, has derived various improved designs. The TSI generally uses a CCD camera with a fisheye lens, which can take high spatial and temporal resolution images of the entire sky range in the visible light band during the day, obtain high-quality cloud cover monitoring data, and automatically upload to the computer for storage and processing. In recent years, the TSI has been greatly improved in quality, resolution (spatial and temporal), and calibration, making it suitable for environmental research and measurement. Based on the ability of the TSI to take high-resolution images of the entire sky range in the visible light band, people have tried to use it to retrieve aerosol optical thickness and other optical properties.

[0005] The high spatial and temporal resolution advantage of the TSI provides new possibilities for aerosol observation, but its existing AOD retrieval method still has obvious limitations. The traditional TSI-based AOD retrieval method usually needs to calculate the radiance of each pixel point, adjust the AOD value iteratively, input the radiative transfer model, and finally select the AOD result that best matches the measured data. This algorithm has high computational complexity, low computational efficiency, and requires large computing power. At the same time, the TSI-based research focuses on AOD retrieval, and the simultaneous retrieval of AOD and SSA has not been achieved, which leads to the inability to analyze the total-scattering properties of aerosols at a fine scale, and severely limits the complete analysis of their radiation effects and climate impact.

[0006] Aerosol optical depth (AOD) is traditionally obtained by measuring the attenuation (extinction ratio) of direct solar radiation. For the TSI, it relies on the overall level of sky background radiance. Even for traditional radiative transfer models, the retrieval of AOD is achieved by adjusting the AOD value to best match the overall level of measured radiance. The TSI can directly reflect the sky brightness, so it is physically natural to use the TSI to retrieve AOD. Single scattering albedo (SSA) is traditionally obtained by measuring the angular distribution of sky radiation (i.e., the distribution of scattered light), which requires multi-angle and multi-position radiance information. The retrieval of SSA is extremely sensitive to the geometric information of the image and the relative distribution (rather than just the absolute value) of sky radiance. Therefore, the retrieval of SSA from the TSI is a technical challenge in the field. SUMMARY

[0007] In order to overcome the problems of high equipment price, low space density of instrument installation and time delay of the sun photometer in the prior art, the present application proposes a machine learning based aerosol optical parameter inversion algorithm for a full-sky imager, which uses a machine learning algorithm to invert the aerosol optical thickness and single scattering albedo based on the full-sky imager data, so as to realize the aerosol optical parameter measurement of the full-sky imager at any place in the global range.

[0008] The machine learning based aerosol optical parameter inversion algorithm for a full-sky imager of the present application comprises the following steps:

[0009] 1) Collecting original sky pictures with time labels;

[0010] 2) Image processing is performed on the original sky pictures to remove all areas affecting the extraction of cloudless sky radiance, wherein an iterative threshold segmentation method is used to remove the light spots generated by the lens module to obtain sky area pictures retaining only clean sky;

[0011] 3) Feature extraction is performed from the sky area pictures in a constant zenith angle scanning mode, and the R, G and B average values of all colored pixel points, the red-blue ratio rbr value, the zenith angle of the sun and the scattering information of the aerosol are extracted, respectively, and the above features are referred to as multi-dimensional features represented by x;

[0012] 4) Aerosol parameters at a set wavelength, including aerosol optical thickness AOD and single scattering albedo SSA, are obtained from a public database, and the above parameters are referred to as target variables represented by y; the geographical position of the aerosol parameters in the public database corresponds to the position of the instrument collecting the original sky pictures;

[0013] 5) The multi-dimensional features and the target variables are matched according to time as a data set, and are divided into a training set and a test set for model training and for evaluating the generalization ability of the model, and are stored in a file, and the data is converted into a DMatrix format special for an ensemble regression model;

[0014] 6) The multi-dimensional features are used as the input of the ensemble regression model, and the target variables are used as the output of the ensemble regression model, the data of step 5) is input into the ensemble regression model, the training set is used for training, and the test set is used for verification and consistency check, and a trained ensemble regression model for this wavelength is obtained;

[0015] 7) The steps 4) to 6) are repeated multiple times by changing the wavelength to obtain multiple trained ensemble regression models for each wavelength;

[0016] 8) Collect the original sky picture of the new time period, input the multi-dimensional features obtained by the picture processing of step 2) and the feature extraction of step 3) into the integrated regression model to obtain the inversion aerosol optical depth AOD and the single scattering albedo SSA of the aerosol.

[0017] In step 1), the instrument position for collecting the original sky picture is fixed, the angle of the lens collected is perpendicular to the ground, the instrument uses a full-sky imager, the original sky picture includes a full-sky image, and the number of original sky pictures is more than 100,000.

[0018] In step 2), the original sky picture is subjected to image processing, and all areas affecting the extraction of cloudless sky radiance are removed, including the following steps:

[0019] a) The relative position of the instrument and the surrounding buildings is fixed, and a black mask is used for trimming to remove the surrounding buildings;

[0020] b) Traverse the image to find the circular area with the maximum brightness, which is considered to be the sun and its surrounding area with relatively high brightness, and a black mask is used to remove the sun and its surrounding area with relatively high brightness;

[0021] c) The image is repeatedly subjected to binary processing using an iterative threshold segmentation method until the threshold value is stable, a black mask is used to cover the area with relatively high brightness, and the cloud and the light spot generated by the lens module are removed.

[0022] The prior art uses computer vision recognition to extract the edge for image segmentation; the present application uses an iterative threshold segmentation method including the following steps: randomly setting an initial threshold value; comparing the pixel points on the picture according to the brightness and the threshold value to divide them into two categories: above the threshold value and below the threshold value; calculating the average brightness of the two categories of pixel points respectively, and then averaging the two average brightness values to obtain a total average value as a new threshold value; re-dividing the pixel points into two categories according to the new threshold value, and repeating the above steps until the threshold value is stable to obtain a stable threshold value; for the pixel points greater than the stable threshold value, a mask is used to cover the pixel points that do not meet the clear sky condition, thereby removing the cloud and the light spot generated by the lens module to obtain a sky area picture that only retains the clean sky. The prior art relies on edge recognition and requires manual parameter adjustment. The present application uses threshold value self-optimization, establishes a clear sky extraction quantization standard, and solves the problems of manual dependence and poor consistency.

[0023] In step 3), all colored pixel points are extracted from the sky region picture, each colored pixel point is divided into three channels of red R, green G and blue B, and the average values of all colored pixel points R, G and B are obtained respectively; the red-blue ratio rbr value is calculated by dividing the average value of R by the average value of B; the solar zenith angle is calculated by combining the R, G and B average values and the rbr value corresponding to the sky region picture with time and latitude and longitude; in order to imitate the optical parameter source of AERONET website, that is, the sun photometer, the Almucantar mode, that is, the fixed zenith angle and the changed azimuth angle, is scanned, 0.6~0.8 times of the radius of the sky region picture is taken as the radius, the center of the circle is located at the center point of the sky region picture, a circle is drawn, and more than 10 pixel points are taken on the circumference, the RGB values of the multiple pixel points are extracted and called the scattering information of aerosol, and the scattering information of aerosol reflects the multi-angle, multi-position information and total brightness related to the scattering phase function of aerosol. The prior art only extracts one-dimensional features and cannot reflect the scattering angle distribution, the present application first obtains multi-angle information and provides key support for SSA inversion.

[0024] In step 4), the spatial difference (horizontal distance) between the geographical position of the aerosol parameter corresponding station in the public database and the position of the instrument collecting the original sky picture is within the spatial threshold, and the spatial position is considered to be consistent, and the spatial threshold is 3~10 km. Aerosol optical thickness reflects the degree of atmospheric pollution, the measurement of single scattering albedo SSA in the prior art is not accurate and has large uncertainty; the present application obtains accurate SSA, thereby reducing the uncertainty of the atmospheric model.

[0025] In step 5), the time of the aerosol parameter of the public database cannot completely match the time of collecting the original sky picture, and the time difference between the collection of the original sky picture and the aerosol parameter of the public database is within the time threshold, which is considered to be time matched; the time threshold is three minutes to ten minutes.

[0026] In step 6), the integrated regression model adopts an extreme gradient boosting regressor XGBoost, includes multiple base learners, takes a root mean square error RMSE as a loss function, determines an optimal boosting iteration number N through K-fold cross-validation and an early stopping strategy, K is a natural number greater than or equal to 5; then, the optimal boosting iteration number N is taken as the number of base learners, and a combination of hyperparameters such as maximum tree depth, learning rate, subsample ratio, column sample ratio and leaf node split minimum loss reduction is set, and an integrated regression model with joint output is trained. The integrated regression model is executed on the test set to obtain a prediction vector pair, and a scatter density plot is drawn with the true value as the horizontal axis and the predicted value as the vertical axis. The integrated regression model is constructed with multi-dimensional features as input and AOD and SSA as output, the RMSE is taken as the loss function, the parameters are determined through K-fold cross-validation, and the samples are matched with spatial difference and 5min time difference. The existing technology is difficult to synchronize in iterative inversion, and the present application realizes synchronous output, clearly defines the sample standard, and greatly improves the efficiency and accuracy.

[0027] Advantages of the present application:

[0028] The prior art uses computer vision recognition to extract edges for image segmentation; the present application uses an iterative threshold segmentation method, the threshold is self-optimized, and a clear-sky extraction quantitative standard is first established to solve the problems of artificial dependence and poor consistency.

[0029] All colored pixel points are extracted from the sky area picture, each colored pixel point is divided into three channels of red R, green G and blue B, and the average values of all colored pixel points R, G and B are obtained respectively; the red-blue ratio rbr value is calculated by dividing the average value of R by the average value of B; the average values of R, G and B corresponding to the sky area picture and the rbr value are combined with time and latitude and longitude to calculate the solar zenith angle; to simulate the optical parameter source of the AERONET website, an equal zenith angle scanning mode is simulated, that is, a sun photometer, 0.6-0.8 times the radius of the sky area picture is taken as the radius, the center of the circle is located at the center point of the sky area picture, a circle is drawn, and more than 10 pixel points are evenly taken on the circumference, and the RGB values of the multiple pixel points are extracted; the prior art only extracts one-dimensional features, which cannot reflect the scattering angle distribution, the present application simulates the equal zenith angle scanning mode to extract features from the sky area picture, thereby obtaining the scattering information of the aerosol reflecting the multi-angle, multi-position information related to the aerosol scattering phase function and the multi-angle information of the total brightness, which provides key support for SSA inversion; the present application constructs an integrated regression model with multi-dimensional features as input and AOD and SSA as output, takes RMSE as the loss function, determines the parameters through K-fold cross-validation, and sets the spatial difference and time difference to match the samples; the existing technology is difficult to synchronize in iterative inversion, and the present application realizes synchronous output, clearly defines the sample standard, and greatly improves the efficiency and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1A flowchart of the aerosol optical parameter inversion algorithm of the all-sky imager based on machine learning of the present application is shown in Figure 1.

[0031] Figure 2 A density scatter plot comparing the SSA and AOD at 440 nm and 675 nm obtained by one embodiment of the aerosol optical parameter inversion algorithm of the all-sky imager based on machine learning of the present application with the data from the Beijing site is shown in Figure 2.

[0032] Figure 3 A density scatter plot comparing the SSA and AOD at 870 nm and 1020 nm obtained by one embodiment of the aerosol optical parameter inversion algorithm of the all-sky imager based on machine learning of the present application with the data from the Beijing site is shown in Figure 3.

[0033] Figure 4 A density scatter plot comparing the SSA and AOD at 440 nm and 675 nm obtained by one embodiment of the aerosol optical parameter inversion algorithm of the all-sky imager based on machine learning of the present application with the data from the SGP site is shown in Figure 4.

[0034] Figure 5 A density scatter plot comparing the SSA and AOD at 870 nm and 1020 nm obtained by one embodiment of the aerosol optical parameter inversion algorithm of the all-sky imager based on machine learning of the present application with the data from the SGP site is shown in Figure 5. DETAILED DESCRIPTION

[0035] The present application will be further described below with reference to the accompanying drawings and specific embodiments.

[0036] The aerosol optical parameter inversion algorithm of the all-sky imager based on machine learning of the present embodiment, as shown in Figure 1, includes the following steps: Figure 1

[0037] 1) The instrument position (39°59'N, 116°18'E) is fixed, the angle of the lens is perpendicular to the ground, the instrument is an all-sky imager, the original sky pictures with time markers are collected, the original sky pictures include images of the entire sky, the number of original sky pictures is 200,000, the image collection time is from April 2018 to January 2020, the resolution is 180 dpi, and the picture shooting time is extracted from the picture name;

[0038] 2) Image processing is performed on the original sky pictures:

[0039] a) The relative position of the instrument and the surrounding buildings is fixed, and a black mask is used to crop and remove the surrounding buildings;

[0040] ​b) Traverse the image to find the circular area with the maximum brightness, which is considered as the sun and its surrounding area with higher brightness, and use a black mask to remove the sun and its surrounding area with higher brightness, the radius of the circular area of the black mask should be 0.3 times the radius of the colored circular area obtained after removing the surrounding buildings in step a);

[0041] c) Use the iterative threshold segmentation method, which includes the following steps: randomly set the initial threshold value, the initial value is 150; compare the pixel points on the picture with the threshold value according to brightness to divide them into two categories: higher than the threshold value and lower than the threshold value; calculate the average brightness of the two categories of pixel points respectively, and then average the two average brightness values to obtain the total average value as the new threshold value; redivide the pixel points into two categories according to the new threshold value, and repeat the above steps until the threshold value is stable to obtain a stable threshold value; for the pixel points greater than the stable threshold value, consider them as pixel points that do not meet the clear sky condition, use a mask to cover them, thereby removing the cloud and the light spot generated by the lens module, removing all areas that affect the extraction of cloudless sky radiance, and obtaining a sky area picture that only retains the clean sky;

[0042] 3) Imitate the equal zenith angle scanning mode to extract all colored pixel points from the sky area picture, each colored pixel point is divided into red R, green G and blue B channels, and the average values of all colored pixel points R, G and B are calculated respectively; calculate the red-blue ratio rbr value by dividing the R average value by the B average value; combine the R, G and B average values of the sky area picture and the rbr value with time and latitude and longitude to calculate the zenith angle of the sun; imitate the optical parameter source of the AERONET website, i.e. the sun photometer, the scanning mode is the equal zenith angle mode, i.e. the fixed zenith angle and the changing azimuth angle, take 0.6~0.8 times the radius of the sky area picture as the radius, the center of the circle is located at the center point of the sky area picture, draw a circle, and take more than 10 pixel points on the circumference on average, extract the RGB values of the multiple pixel points, which are called aerosol scattering information, which can reflect the multi-angle and multi-position information related to the aerosol scattering phase function and the total brightness; the average values of R, G and B of all colored pixel points, the red-blue ratio rbr value, the zenith angle of the sun and the aerosol scattering information are called multi-dimensional features, denoted by x;

[0043] 4) Obtain the aerosol parameters of the set wavelength collected by the sun photometer from the public database AERONET website, including the aerosol optical depth AOD and the single scattering albedo SSA, which are called target variables denoted by y, the latitude and longitude are (40.0N, 116.3E), the data quality level is Level 2.0, and the horizontal distance between the geographical position of the aerosol parameters in the public database and the position of the instrument collecting the original sky picture is less than 5 km, which is considered as the consistent spatial position representation;

[0044] 5) Match the multi-dimensional features with the target variable by time as a dataset, and divide the training set and test set for model training and for evaluating the model generalization ability, store in a file, convert the data into DMatrix format special for ensemble regression model; The time of aerosol parameters in the disclosed database does not completely match the time of collecting the original sky picture, and the time error between the collection of the original sky picture and the aerosol parameters in the disclosed database is within 5 minutes, which is considered to be time matching; Time matching is performed at the same wavelength, that is, the aerosol optical thickness AOD_440nm and the single scattering albedo SSA_440nm measured synchronously by the AERONET sun photometer at the wavelength of 440 nm are taken as the true value label, and a sample pair matched with the input multi-dimensional feature x is established, the matching time window is not more than ±5 min, and a sample set is formed; The data set is divided into a training set and an independent test set in a ratio of 8:2;

[0045] 6) The ensemble regression model adopts an extreme gradient boosting regressor XGBoost, includes multiple base learners, takes the root mean square error RMSE as the loss function, determines the optimal boosting iteration number N through K-fold cross-validation (K=10) and early stopping strategy; Then, with the optimal boosting iteration number N as the base learner number, the maximum tree depth 7, the learning rate 0.08, the subsampling ratio 0.75, the column sampling ratio 1.0, and the leaf node splitting minimum loss reduction 0, the multi-dimensional features are taken as the input of the ensemble regression model, and the target variable is taken as the output of the ensemble regression model, the data in step 5) are input into the ensemble regression model, the training set is used for training, and the test set is used for verification and consistency check, and the ensemble regression model of the joint output (SSA_440nm, AOD_440nm) is trained; The test set is executed to predict the ensemble regression model, and a prediction vector pair is obtained, and a scatter density plot is drawn with the true value as the horizontal axis and the predicted value as the vertical axis; The point cloud density is calculated by using the Gaussian kernel density estimation, and the unit slope reference line y=x and the linear regression fitting line y=kx+b are synchronously superimposed in a visual form; The proportion consistency is quantitatively evaluated by the regression coefficient k; and the trained ensemble regression model for this wavelength is obtained;

[0046] 7) Change the wavelength, repeat steps 4)~6) for 440nm, 675nm, 870nm and 1020nm respectively, and obtain the trained ensemble regression model for each wavelength;

[0047] 8) Collect the original sky picture of the new time period, input into the ensemble regression model after the picture processing in step 2) and the feature extraction in step 3), so as to obtain the inverted aerosol optical thickness AOD and the single scattering albedo SSA of the aerosol, and then verify that the ensemble regression model meets the high-precision inversion requirement of the aerosol multi-optical parameter.

[0048] After the raw sky pictures of ARM_SGP site (36.6N, 97.5W) from July 2024 to July 2025 are matched with the AERONET data, the pictures are processed in step 2) and the features are extracted in step 3), the obtained multi-dimensional features are input into the integrated regression model, and the integrated regression model is tested and verified again using the site, proving that the method of the application is universal for all sites.

[0049] As shown in Figure 2 and 3 , the horizontal axis represents the actual value of the aerosol parameter, the single scattering albedo SSA and the aerosol optical depth AOD of the target wavelength from the AERONET website; the vertical axis represents the predicted value of the aerosol parameter, the inversion result of the XGBoost model after the image processing of step 2) and the feature extraction processing of step 3) of the all-sky imager data of the Beijing site. Sample basis: the data set is divided into a training set and a test set in a ratio of 8:2, Figure 2 and 3 , the data in

[0050] All subgraphs contain two types of key reference lines: the dashed line, that is, the y=x reference line, represents the ideal matching state of "predicted value = actual value", which is used to intuitively judge the deviation direction of the prediction; the solid line, that is, the regression line: the subgraph is labeled with a linear regression equation, such as the SSA870nm subgraph labeled "regression line y=0.83x-0.15" and the AOD1020nm subgraph labeled "regression line: y=0.98x+0.01", which is used to quantitatively evaluate the proportional consistency of the predicted value and the actual value.

[0051] Numerical range: the horizontal axis scale is adjusted according to the parameter type. The SSA subgraph horizontal axis range is concentrated in 0.80-1.00, which is consistent with the physical meaning of SSA: 0≤SSA≤1, and the AOD subgraph horizontal axis range varies with the wavelength, reaching 3.5 at 440nm, about 1.75 at 870nm and 1020nm. The numerical range of the horizontal and vertical axes is consistent with the scale.

[0052] Subgraph grouping: divided into SSA group and AOD group according to the parameter type, listed in two columns; each group is arranged in order of wavelength (440nm, 675nm, 870nm, 1020nm), Figure 2 and 3 , each row has two subgraphs, the left is SSA, and the right is AOD.

[0053] Model effect evaluation based on the Beijing site:

[0054] 1. Figure 2 and 3 , the SSA group includes the left column of four subgraphs, covering 440nm, 675nm, 870nm and 1020nm:

[0055] All wavelength SSA data values are concentrated in 0.80-1.00, consistent with the SSA physical range, and the data points as a whole are distributed along the y=x reference line: 440nm, 675nm, 1020nm have no obvious deviation; although 870nm has a slight low prediction deviation of "y=0.83x-0.15", the accuracy meets the standard.

[0056] 2. Figure 2 And 3 The AOD group in FIG. 4 includes four subgraphs on the right column, covering 440nm, 675nm, 870nm and 1020nm:

[0057] The AOD data points of each wavelength are close to the y=x reference line at low AOD (<1.0), and slightly underestimated at high AOD (>1.0), consistent with the commonality of "slight systematic underestimation of high AOD", and the overall correlation is high.

[0058] As shown in FIG. 5 and FIG. 6, the vertical axis represents the aerosol parameter prediction value, and the XGBoost model inversion result after image processing in step 2) and feature extraction processing in step 3) from the SGP site all-sky imager data; the descriptions of the remaining image elements are consistent with those in Figure 4 and 5 . Figure 2 3

[0059] Model Effect Evaluation Based on SGP Site

[0060] 1. Figure 4 And Figure 5 The SSA group in FIG. 7 includes four subgraphs on the left column, covering 440nm, 675nm, 870nm and 1020nm:

[0061] The actual value and the predicted value of single scattering albedo SSA of each wavelength are concentrated in 0.2-1.0, consistent with the SSA physical range. The data points as a whole are distributed along the y=x reference line, and there is no obvious systematic deviation. The regression results of 440nm (regression line y=0.84x+0.14), 675nm (y=0.84x+0.14), 870nm (y=0.85x+0.13), 1020nm (y=0.87x+0.11) all reflect high correlation.

[0062] 2. Figure 4 And Figure 5 The AOD group in FIG. 8 includes four subgraphs on the right column, covering 440nm, 675nm, 870nm and 1020nm:

[0063] ​​The data points of each wavelength aerosol optical depth AOD are densely distributed, and the low AOD interval is close to the y=x reference line; the coefficients of 440nm (y=0.94x+0.01), 675nm (y=0.96x+0.00), 870nm (y=0.95x+0.00) and 1020nm (y=0.94x+0.00) are close to 1.0, which embodies high correlation, especially the data points of 675nm wavelength almost fit the reference line, and the overall AOD inversion performance of the model is satisfactory.

[0064] The inversion effect of the present application is good, and the effects of simultaneously inverting AOD and SSA in the Beijing site verification set are better than separately inverting the two parameters. When using the XGBoost model to simultaneously invert AOD and SSA, for the Beijing site verification set, the correlation coefficient is 0.93; for the SGP site verification set, the correlation coefficient is 0.94. This proves that the inversion model has good spatiotemporal transferability. Although there is a slight systematic underestimate at high AOD values, the overall performance of the model is still satisfactory, and it can be considered that the accuracy of extracting aerosol optical parameters from the all-sky imager using this algorithm is comparable to AERONET. These findings collectively demonstrate the feasibility of using all-sky imagers to invert aerosol optical parameters, providing a new technical path for future atmospheric remote sensing research. The experimental design is shown in the following table:

[0065]

[0066] Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present application, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present application and the appended claims. Therefore, the present application should not be limited to the disclosed content of the embodiments, and the scope of the present application claimed is the scope defined by the claims.

Claims

1. A method for constructing an aerosol optical parameter retrieval algorithm of an all-sky imager based on machine learning, characterized in that, The inversion algorithm comprises the following steps: 1) Collecting original sky pictures with time labels; 2) Image processing of the original sky pictures to remove all areas affecting the extraction of cloudless sky radiance, wherein an iterative threshold segmentation method is used to remove clouds and lens module generated light spots to obtain a sky area picture retaining only clean sky; 3) Imitating an equal zenith angle scanning mode, feature extraction is performed from the sky area picture to extract all colored pixel point R, G and B average values, red-blue ratio rbr value, solar zenith angle and aerosol scattering information, referred to as multi-dimensional features; 4) Obtaining aerosol parameters of a set wavelength from a public database, including aerosol optical depth AOD and single scattering albedo SSA, referred to as target variables; 5) Matching the multi-dimensional features with the target variables according to time as a data set, and dividing the training set and the test set for model training and for evaluating the model generalization ability; 6) Using an integrated regression model, taking the multi-dimensional features as the input of the integrated regression model and the target variables as the output of the integrated regression model, inputting the data of step 5) into the integrated regression model, training with the training set and verifying and consistency checking with the test set, wherein the consistency checking refers to evaluating the proportional consistency of the predicted value and the actual value, obtaining the trained integrated regression model for this wavelength; 7) Changing the wavelength repeatedly to perform steps 4) to 6) multiple times to obtain multiple trained integrated regression models for each wavelength; 8) Collecting original sky pictures of a new time period, inputting the multi-dimensional features obtained by the picture processing of step 2) and the feature extraction of step 3) into the integrated regression model to obtain the inverted aerosol optical depth AOD and the single scattering albedo SSA of the aerosol.

2. The inversion algorithm of claim 1, wherein, In step 1), the instrument position for collecting the original sky picture is fixed, the angle of the lens collected is perpendicular to the ground, and the original sky picture includes the image of the whole sky.

3. The inversion algorithm of claim 1, wherein, In step 2), the image processing of the original sky picture comprises the following steps: a) Removing the surrounding buildings by black mask cropping; b) Finding the circular area with the largest brightness as the sun and its surrounding area with relatively large brightness, and removing the sun and its surrounding area with relatively large brightness by black mask; c) Using an iterative threshold segmentation method to repeatedly perform binary processing on the image until the threshold is stable, and using a black mask to cover the area with relatively large brightness to remove clouds and lens module generated light spots.

4. The inversion algorithm of claim 1, wherein, In step 3), all colored pixel points are extracted from the sky area picture, each colored pixel point is divided into red R, green G and blue B channels, the R, G and B average values of all colored pixel points are calculated respectively, the red-blue ratio rbr value is calculated by dividing the R average value by the B average value, the solar zenith angle is calculated by combining the R, G and B average values and the rbr value of the sky area picture with time and latitude and longitude, and the scattering information of the aerosol is extracted by drawing a circle with a radius of 0.6-0.8 times the radius of the sky area picture and taking more than 10 pixel points on the circumference as the scattering information of the aerosol.

5. The inversion algorithm of claim 1, wherein, In step 4), the spatial difference between the geographical position of the site corresponding to the aerosol parameter of the public database and the position of the instrument collecting the original sky picture is within the spatial threshold, and the spatial position is considered to be representative of the consistency.

6. The inversion algorithm of claim 1, wherein, In step 5), the time difference between the collection of the original sky picture and the aerosol parameter of the public database is within the time threshold, and it is considered to be time-matched.

7. The inversion algorithm of claim 1, wherein, In step 6), the ensemble regression model adopts an extreme gradient boosting regressor XGBoost, includes multiple base learners, takes the root mean square error RMSE as the loss function, determines the optimal boosting iteration number N through K-fold cross-validation and early stopping strategy, K is a natural number greater than or equal to 5; then set the maximum tree depth, learning rate, subsampling ratio, column sampling ratio and leaf node splitting minimum loss reduction amount as the hyperparameter combination of the base learner number, and train to obtain the ensemble regression model with joint output.

Images