A blue sky classification method and device based on image recognition and machine learning

Abstract

The present invention discloses a blue sky classification method and device based on image recognition and machine learning, which relates to the fields of image processing and machine learning. The method includes: obtaining multiple groups of historical data of sky information in a target area within a preset time period; calculating a classification score index for each group of historical data of sky information; determining the corresponding blue sky level by applying a clustering algorithm according to the classification score index; training a blue sky level classification model with multiple groups of historical data of sky information in the target area within the preset time period as input and the corresponding blue sky level as output; and inputting the blue sky image to be classified, as well as the matched surface meteorological observation data, ERA5 data, surface air quality observation data, and MEIC emission inventory data into the trained blue sky level classification model to obtain the blue sky level of the blue sky image to be classified. The present invention can realize different blue sky level determination technologies through images combined with multi-source data.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and machine learning, and particularly to a blue sky classification method and device based on image recognition and machine learning. Background Art

[0002] In recent years, the increase in the number of blue skies has become an important indicator for improving the quality of life. The appearance of blue skies is mainly caused by physical mechanisms such as Rayleigh scattering and Mie scattering, and is closely related to the concentration of aerosol particles in the air and meteorological conditions. The shorter wavelength blue light, due to its stronger scattering effect, makes the sky appear blue. However, when there are a large number of fine particles suspended in the air, Mie scattering increases, resulting in the sky turning gray and forming haze. Therefore, the appearance of blue skies is usually associated with low cloud cover, high visibility, and low pollutant concentrations. At the same time, Rayleigh scattering and Mie scattering act together on the brightness and color saturation of the sky, giving different degrees of manifestation to blue skies. However, there is still a lack of criteria for determining blue sky levels and automated recognition technologies.

[0003] Images are the most direct reflection of blue skies. The development of image recognition and artificial intelligence has made it possible to automate the process of classifying blue sky levels. The progress of panoramic camera technology, smartphone photography, and deep learning methods has enabled an image-based blue sky recognition system to quickly and intuitively classify blue sky levels at a reduced cost, providing feasibility for real-time monitoring.

[0004] In the prior art, Patent No. ZL202010226425.X has achieved a determination technology for blue skies using long-term meteorological observation data, but this technology can only determine whether it is a blue sky or not, and cannot judge the blue sky level. It is known that the colors presented by blue skies are different, corresponding to different blue sky levels and air pollution conditions, and it is difficult to directly quantify through this method.

[0005] The prior art only relies on meteorological observation data and has a single structure. However, the public's perception of blue skies is mainly based on vision, and at the same time, the impact of air quality needs to be considered. The difference in blue sky colors can only be reflected through photos and cannot be determined through meteorological observations. Therefore, the prior art cannot achieve the determination of blue sky levels.

[0006] Other technologies only study the recognition of air quality, while the blue sky level is not only affected by air quality but also by meteorological conditions. For example, on rainy days or foggy days, the air quality index may be good, but it is not a blue sky. Therefore, it is impossible to directly determine the blue sky level through changes in air quality, and the technical recognition methods of the two are fundamentally different.

[0007] In summary, the prior art all requires the use of expensive meteorological monitoring equipment, and the monitoring range is limited, making it impossible to achieve large-scale data collection and real-time monitoring. Summary of the Invention

[0008] The object of the present invention is to provide a blue sky classification method and device based on image recognition and machine learning, which realizes different blue sky level determination technologies through images combined with multi-source data, and helps to quantify and evaluate the achievements of the blue sky defense war.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A blue sky classification method based on image recognition and machine learning, the method comprising:

[0011] Obtaining multiple sets of historical sky information data of a target area in a preset time period; each set of the historical sky information data includes historical blue sky image data and surface meteorological observation historical data, ERA5 historical data, surface air quality observation historical data, and MEIC emission inventory historical data that match the historical blue sky image data;

[0012] Calculating a classification score index for each set of the historical sky information data;

[0013] According to the classification score index, applying a clustering algorithm to determine the blue sky level corresponding to each set of the historical sky information data of the target area in the preset time period;

[0014] Using the multiple sets of historical sky information data of the target area in the preset time period as input and the corresponding blue sky level as output to train a blue sky level classification model, and obtaining a trained blue sky level classification model; the blue sky level classification model includes a machine learning algorithm and a meta-learner;

[0015] Inputting the to-be-classified blue sky image of the target area in the preset time period and the surface meteorological observation data, ERA5 data, surface air quality observation data, and MEIC emission inventory data that match the to-be-classified blue sky image into the trained blue sky level classification model to obtain the blue sky level of the to-be-classified blue sky image.

[0016] Optionally, the calculating a classification score index for each set of the historical sky information data includes:

[0017] Applying the formula to calculate the classification score index for each set of the historical sky information data;

[0018] where Q is the classification score index; B is the average value of the blue channel of the picture; M is the average value of all meteorological variables; P is the pollutant concentration; B min is the minimum value of the blue channel value; B max is the maximum value of the blue channel value; M min is the minimum value of the meteorological data; M max is the maximum value of the meteorological data; Pmin is the minimum value of the pollutant concentration; P max is the maximum value of the pollutant concentration.

[0019] Optionally, according to the classification scoring index, a clustering algorithm is applied to determine the blue sky level corresponding to each set of historical sky information data of the target area in a preset time period, which specifically includes:

[0020] Judge whether it is raining for the weather corresponding to each set of historical sky information data of the target area in a preset time period;

[0021] If it is raining, the blue sky level is non - blue sky;

[0022] If it is not raining, according to the classification scoring index, a clustering algorithm is applied to determine the primary blue sky level corresponding to the current sky information data.

[0023] Optionally, the clustering algorithm includes K - means, Hierarchical, GMM, and DBSCAN.

[0024] Optionally, the meta - learner is a convolutional neural network.

[0025] Optionally, the process of obtaining the blue sky image to be classified specifically includes:

[0026] Obtain an image taken with a blue sky as the background;

[0027] Cut the image to obtain a blue sky background image;

[0028] Adjust the searched blue sky background image to a standard resolution image;

[0029] Apply color normalization technology to the standard resolution image to obtain a standard image;

[0030] Extract the average value of the blue channel of the standard image to obtain a blue channel value, and use the blue channel value as the blue sky image to be classified.

[0031] Optionally, the training process of the blue sky level classification model specifically includes:

[0032] According to multiple sets of historical sky information data of the target area in a preset time period, apply multiple machine learning algorithms to obtain the first - layer classification results output by each machine learning algorithm;

[0033] Judge the accuracy of the first - layer classification results output by each machine learning algorithm according to the corresponding blue sky level;

[0034] When the accuracy is greater than or equal to a preset accuracy threshold, retain the first - layer classification results of the corresponding machine learning algorithm;

[0035] When the accuracy is less than a preset accuracy threshold, delete the first-layer classification result of the corresponding machine learning algorithm.

[0036] Using the retained first-layer classification results as inputs and the corresponding historical true classification result sample sets of blue sky levels as outputs, train the meta-learner to obtain a trained meta-learner, and connect the output ends of the multiple machine learning algorithms to the input end of the trained meta-learner to obtain a trained blue sky level classification model.

[0037] Optionally, the multiple machine learning algorithms include multi-layer perceptron, random forest, support vector machine, extreme gradient boosting, and light gradient boosting.

[0038] Optionally, use 10-fold cross-validation to verify the blue sky level classification model.

[0039] A computer device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the blue sky classification method based on image recognition and machine learning described in any one of the above.

[0040] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0041] The present invention discloses a blue sky classification method and device based on image recognition and machine learning. By obtaining current sky information data, where the sky information data includes blue sky images, surface meteorological observation data, ERA5 data, and surface air quality observation data; calculating a classification score index of the current sky information data, then, according to the classification score index, applying a clustering algorithm to determine the primary blue sky level corresponding to the current sky information data, and applying the primary blue sky level to label the blue sky image to obtain a labeled blue sky image, and finally, inputting the labeled blue sky image, other sky information data, and the corresponding MEIC emission inventory into a blue sky level classification model for training to obtain a final blue sky level classification model. The present invention realizes different blue sky level determination technologies through images combined with multi-source data, which helps to quantify and evaluate the achievements of the blue sky defense war. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0043] Figure 1Schematic diagram of the blue sky classification method module based on image recognition and machine learning provided by the present invention;

[0044] Figure 2 Schematic diagram of the historical original blue sky sample image set taken for the present invention;

[0045] Figure 3 Schematic diagram of the blue sky background image obtained by cutting the images in the historical original blue sky sample image set of the present invention;

[0046] Figure 4 Schematic diagram of the probability density distribution of the blue channel of the image generated from the blue sky background image of the present invention;

[0047] Figure 5 Schematic diagram of the blue sky classification method process based on image recognition and machine learning provided in Embodiment 1 of the present invention;

[0048] Figure 6 Internal structure diagram of a computer device. Specific implementation manners

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] The purpose of the present invention is to provide a blue sky classification method and device based on image recognition and machine learning, which realizes different blue sky level determination technologies through images combined with multi-source data, and helps to quantify and evaluate the achievements of the blue sky defense war.

[0051] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0052] Embodiment 1

[0053] As Figure 5 shown, the blue sky classification method based on image recognition and machine learning in this embodiment includes:

[0054] S1: Obtain multiple groups of historical data of sky information in a target area within a preset time period; each group of the historical data of sky information includes historical data of blue sky images and surface meteorological observation historical data, ERA5 historical data, surface air quality observation historical data, and MEIC emission inventory historical data that match the historical data of blue sky images.

[0055] S2: Calculate the classification score index for each group of the historical sky information data.

[0056] Specifically, apply the formula to calculate the classification score index for each group of the historical sky information data.

[0057] where Q is the classification score index; B is the average value of the blue channel of the image; M is the average value of all meteorological variables; P is the pollutant concentration; B min is the minimum value of the blue channel value; B max is the maximum value of the blue channel value; M min is the minimum value of the meteorological data; M max is the maximum value of the meteorological data; P min is the minimum value of the pollutant concentration; P max is the maximum value of the pollutant concentration.

[0058] S3: According to the classification score index, apply a clustering algorithm to determine the blue sky level corresponding to each group of the historical sky information data in the target area during the preset time period.

[0059] S3 specifically includes:

[0060] S31: Judge whether it is raining for the weather corresponding to each group of the historical sky information data in the target area during the preset time period;

[0061] S32: If it is raining, the blue sky level is non-blue sky;

[0062] S33: If it is not raining, according to the classification score index, apply a clustering algorithm to determine the primary blue sky level corresponding to the current sky information data.

[0063] Among them, the clustering algorithm includes K-means, Hierarchical, GMM, and DBSCAN.

[0064] S4: Use the multi-group of the historical sky information data in the target area during the preset time period as the input and the corresponding blue sky level as the output to train the blue sky level classification model, and obtain the trained blue sky level classification model; the blue sky level classification model includes a machine learning algorithm and a meta-learner. Among them, the meta-learner is a convolutional neural network.

[0065] S5: Input the to-be-classified blue sky image in the target area during the preset time period, as well as the surface meteorological observation data, ERA5 data, surface air quality observation data, and MEIC emission inventory data matching the to-be-classified blue sky image into the trained blue sky level classification model to obtain the blue sky level of the to-be-classified blue sky image.

[0066] Specifically, the multiple machine learning algorithms include multi-layer perceptron, random forest, support vector machine, extreme gradient boosting, and light gradient boosting. The blue sky level classification model is verified using 10-fold cross-validation.

[0067] The process of obtaining the blue sky image to be classified specifically includes:

[0068] (1) Obtain the captured image with the blue sky as the background.

[0069] (2) Cut the image to obtain the blue sky background image.

[0070] (3) Adjust the searched blue sky background image to a standard resolution image.

[0071] (4) Apply color normalization technology to the standard resolution image to obtain a standard image.

[0072] (5) Extract the average value of the blue channel of the standard image to obtain the blue channel value, and use the blue channel value as the blue sky image to be classified.

[0073] As a specific implementation manner, the training process of the blue sky level classification model specifically includes:

[0074] Step 1: According to multiple groups of historical sky information data of the target area in a preset time period, apply multiple machine learning algorithms to obtain the first-layer classification results output by each machine learning algorithm.

[0075] Step 2: Judge the accuracy of the first-layer classification results output by each machine learning algorithm according to the corresponding blue sky level.

[0076] Step 3: When the accuracy is greater than or equal to the preset accuracy threshold, retain the first-layer classification results of the corresponding machine learning algorithm.

[0077] Step 4: When the accuracy is less than the preset accuracy threshold, delete the first-layer classification results of the corresponding machine learning algorithm.

[0078] Step 5: Use the retained first-layer classification results as the input and the corresponding historical blue sky level true classification result sample set as the output to train the meta-learner to obtain the trained meta-learner, and connect the output ends of the multiple machine learning algorithms to the input end of the trained meta-learner to obtain the trained blue sky level classification model.

[0079] In practical applications, the present invention mainly utilizes open-source meteorological and air quality data, combines the captured image information with it, and applies image recognition technology and deep learning technology to the recognition of the blue sky level, making it possible to intuitively determine the blue sky level. It mainly includes the following main bodies / modules: data acquisition and preprocessing module, historical blue sky level determination module, and blue sky level recognition module. The connection relationships between the various main bodies are as Figure 1 shown.

[0080] Among them, the work content of the data acquisition and preprocessing module mainly includes:

[0081] 1. Image data collection: Collect a large number of original captured blue sky images, determine the latitude and longitude information, time information of each image, and the machine attributes of the captured photos as labels, and file them according to the captured device. The present invention obtains the latitude and longitude information of each image to determine the scope of the target area, and obtains the time information to determine the scope of the preset time period.

[0082] 2. Other data collection: Use web crawler technology to collect meteorological and pollution data corresponding to the latitude and longitude and corresponding time of each image. The hourly dataset of the Chinese ground from the National Meteorological Information Center and the hourly air quality data from the China National Environmental Monitoring Centre will be used, combined with the ERA5 atmospheric reanalysis data from the European Centre for Medium-Range Weather Forecasts, to comprehensively consider the influencing factors of the blue sky.

[0083] In addition, the monthly-scale multi-scale emission inventory model (MEIC) from 2013 to 2020 provided by the Chinese multi-scale emission inventory model is also adopted. The considered species include SO2, NOx, CO, NMVOC, NH3, PM2.5, PM10, BC, OC, with a resolution of 0.25°×0.25°. NCEP FNL provides the WRF-Chem meteorological boundary field and initial conditions, and MOZART-4 provides the chemical initial boundary field. As shown in Table 1 and Table 2.

[0084] Table 1 Statistical table of ground observation data required for this project

[0085]

[0086] Table 2 Statistical table of ERA5 reanalysis data required

[0087]

[0088]

[0089] 3. Image preprocessing: First, divide all the obtained pictures into multiple folders according to the shooting time, set the label of the pictures as the shooting time, and establish classification folders for easy later reading, as Figure 2 shown.

[0090] To avoid interference from non-sky elements such as ground buildings and vegetation in the classification of blue sky levels, only the blue sky part in the pictures is retained in the images, and all pictures need to be image-cut. As Figure 3 shown.

[0091] Then, use Python to adjust all images to a standard resolution with unified pixels and unify the picture sizes for later comparative analysis. At the same time, to reduce the influence of external factors such as the image capture device model and light changes on the images, color normalization technology is applied. All images are converted into a standardized RGB format database IMGdata through cv2.imread to ensure the consistency of the values of each channel at different time periods.

[0092] 4. Color channel processing and blue channel separation: In an RGB image, the value of the blue channel is most directly related to the degree of blue sky. The present invention extracts the average value of the blue channel of the picture information as the main basis for classification. For a single image, the mean value of the blue channel is taken as the representation of the image by reading IMGdata[:,:,2] through Python. Generate the probability density distribution of the blue channels of all images to represent the main blue degree changes and clarify the eigenvalue distribution of different blue sky levels. As Figure 4 shown.

[0093] 5. Data matching and quality control: After the preprocessing of the image data, the processed image data is matched and stored in chronological order with the meteorological observation and air quality data shown in Table 1 and Table 2. Specifically, the image data and the meteorological and observation data at the same time are stored in the same array and arranged in chronological order. All data are normalized to eliminate the dimensional differences between different physical quantities, so that meteorology, pollutant concentration, and image color information can be comprehensively processed to ensure the consistency of the model input.

[0094] The main work content of building a blue sky level classification model mainly includes: constructing a classification scoring index. Define the Q index (Quality Index, abbreviated as Q value) as the comprehensive index for classifying each image, integrating the blue channel value, meteorological data, and air quality data of the image, and reflecting the blue sky level of each image with a unified weight. The construction of the Q value is based on the following formula:

[0095]

[0096] where: B is the average value of the blue channel of the picture, representing the clarity of the blue sky in the image; M is the average value of all meteorological variables (such as the comprehensive index of temperature, humidity, wind speed, etc.), used to quantify the influence of meteorological conditions on the clarity of the blue sky; P is the pollutant concentration (the weighted value of PM2.5 and PM10), representing the influence of air pollution on the clarity of the blue sky; Bmin , B max is the minimum and maximum values of the blue channel value; M min , M max is the minimum and maximum values of the meteorological data; P min , P max is the minimum and maximum values of the pollutant concentration.

[0097] In this formula, all values are normalized so that data between different physical quantities participate in the calculation under the same dimension. The obtained Q value is distributed between 0 and 1. The higher the Q value, the better the blue sky degree, that is, closer to the "deep blue sky"; a lower Q value indicates a poor blue sky degree, which is in the "non-blue sky" state. In particular, if you want to increase the weight of a certain part, you can adjust the ratio of each part. For example, if you want to increase the weight of the image information, set its weight to 1 / 2, and set the parts of meteorology and air quality to 1 / 4 respectively.

[0098] The work content of the historical blue sky level determination module mainly includes:

[0099] 1. Blue sky classification label:

[0100] Before classification, first divide the Q index into rainy days and non-rainy days according to the precipitation information of each day. Rainy days are directly classified into the worst level, representing a state of non-blue sky. Then, use various classification algorithms such as the Gaussian mixture model (GMM) for the Q index of the remaining non-rainy days for preliminary unsupervised classification. The classification results will divide the corresponding dates into one of four levels according to the Q value, thereby generating blue sky level labels for all existing images.

[0101] 2. Result verification and evaluation model:

[0102] To ensure the accuracy and stability of the recognition results of the classification algorithm, the present invention integrates multi-source methods, defines a blue sky classification determination model, and verifies the classification results. This model includes three parts: trend deviation analysis, time correlation evaluation, and variance analysis. For the stability results of the blue sky level, the following are the detailed verification steps:

[0103]

[0104] Among them: Q score is the comprehensive verification score. The closer the value is to 1, the more reliable the classification result of the model; D, R, and S are the trend deviation score, time correlation score, and statistical significance score respectively.

[0105] (1) D is the trend deviation analysis. This score is standardized using an exponential decay function. The higher the score, the more the change trend of the variable conforms to the expectation.

[0106] Trend deviation analysis is used to evaluate whether the change trends of visual and air quality-related variables for each blue sky level in the classification results conform to expectations. For each blue sky level, the median trend of the main visual factors (low cloud amount, total cloud amount, visibility, PM2.5 and PM10 concentrations) changing with the blue sky level is analyzed.

[0107] The trend deviation score is calculated to quantify the degree of deviation between the actual change trend of the variable and the expected trend. The formula for the trend deviation score is as follows:

[0108]

[0109] Where: D is the trend deviation score, ranging from 0 to 1. The closer the score is to 1, the more reasonable the classification trend; ΔX i represents the median difference of the visual factor between adjacent blue sky levels; n is the number of visual factors participating in the analysis.

[0110] (2) Temporal correlation assessment. By calculating the Pearson correlation coefficient, the daily correlation between the blue sky level classification results and visual and air quality factors is evaluated. For a specific factor (such as visibility), there is a negative correlation with the blue sky level. Therefore, its correlation coefficient is reversed so that the results can be uniformly evaluated.

[0111] The formula for calculating the temporal correlation score is as follows:

[0112]

[0113] Where: R is the temporal correlation score, ranging from 0 to 1. The closer the score is to 1, the stronger the correlation between the classification result and the visual factor; r i is the Pearson correlation coefficient of each visual factor; n is the total number of visual factors.

[0114] After standardizing and taking the absolute value of the correlation scores of each factor and then averaging them, the higher the obtained temporal correlation score, the more reasonable the classification result.

[0115] (3) Analysis of variance (ANOVA). To test the significant differences between different blue sky levels, the present invention uses the analysis of variance (ANOVA) method to calculate the between-group variance and within-group variance among each blue sky level.

[0116] The statistical score of ANOVA is obtained through the following statistical test formula:

[0117]

[0118] Where: F is the F-test value, reflecting the ratio of the between-group variance to the within-group variance; MS between is the between-group mean square variance among the blue sky levels; MS withinIt is the within-group mean square deviation within the blue sky level.

[0119] When the F-test value reaches the statistical significance level (usually set at 0.05), it indicates that at least one blue sky level is significantly different from other levels. Further standardize the significance p-value and convert it into a statistical significance score according to the following formula:

[0120] S = 1 – p;

[0121] Where: S is the statistical significance score, and the closer the value is to 1, the more significant the difference; p is the p-value of the ANOVA test.

[0122] Through this multi-dimensional result verification method, the rationality and stability of the blue sky level classification model can be fully evaluated, ensuring that the classification results are highly scientific and consistent in terms of trends, correlations, and significance. If the verification score is poor (less than 0.5), then adjust the adopted meteorological and air quality data and the weights of the Q index until the best score, that is, the combination with the final score closest to 1 is found. Q score Verifies the image results of the classification according to the Q index. By adjusting the adopted meteorological and air quality data and the weights of the Q index until the best score, that is, the combination with the final score closest to 1 is found.

[0123] The main work content of the blue sky level recognition module mainly includes: based on the obtained historical blue sky level database, a future blue sky level recognition model will be established.

[0124] Based on the obtained historical blue sky classification results, add blue sky level labels (simplified to 1, 2, 3, 4, corresponding to non-blue sky to deep blue sky) and corresponding meteorological observations, meteorological reanalysis, and air quality monitoring data to each image except for time information. At the same time, add the MEIC emission inventory to provide auxiliary pollution emission spatial information, and extract relevant time features (day, month). The formed dataset needs to remove samples containing missing values. Divide the dataset into a training set of 70%, a validation set of 20%, and a test set of 10%. It is mainly completed using a stacked ensemble machine learning model. The input of the model is meteorological observations, meteorological reanalysis, air quality monitoring, and the MEIC emission inventory, and the training target is the blue sky level label corresponding to the time.

[0125] The constructed stacked integrated machine learning model consists of two structural layers. In the first layer, five machine learning algorithms are used to establish the blue sky classification model. The selected algorithms include Multi-Layer Perceptron (MLP), Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and LightGBM. After each model is trained and hyperparameter tuned respectively, the prediction performances of different models are compared. Since it is a classification problem, evaluation metrics such as accuracy, recall, and F1 score are used. Models with poor prediction performance (accuracy less than 80%) are deleted.

[0126] In the second layer, a Convolutional Neural Network (CNN) is used as the meta-learner. The prediction results of each model in the previous steps are used as inputs, and the true blue sky level is used as the output for training and optimization. 10-fold cross-validation is used to further verify the stability and reliability of the stacked integrated machine learning model and prevent overfitting. Finally, the test set is used to evaluate the final performance of the model.

[0127] Based on the trained stacked integrated machine learning model, the recognition of the blue sky level of any blue sky image can be achieved.

[0128] As a specific implementation, in addition to the mentioned clustering method, other clustering methods or simple threshold methods can be used to divide the blue sky level, and the divided blue sky levels can be revised according to requirements. The model structure and training data can be adjusted according to specific application requirements to adapt to different scenario needs.

[0129] When it is difficult to accurately capture the blue sky in the image, a simple cutting method can be adopted to only retain the blue sky module in the image that is not contaminated by other information, so as to reduce the interference of surrounding building reflections, sunlight reflections on the lens, etc. on the image.

[0130] The present invention has the following technical effects:

[0131] 1. The present invention realizes the automatic recognition of the blue sky level, divides the blue sky into specific levels, has high-precision classification ability, and the integrated learning model of the system is superior to a single algorithm in terms of accuracy and stability. Through the image-driven monitoring method of this system, low-cost environmental monitoring can be achieved, with a wide range of applications, and it is expected to become a supplementary means for air quality monitoring.

[0132] 2. A comprehensive index Q for blue sky image classification is defined: integrating the blue channel value, meteorological data, and air quality data of the image to reflect the blue sky level of each image on a unified scale.

[0133] 3. Based on the historical blue sky level determination method of images, machine learning algorithms are applied to classify the blue sky information of images and score the results.

[0134] 4. Image-based blue sky level recognition method, which realizes automatic classification of any blue sky level through a stacked model combined with XGBoost, AdaBoost, and logistic regression.

[0135] Based on image recognition technology, deep learning technology, and multi-source data, the present invention can accurately identify the blue sky level. By using deep learning algorithms and combining the captured image information, meteorological observations, and air quality data, it realizes the function of automatically determining the blue sky level (classified as non-blue, light blue, medium blue, and dark blue). It helps to quantify the progress of achieving the goal of "Beautiful China" and the implementation intensity and effectiveness of the "Blue Sky Defense War", and provides blue sky forecast information as part of meteorological forecasts.

[0136] Embodiment 2

[0137] A computer device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the computer program to implement the blue sky classification method based on image recognition and machine learning in Embodiment 1.

[0138] Embodiment 3

[0139] A computer-readable storage medium stores a computer program, which when executed by a processor, implements the blue sky classification method based on image recognition and machine learning in Embodiment 1.

[0140] Embodiment 4

[0141] A computer program product includes a computer program, which when executed by a processor, implements the blue sky classification method based on image recognition and machine learning in Embodiment 1.

[0142] Embodiment 5

[0143] A computer device, which can be a database, and its internal structure diagram can be as Figure 6As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store transactions to be processed. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements the blue sky classification method based on image recognition and machine learning in Embodiment 1.

[0144] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0145] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided by the present invention can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided by the present invention can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0146] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0147] Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A blue sky classification method based on image recognition and machine learning, characterized in that: The method comprises: Acquire multiple groups of sky information historical data of the target area in a preset time period; each group of the sky information historical data includes blue sky image historical data and surface meteorological observation historical data, ERA5 historical data, surface air quality observation historical data and MEIC emission inventory historical data matching the blue sky image historical data; Calculating a classification scoring index for each set of the sky information historical data; According to the classification scoring index, a clustering algorithm is applied to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period; Taking multiple groups of historical sky information data of the target area in a preset time period as input and the corresponding blue sky level as output, a blue sky level classification model is trained to obtain a trained blue sky level classification model; the blue sky level classification model includes a machine learning algorithm and a meta-learner; Inputting the blue sky image to be classified of the target area in a preset time period and the surface meteorological observation data, ERA5 data, surface air quality observation data and MEIC emission inventory data matching the blue sky image to be classified into the trained blue sky grade classification model to obtain the blue sky grade of the blue sky image to be classified; The training process of the blue sky level classification model specifically includes: Applying multiple machine learning algorithms according to multiple sets of historical sky information data of the target area in a preset time period to obtain a first-level classification result output by each machine learning algorithm; Determining the accuracy of the first-level classification results output by each machine learning algorithm according to the corresponding blue sky level; When the accuracy is greater than or equal to a preset accuracy threshold, retaining the first-layer classification result of the corresponding machine learning algorithm; When the accuracy is less than a preset accuracy threshold, deleting the first-layer classification results of the corresponding machine learning algorithm; The meta-learner is trained with the retained first-layer classification results as input and the corresponding historical blue sky level true classification result sample set as output to obtain a trained meta-learner, and the output ends of the multiple machine learning algorithms are connected to the input ends of the trained meta-learner to obtain a trained blue sky level classification model.

2. The blue sky classification method based on image recognition and machine learning according to claim 1 is characterized in that: The calculating of the classification scoring index of each group of the sky information historical data comprises: Apply the formula Calculating a classification scoring index for each set of the sky information historical data; Among them, Q is the classification score index; B is the average value of the blue channel of the image; M is the average value of all meteorological variables; P is the pollutant concentration; B min is the minimum value of the blue channel; B max is the maximum value of the blue channel; M min is the minimum value of meteorological data; M max is the maximum value of meteorological data; P min is the minimum value of pollutant concentration; P max is the maximum value of the pollutant concentration.

3. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: According to the classification scoring index, a clustering algorithm is applied to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period, specifically including: Determine whether the weather corresponding to each set of sky information historical data in the target area during a preset time period is raining; If it is raining, the blue sky level is non-blue sky; If it is not raining, a clustering algorithm is applied according to the classification scoring index to determine the primary blue sky level corresponding to the current sky information historical data.

4. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The clustering algorithms include K-means, Hierarchical, GMM and DBSCAN.

5. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The meta-learner is a convolutional neural network.

6. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The process of acquiring the blue sky image to be classified specifically includes: Get an image taken with a blue sky as the background; Cutting the image to obtain a blue sky background image; Adjust the search blue sky background image to a standard resolution image; Applying a color standardization technique to the standard resolution image to obtain a standard image; An average value of the blue channel of the standard image is extracted to obtain a blue channel value, and the blue channel value is used as the blue sky image to be classified.

7. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: Multiple machine learning algorithms including Multilayer Perceptron, Random Forest, Support Vector Machine, Extreme Gradient Boosting, and Lightweight Gradient Boosting.

8. The blue sky classification method based on image recognition and machine learning according to claim 6, characterized in that: A 10-fold crossover was applied to verify the blue sky grade classification model.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the blue sky classification method based on image recognition and machine learning as described in any one of claims 1-8.

Images