Blue sky classification method and device based on image recognition and machine learning

Abstract

The invention discloses a blue sky classification method and device based on image recognition and machine learning, and relates to the field of image processing and machine learning. The method comprises the following steps: acquiring multiple groups of sky information historical data of a target area in a preset time period; calculating a classification scoring index of each group of sky information historical data; according to the classification scoring index, determining a corresponding blue sky grade by applying a clustering algorithm; training a blue sky grade classification model by taking multiple groups of sky information historical data of the target area in a preset time period as input and taking a corresponding blue sky grade as output; and inputting the to-be-classified blue sky image and the matched surface meteorological observation data, ERA5 data, surface air quality observation data and MEIC emission list data into the trained blue sky grade classification model to obtain the blue sky grade of the to-be-classified blue sky image. According to the invention, different blue sky grade determination technologies can be realized through combination of images and multi-source data.

Description

Technical Field

[0001] The present invention relates to the field of image processing and machine learning, and in particular to a blue sky classification method and device based on image recognition and machine learning. Background Art

[0002] In recent years, the increase in the number of blue skies has become an important indicator for improving the quality of life. The appearance of blue skies is mainly caused by physical mechanisms such as Rayleigh scattering and Mie scattering, and is closely related to the concentration of aerosol particles in the air and meteorological conditions. Shorter wavelength blue light makes the sky appear blue due to a stronger scattering effect. However, when a large number of fine particles are suspended in the air, Mie scattering increases, causing the sky to become gray and form haze. Therefore, the appearance of blue sky is usually associated with low cloud cover, high visibility and low pollutant concentrations. At the same time, Rayleigh scattering and Mie scattering work together on the brightness and color saturation of the sky, giving the blue sky different degrees of expression. However, there is still a lack of criteria for determining the level of blue sky and automated recognition technology.

[0003] Images are the most direct reflection of the blue sky. The development of image recognition and artificial intelligence has made it possible to automate the classification of blue sky levels. Advances in panoramic camera technology, smartphone photography, and deep learning methods have enabled image-based blue sky recognition systems to quickly and intuitively classify blue sky levels while reducing costs, providing feasibility for real-time monitoring.

[0004] Existing technology, patent number ZL202010226425.X, uses long-term meteorological observation data to realize the blue sky determination technology, but this technology can only determine whether it is yes or no, and cannot determine the level of blue sky. It is known that the color of the blue sky varies, corresponding to different levels of blue sky and air pollution conditions, which is difficult to directly quantify through this method.

[0005] Existing technologies only rely on meteorological observation data and have a simple structure, but the public's perception of blue sky is mainly based on vision, and the impact of air quality needs to be considered. The difference in blue sky color can only be reflected through photos, not through meteorological observations, so existing technologies cannot determine the level of blue sky.

[0006] Other technologies only study the identification of air quality, while the level of blue sky is not only affected by air quality, but also by meteorological conditions. For example, on rainy or foggy days, there may be a good air quality index, but it is not a blue sky. Therefore, it is impossible to directly determine the level of blue sky by the change of air quality. The technical identification methods of the two are fundamentally different.

[0007] In summary, the existing technologies all require the use of expensive meteorological monitoring equipment, and the monitoring range is limited, making it impossible to achieve large-scale data collection and real-time monitoring. Summary of the invention

[0008] The purpose of the present invention is to provide a blue sky classification method and device based on image recognition and machine learning, which realizes different blue sky level determination technology by combining images with multi-source data, and helps to quantify and evaluate the results of the blue sky protection campaign.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A blue sky classification method based on image recognition and machine learning, the method comprising:

[0011] Acquire multiple groups of sky information historical data of the target area in a preset time period; each group of the sky information historical data includes blue sky image historical data and surface meteorological observation historical data, ERA5 historical data, surface air quality observation historical data and MEIC emission inventory historical data matching the blue sky image historical data;

[0012] Calculating a classification scoring index for each set of the sky information historical data;

[0013] According to the classification scoring index, a clustering algorithm is applied to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period;

[0014] Taking multiple groups of historical sky information data of the target area in a preset time period as input and the corresponding blue sky level as output, a blue sky level classification model is trained to obtain a trained blue sky level classification model; the blue sky level classification model includes a machine learning algorithm and a meta-learner;

[0015] The blue sky image to be classified in the target area in a preset time period and the surface meteorological observation data, ERA5 data, surface air quality observation data and MEIC emission inventory data matching the blue sky image to be classified are input into the trained blue sky grade classification model to obtain the blue sky grade of the blue sky image to be classified.

[0016] Optionally, the calculating of the classification scoring index of each group of the sky information historical data includes:

[0017] Apply the formula Calculating a classification scoring index for each set of the sky information historical data;

[0018] Among them, Q is the classification score index; B is the average value of the blue channel of the image; M is the average value of all meteorological variables; P is the pollutant concentration; B min is the minimum value of the blue channel; B max is the maximum value of the blue channel; M min is the minimum value of meteorological data; M max is the maximum value of meteorological data; Pmin is the minimum value of pollutant concentration; P max is the maximum value of the pollutant concentration.

[0019] Optionally, applying a clustering algorithm according to the classification scoring index to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period specifically includes:

[0020] Determine whether the weather corresponding to each set of sky information historical data in the target area during a preset time period is raining;

[0021] If it is raining, the blue sky level is non-blue sky;

[0022] If it is not raining, a clustering algorithm is applied according to the classification scoring index to determine the primary blue sky level corresponding to the current sky information data.

[0023] Optionally, the clustering algorithms include K-means, Hierarchical, GMM and DBSCAN.

[0024] Optionally, the meta-learner is a convolutional neural network.

[0025] Optionally, the process of acquiring the blue sky image to be classified specifically includes:

[0026] Get an image taken with a blue sky as the background;

[0027] Cutting the image to obtain a blue sky background image;

[0028] Adjust the search blue sky background image to a standard resolution image;

[0029] Applying a color standardization technique to the standard resolution image to obtain a standard image;

[0030] An average value of the blue channel of the standard image is extracted to obtain a blue channel value, and the blue channel value is used as the blue sky image to be classified.

[0031] Optionally, the training process of the blue sky level classification model specifically includes:

[0032] Applying multiple machine learning algorithms according to multiple sets of historical sky information data of the target area in a preset time period to obtain a first-level classification result output by each machine learning algorithm;

[0033] Determining the accuracy of the first-level classification results output by each machine learning algorithm according to the corresponding blue sky level;

[0034] When the accuracy is greater than or equal to a preset accuracy threshold, retaining the first-layer classification result of the corresponding machine learning algorithm;

[0035] When the accuracy is less than a preset accuracy threshold, deleting the first-layer classification results of the corresponding machine learning algorithm;

[0036] The meta-learner is trained with the retained first-layer classification results as input and the corresponding historical blue sky level true classification result sample set as output to obtain a trained meta-learner, and the output ends of the multiple machine learning algorithms are connected to the input ends of the trained meta-learner to obtain a trained blue sky level classification model.

[0037] Optionally, the multiple machine learning algorithms include multi-layer perceptron, random forest, support vector machine, extreme gradient boosting and lightweight gradient boosting.

[0038] Optionally, a 10-fold crossover is applied to verify the blue sky grade classification model.

[0039] A computer device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned blue sky classification methods based on image recognition and machine learning.

[0040] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0041] The present invention discloses a blue sky classification method and device based on image recognition and machine learning, which obtains current sky information data, wherein the sky information data includes blue sky images, surface meteorological observation data, ERA5 data and surface air quality observation data; calculates the classification score index of the current sky information data, and then, according to the classification score index, applies a clustering algorithm to determine the primary blue sky grade corresponding to the current sky information data, and applies the primary blue sky grade to mark the blue sky image to obtain the marked blue sky image, and finally, inputs the marked blue sky image, other sky information data and the corresponding MEIC emission inventory into the blue sky grade classification model for training to obtain the final blue sky grade classification model. The present invention realizes different blue sky grade determination technology by combining images with multi-source data, which is helpful to quantify and evaluate the results of the blue sky defense war. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1A schematic diagram of a blue sky classification method module based on image recognition and machine learning provided by the present invention;

[0044] Figure 2 Schematic diagram of a set of historical original blue sky sample images taken for the present invention;

[0045] Figure 3 A schematic diagram of a blue sky background image obtained by cutting images in a historical original blue sky sample image set according to the present invention;

[0046] Figure 4 A schematic diagram of the probability density distribution of the blue channel of an image generated for a blue sky background image of the present invention;

[0047] Figure 5 A schematic diagram of a blue sky classification method based on image recognition and machine learning provided in Example 1 of the present invention;

[0048] Figure 6 This is a diagram of the internal structure of a computer device. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The purpose of the present invention is to provide a blue sky classification method and device based on image recognition and machine learning, which realizes different blue sky level determination technology by combining images with multi-source data, and helps to quantify and evaluate the results of the blue sky protection campaign.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example 1

[0053] like Figure 5 As shown, the blue sky classification method based on image recognition and machine learning in this embodiment includes:

[0054] S1: Acquire multiple groups of sky information historical data of the target area in a preset time period; each group of the sky information historical data includes blue sky image historical data and surface meteorological observation historical data, ERA5 historical data, surface air quality observation historical data and MEIC emission inventory historical data matching the blue sky image historical data.

[0055] S2: Calculate the classification score index of each group of the sky information historical data.

[0056] Specifically, applying the formula Calculate a classification score index for each group of the sky information historical data.

[0057] Among them, Q is the classification score index; B is the average value of the blue channel of the image; M is the average value of all meteorological variables; P is the pollutant concentration; B min is the minimum value of the blue channel; B max is the maximum value of the blue channel; M min is the minimum value of meteorological data; M max is the maximum value of meteorological data; P min is the minimum value of pollutant concentration; P max is the maximum value of the pollutant concentration.

[0058] S3: According to the classification scoring index, a clustering algorithm is applied to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period.

[0059] S3 specifically includes:

[0060] S31: Determine whether the weather corresponding to each set of sky information historical data in the target area during a preset time period is raining;

[0061] S32: if it is raining, the blue sky level is non-blue sky;

[0062] S33: If it is not raining, a clustering algorithm is applied according to the classification scoring index to determine the primary blue sky level corresponding to the current sky information data.

[0063] The clustering algorithms include K-means, Hierarchical, GMM and DBSCAN.

[0064] S4: Taking multiple groups of historical sky information data of the target area in a preset time period as input and the corresponding blue sky level as output, a blue sky level classification model is trained to obtain a trained blue sky level classification model; the blue sky level classification model includes a machine learning algorithm and a meta-learner. The meta-learner is a convolutional neural network.

[0065] S5: Input the blue sky image to be classified of the target area in a preset time period and the surface meteorological observation data, ERA5 data, surface air quality observation data and MEIC emission inventory data matching the blue sky image to be classified into the trained blue sky grade classification model to obtain the blue sky grade of the blue sky image to be classified.

[0066] Specifically, multiple machine learning algorithms include multi-layer perceptron, random forest, support vector machine, extreme gradient boosting and lightweight gradient boosting. A 10-fold crossover is applied to verify the blue sky grade classification model.

[0067] The process of acquiring the blue sky image to be classified specifically includes:

[0068] (1) Obtain an image taken with a blue sky as the background.

[0069] (2) Cut the image to obtain a blue sky background image.

[0070] (3) Adjust the search blue sky background image to a standard resolution image.

[0071] (4) Applying color standardization technology to the standard resolution image to obtain a standard image.

[0072] (5) Extracting the average value of the blue channel of the standard image to obtain a blue channel value, and using the blue channel value as the blue sky image to be classified.

[0073] As a specific implementation method, the training process of the blue sky level classification model specifically includes:

[0074] Step 1: Based on multiple sets of historical sky information data of the target area in a preset time period, multiple machine learning algorithms are applied to obtain the first-level classification results output by each machine learning algorithm.

[0075] Step 2: Determine the accuracy of the first-level classification results output by each machine learning algorithm based on the corresponding blue sky level.

[0076] Step 3: When the accuracy is greater than or equal to a preset accuracy threshold, retain the first-layer classification results of the corresponding machine learning algorithm.

[0077] Step 4: When the accuracy is less than a preset accuracy threshold, the first-layer classification results of the corresponding machine learning algorithm are deleted.

[0078] Step 5: Take the retained first-layer classification results as input and the corresponding historical blue sky level true classification result sample set as output, train the meta-learner to obtain a trained meta-learner, connect the output ends of the multiple machine learning algorithms with the input ends of the trained meta-learner to obtain a trained blue sky level classification model.

[0079] In practical applications, the present invention mainly uses open source meteorological and air quality data, combines the captured image information with it, and uses image recognition technology and deep learning technology to apply it to the identification of blue sky levels, making it possible to intuitively determine the blue sky level. It mainly includes the following subjects / modules: data acquisition and preprocessing module, historical blue sky level determination module, blue sky level identification module, and the connection relationship between each subject is as follows: Figure 1 shown.

[0080] Among them, the work content of the data acquisition and preprocessing module mainly includes:

[0081] 1. Image data collection: Collect a large number of original blue sky images, determine the latitude and longitude information and time information of each image and the machine attributes of the photo as tags, and archive them according to the shooting device. The present invention obtains the latitude and longitude information of each image to determine the scope of the target area, and obtains the time information to determine the scope of the preset time period.

[0082] 2. Other data collection: Use crawler technology to collect meteorological and pollution data corresponding to the latitude and longitude of each image and the corresponding time. The hourly ground data set from the National Meteorological Information Center and the hourly air quality data from the China National Environmental Monitoring Center will be used, combined with the ERA5 atmospheric reanalysis data from the European Center for Medium-Range Weather Forecasts, to comprehensively consider the factors affecting blue skies.

[0083] In addition, the monthly scale Multiscale Emission Inventory Model of China (MEIC) for 2013-2020 was used, and the species considered included SO 2 , NOx, CO, NMVOC, NH3, PM2.5, PM10, BC, OC, with a resolution of 0.25°×0.25°. NCEP FNL provides WRF-Chem meteorological boundary fields and initial conditions, and MOZART-4 provides chemical initial boundary fields. As shown in Tables 1 and 2.

[0084] Table 1 Statistics of ground observation data required for this project

[0085]

[0086] Table 2 Statistical table of required ERA5 reanalysis data

[0087]

[0088]

[0089] 3. Image preprocessing: First, all the images are divided into multiple folders according to the shooting time, and the image labels are set to the shooting time. Classification folders are established for easy later reading, such as Figure 2shown.

[0090] In order to avoid the interference of non-sky elements such as ground buildings and vegetation on the blue sky classification, the image only retains the blue sky part of the picture, and all pictures need to be cut. Figure 3 shown.

[0091] Then, Python was used to adjust all images to a standard resolution of uniform pixels and unify the image size for later comparison and analysis. At the same time, in order to reduce the impact of external factors such as image capture equipment model and lighting changes on the image, color standardization technology was applied, and all images were converted into a standardized RGB format database IMGdata through cv2.imread to ensure that the value of each channel is consistent in different time periods.

[0092] 4. Color channel processing and blue channel separation: In RGB images, the value of the blue channel is most directly related to the degree of blue sky. The present invention extracts the average value of the blue channel of the image information as the main basis for classification. Through Python, IMGdata[:,:,2] is read. For a single image, the mean value of the blue channel is taken as the representation of the image. The probability density distribution of the blue channel of all images is generated to characterize the main blue degree changes and clarify the characteristic value distribution of different blue sky levels. Figure 4 shown.

[0093] 5. Data matching and quality control: After the image data is preprocessed, the processed image data is matched and stored in chronological order with the meteorological observation and air quality data shown in Tables 1 and 2. Specifically, the image data and the meteorological and observation data at the same time are stored in the same array in chronological order, and all data are normalized to eliminate the dimensional differences between different physical quantities, so that the meteorological, pollutant concentration, and image color information can be processed comprehensively to ensure the consistency of the model input.

[0094] The work content of constructing the blue sky classification model mainly includes: construction of classification scoring index. The Q index (Quality Index, referred to as Q value) is defined as a comprehensive indicator for the classification of each image, integrating the blue channel value, meteorological data and air quality data of the image, and reflecting the blue sky level of each image with a unified weight. The construction of the Q value is based on the following formula:

[0095]

[0096] Where: B is the average value of the blue channel of the image, representing the clarity of the blue sky in the image; M is the average value of all meteorological variables (such as comprehensive indicators of temperature, humidity, wind speed, etc.), which is used to quantify the impact of meteorological conditions on the clarity of the blue sky; P is the pollutant concentration (weighted values ​​of PM2.5 and PM10), representing the impact of air pollution on the clarity of the blue sky; B min , B max is the minimum and maximum value of the blue channel; M min 、M max is the minimum and maximum value of meteorological data; P min , P max are the minimum and maximum values ​​of pollutant concentration.

[0097] In this formula, all values ​​are standardized so that data between different physical quantities are calculated under the same dimension, and the resulting Q value is distributed between 0 and 1. The higher the Q value, the better the blue sky, that is, closer to the "deep blue sky"; the lower the Q value, the worse the blue sky, which is a "non-blue sky" state. In particular, if you want to increase the weight of a certain part, you can adjust the ratio of each part. For example, if you want to increase the weight of image information, set its weight to 1 / 2, and set the meteorological and air quality parts to 1 / 4 respectively.

[0098] The work content of the historical blue sky level determination module mainly includes:

[0099] 1. Blue Sky Classification Label:

[0100] Before classification, the Q index is first divided into rainy days and non-rainy days based on the daily precipitation information. Rainy days are directly classified as the worst level, representing a non-blue sky condition. Then, the Q index of the remaining non-rainy days is used for preliminary unsupervised classification using multiple classification algorithms such as the Gaussian mixture model (GMM). The classification results will be divided into one of four levels according to the Q value, thereby generating blue sky level labels corresponding to all existing images.

[0101] 2. Result verification and evaluation model:

[0102] In order to ensure the accuracy and stability of the identification results of the classification algorithm, this paper integrates multi-source methods, defines a blue sky classification judgment model, and verifies the classification results. The model includes three parts: trend deviation analysis, time correlation evaluation, and variance analysis. For the stability results of the blue sky level, the following are detailed verification steps:

[0103]

[0104] Where: Q scoreis the comprehensive validation score. The closer the value is to 1, the more reliable the classification result of the model is. D, R, and S are the trend deviation score, time correlation score, and statistical significance score, respectively.

[0105] (1) D is the trend deviation analysis. This score is standardized using an exponential decay function. A higher score means that the trend of the variable is more in line with expectations.

[0106] Trend deviation analysis is used to evaluate whether the trend of visual and air quality related variables for each blue sky level in the classification results is in line with expectations. For each blue sky level, the median trend of the main visual factors (low cloud cover, total cloud cover, visibility, PM2.5 and PM10 concentrations) is analyzed as the blue sky level changes.

[0107] The trend deviation score is calculated to quantify the degree to which the actual change trend of the variable deviates from the expected trend. The formula for the trend deviation score is as follows:

[0108]

[0109] Where: D is the trend deviation score, ranging from 0 to 1. The closer the score is to 1, the more reasonable the classification trend is; ΔX i It represents the median difference of visual factors between adjacent blue sky levels; n is the number of visual Δ factors involved in the analysis.

[0110] (2) Temporal correlation assessment. The daytime correlation between the blue sky classification results and visual and air quality factors was assessed by calculating the Pearson correlation coefficient. Certain factors (such as visibility) were negatively correlated with the blue sky level, so their correlation coefficients were reversed so that the results could be evaluated uniformly.

[0111] The temporal relevance score is calculated as follows:

[0112]

[0113] Where: R is the temporal correlation score, ranging from 0 to 1. The closer the score is to 1, the stronger the correlation between the classification result and the visual factor; r i is the Pearson correlation coefficient of each visual factor; n is the total number of visual factors.

[0114] The correlation scores of each factor are standardized and averaged after taking the absolute values. The higher the time correlation score obtained, the more reasonable the classification result is.

[0115] (3) Analysis of variance (ANOVA): To test the significant differences between different blue sky levels, the present invention uses the analysis of variance (ANOVA) method to calculate the inter-group variance and intra-group variance between each blue sky level.

[0116] The statistical score of ANOVA is obtained by the following statistical test formula:

[0117]

[0118] Among them: F is the F test value, reflecting the ratio of the variance between groups to the variance within groups; MS between is the mean square error between groups of blue sky levels; MS within is the within-group mean square error within the blue sky level.

[0119] When the F test value reaches the statistically significant level (generally set at 0.05), it indicates that at least one blue sky level is significantly different from the other levels. The significant p value is further standardized and converted into a statistically significant score according to the following formula:

[0120] S = 1–p;

[0121] Where: S is the statistically significant score, the closer the value is to 1, the more significant the difference is; p is the p value of the ANOVA test.

[0122] This multi-dimensional result verification method can fully evaluate the rationality and stability of the blue sky classification model, ensuring that the classification results are highly scientific and consistent in terms of trends, correlations, and significance. If the verification score is poor (less than 0.5), the weight of the Q index is adjusted by adjusting the meteorological and air quality data used until the best score is found, that is, the combination with the final score closest to 1. score The image classification results are verified after classification based on the Q index. The weight of the Q index is adjusted by adjusting the meteorological and air quality data used until the best score is found, that is, the combination with the final score closest to 1.

[0123] The work content of the blue sky grade identification module mainly includes: based on the historical blue sky grade database, a future blue sky grade identification model will be established.

[0124] Based on the historical blue sky classification results, a blue sky level label (simplified to 1, 2, 3, 4, corresponding to non-blue sky to deep blue sky) and corresponding meteorological observation, meteorological reanalysis and air quality monitoring data are added to each image in addition to the time information. At the same time, the MEIC emission inventory is added to provide auxiliary pollution emission spatial information and extract relevant time features (day, month). The constructed data set needs to remove samples containing missing values. The data set is divided into 70% training set, 20% validation set and 10% test set. It is mainly completed using a stacked integrated machine learning model. The input of the model is meteorological observation, meteorological reanalysis, air quality monitoring and MEIC emission inventory, and the training target is the blue sky level label of the corresponding time.

[0125] The constructed stacked integrated machine learning model consists of two structural layers. In the first layer, five machine learning algorithms are used to establish a blue sky classification model. The selected algorithms include multi-layer perceptron (MLP), random forest (RF), support vector machine (SVM), extreme gradient boosting (XGBoost) and lightweight gradient boosting machine (LightGBM). After each model is trained and hyperparameter tuned, the prediction performance of different models is compared. Since it is a classification problem, the evaluation indicators use accuracy, recall rate, F1 score, etc. Delete the model with poor prediction performance (accuracy less than 80%).

[0126] In the second layer, a convolutional neural network (CNN) is used as a meta-learner. The prediction results of each model in the previous step are used as input, and the actual blue sky level is used as output for training and optimization. A 10-fold cross validation is used to further verify the stability and reliability of the stacked ensemble machine learning model to prevent overfitting. Finally, the test set is used to evaluate the final performance of the model.

[0127] Based on the trained stacked integrated machine learning model, the blue sky level of any blue sky image can be recognized.

[0128] As a specific implementation method, in addition to the clustering method mentioned above, other clustering methods or simple threshold methods are used to classify the blue sky level, and the blue sky classification level can be revised according to needs. The model structure and training data can be adjusted according to specific application requirements to adapt to different scene requirements.

[0129] When the blue sky in the image is difficult to capture accurately, a simple cutting method can be used to retain only the blue sky modules in the image that are not contaminated by other information, so as to reduce interference with the image caused by reflections from surrounding buildings, reflections of sunlight on the lens, etc.

[0130] The present invention has the following technical effects:

[0131] 1. The present invention realizes the automatic recognition of blue sky levels, divides blue sky into specific levels, and has high-precision classification capabilities. The system's integrated learning model is superior to a single algorithm in terms of accuracy and stability. The image-driven monitoring method of the system can achieve low-cost environmental monitoring, has a wide range of applications, and is expected to become a supplementary means of air quality monitoring.

[0132] 2. Defined a comprehensive index Q for blue sky image classification: It integrates the blue channel value of the image, meteorological data, and air quality data to reflect the blue sky level of each image on a unified scale.

[0133] 3. Based on the historical blue sky level determination method of images, a machine learning algorithm is applied to grade the blue sky information of the image and score the results.

[0134] 4. An image-based blue sky level recognition method realizes automatic classification of any blue sky level through a stacked model composed of XGBoost, AdaBoost and logistic regression.

[0135] Based on image recognition technology, deep learning technology and multi-source data, the present invention can accurately identify the level of blue sky. By using deep learning algorithms, combined with captured image information, meteorological observations and air quality data, the function of automatically determining the level of blue sky (classified as non-blue, light blue, medium blue and dark blue) can be realized. It helps to quantify the progress of the "Beautiful China" goal and the implementation strength and effectiveness of the "Blue Sky Defense War", and provides blue sky forecast information as part of the weather forecast.

[0136] Example 2

[0137] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the blue sky classification method based on image recognition and machine learning in Example 1.

[0138] Example 3

[0139] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the blue sky classification method based on image recognition and machine learning in Example 1.

[0140] Example 4

[0141] A computer program product includes a computer program, which, when executed by a processor, implements the blue sky classification method based on image recognition and machine learning in Example 1.

[0142] Example 5

[0143] A computer device, which may be a database, may have an internal structure as shown in Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the blue sky classification method based on image recognition and machine learning in Example 1 is implemented.

[0144] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A blue sky classification method based on image recognition and machine learning, characterized in that: The method comprises: Acquire multiple groups of sky information historical data of the target area in a preset time period; each group of the sky information historical data includes blue sky image historical data and surface meteorological observation historical data, ERA5 historical data, surface air quality observation historical data and MEIC emission inventory historical data matching the blue sky image historical data; Calculating a classification scoring index for each set of the sky information historical data; According to the classification scoring index, a clustering algorithm is applied to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period; Taking multiple groups of historical sky information data of the target area in a preset time period as input and the corresponding blue sky level as output, a blue sky level classification model is trained to obtain a trained blue sky level classification model; the blue sky level classification model includes a machine learning algorithm and a meta-learner; The blue sky image to be classified in the target area in a preset time period and the surface meteorological observation data, ERA5 data, surface air quality observation data and MEIC emission inventory data matching the blue sky image to be classified are input into the trained blue sky grade classification model to obtain the blue sky grade of the blue sky image to be classified.

2. The blue sky classification method based on image recognition and machine learning according to claim 1 is characterized in that: The calculating of the classification scoring index of each group of the sky information historical data comprises: Apply the formula Calculating a classification scoring index for each set of the sky information historical data; Among them, Q is the classification score index; B is the average value of the blue channel of the image; M is the average value of all meteorological variables; P is the pollutant concentration; B min is the minimum value of the blue channel; B max is the maximum value of the blue channel; M min is the minimum value of meteorological data; M max is the maximum value of meteorological data; P min is the minimum value of pollutant concentration; P max is the maximum value of the pollutant concentration.

3. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The step of applying a clustering algorithm according to the classification scoring index to determine the blue sky level corresponding to each set of sky information historical data of the target area in a preset time period specifically includes: Determine whether the weather corresponding to each set of sky information historical data in the target area during a preset time period is raining; If it is raining, the blue sky level is non-blue sky; If it is not raining, a clustering algorithm is applied according to the classification scoring index to determine the primary blue sky level corresponding to the current sky information data.

4. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The clustering algorithms include K-means, Hierarchical, GMM and DBSCAN.

5. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The meta-learner is a convolutional neural network.

6. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The process of acquiring the blue sky image to be classified specifically includes: Get an image taken with a blue sky as the background; Cutting the image to obtain a blue sky background image; Adjust the search blue sky background image to a standard resolution image; Applying a color standardization technique to the standard resolution image to obtain a standard image; An average value of the blue channel of the standard image is extracted to obtain a blue channel value, and the blue channel value is used as the blue sky image to be classified.

7. The blue sky classification method based on image recognition and machine learning according to claim 1, characterized in that: The training process of the blue sky level classification model specifically includes: Applying multiple machine learning algorithms according to multiple sets of historical sky information data of the target area in a preset time period to obtain a first-level classification result output by each machine learning algorithm; Determining the accuracy of the first-level classification results output by each machine learning algorithm according to the corresponding blue sky level; When the accuracy is greater than or equal to a preset accuracy threshold, retaining the first-layer classification result of the corresponding machine learning algorithm; When the accuracy is less than a preset accuracy threshold, deleting the first-layer classification results of the corresponding machine learning algorithm; The meta-learner is trained with the retained first-layer classification results as input and the corresponding historical blue sky level true classification result sample set as output to obtain a trained meta-learner, and the output ends of the multiple machine learning algorithms are connected to the input ends of the trained meta-learner to obtain a trained blue sky level classification model.

8. The blue sky classification method based on image recognition and machine learning according to claim 7, characterized in that: The multiple machine learning algorithms include multi-layer perceptron, random forest, support vector machine, extreme gradient boosting and lightweight gradient boosting.

9. The blue sky classification method based on image recognition and machine learning according to claim 6, characterized in that: A 10-fold crossover was applied to validate the trained blue sky grade classification model.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the blue sky classification method based on image recognition and machine learning as described in any one of claims 1-9.

Images