Atmospheric monitoring big data analysis system

Abstract

The invention, which belongs to the technical field of atmospheric monitoring, discloses an atmospheric monitoring big data analysis system comprising a demand identification module and a demand analysis module. The demand identification module is used for carrying out real-time demand analysis on the atmospheric monitoring data to obtain a corresponding demand detail list; the demand analysis module is used for carrying out calibration analysis on the demand detail list to obtain application calibration data of a corresponding application demand, supplementing the application calibration data into the demand detail list, and marking the current demand detail list as a demand analysis list; displaying the demand analysis table to a corresponding manager; through mutual cooperation between the demand identification module and the demand analysis module, intelligent analysis of atmospheric monitoring data is realized. The challenges of climate change and continuous change of meteorological monitoring requirements in the urbanization process can be effectively handled; with the continuous evolution of different commercial technologies on atmospheric monitoring data application requirements, the limitation problem of an existing atmospheric monitoring system is broken through.

Description

Technical Field

[0001] The present invention belongs to the technical field of atmospheric monitoring, and specifically relates to a big data analysis system for atmospheric monitoring. Background Art

[0002] With the acceleration of climate change and urbanization, the importance of meteorological monitoring data has become increasingly prominent. Meteorological data is not only related to the basic needs of the public's daily travel, agricultural production, energy management, etc., but also plays a crucial role in commercial decision-making, disaster warning, environmental protection and other fields. However, the current meteorological monitoring system still faces many challenges in meeting these dynamically changing needs; especially with the development of technology, the application requirements for atmospheric monitoring data by different commercial technologies are constantly changing; while the monitoring parameters of the existing atmospheric monitoring system are usually fixed in the system and are difficult to be expanded according to actual needs. This results in that the system may not be able to provide effective monitoring support when facing monitoring needs; it has a great lag; the needs of different users for atmospheric monitoring data vary, but the existing atmospheric monitoring system often has difficulty in meeting these personalized needs; however, the atmospheric monitoring data cannot fully meet the application needs of various users.

[0003] Based on this, in order to solve the above problems, the present invention provides a big data analysis system for atmospheric monitoring. Summary of the Invention

[0004] In order to solve the problems existing in the above solutions, the present invention provides a big data analysis system for atmospheric monitoring.

[0005] The object of the present invention can be achieved by the following technical solutions: A big data analysis system for atmospheric monitoring, including a demand identification module and a demand analysis module; The demand identification module is used to perform real-time demand analysis on atmospheric monitoring data to obtain a corresponding demand list, and the demand list is used to count application requirements and demand description information corresponding to the application requirements; the application requirement is the requirement for having an application for atmospheric monitoring data.

[0006] Further, the demand list is updated in real time.

[0007] Further, the determination of the application requirement includes: Setting demand units, and obtaining potential demands through the demand units in real time; screening the potential demands to determine the application requirements.

[0008] Further, screening the potential demands includes: Establishing an initial screening model, and the expression of the initial screening model is: ; Where: q is the potential demand. The initial screening requirements are that it does not repeat with the potential demand and the application demand corresponding to this potential demand is not counted in the demand details list; the output data is the initial screening value CP(q), and the initial screening value is 1 or 0; Obtain the demand details list, analyze the potential demand according to the initial screening model and the demand details list to obtain the initial screening value of the potential demand; eliminate the potential demand with an initial screening value of 0; Conduct a demand assessment on the remaining potential demands to determine whether the potential demands meet the application criteria; mark the potential demands that meet the application criteria as application demands.

[0009] Furthermore, conducting a demand assessment on the remaining potential demands includes: Establish a demand assessment model, and the expression of the demand assessment model is: ; Where: (p, Q) is the input data, p represents the demand characteristics of the corresponding potential demand; Q is the application criteria; p→Q means that the demand characteristics of the corresponding potential demand meet the application criteria; the output data is the demand assessment value QP(p, Q), and the demand assessment value is 1 or 0; Obtain the application criteria, extract the demand characteristics of the potential demand according to the application criteria to obtain the demand characteristics of the potential demand; Analyze the application criteria and the demand characteristics of the potential demand through the demand assessment model to obtain the demand assessment value of the potential demand; When the demand assessment value is 1, determine that the potential demand meets the application criteria; When the demand assessment value is 0, determine that the potential demand does not meet the application criteria.

[0010] Furthermore, establish a demand blacklist according to the screening records of the potential demand, and conduct priority screening on the potential demand according to the demand blacklist; The demand blacklist is divided into two parts. The first part is the potential demand that is statistically determined not to meet the initial screening requirements, and the second part is the potential demand that is statistically determined not to meet the application criteria; Calibrate the application criteria for the potential demands in the second part according to the preset time plan, determine whether the potential demands meet the application criteria, and update the demand blacklist and the demand details list according to the judgment results.

[0011] The demand analysis module is used to conduct calibration analysis on the demand details list to obtain the application calibration data of the corresponding application demand, supplement the application calibration data to the demand details list, mark the current demand details list as the demand analysis table; display the demand analysis table to the corresponding management personnel.

[0012] Further, calibration analysis is performed on the demand breakdown schedule, including: Establish a calibration analysis library for storing atmospheric monitoring simulation data; Real-time identify each application requirement corresponding to the demand breakdown schedule, perform calibration analysis on the application requirements based on the calibration analysis library, and determine whether the atmospheric monitoring data in the monitoring area can meet the application requirements; When it is determined that the application requirements cannot be met, the calibration result is that the application requirements are not met, identify the corresponding cause data, and integrate the calibration result and the cause data into calibration data; When it is determined that the application requirements are met, the calibration result is that the application requirements are met, evaluate the corresponding application satisfaction rate, and integrate the calibration result and the application satisfaction rate into calibration data.

[0013] Further, the evaluation of the application satisfaction rate includes: Perform simulations based on the atmospheric monitoring simulation data stored in the calibration analysis library, obtain the influence deviation of the atmospheric monitoring simulation data quality on the application requirements during each simulation process, and mark it as a single quality influence value; calculate the data quality influence value based on each single quality influence value; Calculate the corresponding application satisfaction rate according to the satisfaction rate calculation formula, and the satisfaction rate calculation formula is: ; In the formula: ML is the application satisfaction rate; δ is the data quality influence value.

[0014] Further, calculating the data quality influence value according to each single quality influence value includes: Set a representative calculation formula, substitute each single quality influence value into the representative calculation formula for calculation, and obtain the corresponding data quality influence value.

[0015] Further, the representative calculation formula is: ; In the formula: δ is the data quality influence value; DZ i represents the corresponding single quality influence value, i = 1, 2,..., n, and n is the number of single quality influence values.

[0016] Further, the representative calculation formula is: ; In the formula: DZ´ is the combined single quality influence; DZ1 and DZ2 are the two single quality influence values for combined analysis respectively; A1 and A2 are the representative numbers corresponding to the respective single influence values, and the representative number is equal to the number of times the single influence value is combined plus one; Optionally, substitute any two single quality impact values into the representative value formula to calculate the combined single quality impact value; and so on until only one single quality impact value remains, and mark the single quality impact value as the quality impact value.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: Through the mutual cooperation between the demand recognition module and the demand analysis module, intelligent analysis of atmospheric monitoring data is realized; it can effectively cope with the challenges of the continuously changing meteorological monitoring requirements in the face of climate change and urbanization. With the continuous evolution of the application requirements of different commercial technologies for atmospheric monitoring data, the present system breaks through the limitations of the existing atmospheric monitoring system. When facing various emerging monitoring requirements, the system can quickly respond and provide effective monitoring support in a timely manner, greatly reducing the lag of the system and ensuring that the meteorological monitoring data is always synchronized with the actual application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a block diagram of the principle of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The following will clearly and completely describe the technical solutions of the present invention in combination with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0021] As Figure 1 shown, an atmospheric monitoring big data analysis system includes a demand recognition module and a demand analysis module; The demand recognition module is used to perform real-time demand analysis, identify various application requirements with a demand for atmospheric monitoring data, generate a demand detailed list according to the obtained application requirements, and the demand detailed list is used to count each application requirement and the demand description information that the corresponding staff wants to know, such as which fields and uses the application requirement is for; and the demand detailed list is updated in real time according to the update of the application requirement.

[0022] In one embodiment, the determination of the application requirement can be determined in real time through various existing methods such as big data analysis and manual summary of the currently available application requirements.

[0023] In one embodiment, the determination of application requirements includes: Set up a requirements unit for public display. The public can upload various utilization requirements for atmospheric monitoring data through the requirements unit, which are marked as potential requirements. Obtain the existing potential requirements in real time through the requirements unit. For example, perform feature recognition on the requirement data uploaded by the public to obtain corresponding potential requirements. Screen the potential requirements to determine the application requirements.

[0024] In one embodiment, screening of potential requirements can be performed based on existing screening methods.

[0025] In one embodiment, screening of potential requirements includes: Establish an initial screening model, and the expression of the initial screening model is: ; In the formula: q is the potential requirement. The initial screening requirement is that it does not repeat with the potential requirement and the application requirement corresponding to the potential requirement is not counted in the requirement details list. Since the potential requirements are analyzed one by one, for many repeated potential requirements, only one analyzed potential requirement will be retained. The output data is the initial screening value CP(q), and the initial screening value is 1 or 0.

[0026] Obtain the requirement details list. For the case where the requirement details list has not been established, the application requirements not counted in the requirement details list.

[0027] Analyze the potential requirements according to the initial screening model and the requirement details list to obtain the initial screening values of the corresponding potential requirements. Eliminate the potential requirements with an initial screening value of 0.

[0028] Conduct requirement assessment on the remaining potential requirements to determine whether the potential requirements meet the application criteria. Mark the potential requirements that meet the application criteria as application requirements.

[0029] In one embodiment, when conducting requirement assessment on the remaining potential requirements to determine whether the potential requirements meet the application criteria, starting from the perspective of the necessity of the potential requirement. For the public application direction, determine whether the potential requirement is necessary according to the requirements of the corresponding department, that is, without considering the economic perspective, make a judgment according to the requirements of the corresponding department. For the commercial application direction, it can be judged from potential economic perspectives such as the estimated domain output value of the potential requirement, etc., and preset corresponding criteria such as output value for judgment, or it can also be judged from the scope, quantity, scale, etc. of the commercial application audience. When its audience scope, scale, etc. reach the preset requirements, it is considered to meet the application criteria. Based on the above description, make a judgment in combination with various technologies.

[0030] Exemplarily, a demand assessment model is established, and the expression of the demand assessment model is: ; In the formula: (p, Q) is the input data, p represents the demand characteristics of the corresponding potential demand; Q is the application standard; p→Q means that the demand characteristics of the corresponding potential demand meet the application standard; the output data is the demand assessment value QP(p, Q), and the demand assessment value is 1 or 0; Obtain the application standard, and the application standard is set by the management personnel according to the actual needs, and is set specifically according to the above content; extract the characteristics of the corresponding potential demand according to the application standard, and obtain the demand characteristics of the corresponding potential demand, that is, collect the characteristics according to the data that needs to be evaluated according to the application standard; Analyze the corresponding application standard and the demand characteristics of the potential demand through the demand assessment model to obtain the demand assessment value of the corresponding potential demand; When the demand assessment value is 1, it is judged that the corresponding potential demand meets the application standard; When the demand assessment value is 0, it is judged that the corresponding potential demand does not meet the application standard.

[0031] In one embodiment, a demand blacklist can be established according to the unqualified potential demands, and subsequent rapid screening can be carried out according to the demand blacklist; the demand blacklist includes two parts, one part is the potential demands that do not meet the initial screening requirements, and the other part is the potential demands that meet the initial screening requirements but do not meet the application standard; calibrate the application standard for the potential demands in the second part according to the preset time plan, judge whether the potential demands meet the application standard, and update the potential demands in the second part according to the evaluation results. The preset time plan is the setting requirement of the verification time, such as regular verification, real-time verification, verification at intervals of a preset period, etc.; prioritize the screening of potential demands according to the demand blacklist, that is, first screen according to the demand blacklist.

[0032] In one embodiment, demand assessment is performed on the remaining potential demands to judge whether the potential demands meet the application standard, and various current assessment technologies can also be applied for assessment, such as establishing an intelligent model based on deep learning technology for intelligent assessment.

[0033] The demand analysis module is used to calibrate and analyze the demand detail list, determine whether the atmospheric monitoring data in the current monitoring area meets the corresponding application requirements, obtain the application calibration data of the corresponding application requirements, and the monitoring area is the area where atmospheric monitoring needs to be carried out; supplement the application calibration data to the demand detail list, mark the current demand detail list as the demand analysis table; display the demand analysis table to the corresponding management personnel.

[0034] In one embodiment, calibrating and analyzing the demand detail list includes: Build a calibration analysis library, which is used to store atmospheric monitoring simulation data. The atmospheric simulation monitoring data is set according to the historical atmospheric monitoring data in the monitoring area, such as the historical atmospheric monitoring data under various recent weather conditions, to ensure the comprehensiveness and authenticity of the stored atmospheric monitoring simulation data; Identify each application requirement corresponding to the real-time requirement details list, and perform calibration analysis on the application requirement based on the calibration analysis library to determine whether the atmospheric monitoring data in the current monitoring area can meet the application requirement; When it is determined that the application requirement cannot be met, identify the reason data for non-application, that is, what causes the non-meeting of the application requirement; then the application calibration data is the calibration result of non-meeting the application requirement and the reason data for non-application. When it is determined that the application requirement is met, evaluate the corresponding application satisfaction rate, and the calibration data is the calibration result of meeting the application requirement and the application satisfaction rate.

[0035] In one embodiment, determining whether the atmospheric monitoring data in the monitoring area can meet the application requirement includes: Perform simulation analysis on the application requirement according to the calibration analysis library to obtain the corresponding simulation process data, and determine whether the atmospheric monitoring simulation data can meet the application requirement based on the simulation process data to obtain the corresponding calibration result.

[0036] In one embodiment, when performing simulation analysis, it is possible to identify the corresponding application background, application requirements, etc. when identifying that the public submits the corresponding application requirement, and then perform the corresponding simulation judgment.

[0037] In one embodiment, when performing simulation analysis, it is also possible to determine the application requirements that need to be met through manual or intelligent technology, and then perform simulation based on the atmospheric monitoring simulation data.

[0038] In one embodiment, when performing simulation analysis, it is possible to perform simulation analysis based on the existing simulation judgment method, such as using a deep learning algorithm to establish an intelligent simulation judgment model, and training it through a manual method to establish a corresponding training set. The training set includes input data and output data. The input data is the application requirement, and the output data is the calibration result.

[0039] In one embodiment, the evaluation of the application satisfaction rate includes: Simulate according to the atmospheric monitoring simulation data stored in the calibration analysis library, count the influence deviation of the quality of the atmospheric monitoring simulation data for each simulation on the application requirements according to the simulation results, compare the results with the true and accurate data of the corresponding atmospheric simulation data, determine the influence of the atmospheric monitoring data quality on the result of this influence requirement, count the influence deviation corresponding to each simulation, and mark it as a single quality influence value; the data quality refers to the relevant data such as the accuracy and timeliness of the atmospheric monitoring data. For example, due to the monitoring accuracy reason, there are errors in the atmospheric monitoring data, and due to the insufficient coverage rate of the monitoring points, the atmospheric monitoring data cannot accurately represent the accurate atmospheric conditions of the monitoring area; calculate the data quality influence value according to each single quality influence value; Calculate the corresponding application satisfaction rate according to the satisfaction rate calculation formula, and the satisfaction rate calculation formula is: ; In the formula: ML is the application satisfaction rate; δ is the data quality influence value.

[0040] Furthermore, calculate the data quality influence value according to each single quality influence value, including: Set a representative calculation formula, substitute each single quality influence value into the representative calculation formula for calculation, and obtain the corresponding data quality influence value.

[0041] Furthermore, the representative calculation formula is: ; In the formula: δ is the data quality influence value; DZ i represents the corresponding single quality influence value, i = 1, 2,..., n, and n is the number of single quality influence values.

[0042] Furthermore, the representative calculation formula is: ; In the formula: DZ´ is the combined single quality influence; DZ1 and DZ2 are respectively two single quality influence values for combined analysis; A1 and A2 are respectively the representative numbers corresponding to the corresponding single influence values, and the representative number is equal to the combined times corresponding to the single influence value plus one. For example, if the initial combined times are all 0, then the representative number is 1, and after combining once, its representative value is 2; Optionally select two single quality influence values and substitute them into the representative value formula to calculate the combined single quality influence value; and so on until only one single quality influence value remains, and mark this single quality influence value as the quality influence value.

[0043] In one embodiment, it is also possible to intelligently analyze the corresponding auxiliary improvement opinions based on the requirements analysis table to facilitate the improvement and adjustment of atmospheric monitoring.

[0044] The above formulas are all calculated by removing the dimension and taking the numerical value. The formula is obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained by simulating a large amount of data.

[0045] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An atmospheric monitoring big data analysis system, characterized in that, It includes a requirement identification module and a requirement analysis module; The requirement identification module is used to perform real-time requirement analysis on atmospheric monitoring data to obtain a corresponding requirement details list, and the requirement details list is used to count application requirements and requirement description information corresponding to the application requirements; the application requirements are the requirements for applying atmospheric monitoring data; The requirement analysis module is used to perform calibration analysis on the requirement details list to obtain application calibration data for corresponding application requirements, supplement the application calibration data to the requirement details list, mark the current requirement details list as a requirement analysis table; and display the requirement analysis table to corresponding management personnel.

2. The air quality monitoring big data analysis system according to claim 1, wherein Perform real-time update on the requirement details list.

3. The air quality monitoring big data analysis system according to claim 1, characterized in that, The determination of application requirements includes: Set requirement units, and obtain potential requirements through the requirement units in real time; screen the potential requirements to determine application requirements.

4. An air quality monitoring big data analysis system according to claim 3, characterized in that, The screening of potential requirements includes: Establish an initial screening model, and the expression of the initial screening model is: ； In the formula: q is the potential requirement, and the initial screening requirement is not to repeat with the potential requirement and there is no application requirement corresponding to the potential requirement statistically in the requirement details list; the output data is the initial screening value CP(q), and the initial screening value is 1 or 0; Obtain the requirement details list, analyze the potential requirements according to the initial screening model and the requirement details list to obtain the initial screening value of the potential requirements; eliminate the potential requirements with an initial screening value of 0; Perform requirement evaluation on the remaining potential requirements to determine whether the potential requirements meet the application criteria; mark the potential requirements that meet the application criteria as application requirements.

5. An air quality monitoring big data analysis system according to claim 4, characterized in that The requirement evaluation of the remaining potential requirements includes: Establish a requirement evaluation model, and the expression of the requirement evaluation model is: ； In the formula: (p, Q) is the input data, p represents the requirement characteristics of the corresponding potential requirement; Q is the application criteria; p→Q means that the requirement characteristics of the corresponding potential requirement meet the application criteria; the output data is the requirement evaluation value QP(p, Q), and the requirement evaluation value is 1 or 0; Obtain the application criteria, extract the requirement characteristics of the potential requirements according to the application criteria to obtain the requirement characteristics of the potential requirements; Analyze the application criteria and the requirement characteristics of the potential requirements through the requirement evaluation model to obtain the requirement evaluation value of the potential requirements; When the requirement evaluation value is 1, determine that the potential requirement meets the application criteria; When the requirement evaluation value is 0, determine that the potential requirement does not meet the application criteria.

6. An air quality monitoring big data analysis system according to claim 5, characterized in that Establish a requirement blacklist according to the screening records of potential requirements, and preferentially screen potential requirements according to the requirement blacklist; The requirement blacklist is divided into two parts. The first part is the potential requirements that do not meet the initial screening requirements statistically, and the second part is the potential requirements that do not meet the application criteria statistically; Perform application criteria calibration on the potential requirements in the second part according to the preset time plan, determine whether the potential requirements meet the application criteria, and update the requirement blacklist and the requirement details list according to the judgment results.

7. An air quality monitoring big data analysis system according to claim 1, characterized in that, The calibration analysis of the requirement details list includes: Establish a calibration analysis library, and the calibration analysis library is used to store atmospheric monitoring simulation data; Real-time identify each application requirement corresponding to the detailed list of requirements, and based on the calibration analysis library, conduct calibration analysis on the application requirements to determine whether the atmospheric monitoring data in the monitoring area can meet the application requirements; When it is judged that the application requirements cannot be met, the calibration result is that the application requirements are not met, identify the corresponding cause data, and integrate the calibration result and the cause data into calibration data; When it is judged that the application requirements are met, the calibration result is that the application requirements are met, evaluate the corresponding application satisfaction rate, and integrate the calibration result and the application satisfaction rate into calibration data.

8. An air quality monitoring big data analysis system according to claim 7, characterized in that The evaluation of the application satisfaction rate includes: Simulate according to the atmospheric monitoring simulation data stored in the calibration analysis library to obtain the influence deviation of the atmospheric monitoring simulation data quality on the application requirements during each simulation process, and mark it as a single quality influence value; calculate the data quality influence value according to each single quality influence value; Calculate the corresponding application satisfaction rate according to the satisfaction rate calculation formula. The satisfaction rate calculation formula is: ； In the formula: ML is the application satisfaction rate; δ is the data quality influence value.

9. An air quality monitoring big data analysis system according to claim 8, characterized in that, Calculating the data quality influence value according to each single quality influence value includes: Set a representative calculation formula, substitute each single quality influence value into the representative calculation formula for calculation, and obtain the corresponding data quality influence value.

10. An air quality monitoring big data analysis system according to claim 9, characterized in that, The representative calculation formula is: ； In the formula: DZ´ is the combined single quality influence; DZ1 and DZ2 are respectively two single quality influence values for combined analysis; A1 and A2 are respectively the representative numbers corresponding to the respective single influence values, and the representative number is equal to the number of times the single influence value is combined plus one; Optionally select two single quality influence values and substitute them into the representative value formula to calculate the combined single quality influence value; And so on until only one single quality influence value remains, and mark the single quality influence value as the quality influence value.

Images