Ecological environment monitoring quality management method and system based on big data processing

Abstract

The invention relates to the technical field of ecological environment supervision, in particular to an ecological environment monitoring quality management method and system based on big data processing, and the method comprises the steps: collecting sensor monitoring records of an environment management platform; determining the query activeness of the sensor monitoring record according to the association degree between the query times of the sensor monitoring record in a certain period and the query user in the period and the query confidence of the sensor monitoring record; determining a periodic fluctuation coefficient and the periodic fluctuation of the sensor monitoring record according to the record group and information of the sensor monitoring record; according to the fluctuating values monitored and recorded by the sensor and the number of the fluctuating values, the numerical value volatility and the quality early warning priority of the sensor monitoring and recording are determined; and performing quality early warning on the sensor monitoring records according to the quality early warning priorities of the sensor monitoring records. According to the invention, comprehensive evaluation can be carried out on the whole environment ecology, the development trend and existing problems of the environment ecology are monitored and evaluated, and effective quality management is carried out.

Description

Technical Field

[0001] This application relates to the technical field of ecological environment supervision, and particularly to an ecological environment monitoring quality management method and system based on big data processing. Background Art

[0002] An ecological environment monitoring system is a comprehensive technical system for monitoring and evaluating the quality of the ecological environment. It obtains data through sensors, monitoring devices, etc. installed in the ecological environment, and conducts real-time monitoring, analysis, and evaluation of environmental parameters through technical means such as data collection, processing, storage, and analysis.

[0003] For environmental monitoring, physical and chemical indicators and biological indicators are generally used for monitoring under normal circumstances. This local monitoring method only simply measures the harmful factors in the environment and cannot comprehensively evaluate the entire environment. Ecological environment monitoring, on the other hand, can comprehensively evaluate the entire environmental ecology, monitor and evaluate its development trend and existing problems. Existing ecological environment monitoring systems only store, analyze, and publish monitoring data and cannot effectively manage the quality. Summary of the Invention

[0004] To achieve the above object, this application provides the following technical solutions:

[0005] According to the first aspect of the present invention, the present invention claims an ecological environment monitoring quality management method based on big data processing, and the method includes:

[0006] Collect the sensor monitoring records of the environmental management platform;

[0007] Determine the query activity of the sensor monitoring records by the correlation between the number of queries of the sensor monitoring records within a certain period and the query users within that period, and the query confidence of the sensor monitoring records;

[0008] Determine the periodic fluctuation coefficient through the record group and information of the sensor monitoring records, where the record group and information include the type of the sensor monitoring records, the fluctuation degree of the sensor monitoring records, and the safety factor of the sensor monitoring records;

[0009] Determine the periodic volatility of the sensor monitoring records based on the periodic fluctuation coefficient and the query activity;

[0010] Determine the numerical volatility of the sensor monitoring records through the fluctuation values of the sensor monitoring records and the number of the fluctuation values, where the fluctuation values include steady-state monitoring values, peak-state monitoring values, and valley-state monitoring values;

[0011] Determine the quality warning priority of the sensor monitoring records through the ranking of the importance of the sensor monitoring records in the sensor importance, the periodic volatility, and the numerical volatility;

[0012] Perform quality warnings on sensor monitoring records according to the quality warning priorities recorded by the sensors.

[0013] Determine the query activity of sensor monitoring records through the correlation between the number of queries of sensor monitoring records within a certain period and the query users within that period, as well as the query confidence of sensor monitoring records, including:

[0014] Collect the number of queries of sensor monitoring records within a certain period and the real-time upload degree value of the environmental management platform within that period.

[0015] Determine the query confidence of sensor monitoring records based on the number of queries of sensor monitoring records within that period and the real-time upload degree value of the environmental management platform within that period.

[0016] Fit the number of queries of sensor monitoring records within that period and the query users within that period, and output the change broken line of the number of queries of sensor monitoring records and the correlation between the number of queries of sensor monitoring records within that period and the query users within that period.

[0017] Determine the query activity of sensor monitoring records based on the said correlation and the query confidence.

[0018] Determine the query confidence of sensor monitoring records through the following expression:

[0019]

[0020] In the formula, is the query confidence of the v-th sensor monitoring record, G is the number of queries of the v-th sensor monitoring record within that period, is the real-time upload degree value of the v-th sensor monitoring record at the m-th timestamp within that period, M is the number of timestamps within that period, m is the timestamp id within that period, and v is the record id of the sensor monitoring records.

[0021] Determine the query activity of sensor monitoring records through the following expression:

[0022]

[0023] In the formula, is the query activity of the v-th sensor monitoring record, is the correlation between the number of queries of the v-th sensor monitoring record within that period and the query users within that period, It is the query confidence level monitored and recorded by the v-th sensor in the n-th sub-cycle of this cycle. This cycle includes multiple equally divided sub-cycles. N is the number of sub-cycles in this cycle, and v is the number ID of the sensor monitoring records.

[0024] Further, determining the periodic fluctuation coefficient through the record groups and information monitored by the sensor includes:

[0025] Collect the record groups and information of the labeled sensor monitoring records, the periodic volatility marks corresponding to the labeled sensor monitoring records, and the initial clustering model. The initial clustering model is a five-layer fully connected deep learning model;

[0026] Train the initial clustering model according to the record groups and information of the labeled sensor monitoring records and the periodic volatility marks, and output the target clustering model;

[0027] Input the record groups and information of the sensor monitoring records into the target clustering model, and output the periodic fluctuation coefficient.

[0028] Further, through the following expression, determine the periodic volatility of the sensor monitoring records:

[0029]

[0030] In the formula, is the periodic volatility of the v-th sensor monitoring record, is the query activity of the v-th sensor monitoring record, is the periodic fluctuation coefficient of the v-th sensor monitoring record, and v is the number ID of the sensor monitoring records.

[0031] Further, through the following expression, determine the numerical volatility of the sensor monitoring records:

[0032]

[0033] In the formula, is the numerical volatility of the v-th sensor monitoring record, is the number of fluctuation values included in the v-th sensor monitoring record, is the smoothing function, and v is the number ID of the sensor monitoring records.

[0034] Further, through the following expression, determine the quality warning priority of the sensor monitoring records:

[0035]

[0036] In the formula, is the quality warning priority of the v-th sensor monitoring record, The numerical volatility of the monitoring record of the v-th sensor The periodic volatility of the monitoring record of the v-th sensor The importance of the monitoring record of the v-th sensor The id number of the importance of the monitoring record of the v-th sensor in the sensor importance sorted from high to low. The sorting of the sensor importance from high to low is the importance of atmospheric resources, the importance of water resources, the importance of biological resources, and the importance of mineral resources. The corresponding id numbers of the importance of atmospheric resources, water resources, biological resources, and mineral resources of the sensor monitoring record are 1, 2, 3, and 4 respectively.

[0037] According to the second aspect of the present invention, the present invention claims a quality management system for ecological environment monitoring based on big data processing, and the system includes:

[0038] One or more processors;

[0039] A quality early warning device, on which there are one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned quality management method for ecological environment monitoring based on big data processing.

[0040] The quality management method and system for ecological environment monitoring based on big data processing collect the sensor monitoring records of the environmental management platform; determine the query activity of the sensor monitoring records through the correlation between the number of queries of the sensor monitoring records within a certain period and the query users within that period, and the query confidence of the sensor monitoring records; determine the periodic fluctuation coefficient and the periodic volatility of the sensor monitoring records through the record group and information of the sensor monitoring records; determine the numerical volatility and the quality early warning priority of the sensor monitoring records through the fluctuation value and the number of fluctuation values of the sensor monitoring records; and perform quality early warning on the sensor monitoring records according to the quality early warning priority of the sensor monitoring records. The present invention can comprehensively evaluate the entire environmental ecology, monitor and evaluate its development trend and existing problems, and effectively manage the quality. Brief Description of the Drawings

[0041] Figure 1 It is a working flowchart of the quality management method for ecological environment monitoring based on big data processing claimed in the embodiment of the present application;

[0042] Figure 2 It is a structural module diagram of a quality management system for ecological environment monitoring based on big data processing claimed in the embodiment of the present application. Detailed Embodiments

[0043] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. According to the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0044] Next, the specific solution of the ecological environment monitoring quality management method provided by the present invention based on big data processing will be specifically described in conjunction with the accompanying drawings. Please refer to Figure 1 , the method includes the following steps:

[0045] 101, Collect the sensor monitoring records of the environmental management platform.

[0046] The sensor monitoring records of the environmental management platform are the objects of quality early warning. In some embodiments, the sensor monitoring records of the environmental management platform are collected through the following steps:

[0047] 1011, Deploy a lightweight sensor capture port on the environmental management platform, configure the sensor capture port, and make the sensor capture port monitor the specified sensor ecological environment or directory.

[0048] 1012, Real-time monitor the changes in the sensor ecological environment through the sensor capture port. When a new sensor is generated or an existing sensor is updated, capture the sensor monitoring records.

[0049] 1013, Conduct preliminary data filtering and formatting on the locally captured sensor monitoring records through the sensor capture port, and then use the monitoring cloud to temporarily store the sensor monitoring records to balance the load.

[0050] 1014, Integrate the sensor monitoring records in the monitoring cloud and transmit them through a secure transmission protocol.

[0051] Among the sensor monitoring records of the environmental management platform, different sensor monitoring records correspond to their respective sensor texts, and this information data may be frequently queried. The importance and irreplaceability of the sensor monitoring records that are frequently queried are relatively strong. In order to find the sensor monitoring records that are frequently queried, 102 is introduced.

[0052] 102, Determine the query activity of the sensor monitoring records based on the correlation between the number of queries of the sensor monitoring records within a certain period and the query users within that period, as well as the query confidence of the sensor monitoring records.

[0053] The query activity of the sensor monitoring records can reflect the importance and irreplaceability of the sensor monitoring records. Sensor monitoring records with high query activity often require priority quality early warning.

[0054] In some embodiments, 102 may include the following steps:

[0055] 1021, collect the number of queries of the sensor monitoring records in a certain period and the real-time upload degree value of the environmental management platform in this period.

[0056] The number of queries of the sensor monitoring records in this period can reflect the query situation of this sensor monitoring record in this period. The real-time upload degree value of the environmental management platform in this period includes the specific values of real-time upload at each timestamp of the sensor monitoring records in this period, so as to quantify the real-time upload situation of the sensor monitoring records in this period.

[0057] In some embodiments, the real-time upload degree value of the environmental management platform in this period can be collected from the communication status data of the environmental management platform.

[0058] 1022, determine the query confidence of the sensor monitoring records according to the number of queries of the sensor monitoring records in this period and the real-time upload degree value of the environmental management platform in this period.

[0059] The query confidence of the sensor monitoring records can comprehensively evaluate the query situation and real-time upload situation of the sensor monitoring records in this period. Only the sensor monitoring records with a high number of queries and a low real-time upload are more important.

[0060] In some embodiments, through the following expression, determine the query confidence of the sensor monitoring records to implement 1022:

[0061] ;

[0062] In the formula, is the query confidence of the v-th sensor monitoring record, comprehensively reflecting the query situation and real-time upload situation of the v-th sensor monitoring record in this period. G is the number of queries of the v-th sensor monitoring record in this period, reflecting the query situation of the v-th sensor monitoring record in this period. The larger G is, the higher the number of queries of this sensor monitoring record is. is the real-time upload degree value of the v-th sensor monitoring record at the m-th timestamp in this period, reflecting the real-time upload situation of the v-th sensor monitoring record in this period and this value is not zero. The smaller it is, the lower the real-time upload of this sensor monitoring record is. M is the number of timestamps in this period and its value is not zero. m is the timestamp id in this period. M is not zero. v is the id of the number of sensor monitoring records. The larger G is, the smaller it is. For those with a high number of queries and a low real-time upload, then The larger it is, it indicates that there are more queries in the monitoring records of this sensor under better communication conditions, that is, the query confidence of the monitoring records of this sensor is higher.

[0063] 1023, fit the number of queries of the sensor monitoring records in this period and the query users in this period, and output the change broken line of the number of queries of the sensor monitoring records and the correlation degree between the number of queries of the sensor monitoring records in this period and the query users in this period.

[0064] The query users in this period are the period points consistent with the period order in this period. Each query user in this period has the corresponding number of queries of the sensor monitoring records. There is a certain relationship between the number of queries of the sensor monitoring records in this period and the query users in this period. By fitting, the change broken line of the number of queries of the sensor monitoring records is output. The change broken line of the number of queries of the sensor monitoring records can show that there is a certain change relationship between the number of queries of the sensor monitoring records in this period and the query users in this period. And the correlation degree between the number of queries of the sensor monitoring records in this period and the query users in this period output can reflect the change trend of the number of queries of the sensor monitoring records in this period. This correlation degree can be reasonably selected as needed. In some embodiments, the correlation degree can be determined by the Pearson correlation degree between the period sequence of the number of queries on the change broken line of the number of queries and the period sequence of the query users, that is, the quotient value output after dividing the sum value of the numerical value 1 and the Pearson correlation degree by the numerical value 2 is used as this correlation degree.

[0065] 1024, determine the query activity of the sensor monitoring records according to the correlation degree and the query confidence, so as to reflect the importance and irreplaceability of the sensor monitoring records.

[0066] In some embodiments, through the following expression, determine the query activity of the sensor monitoring records to achieve 1024:

[0067] ;

[0068] In the formula, is the query activity of the v-th sensor monitoring record, reflecting the importance and irreplaceability of the sensor monitoring record, is the correlation degree between the number of queries of the v-th sensor monitoring record in this period and the query users in this period, The larger it is, the more the number of queries of the sensor monitoring records in this period shows a continuous upward trend, It is the query confidence of the v-th sensor monitoring record in the n-th sub-cycle of this period. This period includes multiple equally divided sub-cycles. The query confidence of the sub-cycle can be calculated through the expression of 1022. N is the number of sub-cycles in this period and its value is not zero. v is the number id of the sensor monitoring records.

[0069] It is the overall query confidence of the v-th sensor monitoring record in this period, which is calculated and output through the query confidences of multiple sub-cycles. It can make the calculation of the query confidence more accurate and better reflect the actual situation. The larger the larger, then the larger, it indicates that the number of queries of the sensor monitoring records in this period shows a continuous upward trend and the query confidence of the sensor monitoring records is higher. That is to say, it indicates that the query of the v-th sensor monitoring record in the environmental management platform is more frequent, that is the larger, the stronger the importance and irreplaceability of the sensor monitoring records. The sensor monitoring records that are queried more frequently usually have a higher quality warning priority.

[0070] However, relying solely on the frequent query situation is not enough to accurately explain the quality warning priority of the sensor monitoring records. The periodic fluctuation situation and numerical fluctuation situation of the sensor monitoring records also need to be focused on, so as to avoid some sensor monitoring records with infrequent queries but large fluctuations not being given priority quality warnings.

[0071] The sensor monitoring records with periodic fluctuations are the sensor monitoring records that need to be processed or analyzed in a timely manner. They are usually related to system performance, user experience or security events. And this type of sensor monitoring records often requires higher processing requirements and faster response cycles, and need to be given priority quality warnings.

[0072] 103. Determine the periodic fluctuation coefficient through the record group and information of the sensor monitoring records. The record group and information include the type of the sensor monitoring records, the fluctuation degree of the sensor monitoring records, and the safety factor of the sensor monitoring records.

[0073] Determining the periodic fluctuation coefficient is convenient for finding the sensor monitoring records that need to be processed or analyzed in a timely manner.

[0074] It should be noted that the type of the sensor monitoring records can reflect atmospheric resource events, water resource events, etc. Therefore, the type of the sensor monitoring records itself can also reflect the periodic fluctuation degree.

[0075] In some embodiments, 103 is specifically implemented in the following manner:

[0076] 1031. Collect the record group and information monitored and recorded by the annotation sensor, the periodic volatility marker corresponding to the annotation sensor monitoring record, and the initial clustering model. The initial clustering model is a five-layer fully connected deep learning model.

[0077] The record group and information of the annotation sensor monitoring record and the periodic volatility marker corresponding to the annotation sensor monitoring record are learning materials for training the initial clustering model. In some embodiments, the periodic volatility marker corresponding to the annotation sensor monitoring record can be output by manual judgment. The larger the value corresponding to the periodic volatility marker, the higher the periodic volatility of the sensor monitoring record.

[0078] In some embodiments, the record group and information of the annotation sensor monitoring record as the data set and the periodic volatility marker corresponding to the annotation sensor monitoring record are divided according to the training ratio. The ratio of the training set to the test set can be 7:3.

[0079] 1032. Train the initial clustering model based on the record group and information of the annotation sensor monitoring record and the periodic volatility marker, and output the target clustering model.

[0080] The target clustering model is a trained clustering model with accurate prediction, which can be used to judge the periodic fluctuation coefficient of the sensor monitoring record through the record group and information of the sensor monitoring record.

[0081] In some embodiments, the initial clustering model is trained by the gradient descent method until the loss function converges, and the training is completed.

[0082] 1033. Input the record group and information of the sensor monitoring record into the target clustering model, and output the periodic fluctuation coefficient.

[0083] The periodic fluctuation coefficient can reflect the strength of the periodic fluctuation degree.

[0084] 104. Determine the periodic volatility of the sensor monitoring record according to the periodic fluctuation coefficient and the query activity.

[0085] Comprehensively master the periodic volatility of the sensor monitoring record through the periodic fluctuation coefficient and the query activity.

[0086] In some embodiments, through the following expression, determine the periodic volatility of the sensor monitoring record to achieve 104:

[0087] ;

[0088] In the formula, is the periodic volatility of the v-th sensor monitoring record, comprehensively reflecting the fluctuation degree of the sensor monitoring record in the period, is the query activity of the v-th sensor monitoring record, It is the periodic fluctuation coefficient of the v-th sensor monitoring record, reflecting the fluctuation degree of the sensor monitoring record. v is the ID of the number of sensor monitoring records. The larger the larger, it indicates that the query activity of this sensor monitoring record is higher and the possible periodic fluctuation degree of this sensor monitoring record is higher. Then the larger, that is the larger, that is, the fluctuation degree of this sensor monitoring record in terms of period is stronger, and this sensor monitoring record is more likely to belong to the sensor monitoring records with high-quality early warning priority.

[0089] Sensor monitoring records not only have periodic fluctuations. Some sensor monitoring records may include some fluctuating values. Sensor monitoring records with fluctuating values require a higher quality early warning priority to ensure the security of the fluctuating values in the sensor monitoring records. Therefore, it is also necessary to pay attention to the numerical fluctuations of sensor monitoring records, and 105 is introduced.

[0090] 105. Determine the numerical fluctuations of the sensor monitoring records through the fluctuation values and the number of fluctuation values of the sensor monitoring records. The fluctuation values include steady-state monitoring values, peak-state monitoring values, and valley-state monitoring values.

[0091] In some embodiments, the numerical fluctuations of the sensor monitoring records are determined through the following expression to achieve 105:

[0092] ;

[0093] In the formula, is the numerical fluctuation of the v-th sensor monitoring record, reflecting the fluctuation degree of this sensor monitoring record in terms of value, is the number of fluctuation values included in the v-th sensor monitoring record. The larger this value, the more fluctuation values this sensor monitoring record includes, and then the stronger the fluctuation degree of this sensor monitoring record in terms of value, is a smoothing function, and v is the ID of the number of sensor monitoring records. The larger the larger, then the larger, that is, the stronger the fluctuation degree of this sensor monitoring record in terms of value, and a higher quality early warning priority is required to ensure the security of the fluctuating values in the sensor monitoring record.

[0094] After confirming the periodic fluctuations and numerical fluctuations of the sensor monitoring records, sensor monitoring records with higher fluctuations in terms of period or value can be found. In order to determine the order of the quality early warning of these sensor monitoring records, 106 is introduced.

[0095] 106. Determine the quality warning priority of the sensor monitoring record by ranking, periodic volatility, and numerical volatility of the importance of the sensor monitoring record recorded by the sensor in the importance of the sensor.

[0096] Since the monitoring records of different types of sensors have different degrees of importance, and a large number of sensor monitoring records need to be graded for quality warning during the real-time quality warning process to determine the quality warning priority of the sensor monitoring records, so as to more reasonably allocate quality warning resources, reduce the quality warning cost, and also reduce the system response cycle.

[0097] In some embodiments, the quality warning priority of the sensor monitoring record is determined through the following expression to achieve 106:

[0098] ;

[0099] In the formula, is the quality warning priority of the v-th sensor monitoring record, reflecting the priority importance of the quality warning of the v-th sensor monitoring record. is the numerical volatility of the v-th sensor monitoring record. is the periodic volatility of the v-th sensor monitoring record. is the importance of the v-th sensor monitoring record. is the id number of the importance of the v-th sensor monitoring record in the sensor importance ranked from high to low and the value is not zero. The smaller this id number, the higher the importance of the sensor monitoring record. The sensor importance is ranked from high to low as the importance of atmospheric resources, the importance of water resources, the importance of biological resources, and the importance of mineral resources. The id numbers corresponding to the importance of atmospheric resources, water resources, biological resources, and mineral resources of the sensor monitoring record are 1, 2, 3, and 4 respectively. The smaller, The larger, The larger, the Euclidean norm of the periodic volatility and numerical volatility of this sensor monitoring record. The larger, The larger, it means that the system response speed required for this sensor monitoring record is faster and the sensor importance of this sensor monitoring record is higher. Then The larger, that is, The larger, the higher the priority importance of the quality warning of the sensor monitoring record, and it is necessary to give priority to quickly performing quality warning on this sensor monitoring record.

[0100] It should be noted that the importance of sensors in different sensor monitoring records often varies, and the sensor importance is usually used to identify the severity or importance of events. For example, the importance of atmospheric resources indicates that major problems have occurred in the system or application level; the importance of water resources indicates potential problems, which are less severe than those of atmospheric resource sensors; the importance of biological resources is for recording biological resource operations; and the importance of mineral resources is used in the development and mineral resource stages.

[0101] 107. Perform quality warnings on sensor monitoring records according to the quality warning priorities of sensor monitoring records.

[0102] Perform quality warnings on sensor monitoring records according to the quality warning priorities of sensor monitoring records, reasonably allocate quality warning resources. By performing quality warnings on all sensor monitoring records with different priorities, the sensor monitoring records of the environmental management platform can be effectively managed and quality warned, ensuring the security and availability of important sensor monitoring records, and at the same time optimizing the quality warning cost.

[0103] In some embodiments, in response to the quality warning priority of the sensor monitoring record being within the first priority range, quality warn the sensor monitoring record in a high-speed quality warning medium; in response to the quality warning priority of the sensor monitoring record being within the second priority range, quality warn the sensor monitoring record in a medium-speed quality warning medium; in response to the quality warning priority of the sensor monitoring record being within the third priority range, quality warn the sensor monitoring record in a low-speed quality warning medium.

[0104] In summary, the present invention provides an ecological environment monitoring quality management method based on big data processing. This method can quickly respond to sensor monitoring records including fluctuation information or records that need to be processed in a timely manner, perform timely processing, prioritize quality warnings, reasonably allocate quality warning resources, reduce quality warning costs, and ensure the security and availability of important sensor monitoring records.

[0105] The quality warning medium provided in the embodiments of the present application has the same inventive concept as the foregoing embodiments. The values not shown in detail in this quality warning medium can be referred to the foregoing embodiments and will not be elaborated here.

[0106] In the third aspect, as Figure 2 shown, the present invention provides an ecological environment monitoring quality management system based on big data processing. The system includes:

[0107] An acquisition unit 201, configured to acquire sensor monitoring records of an environmental management platform.

[0108] The first determination unit 202 is configured to determine the query activity of the sensor monitoring record by the correlation between the number of queries recorded by the sensor monitoring within a certain period and the query users within that period, as well as the query confidence of the sensor monitoring record.

[0109] The second determination unit 203 is configured to determine the periodic fluctuation coefficient through the record group and information of the sensor monitoring record, where the record group and information include the type of the sensor monitoring record, the degree of fluctuation of the sensor monitoring record, and the safety factor of the sensor monitoring record.

[0110] The third determination unit 204 is configured to determine the periodic volatility of the sensor monitoring record based on the periodic fluctuation coefficient and the query activity.

[0111] The fourth determination unit 205 is configured to determine the numerical volatility of the sensor monitoring record by the fluctuation value of the sensor monitoring record and the number of fluctuation values, where the fluctuation values include the steady-state monitoring value, the peak-state monitoring value, and the valley-state monitoring value.

[0112] The fifth determination unit 206 is configured to determine the quality warning priority of the sensor monitoring record by the ranking of the importance of the sensor monitoring record among the sensor importances, the periodic volatility, and the numerical volatility;

[0113] The quality warning unit 207 is configured to perform quality warning on the sensor monitoring record according to the quality warning priority of the sensor monitoring record.

[0114] In some embodiments, the first determination unit 202 includes:

[0115] The first acquisition module is configured to acquire the number of queries of the sensor monitoring record within a certain period and the real-time upload degree value of the environmental management platform within that period;

[0116] The first determination module is configured to determine the query confidence of the sensor monitoring record based on the number of queries of the sensor monitoring record within that period and the real-time upload degree value of the environmental management platform within that period;

[0117] The first output module is configured to fit the number of queries of the sensor monitoring record within that period and the query users within that period, and output the change broken line of the number of queries of the sensor monitoring record and the correlation between the number of queries of the sensor monitoring record within that period and the query users within that period;

[0118] The second determination module is configured to determine the query activity of the sensor monitoring record based on the correlation and the query confidence.

[0119] In some embodiments, the query confidence of the sensor monitoring record is determined through the following expression:

[0120] ;

[0121] Wherein, is the query confidence level recorded by the v-th sensor, G is the number of queries for the monitoring record of the v-th sensor in this period, is the real-time upload degree value of the m-th timestamp of the monitoring record of the v-th sensor in this period, M is the number of timestamps existing in this period, m is the timestamp id in this period, and v is the id of the number of sensor monitoring records.

[0122] In some embodiments, the query activity of the sensor monitoring record is determined by the following expression:

[0123] ;

[0124] Wherein, is the query activity of the v-th sensor monitoring record, is the correlation degree between the number of queries for the monitoring record of the v-th sensor in this period and the query users in this period, is the query confidence level of the v-th sensor monitoring record in the n-th sub-period in this period. There are multiple equally divided sub-periods in this period, N is the number of sub-periods in this period, and v is the id of the number of sensor monitoring records.

[0125] In some embodiments, the second determination unit 203 includes:

[0126] The second acquisition module is used to acquire the record group and information of the labeled sensor monitoring record, the periodic volatility mark corresponding to the labeled sensor monitoring record, and the initial clustering model. The initial clustering model is a five-layer fully connected deep learning model;

[0127] The second output module is used to train the initial clustering model according to the record group and information of the labeled sensor monitoring record and the periodic volatility mark, and output the target clustering model;

[0128] The third output module is used to input the record group and information of the sensor monitoring record into the target clustering model and output the periodic fluctuation coefficient.

[0129] In some embodiments, the periodic volatility of the sensor monitoring record is determined by the following expression:

[0130] ;

[0131] Wherein, is the periodic volatility of the v-th sensor monitoring record, is the query activity of the v-th sensor monitoring record, The periodic fluctuation coefficient monitored and recorded by the v-th sensor, where v is the number of sensor monitoring records id.

[0132] In some embodiments, the numerical volatility of the sensor monitoring records is determined by the following expression:

[0133] ;

[0134] In the formula, is the numerical volatility of the v-th sensor monitoring record, is the number of fluctuation values included in the v-th sensor monitoring record, is a smoothing function, and v is the number of sensor monitoring records id.

[0135] In some embodiments, the quality warning priority of the sensor monitoring records is determined by the following expression:

[0136] ;

[0137] In the formula, is the quality warning priority of the v-th sensor monitoring record, is the numerical volatility of the v-th sensor monitoring record, is the periodic volatility of the v-th sensor monitoring record, is the importance of the v-th sensor monitoring record, is the id number of the importance of the v-th sensor monitoring record in the sensor importance sorted from high to low. The sensor importance is sorted from high to low as the importance of atmospheric resources, the importance of water resources, the importance of biological resources, and the importance of mineral resources. The id numbers corresponding to the importance of atmospheric resources, water resources, biological resources, and mineral resources of the sensor monitoring records are 1, 2, 3, and 4 respectively.

[0138] The ecological environment monitoring quality management system based on big data processing further includes:

[0139] One or more processors;

[0140] A quality warning device, on which there are one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the ecological environment monitoring quality management method based on big data processing.

[0141] In summary, the present invention provides an ecological environment monitoring quality management system based on big data processing. This system can quickly respond to sensor monitoring records including fluctuation information or those requiring timely processing, and perform timely processing, prioritize quality warnings, reasonably allocate quality warning resources, reduce quality warning costs, and ensure the security and availability of important sensor monitoring records.

[0142] The specific embodiments of the invention have been described in detail above, but they are only examples, and the present application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application. Therefore, equivalent transformations, modifications, improvements, etc. made without departing from the spirit and principles of the present application should all be covered within the scope of the present application.

Claims

1. An ecological environment monitoring quality management method based on big data processing, characterized in that: The method comprises: Collect sensor monitoring records of the environmental management platform; Determine the query activity of the sensor monitoring record by the correlation between the number of queries of the sensor monitoring record in a certain period and the query users in the period and the query confidence of the sensor monitoring record; Determining a periodic fluctuation coefficient through a record group and information of the sensor monitoring record, wherein the record group and information include a type of the sensor monitoring record, a fluctuation degree of the sensor monitoring record, and a safety factor of the sensor monitoring record; Determining the periodic fluctuation of the sensor monitoring record according to the periodic fluctuation coefficient and the query activity; Determine the numerical volatility of the sensor monitoring record by the fluctuation value and the number of fluctuation values, wherein the fluctuation value includes a steady-state monitoring value, a peak-state monitoring value and a valley-state monitoring value; Determining the quality warning priority of the sensor monitoring record by ranking the importance of the sensor monitoring record in the sensor importance, the periodic volatility and the value volatility; According to the quality warning priority of the sensor monitoring records, quality warnings are issued for the sensor monitoring records.

2. The ecological environment monitoring quality management method based on big data processing according to claim 1 is characterized in that: The step of determining the query activity of the sensor monitoring record by the correlation between the number of queries of the sensor monitoring record in a certain period and the query users in the period and the query confidence of the sensor monitoring record includes: Collect the query times of sensor monitoring records within a certain period and the real-time upload value of the environmental management platform within the period; Determine the query confidence of the sensor monitoring record based on the query times of the sensor monitoring record within the period and the upload real-time degree value of the environmental management platform within the period; Fitting the number of queries recorded by the sensor monitoring within the period and the query users within the period, outputting a change line of the number of queries recorded by the sensor monitoring within the period and a correlation between the number of queries recorded by the sensor monitoring within the period and the query users within the period; Determining the query activity of the sensor monitoring record according to the association degree and the query confidence; The query confidence of the sensor monitoring record is determined by the following expression: In the formula, is the query confidence of the vth sensor monitoring record, G is the query number of the vth sensor monitoring record in the cycle, is the upload real-time value of the mth timestamp of the vth sensor monitoring record in the cycle, M is the number of timestamps in the cycle, m is the timestamp id in the cycle, and v is the number id of the sensor monitoring record; The query activity of the sensor monitoring record is determined by the following expression: In the formula, The query activity recorded for the vth sensor, is the correlation between the query times of the vth sensor monitoring record in the period and the query users in the period, is the query confidence of the vth sensor monitoring record in the nth sub-cycle in the cycle, the cycle includes multiple equally divided sub-cycles, N is the number of sub-cycles in the cycle, and v is the number id of the sensor monitoring records.

3. The ecological environment monitoring quality management method based on big data processing according to claim 1 is characterized in that: The recording group and information recorded by the sensor monitoring to determine the periodic fluctuation coefficient includes: Collecting record groups and information of the annotated sensor monitoring records, periodic volatility markers corresponding to the annotated sensor monitoring records, and an initial clustering model, wherein the initial clustering model is a five-layer fully connected deep learning model; According to the record group and information of the annotated sensor monitoring record and the periodic volatility mark, the initial clustering model is trained to output a target clustering model; The record group and information of the sensor monitoring record are input into the target clustering model, and the periodic fluctuation coefficient is output.

4. The ecological environment monitoring quality management method based on big data processing according to claim 1 is characterized in that: The periodic fluctuation of the sensor monitoring record is determined by the following expression: In the formula, is the periodic fluctuation recorded by the vth sensor, The query activity recorded for the vth sensor, is the periodic fluctuation coefficient of the vth sensor monitoring record, and v is the number id of the sensor monitoring records.

5. The ecological environment monitoring quality management method based on big data processing according to claim 1 is characterized in that: The numerical volatility of the sensor monitoring record is determined by the following expression: In the formula, The fluctuation of the value recorded by the vth sensor, is the number of fluctuation values ​​included in the monitoring record of the vth sensor, is a smooth function, and v is the number id of sensor monitoring records.

6. The ecological environment monitoring quality management method based on big data processing according to claim 4 is characterized in that: The quality warning priority of the sensor monitoring record is determined by the following expression: In the formula, The quality warning priority of the vth sensor monitoring record, The fluctuation of the value recorded by the vth sensor, is the periodic fluctuation recorded by the vth sensor, The importance of monitoring records for the vth sensor, is the id number of the importance of the vth sensor monitoring record in the sensor importance sorted from high to low, the sensor importance is sorted from high to low as the importance of air resources, the importance of water resources, the importance of biological resources and the importance of mineral resources, and the id numbers corresponding to the importance of air resources, the importance of water resources, the importance of biological resources and the importance of mineral resources of the sensor monitoring record are 1, 2, 3 and 4 respectively.

7. An ecological environment monitoring quality management system based on big data processing, adopting an ecological environment monitoring quality management method based on big data processing as described in any one of claims 1 to 6, characterized in that: The system comprises: one or more processors; A quality warning device, on which quality warning has one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the ecological environment monitoring quality management method based on big data processing.

Images