An ecological environment monitoring data processing method and system based on big data

Abstract

The application discloses an ecological environment monitoring data processing method and system based on big data, and the method comprises the following steps: acquiring multi-source heterogeneous ecological environment monitoring data of a set regional range, performing data source screening and data preprocessing on the ecological environment monitoring data, extracting multi-dimensional data features from the data source based on the data preprocessing result; constructing a multi-modal knowledge graph, using a CM-GAT cross-modal graph attention network to extract semantic association of the multi-dimensional data features through an attention mechanism, and obtaining an associated feature vector; inputting the associated feature vector into an improved model for prediction, and outputting an environmental state index; judging the pollution degree according to the environmental state index, and analyzing the correlation between the pollutant migration path and the potential pollution source by using an association mining algorithm, and outputting an early warning result. The method can efficiently and quickly analyze, accurately lock the pollution source and clearly depict the pollutant migration path, and realizes active monitoring and early warning.

Description

Technical Field

[0001] This invention relates to the field of ecological environment monitoring technology, and in particular to a method and system for ecological environment monitoring data processing based on big data. Background Technology

[0002] With the rapid development of sensor technology, remote sensing technology, and the Internet of Things (IoT) technology, ecological and environmental monitoring has entered the era of massive big data. Data sources include ground sensor networks, remote sensing satellites, drone aerial photography, and meteorological station data. Currently, traditional ecological and environmental monitoring data processing methods lack efficient fusion frameworks for processing data from different sources with vastly different formats, protocols, and spatiotemporal scales, making it difficult to fully extract the value of the data. Traditional batch processing methods are inefficient and cannot meet the needs of real-time or near-real-time monitoring and early warning, resulting in delayed responses to sudden environmental pollution events. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a method and system for processing ecological environment monitoring data based on big data.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned method for processing ecological environment monitoring data based on big data:

[0005] Acquire multi-source heterogeneous ecological environment monitoring data within a defined geographical area, perform data source screening and data preprocessing on the ecological environment monitoring data, and extract multi-dimensional data features from the data source based on the data preprocessing results;

[0006] A multimodal knowledge graph is constructed, and the semantic associations of the multi-dimensional data features are extracted through the attention mechanism using the CM-GAT cross-modal graph attention network to obtain the associated feature vectors.

[0007] The associated feature vectors are input into the improved Transformer model for prediction, and the output is an environmental state index;

[0008] The pollution level is determined based on the environmental state index, and the FP-Growth association mining algorithm is used to analyze the correlation between pollutant migration paths and potential pollution sources, and early warning results are output.

[0009] Furthermore, in the aforementioned big data-based ecological environment monitoring data processing method, the steps of acquiring multi-source heterogeneous ecological environment monitoring data within a defined geographical area, performing data source screening and data preprocessing on the ecological environment monitoring data, and extracting multi-dimensional data features from the data sources based on the data preprocessing results include:

[0010] Acquire multi-source heterogeneous ecological environment monitoring data within a defined geographical area, including at least real-time sensor data, remote sensing meteorological data, and historical manual data;

[0011] Data sources with less than 90% data integrity were removed, and remote sensing data with a timeliness exceeding 72 hours and sewage discharge data that had no spatial correlation with the monitored area were deleted.

[0012] Missing values ​​were filled using the mean imputation method, outliers were removed using the 3σ principle, data of different formats were uniformly converted into JSON structure, spatial data were aligned to coordinates, temporal data were aggregated, and Z-Score standardization was used to map data of different dimensions to the [-1,1] interval to obtain preprocessed data.

[0013] Extract the indicator dimension, spatial dimension and time dimension data from the preprocessed data to obtain multi-dimensional data features.

[0014] Furthermore, in the aforementioned method for processing ecological environment monitoring data based on big data, the construction of a multimodal knowledge graph and the use of the CM-GAT cross-modal graph attention network to extract the semantic associations of the multi-dimensional data features through an attention mechanism to obtain the associated feature vector include:

[0015] Define entities and relationships. Entities include monitoring sites, pollutants, pollution sources, and areas; relationships are the connections between monitoring sites, monitoring, and pollutants.

[0016] By associating textual, numerical, and image data with entity attributes, and resolving entity ambiguity through string matching, a multimodal knowledge graph is constructed.

[0017] Furthermore, in the aforementioned method for processing ecological environment monitoring data based on big data, the construction of a multimodal knowledge graph and the use of the CM-GAT cross-modal graph attention network to extract the semantic associations of the multi-dimensional data features through an attention mechanism to obtain the associated feature vector include:

[0018] The TransE algorithm is used to generate entity vectors for the knowledge graph. Multi-dimensional data features are embedded and concatenated with the entity vectors to form fused feature data.

[0019] The CM-GAT cross-modal graph attention network is divided into two layers: intra-modal attention and cross-modal attention. Intra-modal attention weights are calculated for numerical features and text features respectively, while cross-modal attention coefficients are used to measure the correlation between features from different modalities.

[0020] The attention-weighted feature data is aggregated to generate associated feature vectors.

[0021] Furthermore, in the aforementioned method for processing ecological environment monitoring data based on big data, the step of inputting the associated feature vector into an improved Transformer model for prediction and outputting an environmental state index includes:

[0022] By adding spatial and temporal attention after the self-attention layer of the traditional Transformer encoder, replacing the fully connected layer of the decoder output layer with a convolutional and fully connected structure, and using 1D convolution to extract local features, an improved Transformer model is obtained.

[0023] Furthermore, in the aforementioned method for processing ecological environment monitoring data based on big data, the step of inputting the associated feature vector into an improved Transformer model for prediction and outputting an environmental state index includes:

[0024] The decoder receives the spatiotemporal and global joint feature vectors output by the encoder, performs feature matching on the historical environment state index sequence through a cross-attention layer, uses the historical index sequence as the key and the encoder output features as the query, and calculates the correlation between the two.

[0025] The feature vector output from the cross-attention layer is input into the 1D convolutional layer, and the local change trend of adjacent feature dimensions is extracted through the sliding convolution operation.

[0026] The 256-dimensional feature vector output from the 1D convolutional layer is input into two fully connected layers, and the output is an environmental state index.

[0027] Furthermore, in the aforementioned big data-based ecological environment monitoring data processing method, the step of judging the pollution level based on the environmental state index and analyzing the correlation between pollutant migration paths and potential pollution sources using the FP-Growth association mining algorithm to output early warning results includes:

[0028] Treat time, region, pollutant type, potential pollution source, and migration path segment as a single transaction to generate multiple transaction data;

[0029] With a minimum support of 5%, the frequent itemsets of multiple transaction data are mined using the FP-Growth association mining algorithm's FP-tree.

[0030] Association rules are extracted from frequent itemsets to determine the relationship between pollutant migration paths and potential pollution sources.

[0031] Furthermore, in a big data-based ecological environment monitoring data processing system, the ecological environment monitoring data processing system includes the following modules:

[0032] The data feature extraction module is used to acquire multi-source heterogeneous ecological environment monitoring data within a set geographical area, perform data source screening and data preprocessing on the ecological environment monitoring data, and extract multi-dimensional data features from the data source based on the data preprocessing results.

[0033] The associated feature fusion module is used to construct a multimodal knowledge graph. It uses the CM-GAT cross-modal graph attention network to extract the semantic associations of the multi-dimensional data features through the attention mechanism to obtain the associated feature vector.

[0034] An environmental state identification module is used to input the associated feature vector into an improved Transformer model for prediction and output an environmental state index.

[0035] The environmental monitoring and early warning module is used to determine the degree of pollution based on the environmental state index, and to analyze the correlation between pollutant migration paths and potential pollution sources using the FP-Growth association mining algorithm, and output early warning results.

[0036] Furthermore, in a big data-based ecological environment monitoring data processing system, the environmental state identification module includes the following sub-modules:

[0037] A submodule is added to add spatial and temporal attention after the self-attention layer of the traditional Transformer encoder, and the fully connected layer of the decoder output layer is replaced with a convolutional and fully connected structure. Local features are extracted using 1D convolution, resulting in an improved Transformer model.

[0038] Furthermore, in a big data-based ecological environment monitoring data processing system, the environmental monitoring and early warning module includes the following sub-modules:

[0039] The generation submodule is used to treat time, region, pollutant type, potential pollution source and migration path segment as a single transaction to generate multiple transaction data;

[0040] The mining submodule is used to set the minimum support to 5% and mine frequent itemsets of multiple transaction data through the FP-Growth association mining algorithm's FP-tree.

[0041] The extraction submodule is used to extract association rules from the frequent itemset to determine the association between pollutant migration paths and potential pollution sources.

[0042] Its beneficial effects lie in the fact that, through systematic data screening and preprocessing, it effectively solves the problem of multi-source heterogeneous data fusion, laying a reliable data foundation for high-quality analysis. The introduction of a multimodal knowledge graph and the CM-GAT cross-modal graph attention network enables deep fusion of deep semantic information from different modalities, accurately uncovering complex and implicit relationships that are difficult to discover using traditional methods, greatly improving the accuracy and interpretability of feature representations. The use of an improved Transformer model for prediction, with its powerful sequence modeling and global dependency capture capabilities, makes the prediction of environmental state indices more accurate and reliable, overcoming the shortcomings of traditional models such as insufficient accuracy and weak generalization ability. Finally, combined with the FP-Growth association rule mining algorithm, it can efficiently and quickly analyze the association rules behind massive amounts of data, thereby accurately locating pollution sources and clearly depicting pollutant migration paths, realizing a shift from passive monitoring to proactive early warning. Attached Figure Description

[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0044] Figure 1 This is a schematic diagram of the first embodiment of a big data-based ecological environment monitoring data processing method according to the present invention;

[0045] Figure 2 This is a schematic diagram of a second embodiment of a big data-based ecological environment monitoring data processing method according to the present invention;

[0046] Figure 3 This is a schematic diagram of the first embodiment of an ecological environment monitoring data processing system based on big data according to the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, an ecological environment monitoring data processing method based on big data is proposed, which includes the following steps:

[0050] Step 101: Obtain multi-source heterogeneous ecological environment monitoring data within a specified geographical area; perform data source screening and data preprocessing on the ecological environment monitoring data; and extract multi-dimensional data features from the data sources based on the data preprocessing results.

[0051] Specifically, in this embodiment, multi-source heterogeneous ecological environment monitoring data within a defined geographical area is acquired, including at least real-time sensor data, remote sensing meteorological data, and historical manual data; data sources with less than 90% data integrity are removed, and remote sensing data with a timeliness exceeding 72 hours and sewage discharge data that are not spatially related to the monitoring area are deleted; missing values ​​are filled using the mean imputation method, outliers are removed using the 3σ principle, data of different formats are uniformly converted into JSON structure, spatial data is aligned to coordinates, temporal data is aggregated, and Z-Score standardization is used to map data of different dimensions to the [-1,1] interval to obtain preprocessed data; indicator dimension, spatial dimension, and temporal dimension data are extracted from the preprocessed data to obtain multi-dimensional data features.

[0052] When acquiring multi-source heterogeneous ecological and environmental monitoring data within a defined geographical area, such as the administrative boundary of a prefecture-level city or a specific watershed, data must be collected through three core channels: real-time sensor data relies on automatic air monitoring stations deployed within the region to collect concentrations of pollutants such as PM2.5, SO2, and NO2, with a sampling frequency of 15 minutes per sampling; surface water monitoring buoys collect water quality indicators such as COD, ammonia nitrogen, and pH value, with a sampling frequency of 1 hour per sampling; and soil moisture sensors collect soil heavy metal content and water content, with a sampling frequency of 6 hours per sampling, and transmit the data in real time to the data center via an IoT gateway; remote sensing meteorological data needs to be integrated with satellite remote sensing platforms. The satellites such as Gaofen and Sentinel acquire spatial image data of the region, including vegetation coverage, water area, and land use type, with a resolution of 30 meters. At the same time, meteorological data such as wind speed, precipitation, and temperature are retrieved from the local meteorological department's database, with a sampling frequency of 1 hour per time. Historical manual data needs to be extracted from the environmental protection department's archives system, including manual sampling and monitoring records for the past 5 years, such as monthly river section manual sampling pollutant concentration data, enterprise pollution discharge declaration reports containing textual and numerical information such as factory name, sewage outlet location, and wastewater / exhaust gas emission volume, and environmental law enforcement inspection records, such as the time, location, and pollutant type of illegal discharge from pollution sources. After data collection is completed, data sources must be screened according to strict standards: First, the percentage of valid data from each data source is statistically analyzed using data integrity verification tools, and data sources with less than 90% integrity are directly eliminated. For example, if an air monitoring station has only 85% valid data for the month due to equipment failure, then the entire data from that station is excluded. For remote sensing data, images with a timeliness exceeding 72 hours are deleted by comparing timestamps. For example, satellite images acquired on Monday are no longer valid by Thursday and are no longer included in the analysis. For enterprise pollution discharge data, data that is not spatially related to the monitoring area is deleted using GIS spatial overlay analysis, such as pollution discharge data from factories whose discharge outlet coordinates are not within the set geographical area.In the data preprocessing stage, the first step is to use mean imputation to fill in missing values. For example, if a water quality monitoring buoy has missing data for the period from 10:00 to 11:00 on a certain day, the mean of the monitoring data from 9:00 to 10:00 and 11:00 to 12:00 on the same day is used to fill in the missing values. Then, the 3σ principle is used to identify and delete outliers that exceed the mean ± 3 times the standard deviation. For example, if a sensor falsely reports a PM2.5 concentration of 1000 μg / m³, which is far beyond the normal range, it is identified as an outlier and removed. Subsequently, data format and spatiotemporal calibration are performed on all data, including CSV format of sensor data and remote sensing data. The TIFF format and the Excel format of the manual data were uniformly converted into JSON structure. The spatial data such as remote sensing images and monitoring station coordinates were uniformly aligned to the WGS84 coordinate system using a coordinate transformation tool. Time data with different sampling frequencies were aggregated at the hourly time granularity, such as aggregating air sensor data of 15 minutes / time into hourly averages. Finally, Z-Score standardization was used to map data of different units such as PM2.5 (μg / m³), precipitation (mm), and COD (mg / L) to the [-1,1] interval to obtain standardized preprocessed data. When extracting multi-dimensional data features based on preprocessed data, the indicator dimension requires calculating the hourly average, daily maximum, and 3-day change rate for each pollutant, such as the daily maximum of PM2.5 and the 3-day change rate of COD. The spatial dimension requires dividing the monitoring area into 1km×1km grids and calculating the number of monitoring stations and the spatial variance of pollutant concentrations in each grid, such as the variance of COD concentrations at all water quality monitoring points in a certain grid. The temporal dimension requires splitting the data into daytime and nighttime periods, 6:00-18:00 as daytime and 18:00-6:00 as nighttime, and extracting the difference in pollutant concentrations between day and night and the duration of pollution, such as the number of consecutive hours when the concentration of a certain pollutant exceeds the standard. Finally, a multi-dimensional data feature set covering the three dimensions of indicators, space, and time is formed.

[0053] Step 102: Construct a multimodal knowledge graph and use the CM-GAT cross-modal graph attention network to extract the semantic associations of multi-dimensional data features through the attention mechanism to obtain the associated feature vectors;

[0054] Specifically, this embodiment defines entities and relationships. Entities include monitoring stations, pollutants, pollution sources, and areas; relationships are the connections between monitoring stations, monitoring, and pollutants. Text data, numerical data, and image data are associated with entity attributes, and string matching is used to resolve entity ambiguities and construct a multimodal knowledge graph. The TransE algorithm is used to generate entity vectors for the knowledge graph, and multi-dimensional data features are embedded and concatenated with entity vectors to form fused feature data. The CM-GAT cross-modal graph attention network is divided into two layers: intra-modal attention and cross-modal attention. Intra-modal attention weights are calculated for numerical and text features respectively, while cross-modal attention coefficients are used to measure the correlation between features from different modalities. The attention-weighted feature data is aggregated to generate associated feature vectors.

[0055] When constructing a multimodal knowledge graph, the specific definitions of entities and relationships must first be clarified. Entities require detailed attribute information. Monitoring station entities should include a unique ID, latitude and longitude coordinates, monitoring equipment model, and main monitoring indicators such as PM2.5 and COD. Pollutant entities should be labeled with chemical formulas such as PM2.5 corresponding to mixtures, SO2 corresponding to sulfur dioxide, pollution categories such as gaseous pollutants, water pollutants, and national limit standards. Pollution source entities should record names such as a chemical enterprise or a farm, discharge type (wastewater / waste gas / solid waste), and discharge outlet coordinates. Regional entities should be divided into grid numbers such as G001-G500 of a 1km×1km grid and their respective administrative levels such as city, district, and street. In addition to covering the core relationship between monitoring stations, monitoring, and pollutants (e.g., monitoring station S01-monitoring-PM2.5), relationship definitions should also include pollution source-emission-pollutant relationships such as chemical plant A-emission-SO2, and region-inclusion-monitoring station relationships such as region G005-inclusion-monitoring station S01, forming a complete knowledge graph topology. Subsequently, multimodal data was associated with entity attributes: textual data, such as the description of Chemical Plant A discharging 50 tons of organic wastewater daily in the enterprise's pollution discharge declaration, and the pollution source's illegal discharge time in environmental law enforcement records, were associated with the pollution source entity's discharge description and violation record attributes. Numerical data, such as the PM2.5 hourly concentration value of 35 μg / m³ collected by monitoring station S01, and the pollution source's daily discharge of 100 kg, were associated with the monitoring station's monitoring values ​​and the pollution source's discharge volume attributes. Image data, such as the abnormal yellow patch in the water body of area G003 in remote sensing imagery, and real-world photos around the monitoring station, were labeled as regional entity spatial images and environmental attributes around the monitoring station. To address entity ambiguity issues, such as Chemical Plant A and Chemical Plant A in a certain region referring to the same pollution source, and PM2.5 and fine particulate matter referring to the same pollutant, a similarity threshold of 85% was set through string fuzzy matching, combined with an environmental term thesaurus such as COD and chemical oxygen demand, to achieve entity normalization. Finally, a knowledge graph containing multimodal information including text, numerical data, and images was constructed.

[0056] Next, the TransE algorithm is used to generate entity vectors for the knowledge graph: based on the translation assumption of TransE, that is, head entity vector + relation vector ≈ tail entity vector, such as pollution source A vector + emission vector ≈ SO2 vector, each entity in the knowledge graph is mapped to a 128-dimensional low-dimensional dense vector. By minimizing the distance loss function of the triple head entity, relation, and tail entity, such as L1 / L2 loss, the vector representation is optimized, so that semantically similar entity vectors are closer in the vector space. For example, the vector distance between chemical plant A and chemical plant B is smaller than the vector distance between chemical plant A and nature reserve. Subsequently, the previously extracted multi-dimensional data features, including the daily peak value of pollutants in the indicator dimension, the grid pollutant variance in the spatial dimension, and the diurnal concentration difference in the time dimension, totaling 256 dimensions, were embedded and spliced ​​with the corresponding entity vectors in 128 dimensions according to entity association relationships. For example, the multi-dimensional features of monitoring station S01 were spliced ​​with the entity vector of monitoring station S01 and the entity vector of the region G005 to which the PM2.5 entity vector it monitors belonged, forming 384-dimensional fused feature data. This ensures that the features contain both the quantitative information of the monitoring data and the semantic association information of the knowledge graph.

[0057] In the CM-GAT cross-modal graph attention network operation stage, the network is first divided into two layers: intra-modal attention and cross-modal attention. The intra-modal attention layer calculates attention weights independently for numerical features in the fused feature data, such as quantitative data like pollutant concentration, emissions, and grid variance, and text features, such as pollution discharge descriptions and violation records. For numerical features, feature importance scoring is used to determine the initial weights by calculating the Pearson correlation coefficient between the features and the environmental state, and then normalization is performed using the softmax function to highlight highly correlated features, such as the daily peak value of PM2.5, which has a higher weight than the less important humidity data. For text features, after being converted into vectors through word embedding, such as Word2Vec, a self-attention mechanism is used to calculate the dependencies between words, giving higher weights to key text fragments such as excessive organic wastewater discharge. The cross-modal attention layer measures the correlation between features from different modalities using attention coefficients. For example, it calculates the correlation strength between the numerical feature vector of wastewater discharge from chemical plant A and the textual feature vector of 'organic wastewater discharge' in the discharge declaration, and the numerical feature vector of COD concentration in region B. The magnitude of the attention coefficient determines the dependency between modalities; for example, the coefficient between discharge and COD concentration is 0.82, indicating a close correlation. Finally, the feature data after intra-modal and cross-modal attention weighting is aggregated using the mean pooling method. This involves averaging multiple sets of weighted features associated with the same entity, compressing the 384-dimensional fused features into a 256-dimensional associated feature vector. Each vector dimension corresponds to a set of entity-feature-modal correlation strengths; for example, dimension 15 corresponds to the correlation value between chemical plant A, wastewater discharge, and region BCOD concentration. This provides semantically and quantitatively combined feature inputs for subsequent model prediction.

[0058] Step 103: Input the associated feature vector into the improved Transformer model for prediction and output the environmental state index;

[0059] Specifically, in this embodiment, spatial and temporal attention are added after the self-attention layer of the traditional Transformer encoder. The fully connected layer of the decoder output layer is replaced with a convolutional and fully connected structure. Local features are extracted using 1D convolution, resulting in an improved Transformer model. The decoder receives the spatiotemporal and global joint feature vector output by the encoder. It performs feature matching on the historical environment state index sequence through a cross-attention layer, using the historical index sequence as the key and the encoder output feature as the query, to calculate the correlation between the two. The feature vector output by the cross-attention layer is input into a 1D convolutional layer, and the local change trend of adjacent feature dimensions is extracted through sliding convolution operation. The 256-dimensional feature vector output by the 1D convolutional layer is input into two fully connected layers to output the environment state index.

[0060] When improving the traditional Transformer model, the focus is first on encoder structure optimization: after the original self-attention layer of the encoder completes global feature dependency capture, two new sub-modules, spatial attention and temporal attention, are added. The spatial attention module first divides the monitoring area into 1km×1km grid cells, uses the feature vectors corresponding to each grid as input, calculates the spatial correlation between features of different grids through cosine similarity, and generates a spatial attention weight matrix. It focuses on strengthening the feature weights of adjacent grids and grids within the same watershed; for example, the weight of region G005 and the adjacent grid G006 is increased to 0.7, while the weight of non-adjacent grids is reduced to below 0.1, thereby capturing the spatial diffusion correlation of pollutants. The temporal attention module... The blocks are organized in units of time steps, such as one time step per hour. A sliding window attention mechanism with a window size of 3 is used to assign differentiated weights to the feature vectors of the current time step and the one time step before and after it. The weight of the current time step is set to 0.6, and the weights of the time steps before and after it are set to 0.2, which highlights the continuity of features in the temporal dimension. Finally, it is fused with the output of the self-attention layer to form a spatiotemporal-global joint feature vector. At the same time, the decoder output layer is reconstructed, replacing the traditional single fully connected layer with a hybrid structure of 1D convolution + fully connected. The 1D convolution layer is set with a kernel size of 3 and an output channel number of 256, which is used for subsequent extraction of local correlation patterns of features, thereby completing the architectural design of the improved Transformer model.

[0061] When the model enters the prediction phase, the decoder first receives the spatiotemporal-global joint feature vector with a dimension of 512 output from the encoder. It then initiates a cross-attention layer for feature matching. This layer inputs a historical environmental state index sequence, which aggregates index data from the past 24 hours at the hourly level, resulting in 24 data points. This sequence is first converted into a 512-dimensional sequence vector as the key, and the spatiotemporal-global joint feature vector output from the encoder is used as the query. The correlation between the two is calculated through scaling dot product attention, for example, identifying the changing patterns of 'factory emissions' and 'wind speed' output from the encoder when PM2.5 levels rise historically, thus achieving deep matching between historical indices and current features. After the cross-attention layer outputs the feature vector, it is immediately input into a pre-defined 1D convolutional layer. Through a sliding convolution operation, the convolution kernel slides along the feature dimension direction with a stride of 1, extracting the local change trends of adjacent feature dimensions segment by segment—such as capturing the adjacent correlation fluctuations between the emissions feature dimension of factory A and the PM2.5 concentration feature dimension of region B. This effectively filters out noise interference generated in the global computation, outputting a local feature vector with a dimension of 256. Finally, the 256-dimensional feature vector is input into two fully connected layers: the first fully connected layer compresses the feature dimension to 64 dimensions using the ReLU activation function to further filter key features; the second fully connected layer uses the Sigmoid activation function to map the output value to the [0,1] interval, and then converts it into an environmental state index in the range of 0-100 with one decimal place using a linear scaling formula. For example, when the Sigmoid output is 0.582, the final index is 58.2, completing the entire prediction process.

[0062] Step 104: Determine the pollution level based on the environmental status index, and use the FP-Growth association mining algorithm to analyze the correlation between pollutant migration paths and potential pollution sources, and output early warning results.

[0063] Specifically, in this embodiment, time, region, pollutant type, potential pollution source, and migration path segment are treated as a single transaction, generating multiple transaction data. The minimum support is set to 5%, and frequent itemsets of the multiple transaction data are mined using the FP-Growth association mining algorithm's FP-tree. Association rules are extracted from the frequent itemsets to determine the association between pollutant migration paths and potential pollution sources.

[0064] When judging the degree of pollution based on the output environmental state index range of 0-100, retaining one decimal place, an index of 0-30 is defined as light pollution, corresponding to good air quality or water quality of Class III or above, requiring no emergency measures and only routine monitoring; an index of 31-60 is defined as moderate pollution; and an index of 61-100 is defined as heavy pollution. At the same time, the threshold is dynamically adjusted for sensitive areas—the threshold for heavy pollution is lowered to 50, and for industrial areas it is raised to 70, to ensure that the classification judgment is consistent with the actual environmental carrying capacity of the area. During the judgment process, the rationality of the index also needs to be verified simultaneously. If the difference between the index of a certain grid and that of the adjacent grid exceeds 25, such as the index of grid G006 being 72.3 and the adjacent index of G007 being 45.1, the monitoring data and model prediction process of that grid need to be re-verified to rule out misjudgments caused by sensor failure or abnormal feature vectors.

[0065] Subsequently, for areas with moderate to severe pollution, the FP-Growth association mining algorithm was used to analyze the relationship between pollutant migration paths and potential pollution sources. First, a transaction dataset was constructed, using timestamp, regional grid ID, pollutant type, potential pollution source candidates, migration path segments, and exponential change values ​​as a complete transaction. Over 100,000 valid transaction data points from the past three months within the region and its surrounding 5km radius were extracted from the historical monitoring database. Next, a minimum support of 5% was set, meaning that the frequency of a certain feature combination in the total number of transactions should not be less than 5%. Combinations below this threshold were considered accidental associations and excluded from the analysis. The process was then constructed using an FP-tree—first, a statistical analysis was performed… The frequency of individual transactions is counted, and the individual features that meet the minimum support are selected to form frequent 1-itemsets. Then, an FP-tree is constructed based on the frequent 1-itemsets, and the transactions are arranged in descending order of feature frequency and inserted into the tree to form branch nodes. Frequent itemsets are mined from the conditional pattern base of the FP-tree, such as the frequent itemsets of PM2.5, chemical plant A, northeast wind, and grid G006. Then, association rules are generated based on the frequent itemsets, and strong association rules are selected by calculating the confidence level. For example, the increase in pollution discharge of chemical plant A is related to the northeast wind level 3, which leads to an increase in the PM2.5 index of grid G006. The migration path of pollutants is estimated by combining wind speed and distance, and the priority of potential pollution sources is determined by calculating the contribution rate.

[0066] Finally, the pollution level assessment results and correlation mining conclusions are integrated to output structured early warning results: the basic information section needs to clearly specify the pollution level, affected area, current environmental status index (e.g., 72.3), and early warning trigger time; the correlation analysis section needs to label the main pollutant types, core potential pollution sources, complete migration paths, and duration of impact; the response recommendations section needs to distinguish between short-term emergency measures and long-term governance plans. At the same time, the early warning information is pushed to the environmental protection department's regulatory platform, the public environmental early warning APP, and relevant enterprise regulatory systems through API interfaces to ensure that the early warning information reaches all relevant stakeholders efficiently and supports pollution prevention and control decisions.

[0067] Its beneficial effects lie in the fact that, through systematic data screening and preprocessing, it effectively solves the problem of multi-source heterogeneous data fusion, laying a reliable data foundation for high-quality analysis. The introduction of a multimodal knowledge graph and the CM-GAT cross-modal graph attention network enables deep fusion of deep semantic information from different modalities, accurately uncovering complex and implicit relationships that are difficult to discover using traditional methods, greatly improving the accuracy and interpretability of feature representations. The use of an improved Transformer model for prediction, with its powerful sequence modeling and global dependency capture capabilities, makes the prediction of environmental state indices more accurate and reliable, overcoming the shortcomings of traditional models such as insufficient accuracy and weak generalization ability. Finally, combined with the FP-Growth association rule mining algorithm, it can efficiently and quickly analyze the association rules behind massive amounts of data, thereby accurately locating pollution sources and clearly depicting pollutant migration paths, realizing a shift from passive monitoring to proactive early warning.

[0068] Please see Figure 2 In a big data-based ecological environment monitoring data processing method, a multimodal knowledge graph is constructed, and the CM-GAT cross-modal graph attention network is used to extract the semantic associations of multi-dimensional data features through the attention mechanism to obtain the associated feature vectors. The steps include:

[0069] Step 201: Use the TransE algorithm to generate entity vectors for the knowledge graph, embed and concatenate multi-dimensional data features with entity vectors to form fused feature data;

[0070] Step 202: Divide the CM-GAT cross-modal graph attention network into two layers: intra-modal attention and cross-modal attention. For intra-modal attention, calculate the attention weights for numerical features and text features respectively. For cross-modal attention, use the attention coefficient to measure the correlation between features from different modalities.

[0071] Step 203: Aggregate the attention-weighted feature data to generate associated feature vectors.

[0072] The above describes an embodiment of the ecological environment monitoring data processing method based on big data according to the present invention. Please refer to [link / reference]. Figure 3 In a big data-based ecological environment monitoring data processing system, the system includes the following modules:

[0073] The data feature extraction module is used to acquire multi-source heterogeneous ecological environment monitoring data within a set geographical area, perform data source screening and data preprocessing on the ecological environment monitoring data, and extract multi-dimensional data features from the data source based on the data preprocessing results.

[0074] The associated feature fusion module is used to construct a multimodal knowledge graph. It uses the CM-GAT cross-modal graph attention network to extract the semantic associations of multi-dimensional data features through the attention mechanism to obtain associated feature vectors.

[0075] The environmental state identification module is used to input the associated feature vectors into the improved Transformer model for prediction and output the environmental state index.

[0076] The environmental monitoring and early warning module is used to determine the degree of pollution based on the environmental status index, and to analyze the relationship between pollutant migration paths and potential pollution sources using the FP-Growth association mining algorithm, and output early warning results.

[0077] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for processing ecological environment monitoring data based on big data, characterized in that, The ecological environment monitoring data processing method includes the following steps: Acquire multi-source heterogeneous ecological environment monitoring data within a defined geographical area, perform data source screening and data preprocessing on the ecological environment monitoring data, and extract multi-dimensional data features from the data source based on the data preprocessing results; A multimodal knowledge graph is constructed, and the TransE algorithm is used to generate entity vectors for the knowledge graph. Multi-dimensional data features are embedded and concatenated with the entity vectors to form fused feature data. The CM-GAT cross-modal graph attention network is divided into two layers: intra-modal attention and cross-modal attention. Intra-modal attention weights are calculated for numerical features and text features respectively, and cross-modal attention coefficients are used to measure the correlation between features of different modalities. The attention-weighted feature data is aggregated to generate associated feature vectors. Spatial and temporal attention are added after the self-attention layer of the traditional Transformer encoder. The fully connected layer of the decoder output layer is replaced with a convolutional and fully connected structure. Local features are extracted using 1D convolution to obtain an improved Transformer model. The associated feature vector is input into the improved Transformer model for prediction, and the output is an environmental state index. The pollution level is determined based on the environmental state index, and the FP-Growth association mining algorithm is used to analyze the correlation between pollutant migration paths and potential pollution sources, and early warning results are output.

2. The method for processing ecological environment monitoring data based on big data as described in claim 1, characterized in that, The process involves acquiring multi-source heterogeneous ecological environment monitoring data within a defined geographical area, filtering data sources and preprocessing the ecological environment monitoring data, and extracting multi-dimensional data features from the data sources based on the preprocessing results, including: Acquire multi-source heterogeneous ecological environment monitoring data within a defined geographical area, including at least real-time sensor data, remote sensing meteorological data, and historical manual data; Data sources with less than 90% data integrity were removed, and remote sensing data with a timeliness exceeding 72 hours and sewage discharge data that had no spatial correlation with the monitored area were deleted. Missing values ​​were filled using the mean imputation method, outliers were removed using the 3σ principle, data of different formats were uniformly converted into JSON structure, spatial data were aligned to coordinates, temporal data were aggregated, and Z-Score standardization was used to map data of different dimensions to the [-1,1] interval to obtain preprocessed data. Extract the indicator dimension, spatial dimension, and temporal dimension data from the preprocessed data to obtain multi-dimensional data features.

3. The method for processing ecological environment monitoring data based on big data as described in claim 1, characterized in that, The construction of the multimodal knowledge graph employs the CM-GAT cross-modal graph attention network to extract semantic associations of the multi-dimensional data features through an attention mechanism, resulting in an associated feature vector, including: Define entities and relationships. Entities include monitoring stations, pollutants, pollution sources, and areas; relationships are the connections between monitoring stations, monitoring, and pollutants. By associating textual, numerical, and image data with entity attributes, and resolving entity ambiguity through string matching, a multimodal knowledge graph is constructed.

4. The method for processing ecological environment monitoring data based on big data as described in claim 1, characterized in that, The step of inputting the associated feature vector into the improved Transformer model for prediction and outputting an environmental state index includes: The decoder receives the spatiotemporal and global joint feature vectors output by the encoder, performs feature matching on the historical environment state index sequence through a cross-attention layer, uses the historical index sequence as the key and the encoder output features as the query, and calculates the correlation between the two. The feature vector output from the cross-attention layer is input into the 1D convolutional layer, and the local change trend of adjacent feature dimensions is extracted through the sliding convolution operation. The 256-dimensional feature vector output from the 1D convolutional layer is input into two fully connected layers, and the output is an environmental state index.

5. The method for processing ecological environment monitoring data based on big data as described in claim 1, characterized in that, The process involves determining the pollution level based on the environmental state index, analyzing the correlation between pollutant migration paths and potential pollution sources using the FP-Growth association mining algorithm, and outputting early warning results, including: Treat time, region, pollutant type, potential pollution source, and migration path segment as a single transaction to generate multiple transaction data; With a minimum support of 5%, the frequent itemsets of multiple transaction data are mined using the FP-Growth association mining algorithm's FP-tree. Association rules are extracted from frequent itemsets to determine the relationship between pollutant migration paths and potential pollution sources.

6. A big data-based ecological environment monitoring data processing system, characterized in that, The ecological environment monitoring data processing system includes the following modules: The data feature extraction module is used to acquire multi-source heterogeneous ecological environment monitoring data within a set geographical area, perform data source screening and data preprocessing on the ecological environment monitoring data, and extract multi-dimensional data features from the data source based on the data preprocessing results. The associated feature fusion module uses the TransE algorithm to generate entity vectors for the knowledge graph, embedding and concatenating multi-dimensional data features with the entity vectors to form fused feature data. The CM-GAT cross-modal graph attention network is divided into two layers: intra-modal attention and cross-modal attention. Intra-modal attention weights are calculated for numerical and textual features respectively, while cross-modal attention coefficients are used to measure the correlation between features from different modalities. The attention-weighted feature data is then aggregated to generate associated feature vectors. The environmental state recognition module is used to add spatial attention and temporal attention after the self-attention layer of the traditional Transformer encoder, replace the fully connected layer of the decoder output layer with a convolutional and fully connected structure, extract local features using 1D convolution to obtain an improved Transformer model, input the associated feature vector into the improved Transformer model for prediction, and output the environmental state index. The environmental monitoring and early warning module is used to determine the degree of pollution based on the environmental state index, and to analyze the correlation between pollutant migration paths and potential pollution sources using the FP-Growth association mining algorithm, and output early warning results.

7. The ecological environment monitoring data processing system based on big data as described in claim 6, characterized in that, The environmental monitoring and early warning module Includes the following sub-modules: The generation submodule is used to treat time, region, pollutant type, potential pollution source and migration path segment as a single transaction to generate multiple transaction data; The mining submodule is used to set the minimum support to 5% and mine frequent itemsets of multiple transaction data through the FP-Growth association mining algorithm's FP-tree. The extraction submodule is used to extract association rules from frequent itemsets to determine the association between pollutant migration paths and potential pollution sources.

Images