Black soil basin ecological safety monitoring method and device based on multi-source data fusion

Abstract

The invention discloses a black soil basin ecological safety monitoring method and device based on multi-source data fusion. The method comprises the steps of obtaining an ecological environment report text, environment parameter time sequence data, structured attribute data and remote sensing image information of a target black soil basin; inputting the remote sensing image information into a pre-trained image-to-text model to obtain an ecological environment description text; performing feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environment parameter time sequence data, the structured attribute data and the remote sensing image information to obtain multi-modal fusion data; inputting the multi-modal fusion data into a pre-trained environmental parameter prediction model to obtain predicted environmental parameters of the target black soil basin in a future time period; and when the predicted environmental parameters exceed a preset safety threshold range, ecological safety early warning information is generated. The accuracy and timeliness of ecological safety monitoring and early warning of the black soil basin are improved, and the method can be widely applied to the technical field of ecological environment monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of ecological environment monitoring, and in particular to a method and device for monitoring ecological security of a black soil watershed based on multi-source data fusion. Background Art

[0002] With global climate change and intensified human activities, ecological security issues in black soil basins are becoming increasingly prominent, profoundly impacting the regional economy, society, and even the global ecosystem. Black soil, one of Earth's most precious soil resources, boasts high yield, high quality, and high efficiency, earning it the nickname "the giant panda of cultivated land." However, due to long-term overexploitation, irrational use, and climate change, black soil basins face a range of ecological challenges, including soil erosion, declining fertility, water shortages, and reduced biodiversity. These challenges pose a serious threat to the sustainable use and ecological security of black soil resources.

[0003] To effectively address ecological security issues in the Black Soil Basin and achieve scientific management and protection, intelligent monitoring and early warning technologies have emerged. By accurately acquiring and analyzing ecological and environmental data in the Black Soil Basin in real time, these technologies provide decision-makers with a scientific basis for formulating appropriate protection and management measures. However, ecological and environmental data in the Black Soil Basin are multi-source, heterogeneous, and complex. Efficiently integrating this multi-source, heterogeneous data to improve the accuracy and timeliness of monitoring and early warning is a key and challenging area of ​​current research.

[0004] Explanation of terms: Transformer: Proposed by Google in 2017, Transformer is a neural network model based on the self-attention mechanism, primarily used to process sequential data such as text and speech. This model has achieved remarkable success in the field of natural language processing (NLP) and has gradually been extended to other fields such as computer vision and audio processing.

[0005] CLIP: CLIP is a multimodal model that achieves cross-modal understanding by associating images and text through contrastive learning, supporting tasks such as zero-shot classification and image generation.

[0006] Cross-Attention: Cross-Attention is a cross-sequence attention mechanism in Transformer. By calculating the correlation between the query (target sequence) and the key value (source sequence), it dynamically integrates information from different sources to achieve contextual interaction and semantic alignment for tasks such as machine translation and text generation.

[0007] WGAN-GP: WGAN-GP is an improved version of Wasserstein GAN. By introducing a gradient penalty term instead of weight clipping, it forces the discriminator to satisfy the Lipschitz continuity constraint, thereby improving training stability and alleviating the mode collapse problem. Summary of the Invention

[0008] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0009] To this end, one purpose of an embodiment of the present invention is to provide a black soil watershed ecological security monitoring method based on multi-source data fusion, which improves the accuracy and timeliness of black soil watershed ecological security monitoring and early warning.

[0010] Another object of an embodiment of the present invention is to provide a black soil watershed ecological security monitoring device based on multi-source data fusion.

[0011] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include: In one aspect, an embodiment of the present invention provides a method for monitoring ecological security of a black soil watershed based on multi-source data fusion, comprising the following steps: Obtain ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information for the target black soil watershed; Inputting the remote sensing image information into a pre-trained image-to-text model to obtain an ecological environment description text; Performing feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain multimodal fusion data; Inputting the multimodal fusion data into a pre-trained environmental parameter prediction model to obtain predicted environmental parameters of the target black soil watershed in the future period; When the predicted environmental parameters exceed the preset safety threshold range, an ecological safety early warning message is generated.

[0012] Furthermore, in one embodiment of the present invention, the image-to-text model is trained by the following steps: Obtain preset remote sensing image samples and determine the corresponding text description information through manual annotation; Normalizing and enhancing the remote sensing image samples to obtain training samples, and mapping the text description information into word sequences; Inputting the training sample and the word sequence into the CLIP large language model for feature extraction to obtain an image embedding vector and a text embedding vector; Constructing a generative adversarial network, inputting the image embedding vector and the text embedding vector into the generative adversarial network for training, and obtaining a trained generator subnetwork and a discriminator subnetwork; The graph-to-text model is composed according to the CLIP large language model and the generator sub-network.

[0013] Furthermore, in one embodiment of the present invention, the inputting the image embedding vector and the text embedding vector into the generative adversarial network for training to obtain a trained generator subnetwork and a discriminator subnetwork specifically includes: Initialize the generator subnetwork and the discriminator subnetwork; Inputting the image embedding vector into the generator sub-network to generate a text sequence; Inputting the generated text sequence and the text embedding vector into the discriminator subnetwork to obtain the authenticity probability of the generated text sequence and the semantic matching score with the image embedding vector; The adversarial loss value is calculated based on the authenticity probability, the semantic alignment loss value is calculated based on the semantic matching score, and the gradient penalty term is determined based on the linear interpolation of the generated text sequence and the text embedding vector. Determining a first loss value of the generator sub-network according to the adversarial loss value and the gradient penalty term, and determining a second loss value of the discriminator sub-network according to the adversarial loss value and the semantic alignment loss value; The generator subnetwork and the discriminator subnetwork are jointly optimized according to the first loss value and the second loss value to obtain the trained generator subnetwork and the discriminator subnetwork.

[0014] Furthermore, in one embodiment of the present invention, the feature extraction and data fusion of the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain multimodal fusion data specifically includes: Performing data cleaning and data alignment on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information; Performing feature extraction on the aligned ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain first text features, second text features, environmental parameter features, attribute value features, and image features; The first text feature, the second text feature, the environmental parameter feature, the attribute numerical feature and the image feature are normalized, and data fusion is performed through a pre-built Transformer model to obtain the multimodal fusion data.

[0015] Furthermore, in one embodiment of the present invention, the data fusion is performed using a pre-built Transformer model to obtain the multimodal fusion data, which specifically includes: Performing feature splicing on the normalized first text feature, the second text feature, the environmental parameter feature, the attribute value feature, and the image feature to obtain a first spliced ​​feature; Performing position encoding on the first splicing feature to obtain a second splicing feature; A multi-head self-attention mechanism is added to the second splicing feature to obtain the multimodal fusion data.

[0016] Furthermore, in one embodiment of the present invention, the environmental parameter prediction model is trained by the following steps: Acquiring historical multimodal fusion data of the target black soil watershed; Sampling the historical multimodal fusion data according to a preset prediction time step to obtain multimodal fusion samples and corresponding environmental parameter labels; Inputting the multimodal fusion samples into a pre-built radial basis function neural network to obtain environmental parameter prediction values; Determining a third loss value according to a root mean square error between the environmental parameter prediction value and the environmental parameter label; The parameters of the radial basis function neural network are updated according to the third loss value to obtain the trained environmental parameter prediction model.

[0017] Furthermore, in one embodiment of the present invention, when the predicted environmental parameter exceeds a preset safety threshold range, generating ecological safety warning information specifically includes: When the predicted environmental parameter exceeds a preset safety threshold range, determining the target parameter index and target time node that exceed the safety threshold range; The ecological safety early warning information is generated according to the target parameter indicators and the target time node.

[0018] On the other hand, an embodiment of the present invention provides a black soil watershed ecological security monitoring device based on multi-source data fusion, comprising: Multi-source data acquisition module, used to obtain ecological environment report text, environmental parameter time series data, structured attribute data and remote sensing image information of the target black soil watershed; A picture-to-text module is used to input the remote sensing image information into a pre-trained picture-to-text model to obtain a text describing the ecological environment; A multi-source data fusion module is used to perform feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data and the remote sensing image information to obtain multimodal fusion data; An environmental parameter prediction module, configured to input the multimodal fusion data into a pre-trained environmental parameter prediction model to obtain predicted environmental parameters of the target black soil watershed in the future period; The ecological safety early warning module is used to generate ecological safety early warning information when the predicted environmental parameters exceed the preset safety threshold range.

[0019] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored on the memory and runnable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the black soil basin ecological security monitoring method based on multi-source data fusion as described above is realized.

[0020] On the other hand, an embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the black soil basin ecological security monitoring method based on multi-source data fusion as described above.

[0021] The advantages and benefits of the present invention will be described in part in the following description and will become apparent from the following description or learned through practice of the present invention: The embodiment of the present invention obtains an ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information for a target black soil watershed. The remote sensing image information is input into a pre-trained graph-based model to obtain an ecological environment description text. Feature extraction and data fusion are performed on the ecological environment description text, ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information to obtain multimodal fusion data. The multimodal fusion data is input into a pre-trained environmental parameter prediction model to obtain predicted environmental parameters for the target black soil watershed in the future. When the predicted environmental parameters exceed a preset safety threshold, an ecological safety warning message is generated. The embodiment of the present invention automatically generates an ecological environment description text based on the remote sensing image information of the target black soil watershed using a graph-based model. The collected ecological environment report text, environmental parameter time series data, and structured attribute data are combined to generate multimodal fusion data. The pre-trained environmental parameter prediction model is used to predict the environmental parameter change trend of the multimodal fusion data. When the predicted environmental parameters exceed the safety threshold, an early warning message is issued, thereby improving the accuracy and timeliness of ecological safety monitoring and early warning in the black soil watershed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A flowchart of a method for monitoring ecological security in a black soil watershed based on multi-source data fusion according to an embodiment of the present invention; Figure 2 A flowchart of the steps for training a graph-to-text model provided in an embodiment of the present invention; Figure 3 A flowchart of step S204 provided in an embodiment of the present invention; Figure 4 A flowchart of step S103 provided in an embodiment of the present invention; Figure 5 A flowchart of step S1033 provided in an embodiment of the present invention; Figure 6 A flowchart of the steps of the training environment parameter prediction model provided by an embodiment of the present invention; Figure 7 A flowchart of step S105 provided in an embodiment of the present invention; Figure 8A schematic diagram of the structure of a black soil watershed ecological security monitoring device based on multi-source data fusion provided by an embodiment of the present invention; Figure 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention; Figure 10 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limitations on the present application. It should be noted that, although the functional modules are divided in the system schematic and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system schematic or the order in the flow chart. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and no limitation is placed on the order between the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0025] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of "first" or "second", it is only used to distinguish technical features and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this document are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0026] The method for monitoring the ecological security of a black soil watershed based on multi-source data fusion provided in the embodiments of the present application can be applied to a terminal, can be applied to a server side, or can be software running on a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the method for monitoring the ecological security of a black soil watershed based on multi-source data fusion, etc., but is not limited to the above forms.

[0027] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0028] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0029] Multi-source heterogeneous data fusion is a data processing technology that integrates and consolidates data from diverse sources, formats, and structures to extract valuable information and patterns. In the field of ecological and environmental monitoring, multi-source heterogeneous data fusion can integrate multiple data sources, such as satellite remote sensing data, ground observation data, meteorological data, soil data, and hydrological data, to form a comprehensive and accurate ecological and environmental monitoring network. Comprehensive analysis of this data can reveal the evolutionary patterns and potential risks of the ecological environment, providing a scientific basis for ecological security early warning.

[0030] like Figure 1 FIG2 is a flowchart of a method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for monitoring ecological security of a black soil watershed based on multi-source data fusion, which specifically includes the following steps: S101. Obtaining ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information of the target black soil watershed; S102, inputting the remote sensing image information into a pre-trained image-to-text model to obtain a description text of the ecological environment; S103, performing feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain multimodal fusion data; S104, inputting the multimodal fusion data into a pre-trained environmental parameter prediction model to obtain the predicted environmental parameters of the target black soil watershed in the future period; S105. When the predicted environmental parameters exceed the preset safety threshold range, an ecological safety warning information is generated.

[0031] The embodiment of the present invention automatically generates ecological environment description text based on the remote sensing image information of the target black soil watershed through a graph-to-text model, combines the collected ecological environment report text, environmental parameter time series data and structured attribute data to generate multimodal fusion data, predicts the environmental parameter change trend of the multimodal fusion data through a pre-trained environmental parameter prediction model, and issues an early warning when the predicted environmental parameters exceed the safety threshold range, thereby improving the accuracy and timeliness of ecological security monitoring and early warning in the black soil watershed.

[0032] The main reference data indicators for intelligent monitoring of ecological security in the Black Soil Basin are divided into four categories: ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information. Among them, remote sensing image information includes but is not limited to vegetation coverage, land use type, land degradation area, water eutrophication Chengdu and flood basin range, and is obtained by remote sensing images taken by high-resolution satellites and drones; ecological environment report text usually exists in unstructured or semi-structured form, including but not limited to expert experience descriptions of the ecological environment, research reports, and monitoring reports; environmental parameter time series data are data recorded in chronological order, including but not limited to precipitation, temperature changes, water quality pH value, dissolved oxygen content, and soil organic matter content, and are obtained by sampling in chronological order by corresponding high-precision sensors; structured attribute data are data stored in tabular form, including numerical and category information, specifically, including but not limited to soil pH value, soil heavy metal content, surface water and groundwater reserves, and soil erosion, and the data are obtained through manual laboratory measurement, monitoring station records, and sensor sampling.

[0033] like Figure 2 The figure shows a flow chart of steps for training a graph-to-text model according to an embodiment of the present invention. Figure 2 As an optional implementation, the graph-to-text model is trained by the following steps: S201, obtaining a preset remote sensing image sample, and determining corresponding text description information through manual annotation; S202, performing normalization and image enhancement on the remote sensing image samples to obtain training samples, and mapping the text description information into a word sequence; S203, inputting the training sample and word sequence into the CLIP large language model for feature extraction to obtain image embedding vectors and text embedding vectors; S204: Build a generative adversarial network, input the image embedding vector and the text embedding vector into the generative adversarial network for training, and obtain a trained generator sub-network and a discriminator sub-network; S205. A graph-to-text model is constructed based on the CLIP large language model and the generator sub-network.

[0034] Specifically, the embodiments of the present invention use an intelligent image-to-text generation method, combined with artificial intelligence, generative AI, and multimodal data interaction technology, to intelligently generate accurate, reliable, and intuitive text descriptions and ecological environment evaluations based on remote sensing image data captured by satellites, drones, or monitoring stations, thereby assisting monitoring personnel in quickly making accurate judgments on ecological security.

[0035] The embodiment of the present invention combines the generative adversarial network and the pre-trained CLIP large language model to train an image-to-text model for generating text from images. The specific process is as follows: 1) Data collection, mainly relying on low-orbit satellites and drones to obtain high-definition remote sensing images and infrared images I ; 2) Build a data set, with each high-definition remote sensing and infrared image given a professional text description by professional ecological security monitoring experts As labels, images and text descriptions need to be strictly aligned. To ensure the semantic consistency between images and text, the CLIP Score indicator is used to quantitatively measure the semantic consistency between images and text descriptions given by experts. The specific calculation formula is:

[0036] in, is the embedding vector of the image; is the embedding vector of the text, is the cosine similarity; 3) Image preprocessing: First, standardize the image. Scaling the captured high-definition remote sensing images and infrared images to a fixed resolution and normalizing them. The normalization operation is based on:

[0037] in, I is the pixel value matrix of the original input image; is the mean of each color channel, which is used to centralize the data to zero mean; is the standard deviation of each color channel, which is used to scale the data to unit variance; Perform image enhancement on the normalized images, including but not limited to random cropping, horizontal flipping, and color jittering, to increase the image diversity of the dataset and prevent overfitting; 4) Text preprocessing, using a pre-trained tokenizer to map the text into a sequence of tokens:

[0038] Text length padding / truncation: unify the length of the mapped word sequence, fill in zeros if the length is insufficient, and truncate the excessive length into two sequences; 5) Cross-modal feature extraction: using the pre-trained CLIP large language model to extract cross-modal features from images and text; Extract image embedding vectors through CLIP’s Vision Transformer:

[0039] Extract text embedding vectors through CLIP's text encoder for supervised training:

[0040] 6) Construct a generative adversarial network, including a generator subnetwork and a discriminator subnetwork. Input the image embedding vector and text embedding vector into the generative adversarial network for training, and obtain the trained generator subnetwork and discriminator subnetwork. 7) A graph-to-text model is constructed based on the CLIP large language model and the generator sub-network.

[0041] like Figure 3 FIG2 is a flowchart of step S204 provided in an embodiment of the present invention, referring to FIG2 Figure 3 As an optional implementation, the image embedding vector and the text embedding vector are input into a generative adversarial network for training to obtain a trained generator subnetwork and a discriminator subnetwork, which specifically includes: S2041. Initialize the generator sub-network and the discriminator sub-network; S2042. Input the image embedding vector into the generator sub-network to obtain a generated text sequence; S2043, inputting the generated text sequence and the text embedding vector into the discriminator sub-network to obtain the authenticity probability of the generated text sequence and the semantic matching score with the image embedding vector; S2044. Calculate the adversarial loss value based on the authenticity probability, calculate the semantic alignment loss value based on the semantic matching score, and determine the gradient penalty term based on the linear interpolation of the generated text sequence and the text embedding vector S2045. Determine a first loss value of the generator sub-network based on the adversarial loss value and the gradient penalty term, and determine a second loss value of the discriminator sub-network based on the adversarial loss value and the semantic alignment loss value; S2046. Jointly optimize the generator sub-network and the discriminator sub-network according to the first loss value and the second loss value to obtain trained generator sub-network and discriminator sub-network.

[0042] Specifically, the generator subnetwork input is the image embedding , the output is the probability distribution of generated text sequence ; The structure of the generator sub-network first includes a conditional injection module, Projection is the initial state of the decoder:

[0043] in, is the image feature extracted by the pre-trained CLIP visual encoder; is the weight matrix, which is also the learnable parameter; It is a bias term and also a learnable parameter.

[0044] Then, we build the generator sub-network architecture by referring to the decoder structure of Transformer and generate text by autoregression:

[0045] in, It is the word unit generated at the last moment, which is converted into a vector through the word embedding layer , as the input of the decoder at the current moment, guiding the generation of the next word; , is the hidden state of the previous moment, which stores the context information of the historical generated content; The cross-attention mechanism is injected into each layer of the decoder to ensure that each generated word is aligned with the image semantics. It is the hidden state at the current moment, which integrates the information of the previous word, historical state and image features; is the weight matrix, and the hidden state Mapped to a probability distribution over the vocabulary; Predict the generated word at the current moment The probability of , and the most likely word is selected after SoftMax normalization.

[0046] At the moment , input start character embedded , combined with generate ;calculate , sample to get word unit ; Repeat this process until a terminator is generated or the maximum length is reached .

[0047] In the Transformer Decoder, the image features and text generation are dynamically aligned through the cross-attention mechanism; the input of the discriminator subnetwork is the text sequence , the output is the probability of authenticity and semantic matching score; the discriminator sub-network architecture first generates text features extracted by the CLIP text encoder:

[0048] in, Generate text features from the network for the generator.

[0049] Calculate the authenticity of the generated text features through the adversarial discriminative head:

[0050] in, The embedding vector of the text generated by the generator after passing through the CLIP text encoder, the dimension is consistent with the output of CLIP; is the weight matrix of the linear layer, mapping the embedding vector to a scalar value; is the bias, which is a scalar value; is a sigmoid function that converts the output of the linear layer into a probability between 0 and 1, indicating the probability that the discriminator believes the text is real.

[0051] Calculate the cosine similarity with the image features to get the semantic alignment score as feedback and scoring for the generator subnetwork:

[0052] in, Image features extracted by CLIP’s Vision Transformer.

[0053] Design the loss function and use joint adversarial loss and CLIP semantic alignment loss as the loss function for model training.

[0054] The goal of the adversarial loss is to make it difficult for the discriminator to identify the generated text as a fake sample. It consists of two parts: Wasserstein GAN loss and gradient penalty term (WGAN-GP). Wasserstein GAN loss is used to improve training stability and is expressed as:

[0055] in, It is the adversarial loss value, reflecting the difference between the generated sample and the real sample from the perspective of the discriminator; To take the expectation of the real text distribution; Take the expectation of the text distribution generated by the generator sub-network; The gradient penalty term expression is:

[0056] in, is the linear interpolation of real and generated samples, Representation Discriminator D Interpolation samples The gradient, represents the L2 norm.

[0057] CLIP semantic alignment loss is used to maximize the cross-modal similarity between generated text and images, expressed as:

[0058] in, N is the batch size, and the discrimination ability is enhanced by negative sample comparison; The joint total loss function includes the total loss of the generator and the total loss of the discriminator. The total loss expression of the generator is:

[0059] The total loss expression of the discriminator is:

[0060] in, and These are typical hyperparameters.

[0061] Formulate a model training strategy and adopt a two-stage model training strategy, including generator pre-training and adversarial fine-tuning. In the generator pre-training stage, only the , use the teacher to force the injection of real words, initialize the reasonable text generation ability, and gradually reduce the teacher force ratio in the subsequent training process to improve the generation robustness; in the adversarial fine-tuning stage, fix the CLIP encoder and jointly optimize and ,Gumbel-SoftMax relaxation is used to solve the gradient break caused by the discreteness of text.

[0062] like Figure 4 FIG. 1 is a flowchart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 4 As an optional implementation method, feature extraction and data fusion are performed on the ecological environment description text, ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information to obtain multimodal fusion data, which specifically includes: S1031. Perform data cleaning and data alignment on ecological environment description texts, ecological environment report texts, environmental parameter time series data, structured attribute data, and remote sensing image information; S1032. Perform feature extraction on the aligned ecological environment description text, ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information to obtain first text features, second text features, environmental parameter features, attribute numerical features, and image features. S1033. Normalize the first text feature, the second text feature, the environmental parameter feature, the attribute numerical feature, and the image feature, and perform data fusion through a pre-built Transformer model to obtain multimodal fusion data.

[0063] Specifically, the present invention uses a multi-source heterogeneous data intelligent fusion method to integrate data from different sources (sensors), formats, and structures, including but not limited to text, images, and time series, into a unified representation to support subsequent downstream tasks. The specific process is as follows: 1) Data cleaning, including missing value processing, noise processing, and outlier processing. The missing value processing method uses the mean method, which adds the values ​​of the two data before and after the missing value and takes the average value as the value of the missing value; Noise processing adopts the data denoising method based on wavelet transform. The specific operations include: (1) Signal decomposition: Perform multi-scale wavelet decomposition on the input signal data to obtain approximate coefficients (low-frequency components) and detail coefficients (high-frequency components). The calculation formula for the decomposition is:

[0064] in, are the wavelet coefficients, It is a set of wavelet functions generated by translating and scaling the wavelet basis function, satisfying , j is the scale parameter, k is the translation parameter; (2) Threshold processing, wavelet coefficients Perform hard threshold processing to remove noise components. The hard threshold calculation expression is:

[0065] in, is the threshold, usually given by the formula Calculated; is the noise standard deviation, is the signal length; (3) Signal reconstruction, using the processed wavelet coefficients Reconstruct the denoised data:

[0066] The outlier processing adopts the multivariate linear regression method to draw the regression graph and remove the outlier values ​​that deviate significantly from the regression curve; 2) Data alignment, which includes time alignment and spatial alignment. Time alignment interpolates or resamples data with different timestamps; spatial alignment aligns or interpolates data with different spatial resolutions. 3) Feature extraction: For text data, word embedding operations are used to convert text into vectors; for image data, convolutional neural networks are used to extract potential image features; for time series data, long short-term memory networks are used to extract time series features; for structured data, numerical features are directly used or standardized; 4) Data standardization: normalize or standardize all features to the same dimension; 5) Perform data fusion through the pre-built Transformer model to obtain multimodal fusion data.

[0067] like Figure 5 FIG. 1 is a flowchart of step S1033 provided in an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, data fusion is performed through a pre-built Transformer model to obtain multimodal fusion data, which specifically includes: S10331, performing feature splicing on the normalized first text feature, the second text feature, the environmental parameter feature, the attribute value feature, and the image feature to obtain a first spliced ​​feature; S10332. Position-encode the first splicing feature to obtain a second splicing feature; S10333. Add a multi-head self-attention mechanism to the second splicing feature to obtain multimodal fusion data.

[0068] Specifically, for each data source, the extracted features are expressed as ,in is the number of samples, It is i The feature dimensions of all data sources are combined into a unified input representation:

[0069] in ; Since the Transformer itself does not have position information, position encoding needs to be added:

[0070] in,P is the positional encoding matrix, which can be generated using sine and cosine functions; the positional encoding is added to the input representation:

[0071] Add a multi-head self-attention mechanism. The multi-head self-attention mechanism captures the dependencies in the data by calculating the relationship between the query (Q), key (K), and value (V). The calculation expression is:

[0072]

[0073]

[0074] in, 、 、 is a learnable weight matrix; Calculate the attention score:

[0075] Calculate the weighted value:

[0076] Use multi-head attention mechanism to capture the characteristics of different subspaces:

[0077] The calculation of each head is:

[0078] is the output weight matrix; Perform nonlinear transformation on the output of multi-head attention through feedforward neural network:

[0079] in, and is bias; By adding residual connections and layer normalization after each sub-layer, a Transformer model can be obtained. The normalized first text features, second text features, environmental parameter features, attribute numerical features, and image features are input into the Transformer model to obtain multimodal fusion data.

[0080] like Figure 6 The figure shows a flow chart of the steps of the training environment parameter prediction model provided by the embodiment of the present invention, referring to Figure 6 As an optional implementation, the environmental parameter prediction model is trained by the following steps: S301, obtaining historical multimodal fusion data of the target black soil watershed; S302: Sampling historical multimodal fusion data according to a preset prediction time step to obtain multimodal fusion samples and corresponding environmental parameter labels; S303, inputting the multimodal fusion sample into a pre-built radial basis function neural network to obtain the predicted value of the environmental parameter; S304, determining a third loss value according to a root mean square error between the environmental parameter prediction value and the environmental parameter label; S305. Update the parameters of the radial basis function neural network according to the third loss value to obtain a trained environmental parameter prediction model.

[0081] Specifically, the present invention uses in-depth mining and learning of the distribution and variation characteristics of multi-source heterogeneous data, especially time series data, to predict the variation trends of ecological security indicators in the black soil watershed based on the current time series data and obtain predicted values ​​of environmental parameters. The specific process is as follows: 1) Obtain historical multimodal fusion data of the target black soil watershed and divide it into training, validation, and test sets in a ratio of 7:2:1; 2) Determine the prediction time step and sample the data at this time interval to obtain multimodal fusion samples and corresponding environmental parameter labels; 3) Build a radial basis function neural network, train the neural network model on the training and validation datasets, save the trained weight model file, load the weight model file, complete the model performance test on the test dataset, and select the neural network model with excellent performance for deployment; 4) The model performance test uses the root mean square error indicator, which is expressed as:

[0082] in, is the number of samples, For the i True value, For the i predicted values.

[0083] like Figure 7 FIG. 1 is a flowchart of step S105 provided in an embodiment of the present invention, referring to FIG. Figure 7 As an optional implementation, when the predicted environmental parameters exceed the preset safety threshold range, an ecological safety warning message is generated, which specifically includes: S1051. When the predicted environmental parameter exceeds a preset safety threshold range, determine the target parameter index and target time node that exceed the safety threshold range; S1052. Generate ecological security early warning information based on target parameter indicators and target time nodes.

[0084] Specifically, when an embodiment of the present invention predicts that a certain parameter indicator may exceed a corresponding safety threshold, an early warning is given in a timely manner, and a specific time point at which the threshold may be exceeded is given to assist monitoring personnel in responding in advance.

[0085] The above describes the method flow of an embodiment of the present invention. It is understood that the embodiment of the present invention automatically generates ecological environment description text based on remote sensing image information of the target black soil watershed through a graph-to-text model, combines the collected ecological environment report text, environmental parameter time series data, and structured attribute data to generate multimodal fusion data, predicts the environmental parameter change trend of the multimodal fusion data through a pre-trained environmental parameter prediction model, and issues an early warning when the predicted environmental parameters exceed the safety threshold range, thereby improving the accuracy and timeliness of ecological security monitoring and early warning in the black soil watershed.

[0086] Compared with the prior art, the embodiments of the present invention also have the following advantages: 1) This embodiment of the present invention proposes an image-to-text generation method based on a combination of a generative adversarial network (GAN) and a pre-trained CLIP large language model. This approach leverages the generative power of GAN and the cross-modal understanding capabilities of CLIP to achieve image-to-text conversion. The generator subnetwork generates text, while the discriminator subnetwork evaluates the authenticity of the generated text and its semantic consistency with the image. Adversarial training improves the quality and reliability of the generated text. Furthermore, the CLIP model extracts features from both the image and text, and calculates the CLIP score, which serves as a measure of semantic consistency, guiding the generator to produce text descriptions that align with the image content. A two-stage training strategy (generator pre-training and adversarial fine-tuning) improves model training efficiency and stability. In the generator pre-training phase, a teacher is used to inject real vocabulary to initialize reasonable text generation capabilities. In the adversarial fine-tuning phase, the generator and discriminator are jointly optimized to further improve the quality of the generated text and its semantic consistency with the image.

[0087] 2) This paper proposes a multi-source heterogeneous data fusion method based on the Transformer model architecture for processing the diverse data types (images, text, time series, and structured data) involved in ecological security monitoring in black soil watersheds. This innovative approach applies the Transformer model, which excels at capturing long-range dependencies and contextual information in sequence data. By leveraging the Transformer model's self-attention mechanism, this method can better mine potential connections between different data sources and achieve deep fusion. Different feature extraction methods (CNN, word embedding, LSTM, etc.) are employed for different types of data (images, text, time series, etc.) and integrated into the Transformer model input to achieve efficient feature fusion.

[0088] 3) This invention uses a radial basis function neural network (RBFNN) to predict the changing trends of ecological security indicators in the black soil watershed and issues early warning information based on the predicted results. RBFNN has strong nonlinear fitting and generalization capabilities, making it effective for predicting time series data. By training on time series data fused from multiple sources of heterogeneous data, the RBFNN learns the changing patterns of indicators and predicts future trends. Based on the predicted results, it sets appropriate early warning thresholds. When a predicted indicator is likely to exceed the threshold, a timely warning is issued, along with the specific time point at which the threshold is likely to be exceeded, providing decision support for monitoring personnel.

[0089] The embodiments of the present invention are of great significance for ensuring the sustainable utilization of black soil resources, promoting the green development of regional economies, and maintaining global ecological and environmental security: 1) Improve the accuracy and timeliness of monitoring and early warning Ecological and environmental data in black soil watersheds is multi-source, heterogeneous, and complex, making it difficult for traditional monitoring and early warning methods to comprehensively and accurately acquire and analyze this data. The embodiments of the present invention can improve the comprehensiveness and accuracy of data by integrating and consolidating multiple data sources. At the same time, by utilizing advanced data processing and analysis technologies, real-time monitoring and early warning of the ecological environment in black soil watersheds can be achieved, enabling timely identification of potential risks and improving the timeliness of early warnings. This is of great significance for preventing and reducing the occurrence of ecological disasters and ensuring the sustainable utilization of black soil resources.

[0090] 2) Promote the innovation and development of ecological environment monitoring technology Research on intelligent monitoring and early warning methods based on multi-source heterogeneous data fusion will promote innovation and development in ecological and environmental monitoring technology. Traditional ecological and environmental monitoring technologies often rely on a single monitoring method and data source, making it difficult to achieve comprehensive and accurate monitoring. Embodiments of the present invention can integrate multiple data sources and monitoring methods to form a comprehensive and accurate ecological and environmental monitoring network. This will provide new ideas and methods for innovation in ecological and environmental monitoring technology, driving its development towards greater efficiency and intelligence.

[0091] 3) Promote green development of the regional economy The Black Soil Basin is an important grain production base and agricultural economic zone, and its ecological security is directly related to the sustainable development of the regional economy. The embodiments of the present invention can achieve real-time monitoring and early warning of the ecological environment in the Black Soil Basin, providing decision-makers with a scientific basis for formulating reasonable protection and management measures. This will help reduce the occurrence of ecological disasters, ensure the stability of agricultural production, and promote the green development of the regional economy. Furthermore, the application of intelligent monitoring and early warning technology can also improve the intelligence level of agricultural production and promote the process of agricultural modernization.

[0092] 4) Improving global ecological and environmental security As one of the planet's most precious soil resources, the ecological security of black soil basins is crucial not only to the sustainable development of regional economies but also to the stability and security of the global ecological environment. The embodiments of the present invention enable real-time monitoring and early warning of the ecological environment in black soil basins, enabling timely identification of potential risks and providing a scientific basis for global ecological and environmental security. This will help promote the protection and management of the global ecological environment and foster its sustainable development.

[0093] 5) Promote interdisciplinary cooperation and communication The embodiments of this invention involve knowledge and technologies from multiple disciplines, including information technology, ecology, geography, and environmental science. This will promote interdisciplinary collaboration and exchange, driving the development and progress of related disciplines. Furthermore, interdisciplinary collaboration and exchange will contribute to more comprehensive and in-depth research results, providing strong support for the improvement and promotion of intelligent monitoring and early warning methods for ecological security in black soil watersheds.

[0094] 6) Provide scientific basis for policy making When formulating policies to protect the ecological security of the Black Soil River Basin, relevant agencies require comprehensive and accurate data support. Embodiments of the present invention can provide real-time, accurate data and information, providing a scientific basis for policymakers. This will facilitate the formulation of more rational and effective policies and measures, promoting the protection and management of the ecological security of the Black Soil River Basin.

[0095] 7) Promoting the development of ecological civilization By monitoring and providing early warnings of ecological and environmental changes in the Black Soil Basin in real time, ecological problems can be promptly identified and addressed, promoting ecosystem restoration and protection. This will contribute to building a more beautiful and livable ecological environment and promoting the development of an ecological civilization.

[0096] In summary, the embodiments of the present invention are of great significance for ensuring the sustainable utilization of black soil resources, promoting the green development of the regional economy, and maintaining global ecological and environmental security.

[0097] like Figure 8 The diagram shows the structure of the black soil basin ecological security monitoring device based on multi-source data fusion provided by the embodiment of the present invention. Figure 8 The embodiment of the present invention provides a black soil watershed ecological security monitoring device based on multi-source data fusion, comprising: Multi-source data acquisition module, used to obtain ecological environment report text, environmental parameter time series data, structured attribute data and remote sensing image information of the target black soil watershed; The image-to-text module is used to input remote sensing image information into a pre-trained image-to-text model to obtain a text describing the ecological environment; Multi-source data fusion module, used to extract features and fuse data from ecological environment description text, ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information to obtain multimodal fusion data; The environmental parameter prediction module is used to input the multimodal fusion data into the pre-trained environmental parameter prediction model to obtain the predicted environmental parameters of the target black soil watershed in the future period; The ecological safety early warning module is used to generate ecological safety early warning information when the predicted environmental parameters exceed the preset safety threshold range.

[0098] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0099] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned method for monitoring ecological security in a black soil watershed based on multi-source data fusion is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0100] like Figure 9 FIG2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention, referring to FIG2 Figure 9 , an embodiment of the present invention provides an electronic device, including: The processor 901 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention. Memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 902 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program code is stored in memory 902 and is called by processor 901 to execute the black soil watershed ecological security monitoring method based on multi-source data fusion according to the embodiments of the present invention. Input / output interface 903, used to implement information input and output; Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 ); The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0101] like Figure 10 FIG2 is a schematic diagram of the structure of the storage medium provided by the embodiment of the present invention, referring to FIG2 Figure 10 An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1001, and the one or more programs 1001 can be executed by one or more processors to implement the above-mentioned black soil basin ecological security monitoring method based on multi-source data fusion.

[0102] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0104] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0105] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0106] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods in various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0107] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0108] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0109] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0110] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0111] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0112] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for monitoring ecological security of black soil watershed based on multi-source data fusion, characterized in that: The following steps are involved: Obtain ecological environment report text, environmental parameter time series data, structured attribute data, and remote sensing image information for the target black soil watershed; Inputting the remote sensing image information into a pre-trained image-to-text model to obtain an ecological environment description text; Performing feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain multimodal fusion data; Inputting the multimodal fusion data into a pre-trained environmental parameter prediction model to obtain predicted environmental parameters of the target black soil watershed in the future period; When the predicted environmental parameters exceed the preset safety threshold range, an ecological safety early warning message is generated.

2. The method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to claim 1, characterized in that: The graph-to-text model is trained by the following steps: Obtain preset remote sensing image samples and determine the corresponding text description information through manual annotation; Normalizing and enhancing the remote sensing image samples to obtain training samples, and mapping the text description information into word sequences; Inputting the training sample and the word sequence into the CLIP large language model for feature extraction to obtain an image embedding vector and a text embedding vector; Constructing a generative adversarial network, inputting the image embedding vector and the text embedding vector into the generative adversarial network for training, and obtaining a trained generator subnetwork and a discriminator subnetwork; The graph-to-text model is composed according to the CLIP large language model and the generator sub-network.

3. The method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to claim 2, characterized in that: The step of inputting the image embedding vector and the text embedding vector into the generative adversarial network for training to obtain a trained generator subnetwork and a discriminator subnetwork specifically includes: Initialize the generator subnetwork and the discriminator subnetwork; Inputting the image embedding vector into the generator sub-network to generate a text sequence; Inputting the generated text sequence and the text embedding vector into the discriminator subnetwork to obtain the authenticity probability of the generated text sequence and the semantic matching score with the image embedding vector; Calculating an adversarial loss value based on the authenticity probability, calculating a semantic alignment loss value based on the semantic matching score, and determining a gradient penalty term based on linear interpolation of the generated text sequence and the text embedding vector; Determining a first loss value of the generator sub-network according to the adversarial loss value and the gradient penalty term, and determining a second loss value of the discriminator sub-network according to the adversarial loss value and the semantic alignment loss value; The generator subnetwork and the discriminator subnetwork are jointly optimized according to the first loss value and the second loss value to obtain the trained generator subnetwork and the discriminator subnetwork.

4. The method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to claim 1, characterized in that: The step of performing feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain multimodal fusion data specifically includes: Performing data cleaning and data alignment on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information; Performing feature extraction on the aligned ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data, and the remote sensing image information to obtain first text features, second text features, environmental parameter features, attribute value features, and image features; The first text feature, the second text feature, the environmental parameter feature, the attribute numerical feature and the image feature are normalized, and data fusion is performed through a pre-built Transformer model to obtain the multimodal fusion data.

5. The method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to claim 4 is characterized in that: The data fusion is performed by using a pre-built Transformer model to obtain the multimodal fusion data, which specifically includes: Performing feature splicing on the normalized first text feature, the second text feature, the environmental parameter feature, the attribute value feature, and the image feature to obtain a first spliced ​​feature; Performing position encoding on the first splicing feature to obtain a second splicing feature; A multi-head self-attention mechanism is added to the second splicing feature to obtain the multimodal fusion data.

6. The method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to claim 1, characterized in that: The environmental parameter prediction model is trained by the following steps: Acquiring historical multimodal fusion data of the target black soil watershed; Sampling the historical multimodal fusion data according to a preset prediction time step to obtain multimodal fusion samples and corresponding environmental parameter labels; Inputting the multimodal fusion samples into a pre-built radial basis function neural network to obtain environmental parameter prediction values; Determining a third loss value according to a root mean square error between the environmental parameter prediction value and the environmental parameter label; The parameters of the radial basis function neural network are updated according to the third loss value to obtain the trained environmental parameter prediction model.

7. A method for monitoring ecological security of a black soil watershed based on multi-source data fusion according to any one of claims 1 to 6, characterized in that: When the predicted environmental parameters exceed the preset safety threshold range, generating ecological safety warning information, which specifically includes: When the predicted environmental parameter exceeds a preset safety threshold range, determining the target parameter index and target time node that exceed the safety threshold range; The ecological safety early warning information is generated according to the target parameter indicators and the target time node.

8. A black soil watershed ecological security monitoring device based on multi-source data fusion, characterized in that: include: Multi-source data acquisition module, used to obtain ecological environment report text, environmental parameter time series data, structured attribute data and remote sensing image information of the target black soil watershed; A picture-to-text module is used to input the remote sensing image information into a pre-trained picture-to-text model to obtain a text describing the ecological environment; A multi-source data fusion module is used to perform feature extraction and data fusion on the ecological environment description text, the ecological environment report text, the environmental parameter time series data, the structured attribute data and the remote sensing image information to obtain multimodal fusion data; An environmental parameter prediction module, configured to input the multimodal fusion data into a pre-trained environmental parameter prediction model to obtain predicted environmental parameters of the target black soil watershed in the future period; The ecological safety early warning module is used to generate ecological safety early warning information when the predicted environmental parameters exceed the preset safety threshold range.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the black soil watershed ecological security monitoring method based on multi-source data fusion as described in any one of claims 1 to 7 are realized.

10. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the black soil basin ecological security monitoring method based on multi-source data fusion as described in any one of claims 1 to 7.

Images