Water conservancy multi-modal intelligent decision-making method and system based on model association protocol

Abstract

The invention discloses a water conservancy multi-modal intelligent decision-making method based on a model association protocol, and the method comprises the following steps: obtaining hydrological, meteorological and remote sensing data, and carrying out the preprocessing and feature extraction respectively; performing structured coding on the extracted multi-modal features by using a data serialization protocol to generate corresponding serialization codes, and performing packaging and transmission by using an MCP protocol; at a receiving end, de-encapsulation is carried out by using a corresponding MCP protocol, the data is restored to original multi-modal features, and each modal feature is fused through a dynamic weight distribution mechanism and a cross-modal attention mechanism; inputting the fused features into a big decision model driven by a knowledge graph, and generating a flood risk level and a corresponding decision through mixed decision reasoning by means of the knowledge graph and a rule engine; according to the method, the problem of data islands is solved, efficient fusion of multi-source data is realized, the decision process is more intelligent and interpretable, and the flood risk level and the prediction accuracy of the corresponding decision are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent water conservancy technology, and particularly to a water conservancy multi-modal intelligent decision-making method and system based on a model association protocol. Background Art

[0002] With the intensification of global climate change and the frequent occurrence of extreme hydrological events, the intelligent early warning and decision-making of flood disasters have become a research hotspot in the field of water conservancy. Traditional flood risk decision-making methods mostly rely on single hydrological monitoring data and use empirical models or statistical regression methods for prediction, suffering from problems such as insufficient integration of multi-source heterogeneous data and single decision-making basis. In the prior art, although attempts have been made to introduce multi-modal data such as meteorological satellites and remote sensing images, due to the differences in spatio-temporal resolution, data format, and semantic expression of hydrological, meteorological, and remote sensing data, it is difficult to effectively align multi-modal features, resulting in a significant data island phenomenon. Especially at the data transmission level, existing systems mostly use customized interfaces to achieve data exchange, lacking a unified structured coding protocol, leading to poor cross-platform data compatibility and low transmission efficiency.

[0003] In terms of decision model construction, the current mainstream methods mainly use machine learning models for end-to-end prediction. Although the prediction accuracy has been improved to a certain extent, there are problems such as insufficient interpretability and difficulty in embedding domain knowledge. Some studies have attempted to introduce knowledge graph technology, but existing solutions mostly use static knowledge representation methods, lacking a dynamic association mechanism with real-time monitoring data and being difficult to achieve effective coordination between knowledge-driven and data-driven. In addition, traditional decision engines mostly rely on a single reasoning mechanism, and when dealing with complex and changeable flood scenarios, problems such as inaccurate decision confidence evaluation and difficult resolution of multi-source evidence conflicts are likely to occur. Summary of the Invention

[0004] The purpose of the present invention is to address the problems existing in the prior art, and provide a water conservancy multi-modal intelligent decision-making method and system based on a model association protocol, which solves the data island problem, realizes the efficient integration of multi-source data, makes the decision-making process more intelligent and interpretable, and improves the prediction accuracy of flood risk levels and corresponding decisions.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A water conservancy multi-modal intelligent decision-making method based on a model association protocol, comprising the following steps:

[0007] S1. Obtain hydrological, meteorological, and remote sensing data, and perform preprocessing and feature extraction respectively;

[0008] S2. Use a data serialization protocol to perform structured coding on the extracted multi-modal features, generate corresponding serialization codes, and encapsulate and transmit them using the MCP protocol;

[0009] S3. At the receiving end, use the corresponding MCP protocol for decapsulation to restore the data to the original multi-modal features, and fuse the features of each modality through a dynamic weight allocation mechanism and a cross-modal attention mechanism;

[0010] S4. Input the fused features into the knowledge graph-driven decision-making large model, and with the help of the knowledge graph and the rule engine, generate the flood risk level and corresponding decisions through hybrid decision-making reasoning.

[0011] In step S1, preprocess and extract features from the hydrological data, specifically including:

[0012] S1.1. Perform OCR recognition and term standardization processing on historical hydrological reports to generate structured text data;

[0013] S1.2. Adopt a dynamic masking strategy to perform targeted masking by identifying hydrological entities to enhance the model's representation ability for professional terms;

[0014] S1.3. Input the processed text data into an improved natural language processing model for multi-granularity encoding to generate semantic vectors; among them, the improved natural language processing model enhances semantic understanding through domain pre-training and realizes multi-granularity feature fusion by combining character-level embedding, sentence-level attention, and document-level Bi-LSTM.

[0015] In step S1, preprocess and extract features from the meteorological data, specifically including:

[0016] S1.4. Construct a basin map structure, use hydrological stations, rainfall stations, and reservoir gates as nodes, and define directed edges based on the water system topological relationship;

[0017] S1.5. Adopt a hybrid architecture of a graph neural network and a time series model to extract node features through spatial aggregation and time series modeling, and output node-level predictions and a risk hot spot map composed of an adjacency list.

[0018] In step S1, preprocess and extract features from the remote sensing data, specifically including:

[0019] S1.6. Perform registration and fusion on multi-source remote sensing images, and extract the water body contour based on the normalized difference water index;

[0020] S1.7. Input the time series image slices into an improved computer vision model, and combine the spatial attention mechanism and the time series Transformer encoder to generate a remote sensing heat map.

[0021] In step S2, use the data serialization protocol to perform structured encoding on the extracted multi-modal features to generate corresponding serialized codes, and use the MCP protocol for encapsulation and transmission, specifically including:

[0022] S2.1. Structurally encode the generated hydrological semantic vectors, risk hotspot maps, and remote sensing thermal maps using ProtocolBuffers respectively to generate serialized codes for each modality;

[0023] S2.2. Package the serialized codes using the MCP protocol. The packaging process includes:

[0024] Add a timestamp accurate to the millisecond level for each data;

[0025] Embed spatial coordinate metadata conforming to the WGS84 standard;

[0026] Attach a confidence score with a scoring range of 0 to 1;

[0027] S2.3. Compress the packaged MCP data packets and add a data check code to the header.

[0028] In step S3, at the receiving end, use the corresponding MCP protocol to unpack, restore the data to the original multi-modal features, and fuse the features of each modality through a dynamic weight allocation mechanism and a cross-modal attention mechanism, specifically including:

[0029] S3.1. The receiving end unpacks and decompresses the data packets through the MCP protocol unpacking module and verifies them to restore the serialized codes of each modality;

[0030] S3.2. Use the corresponding Protocol Buffers decoder at the sending end to deserialize the serialized codes and restore the three original features of hydrological semantic vectors, risk hotspot maps, and remote sensing thermal maps;

[0031] S3.3. Construct a gated neural network and input the current scene parameters, including seasonal features, geographical location, and historical disaster records;

[0032] S3.4. The gated neural network dynamically calculates the weight distribution coefficients of hydrological, meteorological, and remote sensing features according to the scene parameters, where the weight of meteorological features is automatically increased during the flood season, and hydrological features are given priority during the dry season;

[0033] S3.5. Establish cross-modal feature interaction using the multi-head attention mechanism, including:

[0034] Map the features of each modality to a high-dimensional space;

[0035] Calculate the similarity score matrix between modalities;

[0036] Generate attention-weighted features based on the score matrix;

[0037] S3.6. Fuse the weighted multi-modal features through a feature concatenation layer and output a joint feature representation.

[0038] In step S4, the fused features are input into the knowledge graph-driven decision-making large model. With the help of the knowledge graph and the rule engine, the corresponding decision for the flood risk level is generated through dynamic reasoning, specifically including:

[0039] S4.1 Input the fused features into the knowledge graph constructed by the Neo4j graph database. The knowledge graph includes predefined domain entities and relationships.

[0040] S4.2 Perform hybrid decision-making reasoning, including analogical reasoning based on the knowledge graph and threshold judgment based on the rule engine.

[0041] S4.3 Adopt the D-S evidence theory to fuse multi-source evidence and calculate the confidence of each decision option.

[0042] S4.4 Output the graded risk warning and the corresponding decision; send the decision instruction to the execution system through the automated interface.

[0043] A water conservancy multi-modal intelligent decision-making system based on the model association protocol, including:

[0044] A hydrological intelligent agent preprocesses and extracts features from hydrological data; its encoding module uses the data serialization protocol to structurally encode the extracted features, generates the corresponding serialized code, and encapsulates and transmits it using the MCP protocol.

[0045] A meteorological intelligent agent preprocesses and extracts features from meteorological data; its encoding module uses the data serialization protocol to structurally encode the extracted features, generates the corresponding serialized code, and encapsulates and transmits it using the MCP protocol.

[0046] A remote sensing intelligent agent preprocesses and extracts features from remote sensing data; its encoding module uses the data serialization protocol to structurally encode the extracted features, generates the corresponding serialized code, and encapsulates and transmits it using the MCP protocol.

[0047] The multi-modal fusion module uses the corresponding MCP protocol to unpack, restores the data to the original multi-modal features, and fuses the features of each modality through the dynamic weight allocation mechanism and the cross-modal attention mechanism.

[0048] The decision-making large model generates the flood risk level and the corresponding decision through hybrid decision-making reasoning with the help of the knowledge graph and the rule engine.

[0049] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the above method steps.

[0050] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method steps are implemented.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0052] 1. A refined preprocessing process is designed for hydrological, meteorological, and remote sensing data respectively, fully mining the characteristics of each modality data, providing high-quality input for subsequent multi-modal fusion, solving the data island problem, and realizing the efficient fusion of multi-source data.

[0053] 2. The extracted multi-modal features are structurally encoded using a data serialization protocol to generate serialization codes for each modality, and are encapsulated and transmitted through the MCP protocol, ensuring the compatibility and transferability of data between different systems; at the same time, the timestamps, spatial coordinate metadata, and confidence scores added during the encapsulation process provide rich information for subsequent decision-making, making the multi-modal data more standardized and efficient during transmission and processing.

[0054] 3. The fused features are input into a knowledge graph-driven decision-making large model. With the knowledge graph constructed by the Neo4j graph database, rich domain knowledge and semantic information are provided for decision-making reasoning; when performing hybrid decision-making reasoning, analogy reasoning based on the knowledge graph and threshold judgment based on the rule engine are combined, and the D-S evidence theory is used to fuse multi-source evidence to calculate the confidence of each decision option. This way of complementary advantages of multiple decision-making methods makes the decision-making process more intelligent and interpretable, thus improving the prediction accuracy of flood risk levels and corresponding decisions. Description of the Drawings

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 It is a flowchart of the water conservancy multi-modal intelligent decision-making method in the embodiments of the present application. Detailed Embodiments

[0057] The technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0058] The sequence numbers of the steps in the description of this application do not mean the order of execution is prior or subsequent. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0059] In the description of this application specification and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions, and cannot be understood as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used in the text in some embodiments of this application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, the first table can be named the second table, and similarly, the second table can be named the first table, without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0060] The reference to "one embodiment" or "some embodiments" etc. described in this application specification means that a specific feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of this application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all of the embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0061] With the intensification of global climate change and the frequent occurrence of extreme hydrological events, the intelligent early warning and decision-making of flood disasters have become a research hotspot in the water conservancy field. Traditional flood risk decision-making methods mostly rely on single hydrological monitoring data and use empirical models or statistical regression methods for prediction, suffering from problems such as insufficient integration of multi-source heterogeneous data and single decision-making basis. In existing technologies, although attempts have been made to introduce multi-modal data such as meteorological satellites and remote sensing images, due to the differences in spatio-temporal resolution, data formats, and semantic expressions of hydrological, meteorological, and remote sensing data, it is difficult to effectively align multi-modal features, resulting in significant data island phenomena. Especially at the data transmission level, existing systems mostly use customized interfaces to achieve data exchange, lacking a unified structured coding protocol, leading to poor cross-platform data compatibility and low transmission efficiency.

[0062] In terms of decision model construction, the current mainstream methods mainly use machine learning models for end-to-end prediction. Although the prediction accuracy has been improved to a certain extent, there are problems such as insufficient interpretability and difficulty in embedding domain knowledge. Some studies have attempted to introduce knowledge graph technology, but existing solutions mostly use static knowledge representation methods, lacking a dynamic association mechanism with real-time monitoring data and being difficult to achieve effective coordination between knowledge-driven and data-driven. In addition, traditional decision engines mostly rely on a single reasoning mechanism, and when dealing with complex and changeable flood scenarios, problems such as inaccurate decision confidence assessment and difficult resolution of multi-source evidence conflicts are likely to occur.

[0063] To address the above technical problems, as Figure 1 shown, this application provides a water conservancy multi-modal intelligent decision-making method based on a model association protocol, including the following steps S1 - S4.

[0064] S1. Obtain hydrological, meteorological, and remote sensing data, and perform preprocessing and feature extraction respectively.

[0065] In some embodiments, the preprocessing and feature extraction of hydrological data specifically include the following steps S1.1 - S1.3.

[0066] S1.1. Perform OCR recognition and term standardization processing on historical hydrological reports to generate structured text data.

[0067] Specifically, the format of historical hydrological reports can be PDF and plain text files. The Tesseract OCR engine can be used for text extraction. For scanned PDF versions, image enhancement preprocessing can be adopted to improve the recognition accuracy, generating an original text file and retaining structured information such as paragraphs and tables.

[0068] Based on authoritative materials such as the "Hydrological Yearbook" and the bulletins of the Ministry of Water Resources, hydrological professional terms (such as "backwater height", "peak flood discharge") can be sorted out, a standardized mapping table (such as "peak flood discharge" → "standardized peak flood discharge") can be established, and a domain dictionary can be constructed.

[0069] Non-standard terms can be matched using regular expressions or rule-based natural language processing tools (such as SpaCy), and batch replaced according to the mapping table. The consistency of terms can be verified through manual sampling or pre-trained hydrological NER models to ensure the standard expression of key entities (such as "reservoir" and "watershed").

[0070] S1.2. Adopt a dynamic masking strategy, targetedly mask by identifying hydrological entities, and strengthen the model's representation ability for professional terms.

[0071] Specifically, a predefined regular expression library can be used to identify fixed terms. Load a pre-trained hydrological domain NER model fine-tuned based on BERT to identify complex entities, such as "backwater height" and "channel slope", and mask the identified hydrological entities with an 80% probability, and keep 20% of the original words to enhance robustness.

[0072] For the causal logic in long documents, such as "rainfall → water level rise", joint masking can be performed on related entities to force the model to learn the dependency relationship.

[0073] S1.3. Input the processed text data into an improved natural language processing model for multi-granularity encoding to generate semantic vectors; among them, the improved natural language processing model enhances semantic understanding through domain pre-training, and combines character-level embedding, sentence-level attention, and document-level Bi-LSTM to achieve multi-granularity feature fusion.

[0074] Specifically, the standardized hydrological text and general corpus can be used for mixed training. Based on the general BERT, adopt the MLM (masked language modeling) and NSP (next sentence prediction) tasks, and focus on optimizing the embedding representation of hydrological terms.

[0075] The WordPiece tokenizer of BERT can be extended, and rare hydrological words can be added to the vocabulary to avoid incorrect segmentation and achieve character-level embedding.

[0076] A sentence-level attention mechanism can be introduced after the Transformer layer to calculate the correlation weight of each sentence with the CLS token, highlight the key sentences, and achieve sentence-level attention.

[0077] The paragraph encoding can be input into a bidirectional LSTM to capture long-distance dependencies across paragraphs, such as the causal chain of "upstream rainfall → downstream flood peak", and achieve document-level Bi-LSTM.

[0078] Finally, concatenate the character-level, sentence-level, and document-level feature vectors, perform multi-granularity fusion through a fully connected layer, and output 512-dimensional semantic vectors.

[0079] In some embodiments, preprocessing and feature extraction of meteorological data are performed, specifically including the following steps S1.4 - S1.5.

[0080] S1.4. Construct a watershed graph structure, taking hydrological stations, rainfall stations, and reservoir gates as nodes, and defining directed edges based on the water system topological relationship.

[0081] Specifically, NetworkX or PyTorch Geometric can be used to construct the graph structure and store it as an adjacency list.

[0082] In the watershed graph structure, the node types include hydrological stations, rainfall stations, and reservoir gates. The node attributes include temporal features and spatial coordinates. Among them, the temporal features are dynamic data of a 6 - hour sliding window, and the spatial coordinates are longitude and latitude information in the WGS84 standard, which are used for spatial visualization and topological relationship calculation.

[0083] Based on the upstream - downstream relationship of the water system, directed edges are established. For example, A station → B station means that A is located upstream of B.

[0084] Perform dynamic weight calculation, and the formula is as follows:

[0085] Wab = river channel length / (slope * cross - section width), which reflects the water flow propagation efficiency. The larger the value, the more significant the influence between upstream and downstream.

[0086] Use the NetworkX or PyTorch Geometric library to construct an adjacency list and store it as a sparse matrix.

[0087] S1.5. Adopt a hybrid architecture of graph neural network and time - series model, extract node features through spatial aggregation and time - series modeling, and output node - level prediction and a risk hot - spot map composed of the adjacency list.

[0088] Specifically, adopt the graph neural network GraphSAGE, sample 3 - hop neighborhood nodes for each node for neighborhood sampling, calculate the average value of the neighborhood node features, extract the significant signals in the neighborhood features, and generate a 128 - dimensional spatial feature vector to characterize the spatial role of the node in the watershed.

[0089] The time - series data of each node can be input into the time - series model GRU at each time step, and a 64 - dimensional time - series feature vector is output to capture the short - term water level fluctuation pattern.

[0090] Concatenate the 128 - dimensional spatial feature and the 64 - dimensional time - series feature, and input them into a fully - connected layer to map to a 1 - dimensional output, that is, node - level prediction.

[0091] The graph attention network (GAT) is used to calculate the node importance weights to highlight high-risk areas. It can output values in the range of (0 to 1) through the Sigmoid function. Nodes with a probability > 0.7 are marked as high-risk, and a risk hot spot map in GeoJSON format is generated by combining with the adjacency list.

[0092] In some embodiments, preprocessing and feature extraction are performed on meteorological data, specifically including the following steps S1.6 - S1.7.

[0093] S1.6. Register and fuse multi-source remote sensing images, and extract the water body contour based on the normalized difference water index.

[0094] Specifically, the multi-source remote sensing images can be SAR images and multi-spectral images. SAR images have the ability to penetrate clouds, and multi-spectral images have high resolution.

[0095] Quadratic polynomial fitting can be used to align the images, controlling the registration error ≤ 1 pixel. The VV / VH bands of SAR can be superimposed with the infrared band of the multi-spectral image to enhance the water body features.

[0096] The water body contour extraction can be performed using NDWI calculation, and the formula is as follows:

[0097]

[0098] Where Green represents the green band and NIR represents the near-infrared band.

[0099] Set a threshold, such as NDWI > 0.2, for binary segmentation to extract the water body area.

[0100] Cut the image into 256×256 pixel blocks on a weekly basis, including 8 bands (RGB + NIR + SWIR + VV + VH), and generate image time series slices.

[0101] S1.7. Input the time series image slices into an improved computer vision model, combined with a spatial attention mechanism and a temporal Transformer encoder, to generate a remote sensing heat map.

[0102] Specifically, the improved computer vision model includes a spatial attention mechanism and a temporal Transformer encoder.

[0103] The spatial attention mechanism includes channel attention and spatial attention, which are used to calculate the band importance weights and generate a heat map to focus on the water body edge respectively. The time series image feature vectors of 5 consecutive days can be input into the temporal Transformer encoder to capture the water body expansion trend.

[0104] Generate a probability heat map (0-1) through Sigmoid activation. Among them, the Otsu algorithm is used to automatically determine the binarization threshold to distinguish the flooded area from the non-flooded area. The registration parameters are used to convert the heat image pixel coordinates into WGS84 geographic coordinates, the connected region contours are extracted, and the coordinates of the high-risk areas in GeoJSON format are output.

[0105] S2. Structurally encode the extracted multi-modal features using a data serialization protocol to generate corresponding serialization codes, and encapsulate and transmit them using the MCP protocol.

[0106] In some embodiments, step S2 specifically includes the following steps S2.1-S2.3.

[0107] S2.1. Structurally encode the generated hydrological semantic vector, risk hot spot map, and remote sensing heat map respectively using ProtocolBuffers to generate serialization codes for each modality.

[0108] Specifically, define the data format as.proto, and use the Protocol Buffers compiler to compile the.proto file into a serialization class in the target language. Fill the original data into the corresponding serialization class instance, and call the serialization method to generate a binary stream.

[0109] S2.2. Use the MCP protocol to encapsulate the serialization codes. The encapsulation process includes: adding a timestamp accurate to the millisecond level to each data; embedding spatial coordinate metadata conforming to the WGS84 standard; and attaching a confidence score with a scoring range of 0-1.

[0110] S2.3. Compress the encapsulated MCP data packet and add a data check code to the header.

[0111] S3. At the receiving end, use the corresponding MCP protocol to de-encapsulate, restore the data to the original multi-modal features, and fuse the features of each modality through a dynamic weight allocation mechanism and a cross-modal attention mechanism.

[0112] In some embodiments, step S3 specifically includes the following steps S3.1-S3.6.

[0113] S3.1. The receiving end decompresses and verifies the data packet through the MCP protocol de-encapsulation module to restore the serialization codes of each modality.

[0114] S3.2. Use the ProtocolBuffers decoder corresponding to the sending end to deserialize the serialization codes to restore the three original features of the hydrological semantic vector, risk hot spot map, and remote sensing heat map.

[0115] S3.3. Construct a gated neural network and input the current scene parameters, including seasonal features, geographical location, and historical disaster records.

[0116] Specifically, the construction of the gated neural network can include an input layer, a hidden layer, and an output layer. The input layer encodes the scene parameters into dense vectors. The hidden layer includes 2 fully connected layers (ReLU activation), and the output dimension is the same as the number of modalities. The output layer uses Softmax activation to generate weight allocation coefficients.

[0117] S3.4. The gated neural network dynamically calculates the weight allocation coefficients of hydrological, meteorological, and remote sensing features according to the scene parameters, where the weight of meteorological features is automatically increased during the flood season, and hydrological features are given priority during the dry season.

[0118] Specifically, the weighted feature = weight × original feature, and each modality feature vector is scaled according to the weight coefficient.

[0119] S3.5. Use the multi-head attention mechanism to establish cross-modal feature interactions, including:

[0120] Project each modality feature to a unified dimension through a linear layer.

[0121] Queries (Q), keys (K), values (V), and each modality feature generates Q / K / V matrices.

[0122] Scaled dot-product attention, normalize the scaled similarity matrix row by row to generate attention weights, and the formula is as follows:

[0123]

[0124] Calculate the attention-weighted features generated by each modality.

[0125] Apply the attention weights to the value matrix V to generate the features after cross-modal interaction and establish cross-modal interaction. For example, if the hydrological feature shows a sudden rise in water level, the attention mechanism automatically enhances the feature weights of "rainfall" in the meteorological modality and "upstream flooded area" in the remote sensing modality.

[0126] S3.6. Fusion the weighted multi-modal features through a feature concatenation layer and output the joint feature representation.

[0127] Specifically, the weighted multi-modal features can be concatenated along the feature dimension. For example, input hydrology (512 dimensions), meteorology (512 dimensions), and remote sensing (512 dimensions). Output a 1536-dimensional vector.

[0128] The fully connected layer then compresses the 1536-dimensional vector to 512 dimensions and outputs the joint feature representation.

[0129] S4. Input the fused features into the knowledge graph-driven decision-making large model. With the aid of the knowledge graph and the rule engine, generate the flood risk level and corresponding decisions through hybrid decision-making reasoning.

[0130] In some embodiments, step 4 specifically includes the following steps S4.1 - S4.4.

[0131] S4.1. Input the fused features into the knowledge graph constructed by the Neo4j graph database. The knowledge graph includes predefined domain entities and relationships.

[0132] Specifically, the entity types may include: disaster events, monitoring stations, regulation facilities, natural factors, and emergency measures.

[0133] The relationship types may include: cause (heavy rain → flood), influence (gate opening → water level), protection (dike → residential area).

[0134] The cosine similarity can be calculated between the 512-dimensional joint feature vector and the entity embeddings in the knowledge graph (pre-generated by GraphSAGE) to match the most relevant entities.

[0135] For example, if the fused features include "rising water level + upstream rainfall", then match the "heavy rain" and "flood" nodes in the knowledge graph.

[0136] Temporary relationships can be dynamically created, and the weights are determined by the feature similarity.

[0137] Sensor data can be dynamically written through the Neo4j plugin to update the node attributes.

[0138] S4.2. Performing hybrid decision-making reasoning includes: analogical reasoning based on the knowledge graph and threshold judgment based on the rule engine.

[0139] Among them, for the analogical reasoning based on the knowledge graph, the Cypher query language can be used to retrieve similar historical events, and the retrieved case strategies are weighted and fused (similarity as the weight) to output a recommended solution (such as "the flood discharge flow control is the historical average ± 10%").

[0140] For the threshold judgment based on the rule engine, a static rule library can be adopted, such as IF water level > warning line AND rainfall > 50 mm / h THEN initiate a level II response; open the flood discharge sluice (flow = 300 m 3 / s), evacuate the residents in low-lying areas.

[0141] The rule parameters can be updated according to the entity status in the knowledge graph. For example, the warning line value can be adjusted in real time.

[0142] Priority settings can be carried out, such as static rules > case reasoning, but manual intervention is allowed to override.

[0143] S4.3. Use the D-S evidence theory to fuse multi-source evidence and calculate the confidence of each decision option.

[0144] Specifically, fuzzy weighting can be performed on conflicting evidence, such as "high water level but low rainfall".

[0145] According to the fuzzy weighting, calculate the confidence interval of each decision option, such as "Flood discharge plan: confidence = 0.85 ± 0.05".

[0146] S4.4. Output the graded risk warning and the corresponding decision; send the decision instruction to the execution system through the automated interface.

[0147] Specifically, a 5-level classification of risk warning and implementation plan can be output, such as "Red warning: flood discharge 500m 3 / s, evacuate Area A / B".

[0148] In summary, this application designs a refined preprocessing process for hydrological, meteorological and remote sensing data respectively; for hydrological data, through OCR recognition and term standardization processing, historical hydrological reports are transformed into structured text data, and a standardized mapping table is established, effectively solving the problem of non-standard hydrological terms and improving the consistency and usability of data; for meteorological data, a basin map structure is constructed, comprehensively considering the water system topological relationship, temporal characteristics and spatial coordinates, providing a rich information basis for subsequent analysis; for remote sensing data, registration fusion and normalized water body index are used to extract the water body contour, enhancing the ability to identify water body characteristics.

[0149] In the feature extraction stage, hydrological data strengthens the model's representation ability of professional terms through the dynamic mask strategy, combines with the improved natural language processing model to achieve multi-granularity coding, and generates vectors with more semantic information; meteorological data uses a hybrid architecture of graph neural network and time series model to extract node features and generate risk hot spot maps, fully considering spatial aggregation and time series modeling; remote sensing data combines spatial attention mechanism and temporal Transformer encoder to generate remote sensing heat maps, effectively capturing the water body expansion trend; through the above methods, the features of each modality data are fully mined, providing high-quality input for subsequent multi-modal fusion.

[0150] Use the data serialization protocol to perform structured encoding on the extracted multi-modal features, generate serialization codes for each modality using ProtocolBuffers, and encapsulate and transmit them through the MCP protocol. The standardized encoding method ensures the compatibility and transferability of data between different systems. At the same time, the timestamp, spatial coordinate metadata and confidence score added during the encapsulation process provide rich information for subsequent decisions.

[0151] At the receiving end, by constructing a gated neural network, the weight distribution coefficients of hydrological, meteorological, and remote sensing features are dynamically calculated according to the current scene parameters. The adaptive weight distribution mechanism can automatically adjust the importance of each modal feature according to different scenarios. The multi-head attention mechanism is used to establish cross-modal feature interactions. Each modal feature is projected to a unified dimension through a linear layer, and query (Q), key (K), and value (V) matrices are generated. The attention weights are calculated using scaled dot-product attention to achieve interactions between modalities. The cross-modal interaction mechanism can capture the correlations and dependencies between different modal features.

[0152] The fused features are input into a knowledge graph-driven decision-making large model. With the knowledge graph constructed by the Neo4j graph database, the cosine similarity between the joint feature vector and the entity embeddings in the knowledge graph is calculated to match the most relevant entity, and temporary relationships are dynamically created. The introduction of the knowledge graph provides rich domain knowledge and semantic information for decision-making reasoning, making the decision-making process more intelligent and interpretable.

[0153] When performing hybrid decision-making reasoning, it combines analogy reasoning based on the knowledge graph and threshold judgment based on a rule engine. Analogy reasoning outputs a recommended solution by retrieving similar historical events and weighted-fusing case strategies; threshold judgment uses a static rule base and updates the rule parameters according to the entity status in the knowledge graph. At the same time, the D-S evidence theory is used to fuse multi-source evidence, calculate the confidence of each decision option, realizing the complementary advantages of multiple decision-making methods and improving the accuracy and reliability of decision-making.

[0154] Finally, a hierarchical risk warning and corresponding decisions are output, and the decision-making instructions are sent to the execution system through an automated interface. It can output risk warnings and execution plans with 5-level classification.

[0155] In the second aspect of this application, a water conservancy multi-modal intelligent decision-making system based on a model association protocol is provided, including:

[0156] A hydrological intelligent agent that preprocesses and extracts features from hydrological data; its encoding module uses a data serialization protocol to structurally encode the extracted features, generates corresponding serialization codes, and encapsulates and transmits them using the MCP protocol;

[0157] A meteorological intelligent agent that preprocesses and extracts features from meteorological data; its encoding module uses a data serialization protocol to structurally encode the extracted features, generates corresponding serialization codes, and encapsulates and transmits them using the MCP protocol;

[0158] A remote sensing intelligent agent that preprocesses and extracts features from remote sensing data; its encoding module uses a data serialization protocol to structurally encode the extracted features, generates corresponding serialization codes, and encapsulates and transmits them using the MCP protocol;

[0159] The multimodal fusion module uses the corresponding MCP protocol for decompression and restores the data to the original multimodal features, and fuses the features of each modality through a dynamic weight allocation mechanism and a cross-modal attention mechanism;

[0160] The decision-making large model, with the help of the knowledge graph and the rule engine, generates the flood risk level and the corresponding decision through hybrid decision-making reasoning.

[0161] In the third aspect of the present application, a computer-readable storage medium is provided. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the above method steps are implemented.

[0162] In the fourth aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above method steps are implemented.

[0163] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A water conservancy multimodal intelligent decision-making method based on a model association protocol, characterized in that, It includes the following steps: S1. Obtain hydrological, meteorological and remote sensing data, and perform preprocessing and feature extraction respectively; S2. Use a data serialization protocol to structurally encode the extracted multi-modal features, generate corresponding serialized codes, and encapsulate and transmit them using the MCP protocol; S3. At the receiving end, use the corresponding MCP protocol to unpack, restore the data to the original multi-modal features, and fuse the features of each modality through a dynamic weight allocation mechanism and a cross-modal attention mechanism; S4. Input the fused features into a knowledge graph-driven decision-making large model, and generate flood risk levels and corresponding decisions through hybrid decision-making reasoning with the help of the knowledge graph and the rule engine.

2. The water conservancy multi-modal intelligent decision-making method based on the model association protocol according to claim 1, characterized in that, In step S1, when performing preprocessing and feature extraction on hydrological data, it specifically includes: S1.

1. Perform OCR recognition and term standardization processing on historical hydrological reports to generate structured text data; S1.

2. Adopt a dynamic masking strategy to perform targeted masking by identifying hydrological entities; S1.

3. Input the processed text data into an improved natural language processing model for multi-granularity encoding to generate semantic vectors; among them, the improved natural language processing model enhances semantic understanding through domain pre-training, and realizes multi-granularity feature fusion by combining character-level embedding, sentence-level attention and document-level Bi-LSTM.

3. A water conservancy multi-modal intelligent decision-making method based on a model association protocol according to claim 1, characterized in that In step S1, when performing preprocessing and feature extraction on meteorological data, it specifically includes: S1.

4. Construct a basin map structure, use hydrological stations, rain gauges and reservoir gates as nodes, and define directed edges based on the water system topology relationship; S1.

5. Adopt a hybrid architecture of a graph neural network and a time series model to extract node features through spatial aggregation and time series modeling, and output node-level predictions and a risk hot spot map composed of an adjacency list.

4. A water conservancy multi-modal intelligent decision-making method based on a model association protocol according to claim 1, characterized in that In step S1, when performing preprocessing and feature extraction on remote sensing data, it specifically includes: S1.

6. Register and fuse multi-source remote sensing images, and extract water body contours based on the normalized difference water index; S1.

7. Input time series image slices into an improved computer vision model, and combine a spatial attention mechanism and a time series Transformer encoder to generate a remote sensing heat map.

5. A water conservancy multi-modal intelligent decision-making method based on a model association protocol according to claim 1, characterized in that, In step S2, when using a data serialization protocol to structurally encode the extracted multi-modal features, generate corresponding serialized codes, and encapsulate and transmit them using the MCP protocol, it specifically includes: S2.

1. Respectively use ProtocolBuffers to structurally encode the generated hydrological semantic vectors, risk hot spot maps, and remote sensing heat maps to generate serialized codes for each modality; S2.

2. Use the MCP protocol to encapsulate the serialized codes, and the encapsulation process includes: Add a time stamp accurate to the millisecond level for each data; Embed spatial coordinate metadata that conforms to the WGS84 standard; Attach a confidence score, and the score range is 0 to 1; S2.

3. Compress the encapsulated MCP data packet and add a data check code to the header.

6. The multi-modal intelligent decision-making method for water conservancy based on the model association protocol according to claim 1, wherein, In step S3, at the receiving end, use the corresponding MCP protocol to unpack, restore the data to the original multi-modal features, and fuse the features of each modality through a dynamic weight allocation mechanism and a cross-modal attention mechanism, specifically including: S3.

1. The receiving end uses the MCP protocol decompression and encapsulation module to decompress and verify the data packet, and restore the serialized codes of each modality; S3.

2. Use the corresponding ProtocolBuffers decoder at the sending end to deserialize the serialized code, and restore the three original features of the hydrological semantic vector, risk hotspot map, and remote sensing thermal map; S3.

3. Construct a gated neural network and input the current scene parameters, including seasonal features, geographical location, and historical disaster records; S3.

4. The gated neural network dynamically calculates the weight distribution coefficients of hydrological, meteorological, and remote sensing features according to the scene parameters. During the flood season, the weight of meteorological features is automatically increased, and during the dry season, hydrological features are given priority; S3.

5. Establish cross-modal feature interaction using the multi-head attention mechanism, including: Map the features of each modality to a high-dimensional space; Calculate the similarity score matrix between modalities; Generate attention-weighted features based on the score matrix; S3.

6. Fusion the weighted multi-modal features through the feature splicing layer and output the joint feature representation.

7. A water conservancy multi-modal intelligent decision-making method based on a model association protocol according to claim 1, characterized in that, In step S4, input the fused features into the knowledge graph-driven decision-making large model. With the help of the knowledge graph and the rule engine, generate the decision corresponding to the flood risk level through dynamic reasoning, specifically including: S4.

1. Input the fused features into the knowledge graph constructed by the Neo4j graph database. The knowledge graph includes predefined domain entities and relationships; S4.

2. Perform hybrid decision-making reasoning, including analogical reasoning based on the knowledge graph and threshold judgment based on the rule engine; S4.

3. Use the D-S evidence theory to fuse multi-source evidence and calculate the confidence of each decision option; S4.

4. Output the hierarchical risk warning and the corresponding decision; send the decision instruction to the execution system through the automation interface.

8. A water conservancy multi-modal intelligent decision-making system based on a model association protocol, characterized in that, Including: Hydrological agent, which preprocesses and extracts features from hydrological data; Its encoding module uses the data serialization protocol to structurally encode the extracted features, generate the corresponding serialized code, and use the MCP protocol for encapsulation and transmission; Meteorological agent, which preprocesses and extracts features from meteorological data; Its encoding module uses the data serialization protocol to structurally encode the extracted features, generate the corresponding serialized code, and use the MCP protocol for encapsulation and transmission; Remote sensing agent, which preprocesses and extracts features from remote sensing data; Its encoding module uses the data serialization protocol to structurally encode the extracted features, generate the corresponding serialized code, and use the MCP protocol for encapsulation and transmission; Multi-modal fusion module, which uses the corresponding MCP protocol to decompress and restore the data to the original multi-modal features, and fuse the features of each modality through the dynamic weight distribution mechanism and the cross-modal attention mechanism; Decision-making large model, which generates the flood risk level and the corresponding decision through hybrid decision-making reasoning with the help of the knowledge graph and the rule engine.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method steps described in any one of claims 1 to 7 are implemented.

Images