A Knowledge Graph-Based Embedded Intelligent Risk Control Method for Water Conservancy Projects

Abstract

This invention provides a knowledge graph-based embedded intelligent risk control method for water conservancy projects. The method includes: Step S1: collecting and organizing relevant data of the target water conservancy project; Step S2: constructing a time-series-based multimodal data knowledge graph using text and image data as multimodal inputs; Step S3: using the output recommendations based on the multimodal knowledge graph as input to a large language model; Step S4: generating multimodal data. This invention establishes a multi-disciplinary risk transmission chain based on knowledge graphs to comprehensively traverse emergency plans for water conservancy projects, thereby achieving multi-disciplinary and cross-professional coupled risk analysis and response strategy selection, improving the overall collaborative nature of water conservancy hub safety management and the comprehensive scheduling capability of the basin.

Description

Technical Field

[0001] The present invention relates to the field of water conservancy and hydropower engineering, and in particular to a method for intelligent risk control of water conservancy projects based on knowledge graph embedding. Background Art

[0002] As a comprehensive building with macro-control at the basin level, water conservancy projects undertake economic and social benefits such as flood control, power generation, water supply, shipping, ecology, irrigation, and landscape. Large-scale water conservancy projects can regulate the runoff of natural rivers through themselves and their ancillary buildings, thereby affecting the natural climate conditions of the entire basin and its radiation area. At the same time, as a complex building with a complex structure, the construction and operation of water conservancy projects span multiple disciplines and are affected by multiple internal and external factors. Water conservancy projects are prone to chain risk events under the superposition of coupled fault inducements, and once the risk chain has misjudgment, risk emergency plan deviation, and risk emergency disposal errors, it will bring immeasurable losses. Therefore, it is urgent to develop an intelligent decision-making method for emergency plans for large-scale water conservancy hubs that spans multiple disciplines and covers multi-dimensional risks.

[0003] Large language models have been widely promoted and applied in many fields due to their outstanding performance in various natural language processing tasks, but their operating effects have been questioned to a certain extent due to their lack of factual knowledge. Relevant studies have shown that large language models may not be able to trace back facts and obtain virtual expected results by generating false information statements. In addition, due to the lack of knowledge in specific fields and new training data, large language models cannot obtain good generalization effects to adapt to specific fields or new knowledge. The embedding integration of knowledge graphs and large language models is one of the effective ways to solve the above problems. Knowledge graphs can store a large number of facts in the form of triples and represent the relationship between entities in a structured form. Compared with large language models, knowledge graphs can provide accurate explicit knowledge, and their reasoning ability and interpretability are stronger. At the same time, as new knowledge continues to expand, knowledge graphs can evolve and build a knowledge base in a specific field. Therefore, knowledge graphs and large language models can promote each other, and studies have shown that there is an intrinsic connection between the two. Knowledge graphs can analyze and provide interpretability for large language models. On the contrary, large language models can construct knowledge graphs, generate texts, and improve their operating performance.

[0004] At the current stage, research on water conservancy project scheduling is mainly focused on benefit optimization, while there are obvious deficiencies in the decision-making methods of safety scheduling plans based on full-chain and multi-dimensional risks. At the macro level, the present invention proposes an intelligent decision-making system for water conservancy project safety scheduling plans based on a knowledge graph embedded large language model. At the specific level, the system proposed by the present invention uses water conservancy project operation monitoring data, operation log text files, emergency response measures and plan text files, event research report text files and the image information contained therein as the data basis, combined with the knowledge graph, to improve the reasoning ability of the large language model in the field of water conservancy project risk management and control, the coverage of risk identification and transmission, and the interpretability of emergency plan calls. At the same time, the present invention uses Word2Vec and ELMO-SIFRank systems to improve the extraction accuracy of water conservancy project risk keywords and uses a deep residual shrinkage network to enhance the recognition accuracy of key risk features in image data; performs serial linear processing on entity triples to improve the completeness of their transmitted content; introduces a translation scoring function and an output recommendation mechanism to achieve self-evaluation correction of model errors; proposes a two-stage multimodal training-output strategy, and introduces a diffusion loss mechanism for text and images to reduce the impact of errors on the final output of the model while improving the robustness of the model under small sample data. The present invention can effectively solve the deficiencies of existing water conservancy project safety management and control strategies, such as defects in risk transfer analysis between professions, conflicts in priorities between emergency response measures and plans, and incompleteness of the risk transfer chain. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a method for intelligent risk control of water conservancy projects based on knowledge graph embeddedness, which overcomes the above problems or at least partially solves the above problems.

[0006] According to one aspect of the present invention, a method for intelligent risk control of water conservancy projects based on knowledge graph embedding is provided, and the method for intelligent risk control includes:

[0007] Step S1: Collect and organize relevant data of the target water conservancy project;

[0008] Step S2: Using text and image data as multimodal input, construct a time-series-based multimodal data knowledge graph;

[0009] Step S3: using the output recommendation based on the multimodal knowledge graph as input to the large language model;

[0010] Step S4: Generate multi-modality.

[0011] Optionally, the step S1: collecting and organizing relevant data of the target water conservancy project specifically includes: water conservancy project operation monitoring data, investigation reports, research literature, emergency plans, scheduling procedure documents, operation monitoring images, on-site images and water conservancy project risk-related literature.

[0012] Optionally, the step S2: using text and image data as multimodal input to construct a time-series-based multimodal data knowledge graph specifically includes:

[0013] The collected data is structured into triplets, and the head entity, relation, and tail entity are converted into independent embedding vectors, which are passed through the embedding layer to generate dense vectors;

[0014] The dense vector is embedded into the deep layer of the model through high-order connections;

[0015] TransE is used to learn the structured representation of the knowledge graph to achieve information aggregation of multimodal adjacent entities in entity text and enrich entity representation;

[0016] Through the entity encoder and attention layer, each entity obtains a new entity representation, which is embedded into the knowledge graph through the translation scoring function, transforming entities and relations into parameterized vectors and retaining the relational reasoning therein;

[0017] Output recommendation layer.

[0018] Optionally, the output recommendation layer specifically includes:

[0019] Each entity obtains the corresponding input through the knowledge graph embedding module and pushes it to the recommendation module, while the attention mechanism is used to aggregate the information;

[0020] The layer aggregation mechanism is used to connect the representation of each step into a single vector, and its mathematical expression is as follows:

[0021]

[0022] Where L is the number of attention layers of the multimodal knowledge graph.

[0023] Optionally, the method of obtaining a new entity representation for each entity through the entity encoder and the attention layer, embedding the new entity representation into the knowledge graph through the translation scoring function, converting the entity and the relationship into a parameterized vector and retaining the relationship reasoning therein specifically includes:

[0024] The expression of the translation score function is shown in formula (6):

[0025] goal(c,d,h)=||f c +f d -f h || 2 (6)

[0026] During the knowledge graph training process, the loss pair rankings of correct triples and incorrect triples are sorted in pairs using formula (7) to obtain effective training results;

[0027] α KG =∑ (c,d,h,h')∈δ -lnθ(goal(c,d,h')-goal(c,d,h)) (7)

[0028] In the formula, (c, d, h') is an incorrect triplet formed by randomly replacing entities in the correct triplet, and θ represents the sigmiod function;

[0029] The layer models entities and relations at the granularity of triples for regularization, inputting direct connections into the representation.

[0030] Optionally, the step S3: using the output recommendation based on the multimodal knowledge graph as the input of the large language model specifically includes:

[0031] Multimodal encoding, the text tags output by the knowledge graph are embedded into the vector e t ∈R D , after pre-training and visual encoder, each image input is converted into feature e img ∈R D×32 , the embeddings are concatenated to create input prompt features;

[0032] Vokens added, added special mark M img As the generation call function of the large language model corpus, where M img Specifically expressed as:

[0033] M img ={img1, img2, img3,..., imgn} (9)

[0034] The hidden output states of the variables are used for subsequent image generation, and the positions of the variables can be used to represent the insertion of interlaced images.

[0035] Optionally, the step S4: generating multi-modality specifically includes:

[0036] Text generation, following the strategy of stochastic language modeling, jointly generates text and virtual expectations in the text space, and adds "function words" to the positions of actual images during training, while training the model to predict "function words" during text generation;

[0037] The generated chunk is represented as:

[0038] Tk={t1,t2,t3,...,t n}t i ∈V∪V img (10)

[0039] The text-image loss is defined as:

[0040]

[0041] Where, t i ∈V∪V img ;

[0042] Establish Voken features for image generation.

[0043] Optionally, the step of establishing Voken features to generate an image specifically includes:

[0044] Align the output hidden state value with the text-conditional feature space of the text-to-image generation model and construct a feature mapper module containing a 2-layer MLP model θ mlp 、A 4-layer encoder-decoder model based on Transformer enc-dec And a learnable decoding feature sequence q, where the mathematical expression of the mapping feature is as follows:

[0045]

[0046] The latent diffusion loss model (LDM) is used as a guide to ensure the authenticity of the generated images;

[0047] In the process, the pre-trained VAE transforms the real image into the latent feature z0, and then adds noise to z0 to obtain the latent feature z of the noise. t , using the pre-trained U-Net model ∈ ρ The principle of calculating LDM loss is shown in formula (13);

[0048]

[0049] The present invention provides a method for intelligent risk control of water conservancy projects based on knowledge graph embedding, the method comprising: step S1: collecting and organizing relevant data of target water conservancy projects; step S2: using text and image data as multimodal inputs to construct a multimodal data knowledge graph based on time series; step S3: using the output recommendation based on the multimodal knowledge graph as a large language model input; step S4: generating multimodal. The present invention builds a multi-professional risk transmission chain based on the knowledge graph in order to traverse the comprehensive emergency plan of water conservancy projects, thereby realizing multi-disciplinary and cross-professional coupled risk analysis and disposal strategy selection, which can improve the overall collaboration of water conservancy hub safety management and control and the comprehensive dispatching capacity of the basin. At the same time, the present invention embeds the knowledge graph into the large language model, uses the professional knowledge of water conservancy projects as training data, and relies on the logical reasoning ability of the knowledge graph to improve the professionalism of the large language model in the field of water conservancy safety and the interpretability of the operation process. The combination of the two can achieve clear visualization of risk points and risk transmission chains of water conservancy projects, ensure the accuracy and reliability of risk emergency management plans, and provide technical support for dispatching and operating personnel to carry out safe dispatching and disposal under sudden and multi-dimensional risk superposition conditions.

[0050] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0052] Figure 1 A knowledge graph framework diagram based on multimodal input and deep residual shrinkage network provided in an embodiment of the present invention;

[0053] Figure 2 The basic structure diagram of ELMo provided by the embodiment of the present invention;

[0054] Figure 3 A schematic diagram of a high-order transfer mechanism based on an attention mechanism provided in an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of the self-attention and Transformer mechanism provided in an embodiment of the present invention;

[0056] Figure 5A framework diagram of a multimodal large language model with a generation loss module provided in an embodiment of the present invention;

[0057] Figure 6 A schematic diagram of the output results of the model under given text and images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0059] The terms "comprises" and "having" and any variations thereof in the description embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0060] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0061] Example 1

[0062] S1. Data collection. Collect and organize water conservancy project operation monitoring data and work logs, emergency response measures, risk control plans, unit monitoring images and other parameters;

[0063] S2. Graph construction and hardware settings. In the multimodal learning stage, LoRA is used to generate tokens and vokens, the parameters and decoder in the MLP are kept unchanged, and the AdamW optimizer is used for training. The hardware equipment is equipped with two NVIDIA RTX-A6000 graphics cards (the graphics card selection is based on the model parameter calculation). In the alignment stage, the AdamW optimizer can also be used for text semantic alignment.

[0064] S3. Generate entity triples from the hydropower station operation monitoring data, work logs, emergency response measures and plans, and monitoring images with text descriptions, and input them into the knowledge graph used in the present invention. The knowledge graph outputs recommended plans (text format) and risk image predictions (image format with text descriptions).

[0065] S4. Input the recommendation results given by the knowledge graph into the large language model with multimodal attention mechanism.

[0066] a. Text generation stage: input <historical water level data> - next scene image <image> <imagehere> , output <text description>;

[0067] b. Image generation stage: input <historical water level data> - generate images [text description] of surrogate scene description, output [image1]…[imagen];

[0068] c. Text-image generation: Input <historical water level data> - water level increase from 8:00 to 20:00 in the future, the current status of each risk point, and what emergency plans should be taken? Output <text description> [image1]…[imagen];

[0069] S5. Model generation results are shown in Figure 6 shown.

[0070] The present invention proposes an intelligent decision-making method for emergency plans based on the embedding of a large model knowledge graph of water conservancy projects. The method can cover the scope of multidisciplinary risk reasoning of water conservancy projects, and realize risk emergency disposal and auxiliary decision-making of emergency plans in combination with emergency plans, thereby realizing accurate reasoning of interdisciplinary and multidimensional safety risks of water conservancy hubs and efficient decision-making of safety scheduling plans.

[0071] Example 2

[0072] A decision-making method for multi-dimensional safety scheduling of water conservancy projects based on risk reasoning

[0073] S1. Collect and organize relevant data of the target water conservancy project, including: water conservancy project operation monitoring data (including digital text and time records such as operation logs), investigation reports, research documents, emergency plans, dispatching procedures, operation monitoring images, on-site images and water conservancy project risk-related documents;

[0074] S2. Taking text and image data as multimodal input, a time-series-based multimodal data knowledge graph is constructed, which includes a multimodal knowledge graph embedding module and an auxiliary suggestion module;

[0075] a. Figure 1 As shown in the figure, the collected data is structured into triplets, and the head entity, relationship, and tail entity are converted into independent embedding vectors, which are then passed through the embedding layer to generate dense vectors. Text part: According to the operational requirements and dispatching procedures of the water conservancy hub management personnel, Word2Vec is used to train word vectors, and the ELMO-SIFRank model is used to accurately capture the core topics of the text. The structure of the ELMo model is as follows: Figure 2 As shown in the figure. Image part: The image data is processed using a deep residual contraction network, which is pre-trained on the basis of Imagenet3. Subsequently, all modes of the data entity are normalized to the same dimension through a dense layer.

[0076] b. Multimodal attention layer. The input data processed in step a will be embedded into the deep layer of the model through high-order connections. In the multimodal attention layer, cascaded attention weights are used to reveal the weight ratios of different types of risks among different responsible departments. In addition, the attention mechanism can significantly reduce the impact of noise. Figure 3 Demonstrated a high-order transfer mechanism based on the attention mechanism.

[0077] c. Transmission layer. TransE is used to learn the structured representation of the knowledge graph. In order to achieve information aggregation of multimodal adjacent entities in entity text and enrich entity representation, F is introduced. c The aggregation of adjacent triples is connected to entity c, where the triple is represented as c+d≈h, c represents the candidate entity, and f agg The representation vector representing the aggregation of adjacent entities represents the linear combination of each triplet. Its calculation formula is shown in formula (1):

[0078]

[0079] Where f(c, d, h) represents the embedding of the triplet, and π(c, d, h) represents the attention score of each triplet embedding to achieve control over the amount of information transmitted by the triplet.

[0080] In order to retain the relational information in the parameters, the embedded serial linear variation learning of the head, tail and relation of the triples is performed to achieve the complete transmission of relational information in the knowledge graph. The formula of serial linear expression is as follows:

[0081] f(c, d, h) = W1(f c ||f d ||f h ) (2)

[0082] In the formula, f c and f h denote the embedding of the head entity and the tail entity respectively, and f d Represents relation embedding.

[0083] The relational attention mechanism is used to allocate the triplet attention score. The calculation process is shown in formula (3):

[0084] π(c,d,h)=LeakyReLU(W1f(c,d,h)) (3)

[0085] In the formula, LeakyReLU is a nonlinear activation function, and then the softmax function is used to normalize the above connected triple coefficients:

[0086]

[0087] Aggregation layer. To avoid the loss of initial information, the entity expression f is realized by serial aggregation method, that is, formula (5) c and corresponding to f agg Aggregation of entity expressions.

[0088] f concat =W c (f c ||f agg ) (5)

[0089] In the formula, || represents the concatenation operation, W c Represents the trainable parameters of the model. Then high-order propagation is achieved by stacking propagation and aggregation layers.

[0090] d. Knowledge graph embedding. Through the entity encoder and attention layer, each entity obtains a new entity representation, and then the new entity representation is embedded into the knowledge graph through the translation score function, which transforms the entity and relationship into a parameterized vector and retains the relationship reasoning therein. The expression of the translation score function is shown in formula (6):

[0091] goal(c, d, h) = || f c +f d -f h || 2 (6)

[0092] During the knowledge graph training process, the loss pair rankings of correct triples and incorrect triples are sorted in pairs using formula (7) to obtain effective training results.

[0093] α KG =∑ (c，d，h，h′)∈δ -lnθ(goal(c,d,h′)-goal(c,d,h)) (7)

[0094] In the formula, (c, d, h′) is an incorrect triplet formed by randomly replacing entities in the correct triplet, and θ represents the sigmiod function.

[0095] The corresponding layer models entities and relations at the granularity of triples for regularization, and then directly inputs the representation.

[0096] e. Output recommendation layer. Subsequently, each entity obtains the corresponding input through the knowledge graph embedding module and pushes it to the recommendation module, while the attention mechanism is used to aggregate the information. In this step, the layer aggregation mechanism is used to connect the representation of each step into a single vector, and its mathematical expression is as follows:

[0097]

[0098] Where L is the number of attention layers of the multimodal knowledge graph. The above steps can not only enrich the initial embedding, but also control the propagation intensity by adjusting L.

[0099] S3. Using the output recommendation (text / image) based on the multimodal knowledge graph as the input of the large language model. Here, the present invention uses a two-stage training method to achieve the robustness of the model under the condition of few sample data.

[0100] a. Multimodal encoding. The text tags output by the knowledge graph are embedded into the vector e t ∈R D , after pre-training and visual encoder, each image input (or the entire knowledge graph) is converted into feature e img ∈R D×32 , the above embeddings are concatenated to create input prompt features.

[0101] b. Vokens added. Add special mark M img As the generation call function of the large language model corpus, where M img Specifically expressed as:

[0102] M img ={img1, img2, img3,..., imgn} (9)

[0103] The hidden output states of the above variables are used for subsequent image generation, and the positions of the above variables can be used to indicate the insertion of interlaced images.

[0104] S4. Multimodal generation.

[0105] A compact mapping module is used for dimension matching to achieve alignment between generated tags and generated models. In this process, the spatial loss of text and image and the risk potential diffusion loss are added to the model as supervised training losses.

[0106] a. Text generation. According to the strategy of random language modeling, text and virtual expectations are jointly generated in the text space, and "function words" are added to the positions of actual images during training. At the same time, the training model predicts "function words" during the text generation process. In the above steps, the generated word block can be expressed as:

[0107] Tk={t1,t2,t3,…,t n }t i ∈V∪V img (10)

[0108] The text-image loss is defined as:

[0109]

[0110] Where, t i ∈V∪V img .

[0111] b. Establish Voken features for image generation. Align the output hidden state value with the text-conditional feature space of the text-to-image generation model. Here, construct a feature mapper module, which contains a 2-layer MLP model θ mlp 、A 4-layer Transformer-based encoding-decoding modelθ enc-dec And a learnable decoding feature sequence q, where the mathematical expression of the mapping feature is as follows:

[0112]

[0113] In the above steps, the latent diffusion loss model (LDM) is used as a guide to ensure the authenticity of the generated image. In this process, the VAE is first pre-trained to transform the real image into the latent feature z0, and then noise is added to z0 to obtain the hidden feature zt of the noise. The pre-trained U-Net model ∈ ρ The principle of calculating LDM loss is shown in formula (13). Figure 4 The mechanism principle diagram of self-attention mechanism and Transformer is given.

[0114]

[0115] The above strategy will ensure that the text and images generated by the large language model remain consistent. In addition, in S4, in order to solve the problem that the alignment failure of text and image data training may lead to the degradation of image quality, a two-stage training strategy is adopted, that is, the first stage is for rough feature alignment, and the second stage is for fine-tuning and learning complex features. Since the input of the large language model comes from the knowledge graph, its data will contain image description information, so it is necessary to use auxiliary loss to assist voken in alignment, thereby minimizing the feature generation in the text-to-image generation model. And the feature τ in the text encoder σ The distance between them is expressed as:

[0116]

[0117] The final output of the present invention can be converted into text, image, image with text description, etc. Figure 5 A multimodal large language model framework with a generation loss module is given.

[0118] Beneficial effects: The present invention provides an intelligent decision-making method for risk emergency dispatch plans for water conservancy projects based on knowledge graphs. This method improves the shortcomings of existing water conservancy project risk management and control that rely only on a single data source, resulting in omissions in details in the risk emergency response plan decision-making process. Using images and images containing text descriptions as data input and obtaining corresponding prediction results can significantly improve the authenticity and accuracy of the plan decision-making process. In addition, the present invention builds a multi-professional risk transmission chain based on the knowledge graph in order to traverse the comprehensive emergency plan for water conservancy projects, thereby realizing multi-disciplinary and cross-professional coupled risk analysis and disposal strategy selection, which can improve the overall collaboration of water conservancy hub safety management and control and the comprehensive dispatching capabilities of the basin.

[0119] The present invention embeds the knowledge graph into the large language model, uses the professional knowledge of water conservancy projects as training data, and leverages the logical reasoning ability of the knowledge graph to improve the professionalism of the large language model in the field of water conservancy safety and the interpretability of the operation process. The combination of the two can achieve clear visualization of risk points and risk transmission chains in water conservancy projects, ensure the accuracy and reliability of risk emergency management plans, and provide technical support for dispatching and operating personnel to perform safe dispatching and disposal under sudden and multi-dimensional risk superposition conditions.

[0120] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.< / imagehere>

Claims

1. A method for intelligent control of water conservancy project risks based on knowledge graph embedding, characterized in that: The risk intelligent control method comprises: Step S1: Collect and organize relevant data of the target water conservancy project; Step S2: Using text and image data as multimodal input, construct a time-series-based multimodal data knowledge graph, including: The collected data is structured into triplets, and the head entity, relation, and tail entity are converted into independent embedding vectors, which are passed through the embedding layer to generate dense vectors; The dense vector is embedded into the deep layer of the model through high-order connections; TransE is used to learn the structured representation of the knowledge graph to achieve information aggregation of multimodal adjacent entities in entity text and enrich entity representation; Through the entity encoder and attention layer, each entity obtains a new entity representation, which is embedded into the knowledge graph through the translation scoring function, transforming the entities and relations into parameterized vectors and retaining the relational reasoning therein; Output recommendation layer; Step S3: using the output recommendation based on the multimodal knowledge graph as input to the large language model; Step S4: Generate multi-modality.

2. According to claim 1, a method for intelligent control of water conservancy project risks based on knowledge graph embedding is characterized in that: The step S1: collecting and organizing relevant data of the target water conservancy project specifically includes: water conservancy project operation monitoring data, investigation reports, research documents, emergency plans, scheduling procedure documents, operation monitoring images, on-site images and water conservancy project risk-related documents.

3. According to claim 1, a method for intelligent control of water conservancy project risks based on knowledge graph embedding is characterized in that: The output recommendation layer specifically includes: Each entity obtains the corresponding input through the knowledge graph embedding module and pushes it to the recommendation module, while the attention mechanism is used to aggregate the information; The layer aggregation mechanism is used to connect the representation of each step into a single vector, and its mathematical expression is as follows: Where L is the number of attention layers of the multimodal knowledge graph.

4. According to claim 1, a method for intelligent control of water conservancy project risks based on knowledge graph embedding is characterized in that: The method of obtaining a new entity representation for each entity through the entity encoder and the attention layer, embedding the new entity representation into the knowledge graph through the translation scoring function, converting the entity and the relationship into a parameterized vector and retaining the relationship reasoning therein specifically includes: The expression of the translation score function is shown in formula (7): goal(c,d,h)=||f c +f d -f h || 2 (7) During the knowledge graph training process, the loss pair rankings of correct triples and incorrect triples are sorted in pairs using formula (7) to obtain effective training results; α KG =∑ (c,d,h,h′)∈δ -lnθ(goal(c,d,h′)-goal(c,d,h)) (8) In the formula, (c, d, h′) is an incorrect triplet formed by randomly replacing entities in the correct triplet, and θ represents the sigmiod function; The layer models entities and relations at the granularity of triples for regularization, inputting direct connections into the representation.

5. According to claim 1, a method for intelligent control of water conservancy project risks based on knowledge graph embedding is characterized in that: The step S3: using the output recommendation based on the multimodal knowledge graph as the input of the large language model specifically includes: Multimodal encoding, the text tags output by the knowledge graph are embedded into the vector e t ∈R D , after pre-training and visual encoder, each image input is converted into feature e img ∈R D×32 , the embeddings are concatenated to create input prompt features; Vokens added, added special mark M img As the generation call function of the large language model corpus, where M img Specifically expressed as: M img ={img1,img2,img3,…,imgn} (9) The hidden output states of the variables are used for subsequent image generation, and the positions of the variables can be used to represent the insertion of interlaced images.

6. According to claim 1, a method for intelligent control of water conservancy project risks based on knowledge graph embedding is characterized in that: The step S4: generating multi-modality specifically includes: Text generation, following the strategy of stochastic language modeling, jointly generates text and virtual expectations in the text space, and adds "function words" to the positions of actual images during training, while training the model to predict "function words" during text generation; The generated chunk is represented as: Tk={t1,t2,t3,…,t n } t i ∈V∪V img (10) The text-image loss is defined as: Where, t i ∈V∪V img ; Establish Voken features for image generation.

7. According to claim 6, a method for intelligent control of water conservancy project risks based on knowledge graph embedding is characterized in that: The establishment of Voken features for image generation specifically includes: Align the output hidden state value with the text-conditional feature space of the text-to-image generation model and construct a feature mapper module containing a 2-layer MLP model θ mlp 、A 4-layer encoder-decoder model based on Transformer enc-dec And a learnable decoding feature sequence q, where the mathematical expression of the mapping feature is as follows: The latent diffusion loss model (LDM) is used as a guide to ensure the authenticity of the generated images; In the process, the pre-trained VAE transforms the real image into the latent feature z0, and then adds noise to z0 to obtain the latent feature z of the noise. t , using the pre-trained U-Net model ∈ ρ The principle of calculating LDM loss is shown in formula (13);

Images