A Method and System for Constructing a Knowledge Graph for Flood Control Scheduling Based on a Large Language Model and OpenSPG

Abstract

This invention discloses a method and system for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG, relating to the field of flood control scheduling technology. The specific steps of the detection method are as follows: S100, Multi-source heterogeneous real-time data acquisition and integration: Collecting structured, semi-structured and unstructured data from real-time hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments, and government open websites. This invention not only enriches the data sources by collecting structured, semi-structured and unstructured data from multiple channels, such as real-time hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments, and government open websites, but also effectively improves the quality and usability of the data through advanced data preprocessing techniques, such as using the pandas library to clean structured data, using parsing tools to process semi-structured data, and using OCR technology to process unstructured data.

Description

Technical Field

[0001] This invention relates to the field of flood control scheduling technology, specifically to a method and system for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG. Background Technology

[0002] With the acceleration of global climate change and urbanization, extreme weather events are becoming more frequent, and floods pose a huge threat to human society and the natural environment. Flood control scheduling, as a key measure to mitigate the impact of floods, is directly related to the safety of people's lives and property and the stable development of the social economy. In recent years, with the rapid development of information technology, big data and artificial intelligence technologies have provided new ideas and methods for flood control scheduling. In particular, knowledge graph technology, by constructing a structured knowledge system, can integrate, analyze, and utilize massive amounts of flood control scheduling data, providing more scientific and accurate support for flood control scheduling decisions.

[0003] Compared with the prior art document "202210414164.3 A Method for Constructing a Flood Control Dispatch Knowledge Graph," although the prior art document also proposes a construction idea for a flood control dispatch knowledge graph, its limitations are obvious: in terms of data sources, it may rely too much on a single or a few data sources, resulting in insufficient data coverage and difficulty in comprehensively and accurately reflecting the actual situation of flood control dispatch; in terms of data processing capabilities, the prior art document may have limitations in processing semi-structured and unstructured data, causing a large amount of valuable information to be overlooked; in terms of intelligence level, the prior art document may have limited capabilities in entity recognition, relation extraction, and knowledge reasoning, making it difficult to automatically and efficiently mine valuable information; in addition, the prior art document lacks an effective dynamic update mechanism for the knowledge graph, making it difficult to keep the information in the knowledge graph timely.

[0004] Therefore, the development of a flood control scheduling knowledge graph construction method and system based on large language models and OpenSPG not only improves the scientificity and effectiveness of flood control scheduling work, but also brings new technological breakthroughs to the field of flood control and disaster reduction. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method and system for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG. Through multi-source heterogeneous data collection, preprocessing, knowledge construction, storage and application steps, it realizes the comprehensive integration of flood control scheduling knowledge. This method innovatively integrates structured, semi-structured and unstructured data, uses advanced technologies to improve data quality and usability, and constructs a high-quality knowledge graph, providing scientific support for flood control scheduling decisions.

[0006] To solve the above-mentioned technical problems, this invention provides the following technical solution: On the one hand, a method for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG, the specific steps of which are as follows:

[0007] S100, multi-source heterogeneous real-time data acquisition and integration: real-time acquisition of structured, semi-structured and unstructured data from hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments and government open websites, and preliminary processing of different data during acquisition;

[0008] S200, Data Preprocessing: Use the pandas library to clean structured data, use parsing tools to process semi-structured data to extract key data, use OCR technology to process unstructured scanned documents, and use natural language processing toolkits to extract key information after manual verification.

[0009] S300, Knowledge Modeling: Utilizes a large language model to assist OpenSPG's SPG-Schema semantic function modeling, uses the model for entity recognition and relation extraction, implements a multi-source data fusion strategy, and constructs a knowledge graph based on OpenSPG;

[0010] S400, Knowledge Building: Install and configure Neo4j graph database, set parameters according to hardware and data scale, import flood control scheduling knowledge graph after initialization, optimize index using Cypher query language, and monitor and optimize database performance regularly;

[0011] S500, Knowledge Application: Collect and label user questions in the field of flood control scheduling, select algorithms to build and train a secondary intent classification model and optimize the model, while using a large language model to extract key information of the questions and match them with entities in the knowledge graph, combine user intent to retrieve answers, and output them in natural language through the Django framework. At the same time, a knowledge query interface and decision support model are developed and updated regularly.

[0012] Furthermore, in S100, the method for acquiring various types of data in multi-source heterogeneous real-time data acquisition and integration:

[0013] Real-time hydrological monitoring data acquisition: Write a program according to the interface specifications of hydrological monitoring stations to periodically request and obtain data on water level, flow rate, and rainfall, and then perform format conversion and verification after acquisition;

[0014] Real-time weather warning data: A Websocket client is built according to the interface type and specifications of the meteorological department to receive weather warning data in real time, and then parse and extract key information.

[0015] Real-time flood control engineering facility operation status data: Write SQL query statements to obtain data on reservoir water level and gate opening, and verify data integrity and consistency during reading;

[0016] Historical flood control scheduling case documents, policy regulations and procedures: Documents are sorted and recorded in the internal file system of the water resources department by year and region; web crawling technology is used to collect documents from government websites based on keywords and document types, and then the collected documents are screened and organized.

[0017] Furthermore, in S100, the specific content of the structured, semi-structured, and unstructured data in the multi-source heterogeneous real-time data acquisition and integration is as follows:

[0018] Structured data includes: water level, flow rate, and rainfall data collected by real-time hydrological monitoring stations; reservoir water level and gate opening data obtained by automated monitoring systems for flood control engineering facilities; and historical hydrological data, meteorological data, flood control engineering design parameters, and land use information stored in databases.

[0019] Semi-structured data: configuration files, log files, and some data in XML and JSON formats;

[0020] Unstructured data: exists in historical flood control scheduling case documents, policy and regulatory documents, and unformatted text records.

[0021] Furthermore, in S200, the processing of various types of data during data preprocessing includes:

[0022] For processing structured data: fill in missing values, remove outliers, and delete duplicate data;

[0023] For semi-structured data processing: For XML format data, the ElementTree parser is used to extract key data based on the document structure and tag definitions; for JSON format data, Python's json library is used to convert it into a dictionary or list structure before extracting key information.

[0024] For processing unstructured data: the scanned document is preprocessed, the brightness, contrast and resolution parameters of the image are adjusted, OCR technology is used to recognize the text in the scanned document, key information is manually verified after recognition, and then NLTK is used for part-of-speech tagging and named entity recognition to extract key information related to flood control scheduling.

[0025] Furthermore, in S300, the strength of relationships between entities is measured using an entity relationship strength formula in knowledge modeling. The formula is: Relationship Strength Among them, E i and E jThere are two entities, A i ,a and A j,a These are their a-th attributes, respectively. The Sim function is used to calculate the similarity between two attributes, where n is the total number of attributes, and Co(E) = ... i E j ) represents entity E i and E j The co-occurrence frequency, Do(E) i E j The relation weights are assigned based on domain knowledge. α, β, and γ are weight coefficients used to adjust the importance of each part in the relation strength calculation, and α+β+γ=1.

[0026] Furthermore, in S300, the multi-source data fusion strategy in knowledge modeling is as follows:

[0027] Data standardization: Standardize structured, semi-structured, and unstructured data from different data sources: For structured data, unify data format, encoding method, and data type; for semi-structured data, extract key information and perform structured transformation; for unstructured data, convert it to text format and perform information extraction and structured representation.

[0028] Entity linking: Linking the same entity from different data sources by establishing a unified entity identifier;

[0029] Attribute alignment: Align and integrate the attributes of the same entity in different data sources. At the same time, for missing attribute values ​​in different data sources, fill or estimate them.

[0030] Data conflict resolution: During the data fusion process, if the attribute values ​​of the same entity are inconsistent in different data sources, judgment and selection are made based on factors such as data reliability and timeliness.

[0031] Furthermore, in S300, the specific steps for constructing the knowledge graph in knowledge modeling are as follows: the data after entity recognition, relation extraction, and data fusion processing is imported into the OpenSPG platform. Using the SPG-Builder function of OpenSPG, a flood control scheduling knowledge graph is constructed based on the constructed SPG-Schema semantic model and the fused data. During the construction process, entities are used as nodes, relations as edges, and attributes as additional information for nodes and edges to form a complete knowledge graph structure. The knowledge graph is then reasoned and improved using the knowledge reasoning function of OpenSPG, and the constructed flood control scheduling knowledge graph is evaluated and optimized.

[0032] Furthermore, in S500, the construction of the secondary intent classification model in knowledge application is based on the following formula: Among them, P(Ck |x) represents the intent category C given the user question x. k The probability, where m is the total number of intent categories, and z is the probability. k Let represent the linear combination score of the k-th intent category, consisting of feature weights and bias terms, calculated using the following formula: x j It is the j-th feature after feature extraction, w k,j It is the weight connecting the j-th feature and the k-th intent category, b k It is a bias term, and the weights w are continuously adjusted through training. k,j and bias term b k This makes the probability prediction more accurate.

[0033] Furthermore, in S500, the knowledge application evaluates the degree of matching between the extracted key information and entities in the knowledge graph using a comprehensive matching degree formula. The formula is: Comprehensive matching degree (Q,E)=λ1·Sem(Q,E)+λ2·P(C|Q)·Int(C,E)+λ3·Ent(Q,E) where Q is the user question, E is the entity in the knowledge graph, Sem(Q,E) is the semantic similarity between the question and the entity, P(C|Q) is the probability that the question obtained from intent classification belongs to intent category C, Int(C,E) measures the degree of matching between the intent category and the entity, Ent(Q,E) represents the degree of association between the question and the entity, and λ1, λ2 and λ3 are weight coefficients, and λ1+λ2+λ3=1, which are adjusted according to the actual application scenario and needs.

[0034] On the other hand, a flood control scheduling knowledge graph construction system based on a large language model and OpenSPG is provided. The system includes: a multi-source heterogeneous data acquisition module, a data preprocessing module, a knowledge construction module based on a large language model and OpenSPG, a knowledge storage module, and a knowledge application module.

[0035] The multi-source heterogeneous data acquisition module: collects structured, semi-structured and unstructured data in real time from hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments and government websites; writes programs according to the interface specifications of hydrological monitoring stations to obtain hydrological data; builds a Websocket client to receive meteorological early warning data; writes SQL statements to obtain engineering facility operation data; and collects document data from internal systems and websites.

[0036] The data preprocessing module uses the pandas library to clean structured data, uses parsing tools to process semi-structured data, and uses OCR technology to process unstructured scanned documents.

[0037] The knowledge construction module based on the large language model and OpenSPG: uses the large language model to assist OpenSPG's SPG-Schema semantic modeling, performs entity recognition and relation extraction, uses formulas to measure the strength of entity relations, implements a multi-source data fusion strategy, and then constructs a knowledge graph based on OpenSPG, and performs reasoning, evaluation and optimization.

[0038] The knowledge storage module includes: installing and configuring the Neo4j graph database, setting parameters according to hardware and data scale, importing the flood control scheduling knowledge graph after initialization, optimizing the index using the Cypher query language, and regularly monitoring and optimizing database performance.

[0039] The knowledge application module collects and labels user questions in the field of flood control scheduling, uses algorithms to build and train a secondary intent classification model, extracts key information of questions and matches them with entities in the knowledge graph using a large language model, evaluates the degree of matching through a comprehensive matching degree formula, retrieves answers based on user intent, outputs the results in natural language using the Django framework, develops a knowledge query interface and decision support model, and regularly updates the knowledge graph.

[0040] Compared with existing technologies, this method and system for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG has the following advantages:

[0041] I. This invention collects structured, semi-structured, and unstructured data from multiple channels, including real-time hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments, and government websites. This not only enriches the data sources but also effectively improves data quality and usability through advanced data preprocessing techniques, such as using the pandas library to clean structured data, using parsing tools to process semi-structured data, and using OCR technology to process unstructured data. More importantly, the multi-source data fusion strategy adopted in this invention achieves seamless connection and deep integration between different data sources through data standardization, entity linking, attribute alignment, and data conflict resolution steps. This deep integration not only eliminates data silos but also makes the information in the knowledge graph more comprehensive and accurate.

[0042] Second, this invention introduces a large language model to assist OpenSPG's SPG-Schema semantic modeling, achieving efficient entity recognition and relation extraction. It can automatically mine valuable information from data and construct a well-structured and content-rich flood control scheduling knowledge graph. This not only improves the efficiency and accuracy of knowledge construction but also makes the knowledge graph more scalable. In terms of knowledge application, this invention collects and labels user questions in the field of flood control scheduling, trains a secondary intent classification model, and uses a large language model to extract key information from the questions and match it with entities in the knowledge graph. This achieves accurate answers to user questions and decision support. At the same time, by developing a knowledge query interface and decision support model, users can more conveniently obtain the information they need, providing powerful intelligent support for flood control scheduling work.

[0043] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0045] Figure 1 The flowchart shows the method for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG.

[0046] Figure 2 A schematic diagram of the structure of a flood control scheduling knowledge graph construction system based on a large language model and OpenSPG;

[0047] Figure 3 The framework diagram of the flood control scheduling knowledge graph construction method and system based on large language model and OpenSPG provided by this invention;

[0048] Figure 4 The flowchart of the information extraction method based on large language model and OpenSPG provided by this invention;

[0049] Figure 5 This invention provides a flowchart of knowledge application based on flood control scheduling knowledge graph.

[0050] Figure 6 This is an example of a flood control scheduling knowledge graph built based on a large language model and OpenSPG, provided by the present invention. Detailed Implementation

[0051] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0052] Example 1:

[0053] Flood control planning and facility management.

[0054] In the scenario of flood control planning and comprehensive management of flood control facilities in a certain river basin, a flood control scheduling knowledge graph construction method based on a large language model and OpenSPG is applied. The specific steps are as follows:

[0055] Data Acquisition and Integration: Real-time hydrological monitoring data is acquired periodically via a program written according to the interface specifications of hydrological monitoring stations, including water level, flow rate, and rainfall data. After acquisition, the data undergoes format conversion and verification. Real-time meteorological early warning data is received in real-time via a built WebSocket client, and key information is extracted after parsing. Real-time flood control engineering facility operation status data is obtained by writing SQL queries to retrieve reservoir water level and gate opening data, which are then recorded after verifying completeness and consistency. Simultaneously, historical flood control scheduling case documents, policies, regulations, and procedures are carefully reviewed within the water resources department's internal file system, organized by year and region, and metadata is recorded. Web crawling technology is used to collect files from publicly available government websites based on relevant keywords and file types. These collected files are then filtered and organized, containing information such as implementation experience of flood control plans in different regions, construction standards for flood control facilities, and maintenance records. Figure 1 As shown.

[0056] Data preprocessing: For structured data, the pandas library is used for cleaning, filling missing values, removing outliers, and deleting duplicate data. For semi-structured data, such as XML data, the ElementTree parser is used to extract key data based on document structure and tag definitions. For JSON data, Python's json library is used to convert it into a dictionary or list structure before extracting key information. For unstructured data, scanned documents are preprocessed by adjusting image brightness, contrast, and resolution parameters. OCR technology is used to recognize the text in the scanned documents. After recognition, key information is manually verified. Then, NLTK is used for part-of-speech tagging and named entity recognition to extract key information related to flood control planning and facility management, such as the type of flood control facilities, construction time, and maintenance requirements. Figure 4 As shown.

[0057] Knowledge Construction: Utilizing a large language model to assist OpenSPG's SPG-Schema semantic modeling, entity recognition and relation extraction are performed to identify entities such as "flood control planning area," "flood control facility type," "maintenance cycle," and "construction standards," as well as the relationships between them. For example, there is an association between "flood control planning area" and "flood control facility type," and different planning areas may be suitable for different types of flood control facilities. Figure 3 As shown, the strength of the relationship between entities is measured using the entity relationship strength formula, which is: Relationship Strength Among them, E i and E j Representing two entities, A i,a and A j,a These are their a-th attributes, respectively. The Sim function is used to calculate the similarity between two attributes, where n is the total number of attributes, and Co(E) = ... i E j ) represents entity E i and E j The co-occurrence frequency, Do(E) i E j The relation weights are assigned based on domain knowledge. α, β, and γ are weight coefficients, and α + β + γ = 1. This formula can be used to determine, for example, the closeness of the relationship between "flood control facility type" and "maintenance cycle". Figure 6 As shown.

[0058] A multi-source data fusion strategy is adopted. First, structured, semi-structured, and unstructured data from different data sources are standardized. The format, encoding method, and data type of structured data are unified. Key information is extracted from semi-structured data and transformed into a structured form. Unstructured data is converted into text format and information is extracted and represented in a structured manner. Next, a unified entity identifier is established to link the same entity from different data sources. Then, the attributes of the same entity from different data sources are aligned and integrated. Missing attribute values ​​are handled using imputation and estimation methods. During the data fusion process, if the attribute values ​​of the same entity are inconsistent in different data sources, [further steps are taken]. Based on factors such as data reliability and timeliness, the data is selected and then imported into the OpenSPG platform after entity recognition, relation extraction, and data fusion processing. Using OpenSPG's SPG-Builder function, a flood control scheduling knowledge graph is constructed based on the previously built SPG-Schema semantic model and the fused data. During the construction process, entities are used as nodes, relations as edges, and attributes as additional information for nodes and edges to form a complete knowledge graph structure. OpenSPG's knowledge reasoning function is used to reason and improve the knowledge graph, and the constructed flood control scheduling knowledge graph is evaluated and optimized.

[0059] Knowledge storage: Install and configure the Neo4j graph database, set parameters according to hardware and data scale, import the flood control scheduling knowledge graph after initialization, optimize the index using Cypher query language, and regularly monitor and optimize database performance to ensure efficient and stable storage and querying of the knowledge graph.

[0060] Knowledge Application: Collect and label user questions related to flood control planning and facility management in the field of flood control scheduling, such as "When conducting flood control planning in a specific area, which flood control facility should be given priority?" and "How to formulate a reasonable maintenance plan based on the existing flood control facilities?". Use an adaptation algorithm to train a secondary intent classification model, with the formula: Among them, P(C k |x) represents the intent category C given the user question x. k The probability, where m is the total number of intent categories, and z is the probability. k The calculation formula is: x j It is the j-th feature after feature extraction, w k,j It is the weight connecting the j-th feature and the k-th intent category, b k It is a bias term, and the weights w are continuously adjusted through training. k,j and bias term b k This makes the probability prediction more accurate, such as Figure 5 As shown, this is used to accurately determine the intent category of the user's question.

[0061] Using a large language model, key information from questions is extracted. For example, for the question "When conducting flood control planning in a specific area, which flood control facility should be given priority?", the key information "specific area," "flood control planning," and "flood control facility" is extracted and matched with entities in a knowledge graph. The degree of matching between the extracted key information and the entities in the knowledge graph is evaluated using a comprehensive matching degree formula: Comprehensive Matching Degree (Q,E) = λ1·Sem(Q,E) + λ2·P(C|Q)·Int(C,E) + λ3·Ent(Q,E), where Q is the user question, E is the entity in the knowledge graph, Sem(Q,E) is the semantic similarity between the question and the entity, P(C|Q) is the probability that the question belongs to intent category C obtained from intent classification, Int(C,E) measures the degree of matching between the intent category and the entity, Ent(Q,E) represents the degree of association between the question and the entity, and λ1, λ2, and λ3 are... The weighting coefficients, λ1+λ2+λ3=1, are adjusted according to actual application scenarios and needs. The answers are retrieved based on user intent and output in natural language using the Django framework. For example, "When conducting flood control planning in a specific area, considering the area's topography and historical flood data, priority should be given to constructing dike-type flood control facilities, which can effectively block flood intrusion and ensure regional safety." A knowledge query interface is also developed to facilitate quick access to relevant knowledge on flood control planning and facility management. A decision support model is established to provide a scientific basis for flood control planning and facility management decisions. For example, based on data and analysis results in the knowledge graph, specific maintenance times and content suggestions are provided for facility maintenance plans. The knowledge graph is regularly updated to incorporate new flood control technologies, policies, regulations, and practical case experience to ensure the timeliness and accuracy of knowledge, better serving flood control planning and facility management.

[0062] In summary, for flood control planning and facility management scenarios, this invention utilizes, as follows: Figure 2 The system, as shown, collects data from multiple channels, including real-time monitoring and historical documents. Data preprocessing is performed on various types of data. Knowledge construction identifies key entities and relationships, uses formulas to evaluate relationship strength, and integrates data to build a knowledge graph. The Neo4j graph database stably stores knowledge. In the knowledge application stage, the model is trained to understand user intent, and answers are selected through comprehensive matching. The results are then output in natural language using the Django framework. This series of operations helps to formulate scientific flood control plans and achieve efficient management of flood control facilities.

[0063] Example 2:

[0064] Flood warning and dispatch decision support.

[0065] Data Acquisition and Integration: For real-time hydrological monitoring data, programs were written according to the interface specifications of hydrological monitoring stations to periodically request and obtain water level, flow, and rainfall data, which were then converted and verified. Real-time meteorological early warning data was received in real time through a WebSocket client, and key information was extracted after receiving the data. Real-time flood control engineering facility operation status data was obtained by writing SQL queries to obtain reservoir water level and gate opening data, and the integrity and consistency were verified. Historical flood control scheduling case documents, policies, regulations, and procedures were organized and recorded in the water resources department's internal file system by year and region for metadata, and collected and filtered from government websites using web crawlers.

[0066] Data preprocessing: Structured data is cleaned using the pandas library, which fills in missing values, removes outliers, and deletes duplicate data; in semi-structured data, XML format is parsed using the ElementTree parser to extract key data, and JSON format is transformed using the json library to extract information; unstructured scanned documents are first preprocessed for image parameters, then OCR technology is used to recognize text, and after manual verification, NLTK is used for part-of-speech tagging and named entity recognition to extract key information.

[0067] Knowledge Construction: Utilizing a large language model to assist OpenSPG's SPG-Schema semantic modeling, entity recognition and relation extraction are performed. For example, the entities "reservoir," "water level," and "rainfall" are identified, along with their relationships. Relationship strength is calculated using the entity relationship strength formula: Relationship Strength This is used to measure the degree of correlation between entities. A multi-source data fusion strategy is adopted to standardize the processing of data from different data sources, establish a unified entity identifier for entity linking, align and integrate the same entity attributes, resolve data conflicts based on reliability and timeliness, import the processed data into the OpenSPG platform, use the SPGBuilder function to build a flood control scheduling knowledge graph, use the knowledge reasoning function to improve the graph and evaluate and optimize it.

[0068] Knowledge storage: Install and configure the Neo4j graph database, set parameters according to hardware and data scale, import the knowledge graph after initialization, optimize the index using Cypher query language, and regularly monitor and optimize database performance.

[0069] Knowledge Application: Collect and label user questions in the field of flood control scheduling, such as "Under the current rainfall conditions, how should a reservoir's gates be managed?", and use these to train a secondary intent classification model. The formula is as follows: The weights and biases are continuously adjusted to improve prediction accuracy. Key information from questions, such as "rainfall," "reservoir," and "gate scheduling," is extracted using a large language model and matched with entities in the knowledge graph. The degree of matching is evaluated using a comprehensive matching degree formula: Comprehensive Matching Degree (Q,E) = λ1·Sem(Q,E) + λ2·P(C|Q)·Int(C,E) + λ3·Ent(Q,E). The answer is retrieved based on the user's intent and output in natural language using the Django framework. A knowledge query interface is also developed to facilitate user queries. A decision support model is established to provide a reference for scheduling decisions, and the knowledge graph is updated regularly to ensure information accuracy.

[0070] In summary, in the scenario of flood warning and dispatch decision support, this invention, through the collection and integration of multi-source heterogeneous data, gathers real-time hydrological, meteorological, engineering facility data and historical documents. Data preprocessing ensures data quality. Knowledge construction utilizes a large language model and OpenSPG to clarify entity relationships and integrate multi-source data to construct a knowledge graph. Knowledge is stored in an optimized Neo4j graph database for convenient and efficient querying. In terms of knowledge application, a secondary intent classification model is trained to understand user needs, and answers are accurately retrieved using a comprehensive matching degree formula. The output is achieved using the Django framework. The entire process forms a closed loop, providing scientific and accurate support for flood warning and dispatch decision support.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG, characterized in that, The specific steps of this construction method are as follows: S100, multi-source heterogeneous real-time data acquisition and integration: real-time acquisition of structured, semi-structured and unstructured data from hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments and government open websites, and preliminary processing of different data during acquisition; S200, Data Preprocessing: Use the pandas library to clean structured data, use parsing tools to process semi-structured data to extract key data, use OCR technology to process unstructured scanned documents, and use natural language processing toolkits to extract key information after manual verification. S300, Knowledge Modeling: Utilizes a large language model to assist OpenSPG's SPG-Schema semantic function modeling, uses the model for entity recognition and relation extraction, implements a multi-source data fusion strategy, and constructs a knowledge graph based on OpenSPG; S400, Knowledge Building: Install and configure Neo4j graph database, set parameters according to hardware and data scale, import flood control scheduling knowledge graph after initialization, optimize index using Cypher query language, and monitor and optimize database performance regularly; S500, Knowledge Application: Collect and label user questions in the field of flood control scheduling, select an algorithm to build and train a secondary intent classification model, and optimize the model. The calculation formula for the secondary intent classification model is: ,in, Indicates a given user question In this case, it belongs to the intent category. The probability, It is the total number of intent categories. The linear combination score of the k-th intent category is composed of feature weights and bias terms, and is calculated using the following formula: , It is the first after feature extraction One characteristic, It is the connection of the first The first feature and the second The weights of each intent category It is a bias term, and its weights are continuously adjusted through training. and bias terms This makes the predicted probability more accurate. Simultaneously, it utilizes a large language model to extract key information from the question and evaluates the degree of matching between the extracted key information and entities in the knowledge graph using a comprehensive matching degree formula. The mathematical formula for the comprehensive matching degree is: Comprehensive Matching Degree. Sem Int Ent ,in, It's a user issue. It is an entity in a knowledge graph, Sem It is the semantic similarity between the question and the entity. The question obtained from intent classification belongs to the intent category. The probability of Int It measures the degree of matching between intent categories and entities. This indicates the degree of association between the question and the entity. , and It is a weighting coefficient, and The system is adjusted according to actual application scenarios and needs, combines user intent to retrieve answers, and outputs them in natural language through the Django framework. At the same time, a knowledge query interface and decision support model are developed and updated regularly.

2. The flood control scheduling knowledge graph construction method based on a large language model and OpenSPG according to claim 1, characterized in that, S100, the method for acquiring various types of data in multi-source heterogeneous real-time data acquisition and integration: Real-time hydrological monitoring data acquisition: Write a program according to the interface specifications of hydrological monitoring stations to periodically request and obtain data on water level, flow rate, and rainfall, and then perform format conversion and verification after acquisition; Real-time weather warning data: A Websocket client is built according to the interface type and specifications of the meteorological department to receive weather warning data in real time, and then parse and extract key information. Real-time flood control engineering facility operation status data: Write SQL query statements to obtain data on reservoir water level and gate opening, and verify data integrity and consistency during reading; Historical flood control scheduling case documents, policy regulations and procedures: Documents are sorted and recorded in the internal file system of the water resources department by year and region; web crawling technology is used to collect documents from government websites based on keywords and document types, and then the collected documents are screened and organized.

3. The flood control scheduling knowledge graph construction method based on a large language model and OpenSPG according to claim 1, characterized in that, In S100, the specific content of the structured, semi-structured, and unstructured data in the multi-source heterogeneous real-time data acquisition and integration is as follows: Structured data includes: water level, flow rate, and rainfall data collected by real-time hydrological monitoring stations; reservoir water level and gate opening data obtained by automated monitoring systems for flood control engineering facilities; and historical hydrological data, meteorological data, flood control engineering design parameters, and land use information stored in databases. Semi-structured data: configuration files, log files, and some data in XML and JSON formats; Unstructured data: exists in historical flood control scheduling case documents, policy and regulatory documents, and unformatted text records.

4. The flood control scheduling knowledge graph construction method based on a large language model and OpenSPG according to claim 1, characterized in that, S200, the processing of various types of data in data preprocessing: For processing structured data: fill in missing values, remove outliers, and delete duplicate data; For semi-structured data processing: For XML format data, the ElementTree parser is used to extract key data based on the document structure and tag definitions; for JSON format data, Python's json library is used to convert it into a dictionary or list structure before extracting key information. For processing unstructured data: the scanned document is preprocessed, the brightness, contrast and resolution parameters of the image are adjusted, OCR technology is used to recognize the text in the scanned document, key information is manually verified after recognition, and then NLTK is used for part-of-speech tagging and named entity recognition to extract key information related to flood control scheduling.

5. The flood control scheduling knowledge graph construction method based on a large language model and OpenSPG according to claim 1, characterized in that, In S300, knowledge modeling uses an entity relationship strength formula to measure the strength of relationships between entities. The formula is: Relationship Strength Co Do ,in, and There are two entities. and They are respectively their first The Sim function is used to calculate the similarity between two attributes. It is the total number of attributes, Co Representing entities and Co-occurrence frequency, Do The relation weights are assigned based on domain knowledge. , γ and γ are weighting coefficients used to adjust the importance of each component in the relationship strength calculation, and .

6. The method for constructing a flood control scheduling knowledge graph based on a large language model and OpenSPG according to claim 1, characterized in that, In S300, the multi-source data fusion strategy in knowledge modeling is as follows: Data standardization: Standardize structured, semi-structured, and unstructured data from different data sources: For structured data, unify data format, encoding method, and data type; for semi-structured data, extract key information and perform structured transformation; for unstructured data, convert it to text format and perform information extraction and structured representation. Entity linking: Linking the same entity from different data sources by establishing a unified entity identifier; Attribute alignment: Align and integrate the attributes of the same entity in different data sources. At the same time, for missing attribute values ​​in different data sources, fill or estimate them. Data conflict resolution: During the data fusion process, if the attribute values ​​of the same entity are inconsistent in different data sources, judgment and selection are made based on factors such as data reliability and timeliness.

7. The flood control scheduling knowledge graph construction method based on a large language model and OpenSPG according to claim 1, characterized in that, The specific steps for knowledge graph construction in S300 knowledge modeling are as follows: Data processed by entity recognition, relation extraction, and data fusion is imported into the OpenSPG platform. Using OpenSPG's SPG-Builder function, a flood control scheduling knowledge graph is constructed based on the built SPG-Schema semantic model and the fused data. During the construction process, entities are used as nodes, relations as edges, and attributes as additional information for nodes and edges, forming a complete knowledge graph structure. OpenSPG's knowledge reasoning function is used to reason about and improve the knowledge graph, and the constructed flood control scheduling knowledge graph is evaluated and optimized.

8. The flood control scheduling knowledge graph construction system based on a large language model and OpenSPG, characterized in that, The system is applicable to the flood control scheduling knowledge graph construction method based on large language model and OpenSPG as described in any one of claims 1-7. The system includes: a multi-source heterogeneous data acquisition module, a data preprocessing module, a knowledge construction module based on large language model and OpenSPG, a knowledge storage module, and a knowledge application module. The multi-source heterogeneous data acquisition module: collects structured, semi-structured and unstructured data in real time from hydrological monitoring stations, meteorological departments, automated monitoring systems for flood control engineering facilities, internal file systems of water conservancy departments and government websites; writes programs according to the interface specifications of hydrological monitoring stations to obtain hydrological data; builds a Websocket client to receive meteorological early warning data; writes SQL statements to obtain engineering facility operation data; and collects document data from internal systems and websites. The data preprocessing module uses the pandas library to clean structured data, uses parsing tools to process semi-structured data, and uses OCR technology to process unstructured scanned documents. The knowledge construction module based on the large language model and OpenSPG: uses the large language model to assist OpenSPG's SPG-Schema semantic modeling, performs entity recognition and relation extraction, uses formulas to measure the strength of entity relations, implements a multi-source data fusion strategy, and then constructs a knowledge graph based on OpenSPG, and performs reasoning, evaluation and optimization. The knowledge storage module includes: installing and configuring the Neo4j graph database, setting parameters according to hardware and data scale, importing the flood control scheduling knowledge graph after initialization, optimizing the index using the Cypher query language, and regularly monitoring and optimizing database performance. The knowledge application module collects and labels user questions in the field of flood control scheduling, uses algorithms to build and train a secondary intent classification model, extracts key information of questions and matches them with entities in the knowledge graph using a large language model, evaluates the degree of matching through a comprehensive matching degree formula, retrieves answers based on user intent, outputs the results in natural language using the Django framework, develops a knowledge query interface and decision support model, and regularly updates the knowledge graph.

Images