Water conservancy natural language model training data automatic generation system and method

Abstract

The invention relates to a water conservancy natural language model training data automatic generation system and method, and belongs to the technical field of data processing. A front-end application layer of the system comprises a system text uploading module, a health examination module and a storage downloading module, a rear-end application layer comprises a text analysis module, a text standardization module and a standardized natural language processing annotation data generation module, and a large model layer comprises a text intelligent extraction module. A user uploads an initial text, the text is sent to a text analysis module to be analyzed, then the text is sent to a text standardization module to be spliced into an adjustment and optimization statement, and the adjustment and optimization statement is subjected to request interaction with a text intelligent extraction module to generate a standardized text; and the text is transmitted to the health examination module for examination, the data is transformed into natural language processing training data in a standard format after passing the examination, and finally the data is transmitted to the storage downloading module. According to the method, the burden of manual auditing is reduced, and the negative influence of data errors on model training is fundamentally reduced.

Description

Technical Field

[0001] The present invention relates to a system and method for automatically generating water conservancy natural language model training data, and belongs to the technical field of smart water conservancy and data processing. Background Art

[0002] Currently, the models widely used in the field of natural language processing (NLP) are deeply constrained by the unified and finely labeled data format of the relationship extraction task. The construction process of this data format is extremely cumbersome. Traditional methods rely on manual identification and accurate extraction of relationships from the vast ocean of text, and manual organization of them into a standardized data format, which is often accompanied by a lot of repetitive work. This process is not only inefficient and time-consuming, but also very easy to introduce errors due to human factors, which in turn has an adverse effect on the training effect and accuracy of the model. Therefore, exploring more efficient and accurate data processing methods has become the key to promoting further development in the field of natural language processing (NLP). Summary of the invention

[0003] The object of the present invention is to overcome the above-mentioned shortcomings and to provide a system and method for automatically generating training data for a water conservancy natural language model, so as to improve the generation efficiency of training data.

[0004] The technical solution adopted by the present invention is: A water conservancy natural language model training data automatic generation system includes a front-end application layer, a back-end application layer and a large model layer. The front-end application layer includes a system upload text module, a health check module and a storage download module. The system upload text module is used for users to upload original text documents and transmit the text documents to the back-end application layer. The health check module is used to conduct a secondary review of the data standardized by the back-end application layer to ensure that the data is accurate and meets business standards. The storage download module is used to generate and provide file download services in real time after data processing and verification are completed; The back-end application layer includes a text parsing module, a text standardization module, and a standardized natural language processing (NLP) annotated data generation module. The text parsing module is responsible for parsing and processing the document data uploaded from the front-end application layer, segmenting and structuring the text, and converting it into a format suitable for subsequent processing; the text standardization module is used to tune the sentence splicing for each segmented data after completing the text parsing, and call the text intelligent extraction function of the large model layer to generate standardized text data; the standardized natural language processing (NLP) annotated data generation module is used to process the text data generated by the large model layer after verification by the health detection module of the front-end application layer, and generate a standardized natural language processing (NLP) annotated data format for information extraction tasks in natural language processing (NLP); The big model layer contains a text intelligent extraction module, which mainly uses the big model to split the sentence into two arrays of parameters and relations according to the required format and returns them to the text standardization module.

[0005] The health check module includes a parameter processor and a triples processor. The parameter processor is mainly used for fine structure processing of the data after model parsing, and the triples processor is mainly used for accurate correction and beautification of the data.

[0006] The text parsing module includes two modules, readWord and readText, which are used to read and process two text formats, Word and Text, respectively, and finally divide the contents of the two documents into paragraphs. The readWord module uses the Apache POI library to parse the paragraph structure in the document, and extracts the text of each paragraph and appends it to the string builder by traversing all the paragraphs of the document, and uses line breaks to isolate and filter out empty paragraphs; the readText module uses a buffer reader to read the text in the Text file paragraph by paragraph, and appends each paragraph of text to the string builder, and uses line breaks to isolate and filter out empty paragraphs.

[0007] The text standardization module communicates with the big model layer by asynchronously calling the big model interface through RestTemplate (REST client template). The information extracted by the big model layer text intelligent extraction module uses the ObjectMapper (object mapping tool) in the text standardization module to serialize the object into a JSON string and extract the standardized text therein.

[0008] The standardized natural language processing annotation data generation module includes a parameter processor, a sents (character sequence set) and a vertexSet (entity set) processor, and a labels (relationship label) processor. The parameter processor is used to extract entities from the data and deduplicate them, and at the same time verify the inclusion relationship between the entities and the original sentences; the sents (character sequence set) and vertexSet (entity set) processors are used to perform sentence segmentation, entity position recording, composite entity processing, and standardized splicing of original sentences and entities; the labels (relationship label) processor is used to record the position of the entities in the relationship in the vertexSet (entity set), standardize the splicing of the title, entity set and relationship label, and generate a standard natural language processing (NLP) annotation data format.

[0009] A method for automatically generating water conservancy natural language model training data comprises the following steps: First, the user uploads the initial text in the system upload text module of the front-end application layer. After the front-end application layer establishes a connection with the server, the text is transmitted to the text parsing module of the back-end application layer; the text parsing module parses the selected articles respectively, divides them into multiple independent text paragraphs according to the article paragraphs, and then transmits them to the text standardization module. The text standardization module splices each segmented text paragraph into a tuning statement that meets the needs of water conservancy business, and interacts with the text intelligent extraction module of the large model layer to generate standardized text; the standardized text generated by the large model layer is transmitted to the front-end application layer for inspection through the health check module. If it fails, the relationship data will be deleted. If it passes, the processed parameters will be transmitted to the standardized natural language processing (NLP) annotation data generation module of the back-end application layer to transform the data into natural language processing training data in a standard format. Finally, the data in the standard format is transmitted to the front-end application layer and stored and downloaded through the storage and download module.

[0010] The data processing process of the health check module is as follows: the parameter processor processes the refined structure of the data after model parsing, and uses regular expressions to accurately remove escape characters in the data, and then converts the data into object form through string segmentation technology to achieve effective data organization, and double-backs up the data, and sends one copy to the triples processor for further processing; the triples processor accurately corrects and beautifies the data, first determining the verification entity of the triples array after the large model end layer parsing, traversing the relationship data, and performing strict data type verification on the entities involved, eliminating non-compliant data, removing escape spaces remaining in the data, and verifying whether the entities mentioned in the relationship array actually exist in the original text, and then adding visually friendly beautification escape characters, and finally updating the stored second copy of the data content to these accurately corrected and beautified text data to ensure that the output parameter data is both standard and reliable.

[0011] The data processing process of the standardized natural language processing annotation data generation module is as follows: the parameter processor extracts entities from the data and deduplicates them, and further verifies the inclusion relationship between the entities and the original sentences, and inputs the parameter data into the sents (character sequence set) and vertexSet (entity set) processors; the sents (character sequence set) and vertexSet (entity set) processors perform sentence segmentation, intra-sentence character segmentation, entity position recording, composite entity processing, and standardized splicing of original sentences and entities on the data; then the data enters the labels (relationship label) processor, the labels (relationship label) processor records the position of the entities in the relationship in the vertexSet (entity set), and performs standardized splicing of the title, entity set, and relationship label to generate a standard natural language processing (NLP) annotation data format.

[0012] The present invention seamlessly uploads the original data text to the back-end application layer, which then intelligently reads and processes these text files, and uses the large model layer for in-depth analysis. After the analysis is completed, the analysis results are quickly fed back to the front-end application layer through a stable long connection mechanism. The front-end application layer uses a sophisticated health check mechanism to ensure the accuracy of the data format and the correctness of the content, and then transfers the data flow back to the back-end application layer. Finally, the back-end application layer converts this series of carefully processed data into a natural language processing (NLP) data model training format that meets standard specifications, and stores it securely on the front end, achieving comprehensive optimization and efficient use of data.

[0013] The beneficial effects of the present invention are: (1) The text standardization module uses the sentences tuned by the text standardization module to interact with the big model, allowing the big model to generate training data text. This can automatically generate a large amount of diverse high-quality training data, effectively solving the problem of long data production cycle and low efficiency under traditional methods. (2) During the data generation process, the health check module is used for review, the standardized natural language processing (NLP) annotation data generation module is used for secondary verification, and the text verification and error correction mechanism is used to ensure that the result data is completely consistent with the text format of the training data; (3) The generation method requires less manual processing and intervention. The core work is replaced by computers instead of humans. The standardized processing flow not only reduces the burden of manual review, but also fundamentally reduces the negative impact of data errors on model training, thereby improving efficiency and reducing the probability of errors. It can improve work efficiency by more than 50% based on the traditional manual processing flow. For NLP models that require a large amount of training data, it can save a lot of labor costs and maximize cost-effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a structural framework diagram of the system of the present invention; Figure 2 The data processing flow of the front-end application layer in the method of the present invention; Figure 3 The process of generating standardized text at the back-end application layer of the present invention; Figure 4 The data processing flow of the natural language processing (NLP) annotation data generation module of the back-end application layer of the present invention is standardized. DETAILED DESCRIPTION

[0015] The present invention is described in detail below with reference to specific embodiments.

[0016] Example 1 A system for automatically generating water conservancy natural language model training data, such as Figure 1 As shown, it includes a front-end application layer, a back-end application layer and a large model layer. The front-end application layer includes a system upload text module, a health check module and a storage download module. The system upload text module is used for users to upload original text documents and transmit the text documents to the back-end application layer. The health check module is used to conduct a secondary review of the data that has been initially parsed and standardized by the back-end application layer to ensure that the data is accurate and meets business standards. The storage download module is used to generate and provide file download services in real time after data processing and verification are completed.

[0017] The system upload text module provides users with a convenient way to safely upload original text documents to the server and automatically complete the stub record of the document. This function significantly reduces the risk of users being unable to recover files when files are accidentally lost.

[0018] The health check module integrates two core components: the parameter processor and the triples processor. The parameter processor is mainly used for fine-grained structural processing of data after model parsing, while the triples processor is mainly used for accurate correction and beautification of data.

[0019] The storage and download module is divided into two functional blocks: storage and download. In terms of storage, the module cleverly uses the localStorage technology of the web page to achieve seamless preservation and fast reading of user data, ensuring the persistence and accessibility of data. As for the download function, the module uses Blob object technology to efficiently convert the required data into a binary data stream and construct a file entity based on it. Subsequently, by dynamically generating a link and simulating user click operations, convenient file download is achieved, providing users with a smooth data management experience.

[0020] The system upload text module uses efficient long connection technology to securely and stably transmit this data to the back-end application layer for processing. The health check module ensures that every piece of data is accurate and meets the strict standards of business requirements. The storage and download module allows users to easily obtain the final results.

[0021] The back-end application layer includes a text parsing module, a text standardization module, and a standardized natural language processing (NLP) annotation data generation module. The text parsing module is responsible for parsing and processing the document data uploaded from the front-end application layer, segmenting and structuring the text, and converting it into a format suitable for subsequent processing; the text standardization module is used to optimize the sentence splicing for each segmented data after completing the text parsing, and call the text intelligent extraction function of the large model layer to generate standardized text data; the standardized natural language processing (NLP) annotation data generation module is used to process the text data generated by the large model layer after verification by the health detection module of the front-end application, and generate a standardized natural language processing (NLP) annotation data format for information extraction tasks in natural language processing (NLP).

[0022] Text parsing modules, such as Figure 3 In the WebSocket protocol, WebSocket is responsible for establishing a long connection with the client, including maintaining heartbeats to prevent connection timeouts, receiving document path information from the front end, and gradually transmitting the processed triplet data to the client to ensure real-time and interactivity. The text parsing module includes two modules, readWord and readText, which are used to read and process the two text formats of Word and Text, respectively, and finally divide the contents of the two documents into paragraphs. The readWord module uses the Apache POI (complex text file parsing library) library to parse the paragraph structure in the document. By traversing all the paragraphs of the document, the text of each paragraph is extracted and appended to the StringBuilder, and the line breaks are used to isolate and filter out empty paragraphs; the readText module uses BufferedReader to read the text in the Text file paragraph by paragraph, and appends each paragraph of text to the StringBuilder, and the line breaks are used to isolate and filter out empty paragraphs.

[0023] Text normalization modules, such as Figure 3In the process, each piece of data after document parsing will be spliced ​​with the predefined tuning statement to generate a complete request text, and the big model interface will be communicated with through the RestTemplate (REST client template) asynchronous call (GLM), and the object will be serialized into a JSON string using ObjectMapper (object mapping tool) as the content of the HTTP request body. Each piece of text data of the sent document is used as the text generated by the request big model. After receiving the response, use ObjectMapper (object mapping tool) to parse the JSON string of the response body, extract the standardized text, replace the line breaks in the returned text with __NEWLINE__ (to ensure that the format is maintained during WebSocket transmission), and send the processed results back to the front end through the session's send message method.

[0024] Standardized Natural Language Processing (NLP) annotation data description: (1) Title: used to explain the meaning of this section of labeled data; (2) sents (character sequence set): This is a two-dimensional list. Each inner list represents a sentence, and the characters in the sentence are split into individual characters or tokens. For example: sents[0] (character sequence set) represents the character sequence of sentence 0. sents[1] (character sequence set) represents the character sequence of sentence 1. This part stores the sentence information of the document or paragraph, which is used for subsequent entity recognition and relationship extraction tasks. When annotating, Chinese characters and punctuation marks are divided separately, and the numerical part and English words are divided as a whole. For example, 1600km / s needs to be split into "1600", "km", " / ", "s"; (3) vertexSet (entity set): This is a multidimensional list, each inner list contains multiple dictionaries, representing different mentions of a certain entity. Each dictionary contains the following information: name: represents the name of the entity mentioned in the text. sent_id: indicates the sentence in which the mention appears. pos: This is a list that represents the character position range (start, end) of the entity mention in the sentence. Used to mark the exact position of the mention. The entire vertexSet (entity set) can be regarded as a set of all entities in the document and their different mention positions. For example, vertexSet[0] (entity set) represents all mentions of the first entity, which may appear in different sentences (i.e., cross-sentence entity mentions), while vertexSet[1] (entity set) represents all mentions of the second entity; (4) labels: This is a list of relation labels. Each label represents the relationship between a pair of entities in the document. Each relation label contains the following fields: r: represents the relationship between entities. This field stores the type of relationship between two entities, which depends on the specific definition of the dataset. h: This is the index of the head entity, pointing to an entity in the vertexSet. t: This is the index of the tail entity, also pointing to an entity in the vertexSet. labels define all the annotated entity relationships in the document. The goal of the relation extraction model is to automatically identify these entity pairs and the relationships between them.

[0025] These data formats are mainly used for the following natural language processing tasks: Relation Extraction: Identify entities and their semantic relationships from text. Knowledge Graph Construction: Build a structured knowledge graph based on extracted entities and relationships. This type of data can be used to train a variety of information extraction models, such as Transformer-based models, Graph Neural Networks (GNNs), Joint Learning Models, and Sequence Labeling Models.

[0026] The standardized natural language processing annotated data generation module includes a parameter processor, sents (character sequence set) and vertexSet (entity set) processors, and labels (relationship label) processors. The parameter processor is used to extract entities from the data and deduplicate them, and verify the inclusion relationship between the entities and the original sentences; the sents (character sequence set) and vertexSet (entity set) processors are used to perform sentence segmentation, entity position recording, composite entity processing, and standardized splicing of original sentences and entities; the labels (relationship label) processor is used to record the position of entities in the relationship in vertexSet (entity set), standardize the splicing of titles, entity sets, and relationship labels, and generate standard natural language processing (NLP) annotated data formats.

[0027] The big model layer contains a text intelligent extraction module, which mainly uses the big model to generate and split the text data into two arrays of parameters and relationships according to the requirements of the tuning statement and return them to the text standardization module.

[0028] The large model layer uses a widely recognized open source model framework to input the original text data of the water conservancy industry into the neural network, and uses its powerful semantic understanding and analysis capabilities to conduct in-depth analysis of the data. Through the complex calculations of the neural network, we can accurately capture the key information in the data and intelligently process this information based on the actual needs and standards of the water conservancy industry. Finally, the system can automatically output data that meets specific requirements and formats. These data structures are clear and easy to understand, which will escort the final standardized natural language processing (NLP) annotation data.

[0029] Example 2 A method for automatically generating water conservancy natural language model training data using the system described in Example 1 is as follows: First, the user uploads the initial text in the system upload text module of the front-end application layer. After the front-end application layer establishes a connection with the server, the text is transmitted to the text parsing module of the back-end application layer; the text parsing module parses the selected articles respectively, divides them into multiple independent text paragraphs according to the article paragraphs, and then transmits them to the text standardization module. The text standardization module splices each segmented text paragraph into a tuning statement that meets the needs of water conservancy business, and interacts with the text intelligent extraction module of the large model layer to generate standardized text; the standardized text generated by the large model layer is transmitted to the front-end application layer for inspection through the health check module. If it fails, the relationship data will be deleted. If it passes, the processed parameters will be transmitted to the standardized natural language processing (NLP) annotation data generation module of the back-end application layer to transform the data into natural language processing training data in a standard format. Finally, the data in the standard format is transmitted to the front-end application layer and stored and downloaded through the storage and download module.

[0030] The data processing flow of the health check module is as follows Figure 2As shown in the figure, the parameter processor focuses on the refined structural processing of the data after model parsing: in the initial step, it uses regular expressions to accurately remove escape characters in the data, and then converts the data into object form through string segmentation technology to achieve effective data organization. After that, the data is double-backed up, and one copy is immediately sent to the triples processor for further processing. The triples processor focuses on the precise correction and beautification of the data. We convert the subject and object after sentence parsing into entity A and entity B, which are the first and third sub-elements of each item in the triples array after the large model layer is parsed. First, the relationship data is traversed, and strict data type verification is performed on the involved entities A and B. Any data that does not meet the specifications will be eliminated to ensure data quality. Then the processor removes the remaining escape spaces in the data and verifies whether the entities A and B mentioned in the relationship array actually exist in the original text, so as to enhance the accuracy of the data. On this basis, it also cleverly adds visually friendly beautification escape characters to make the data clearer and easier to read, which is convenient for subsequent review and modification. Finally, the triples processor updates the second data content that was originally stored with these carefully processed text data, ensuring that the output parameter data is both standard and reliable.

[0031] The function of the standardized natural language processing (NLP) annotation data generation module is to generate the standardized natural language processing (NLP) annotation data described above from the standardized parameters. The processing flow is as follows: Figure 4 middle: The parameter processor extracts two entities A and B in the ternary relationship from labels (relationship labels) and stores these entities in an intermediate structure vertexSet (entity set). Then the parameter processor verifies whether the entities A and B extracted from labels (relationship labels) exist in sents (character sequence set). If an entity does not exist, the system will return an error message indicating that the entity is missing. VertexSet (entity set) is then deduplicated, and the processor passes the deduplicated entity set and the unprocessed sents (character sequence set) together to the next processing module.

[0032] In the sents (character sequence set) and vertexSet (entity set) processors, the sents (character sequence set) and vertexSet (entity set) processors segment the input sents (character sequence set) text into sentences. This step includes the recognition and processing of sentence ending symbols (such as periods, question marks, and exclamation marks). The processors segment the Chinese characters and punctuation marks in each sentence independently, while processing the numeric part of the sentence and the complex document grammatical structure, and record the segmentation position to ensure that the processed sents (character sequence set) and vertexSet (entity set) can match. Then the entity part in vertexSet (entity set) is processed, and the starting and ending positions of each entity in sents (character sequence set) are recorded to ensure that the position of each entity in sents (character sequence set) can be correctly marked. Use regular expressions to process complex structures in vertexSet (entity set). For example, 1600km / s needs to be split into "1600", "km", " / ", and "s". The processor concatenates the processed sents (character sequence set) and vertexSet (entity set) to form a standardized sents (character sequence set) and vertexSet (entity set) format, and then enters the labels (relationship labels) processor.

[0033] In the labels processor, for all entities in the ternary relationship labels, the processor records their positions in the vertexSet to ensure that the entity positions of each relationship are accurately labeled. The processor then standardizes the splicing of the title, entity set, and relationship label to form the final output data structure. Finally, the processor returns the generated standardized NLP annotation data to the front-end application layer. The standard format of the data for the annotated relation extraction task is used for information extraction tasks in natural language processing (NLP).

[0034] The above is a detailed description of the present invention in combination with the embodiments, and the protection scope of the present invention is not limited thereto.

Claims

1. A water conservancy natural language model training data automatic generation system, including a front-end application layer, a back-end application layer and a large model layer, characterized in that the front-end The application layer includes a system text upload module, a health check module, and a storage download module. The system text upload module is used by users to upload original text documents and transmit the text documents to the back-end application layer. The health check module is used to conduct a secondary review of the data that has been standardized by the back-end application layer to ensure that the data is accurate and meets business standards. The storage download module is used to generate and provide file download services immediately after data processing and verification are completed; The back-end application layer includes a text parsing module, a text standardization module, and a standardized natural language processing annotation data generation module. The text parsing module is responsible for parsing and processing the document data uploaded from the front-end application layer, segmenting and structuring the text, and converting it into a format suitable for subsequent processing; the text standardization module is used to tune the sentence splicing for each segmented data after completing the text parsing, and call the text intelligent extraction function of the large model layer to generate standardized text data; the standardized natural language processing annotation data generation module is used to process the text data generated by the large model layer after verification by the health detection module of the front-end application layer, and generate a standardized natural language processing annotation data format for information extraction tasks in natural language processing; The big model layer contains a text intelligent extraction module, which mainly uses the big model to split the sentence into two arrays of parameters and relations according to the required format and returns them to the text standardization module.

2. According to claim 1, a water conservancy natural language model training data automatic generation system is characterized in that: The health check module includes a parameter processor and a triplet processor. The parameter processor is mainly used for fine structure processing of the data after model parsing, and the triplet processor is mainly used for accurate correction and beautification of the data.

3. The automatic generation system of water conservancy natural language model training data according to claim 1 is characterized in that: The text parsing module includes two modules, readWord and readText, which are used to read and process two text formats, Word and Text, respectively, and finally divide the contents of the two documents into paragraphs. The readWord module uses the Apache POI library to parse the paragraph structure in the document, and extracts the text of each paragraph and appends it to the string builder by traversing all the paragraphs of the document, and uses line breaks to isolate and filter out empty paragraphs; the readText module uses a buffer reader to read the text in the Text file paragraph by paragraph, and appends each paragraph of text to the string builder, and uses line breaks to isolate and filter out empty paragraphs.

4. The automatic generation system of water conservancy natural language model training data according to claim 1 is characterized in that: The text standardization module communicates with the big model layer by asynchronously calling the big model interface through the REST client template. The information extracted by the big model layer text intelligent extraction module uses the object mapping tool in the text standardization module to serialize the object into a JSON string and extract the standardized text therein.

5. The automatic generation system of water conservancy natural language model training data according to claim 1 is characterized in that: The standardized natural language processing annotation data generation module includes a parameter processor, a character sequence set and entity set processor, and a relationship label processor. The parameter processor is used to extract entities from the data and deduplicate them, and at the same time verify the inclusion relationship between the entities and the original sentences; the character sequence set and entity set processors are used to perform sentence segmentation, entity position recording, composite entity processing, and standardized splicing of original sentences and entities; the relationship label processor is used to record the position of the entities in the relationship in the entity set, and to perform standardized splicing of the title, entity set and relationship label to generate a natural language processing annotation data format that meets the standard.

6. A method for automatically generating water conservancy natural language model training data, using the system according to any one of claims 1 to 5, characterized in that: The steps include: First, the user uploads the initial text in the system upload text module of the front-end application layer. After the front-end application layer establishes a connection with the server, the text is transmitted to the text parsing module of the back-end application layer; the text parsing module performs text parsing on the selected articles respectively, divides them into multiple independent text paragraphs according to the article paragraphs, and then transmits them to the text standardization module. The text standardization module splices each segmented text paragraph into a tuning statement that meets the needs of water conservancy business, and interacts with the text intelligent extraction module of the large model layer to generate standardized text; the standardized text generated by the large model layer is transmitted to the front-end application layer for inspection through the health check module. If it fails, the relationship data will be deleted. If it passes, the processed parameters will be transmitted to the standardized natural language processing annotation data generation module of the back-end application layer to transform the data into natural language processing training data in a standard format. Finally, the data in the standard format is transmitted to the front-end application layer and stored and downloaded through the storage and download module.

7. The method for automatically generating water conservancy natural language model training data according to claim 6 is characterized in that: The data processing process of the health check module is as follows: the parameter processor processes the refined structure of the data after model parsing, and uses regular expressions to accurately remove escape characters in the data, and then converts the data into object form through string segmentation technology to achieve effective data organization, and double-backs up the data, and sends one copy to the triple processor for the next step of processing; the triple processor accurately corrects and beautifies the data, first determining the verification entity of the triple array after the large model end layer is parsed, traversing the relationship data, and performing strict data type verification on the entities involved, eliminating non-compliant data, removing escape spaces remaining in the data, and verifying whether the entities mentioned in the relationship array actually exist in the original text, and then adding visually friendly beautification escape characters, and finally updating the content of the second stored data to these accurately corrected and beautified text data to ensure that the output parameter data is both standard and reliable.

8. The method for automatically generating water conservancy natural language model training data according to claim 6 is characterized in that: The data processing process of the standardized natural language processing annotation data generation module is as follows: the parameter processor extracts entities from the data and deduplicates them, and further verifies the inclusion relationship between the entities and the original sentences, and inputs the parameter data into the character sequence set and entity set processor; the character sequence set and entity set processor performs sentence segmentation, intra-sentence character segmentation, entity position recording, composite entity processing, and standardized splicing of original sentences and entities on the data; then the data enters the relationship label processor, which records the position of the entities in the relationship in the entity set, and performs standardized splicing of the title, entity set, and relationship label to generate a natural language processing annotation data format that meets the standard.

Images