Text data processing method and system for gas turbine power plant
In the engineering management text data processing of gas turbine power plants, data augmentation technology and open source data sets are used to pre-train and adjust the language model to form a hybrid model, which solves the problem of insufficient model generalization capabilities in the existing technology and achieves more efficient data processing and utilization.
Patent Information
- Application Number
- CN202411602406.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-02-28
AI Technical Summary
In the prior art, in the engineering management text data processing of gas turbine power plants, the high dependence of artificial parameters leads to insufficient generalization capabilities of the model, making it difficult to adapt to different scenarios and diversified data characteristics.
By obtaining the engineering management text information of the gas turbine power plant for preprocessing, the initial data set is generated using data augmentation technology, and the language model is pre-trained and model parameter adjustment is adjusted using open source data sets. Finally, the adjusted model is spliced to form a hybrid model for data analysis.
It reduces the impact of manual parameters on the model, improves the generalization ability of the model, can process the engineering management text data of the gas turbine power plant more scientifically and efficiently, and improves the maintenance and utilization efficiency of data.
Smart Images
Figure CN119538907A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of text data processing in a gas turbine power plant, and in particular to a text data processing method and system for a gas turbine power plant. Background Art
[0002] During the construction of a gas turbine power plant, system integration and engineering management activities generate a large amount of text data, covering design plans, construction records, quality control reports, equipment commissioning logs and other information. These data contain rich knowledge and potential value, and can be used to guide engineering decisions, optimize construction processes, and improve equipment operating efficiency. However, during the actual construction phase, due to insufficient manpower and limited resources, these valuable data are often not fully organized, analyzed and utilized, resulting in waste of resources and low construction efficiency.
[0003] In existing technical solutions, language model training mostly relies on manually selected high-quality training data. This process is usually carried out on a single neural network structure, and the model performance is optimized by repeatedly adjusting the network parameters. Although this method can achieve good results on specific data sets, due to its high reliance on manually selected network parameters, the model's generalization ability has obvious defects and is difficult to adapt to different scenarios and diverse data features. Summary of the invention
[0004] In view of the above-mentioned problems, the present invention is proposed.
[0005] Therefore, the technical problem solved by the present invention is: how to scientifically and efficiently process the engineering management text data of a gas turbine power plant, reduce the influence of artificial parameters on the model, and improve the generalization ability of the model.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] In a first aspect, an embodiment of the present invention provides a method for processing text data of a gas turbine power plant, comprising:
[0008] Obtain and preprocess the engineering management text information of the gas turbine power plant, and generate the initial engineering management data set through data augmentation technology;
[0009] Use open source datasets to pre-train a group of language models at the same time, and use the initial engineering management dataset to adjust the model parameters of the language model;
[0010] The models after parameter adjustment are spliced to obtain a hybrid model;
[0011] Data analysis was performed on the initial engineering management dataset using a hybrid model.
[0012] As a preferred solution for text data processing method of gas turbine power plant, wherein:
[0013] The obtaining of the gas turbine power plant engineering management text information includes:
[0014] Enter the interactive texts of the construction units generated during the construction of completed gas turbine power plants and gas turbine power plants under construction, including project notifications and progress feedback documents; including project archive information and engineering information of monitoring and analysis reports.
[0015] As a preferred solution for text data processing method of gas turbine power plant, wherein:
[0016] The preprocessing to generate the initial engineering management data set by data augmentation technology includes:
[0017] The engineering management text information of the gas turbine power plant is converted into tabular data according to a pre-set format, the text is encoded using professional vocabulary, and data augmentation is added as the initial engineering management data set.
[0018] As a preferred solution for text data processing method of gas turbine power plant, wherein:
[0019] The language model includes a traditional convolutional neural network language model, a neural network language model with a self-attention mechanism, and a time-recurrent neural network model.
[0020] As a preferred solution for text data processing method of gas turbine power plant, wherein:
[0021] The use of open source datasets to simultaneously pre-train a set of language models includes:
[0022] The initial engineering management data set is used to adjust the language model in the form of LORA to obtain models A1, A2, and A3, whose output prediction probabilities are PA1, PA2, and PA3 respectively;
[0023] Introduce a low-rank matrix W between each layer of the model lora , adjust the weight of the model to reduce the consumption of computing resources; let the original weight matrix be W, and the introduced weight matrix be W′, then:
[0024] W′=W+W lora
[0025] Among them, W lora is a low-rank matrix;
[0026] The output prediction probability of each model is determined by the cross-validation method to ensure the generalization ability of the model on different data sets; the specific formula is:
[0027] PA i =Softmax(f i (X; Wi ′))
[0028] Among them, PA i For Model A i The output prediction probability of , i is the model number (1,2,3), f i Represents model A i The prediction function, X is the input data, W i ′ is the weight matrix after the low-rank matrix is introduced.
[0029] As a preferred solution for text data processing method of gas turbine power plant, wherein:
[0030] The splicing of the adjusted models includes:
[0031] After normalizing the output layers of the neural network training A1, A2, and A3, adaptive parameters α1, α2, and α3 are introduced, the model output layers are spliced, and the mixed output is set to P = (α1*PA1+α2*PA2+α3*PA3) for model splicing.
[0032] As a preferred solution for text data processing method of gas turbine power plant, wherein:
[0033] The data analysis of the initial engineering management data set using the hybrid model includes:
[0034] The text information of the initial engineering management dataset is rewritten as "archival text classification; file requirement information extracted from file content; file requirement processing results extracted from file content; manual evaluation of files". The rewritten text information is converted into a structured engineering management dataset using a professional vocabulary library and encoding rules. The encoding steps are: encode the classification results using One-Hot encoding or TF-IDF encoding; encode the file requirement information using a sequence labeling method; and binary encode the file requirement processing results and manual evaluation results.
[0035] In a second aspect, an embodiment of the present invention provides a text data processing system for a gas turbine power plant, comprising:
[0036] The initial data set acquisition module is used to acquire and pre-process the text information of the gas turbine power plant engineering management, and generate the initial engineering management data set through data augmentation technology;
[0037] The parameter tuning module is used to pre-train a group of language models simultaneously using open source datasets, and to tune the model parameters of the language models using the initial engineering management dataset;
[0038] The hybrid model building module is used to splice the models after parameter adjustment to obtain a hybrid model;
[0039] A data analysis module is used to perform data analysis on the initial engineering management dataset using a hybrid model.
[0040] In a third aspect, an embodiment of the present invention provides a computing device, including:
[0041] Memory and processor;
[0042] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the one or more programs are executed by the one or more processors, the one or more processors implement the text data processing method for a gas turbine power plant as described in any embodiment of the present invention.
[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the text data processing method for a gas turbine power plant.
[0044] Beneficial effects of the present invention: The present invention uses the engineering management text data of the gas turbine power plant construction period as a unified training set, sets parameters to use the training set to pre-train a language model on complex data, and after the pre-model training is completed, uses actual demand as a test machine to test the fusion model to obtain test results, and uses the fusion model to perform data analysis on the gas turbine power plant engineering management data when the test results of the fusion model meet the preset conditions. After the present invention uses natural language processing technology to vectorize the engineering management data of the gas turbine power plant, it can then use the fusion pre-training model to quickly provide solutions to engineering management problems, analyze and interpret real-time updated engineering data, further improve the maintenance and utilization efficiency of gas turbine power plant engineering management data, and provide effective support for engineering management. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0046] Figure 1 It is an overall flow chart of the text data processing method of a gas turbine power plant described in the first embodiment of the present invention. DETAILED DESCRIPTION
[0047] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0050] Example 1
[0051] Reference Figure 1 , which is the first embodiment of the present invention, and provides a text data processing method for a gas turbine power plant, comprising:
[0052] S1: Obtain and preprocess the engineering management text information of the gas turbine power plant, and generate the initial engineering management data set through data augmentation technology;
[0053] S2: Use open source datasets to pre-train a group of language models at the same time, and use the initial engineering management dataset to adjust the model parameters of the language model;
[0054] S3: Connect the models after parameter adjustment to obtain a hybrid model;
[0055] S4: Data analysis of the initial engineering management dataset using a hybrid model.
[0056] In the embodiment of the present application, the acquisition and preprocessing of the gas turbine power plant engineering management text information in the above step S1 includes:
[0057] Input the interactive texts of the construction units generated during the construction of completed gas turbine power plants and gas turbine power plants under construction, including project notifications, progress feedback documents; including project archive information, and engineering information of monitoring and analysis reports;
[0058] The engineering management text information of the gas turbine power plant is converted into tabular data in the format of "archival text classification; all file contents; manual evaluation of files" and the text is encoded using the manually annotated thermal power engineering professional vocabulary quasi-partition word vectors, and the data augmentation method is added as the initial engineering management data set D1;
[0059] In the embodiment of the present application, the data augmentation method used is to randomly mask or adjust the text sequence;
[0060] In another possible implementation, the data augmentation method used can also be data mixing or word replacement. Data mixing can be mixing text information from different sources in a certain proportion to increase the diversity and complexity of the data, thereby improving the robustness of the model. Word replacement is to randomly replace certain words in the text without changing the semantics, so as to simulate the input changes that may occur in the real world. These data augmentation methods help to improve the model's understanding and generalization ability of different text representations, and further optimize the performance of the model.
[0061] In the embodiment of the present application, the above step S2 uses an open source dataset to pre-train a group of language models simultaneously, including:
[0062] A set of language models includes traditional convolutional neural network language model, neural network language model with self-attention mechanism, and time recurrent neural network model. The initial engineering management data set is used to adjust the language model parameters in the form of LORA to obtain models A1, A2, and A3, whose output prediction probabilities are PA1, PA2, and PA3 respectively.
[0063] Introduce a low-rank matrix W between each layer of the model lora , adjust the weight of the model to reduce the consumption of computing resources; let the original weight matrix be W, and the introduced weight matrix be W′, then:
[0064] W′=W+W lora
[0065] Among them, W lora is a low-rank matrix;
[0066] The output prediction probability of each model is determined by the cross-validation method to ensure the generalization ability of the model on different data sets; the specific formula is:
[0067] PA i =Softmax(f i (X; W i ′))
[0068] Among them, PA i For Model Ai The output prediction probability of , i is the model number (1,2,3), f i Represents model A i The prediction function, X is the input data, W i ′ is the weight matrix after the low-rank matrix is introduced.
[0069] It should be noted that LORA (Low-Rank Adaptation) is an efficient model parameter adjustment method, which is usually used to fine-tune pre-trained language models (such as GPT, BERT, etc.). It adjusts the weights of the model by introducing low-rank matrices between the internal layers of the model, thereby achieving adaptation to specific tasks without changing the original parameters of the model. The advantage of the LORA method is that it requires fewer computing resources and can quickly adjust the model with a small amount of data to make it better adapted to new tasks or data sets.
[0070] In another possible implementation, random search can also be used to adjust parameters:
[0071] Random search randomly selects a combination of hyperparameters for training. Compared with grid search, it is more likely to find a parameter combination with better performance in the same number of trainings.
[0072] It is suitable for optimizing multiple hyperparameters simultaneously, with simple programming and high efficiency.
[0073] In another possible implementation, Bayesian Optimization can also be used to adjust parameters:
[0074] Bayesian optimization is a surrogate model-based approach that guides subsequent training based on the results of previous training.
[0075] Choosing parameters by predicting the uncertainty of the function works well in some specific cases, but may not be suitable for all scenarios.
[0076] In the embodiment of the present application, the open source dataset used may be OpenAssistant ConversationsDataset (OASST1)
[0077] In another possible implementation, the open source dataset used may also be Hugging FaceDatasets:
[0078] WikiText: A large dataset of Wikipedia text suitable for language modeling tasks.
[0079] GLUE: General Language Understanding Evaluation dataset, including multiple natural language understanding tasks.
[0080] SQuAD: The Stanford Question Answering Dataset, designed for machine reading comprehension tasks.
[0081] In another possible implementation, the open source dataset used may also be Google Research Datasets:
[0082] C4 (Colossal Clean Crawled Corpus): A huge, cleaned web text dataset suitable for pre-training language models.
[0083] TensorFlow Datasets: Provides a variety of datasets, such as MNLI (Natural Language Inference), SNLI (Sentence Entailment), etc.
[0084] In another possible implementation, the open source dataset used may also be TimeseriesBenchmarking:
[0085] UCR Time Series Archive: Contains a dataset for multiple time series classification tasks, suitable for studying time-recurrent neural network models.
[0086] In the embodiment of the present application, the step S3 of splicing the adjusted models includes:
[0087] After normalizing the output layers of the neural network training A1, A2, and A3, adaptive parameters α1, α2, and α3 are introduced, the model output layers are spliced, and the mixed output is set to P = (α1*PA1+α2*PA2+α3*PA3) for model splicing.
[0088] Furthermore, in order to reduce the training time, the training results of A1, A2, and A3 are used, and the average α=0.3 is taken as the initial parameter. The pre-training process of the open source dataset and the training set A1 is re-executed. After multiple rounds of training, the output of the hybrid model A4 can fit itself based on the adaptive parameters to obtain better convergence results than a single neural network.
[0089] In the embodiment of the present application, the above step S4 uses the hybrid model to perform data analysis on the initial engineering management data set, including:
[0090] The text information of the initial engineering management dataset is rewritten as "archival text classification; file requirement information extracted from file content; file requirement processing results extracted from file content; manual evaluation of files". The rewritten text information is converted into a structured engineering management dataset using a professional vocabulary library and encoding rules. The encoding steps are: encode the classification results using One-Hot encoding or TF-IDF encoding; encode the file requirement information using a sequence labeling method; and binary encode the file requirement processing results and manual evaluation results.
[0091] For example, in the usage phase, when the user inputs "generate a list of engineering progress measures and engineering plans under certain conditions", the hybrid language model will retrieve the corresponding data material from the engineering management data set, parse it into "a list of engineering progress measures and engineering plans under multiple similar conditions", and provide a "list of engineering progress measures and engineering plans" synthesized after hybrid calculation, providing users with high-quality engineering management suggestions and engineering data analysis results.
[0092] The above is a schematic scheme of the text data processing method for a gas turbine power plant of this embodiment. It should be noted that the technical scheme of the text data processing system for a gas turbine power plant and the technical scheme of the text data processing method for a gas turbine power plant are of the same concept, and the details not described in detail in the technical scheme of the text data processing system for a gas turbine power plant in this embodiment can be referred to the description of the technical scheme of the text data processing method for a gas turbine power plant.
[0093] The text data processing system of a gas turbine power plant in this embodiment includes:
[0094] The initial data set acquisition module is used to acquire and pre-process the text information of the gas turbine power plant engineering management, and generate the initial engineering management data set through data augmentation technology;
[0095] The parameter tuning module is used to pre-train a group of language models simultaneously using open source datasets, and to tune the model parameters of the language models using the initial engineering management dataset;
[0096] The hybrid model building module is used to splice the models after parameter adjustment to obtain a hybrid model;
[0097] A data analysis module is used to perform data analysis on the initial engineering management dataset using a hybrid model.
[0098] This embodiment also provides a computing device, which is applicable to the text data processing method of a gas turbine power plant, and includes:
[0099] A memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the text data processing method of a gas turbine power plant proposed in the above embodiment.
[0100] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the text data processing method for a gas turbine power plant proposed in the above embodiment is implemented.
[0101] The storage medium proposed in this embodiment and the text data processing method for a gas turbine power plant proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
[0102] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A text data processing method for a gas turbine power plant, characterized in that: include: Obtain and preprocess the engineering management text information of the gas turbine power plant, and generate the initial engineering management data set through data augmentation technology; Use open source datasets to pre-train a group of language models at the same time, and use the initial engineering management dataset to adjust the model parameters of the language model; The models after parameter adjustment are spliced to obtain a hybrid model; Data analysis was performed on the initial engineering management dataset using a hybrid model.
2. The text data processing method for a gas turbine power plant according to claim 1, characterized in that: The obtaining of the gas turbine power plant engineering management text information includes: Enter the interactive texts of the construction units generated during the construction of completed gas turbine power plants and gas turbine power plants under construction, including project notifications and progress feedback documents; including project archive information and engineering information of monitoring and analysis reports.
3. The text data processing method for a gas turbine power plant according to claim 2, characterized in that: The preprocessing to generate the initial engineering management data set by data augmentation technology includes: The engineering management text information of the gas turbine power plant is converted into tabular data according to a pre-set format, the text is encoded using professional vocabulary, and data augmentation is added as the initial engineering management data set.
4. The text data processing method for a gas turbine power plant according to claim 3, characterized in that: The language model includes a traditional convolutional neural network language model, a neural network language model with a self-attention mechanism, and a time-recurrent neural network model.
5. The text data processing method for a gas turbine power plant according to claim 4, characterized in that: The use of open source datasets to simultaneously pre-train a set of language models includes: The initial engineering management data set is used to adjust the language model in the form of LORA to obtain models A1, A2, and A3, whose output prediction probabilities are PA1, PA2, and PA3 respectively; Introduce a low-rank matrix W between each layer of the model lora , adjust the weight of the model to reduce the consumption of computing resources; let the original weight matrix be W, and the introduced weight matrix be W′, then: W′=W+W lora Among them, W lora is a low-rank matrix; The output prediction probability of each model is determined by the cross-validation method to ensure the generalization ability of the model on different data sets; the specific formula is: NETWORK i =Softmax(f i (X:W i ′)) Among them, PA i For Model A i The output prediction probability of , i is the model number (1,2,3), f i Represents model A i The prediction function, X is the input data, W i ′ is the weight matrix after the low-rank matrix is introduced.
6. The text data processing method for a gas turbine power plant according to claim 5, characterized in that: The splicing of the adjusted models includes: After normalizing the output layers of the neural network training A1, A2, and A3, adaptive parameters α1, α2, and α3 are introduced, the model output layers are spliced, and the mixed output is set to P = (α1*PA1+α2*PA2+α3*PA3) for model splicing.
7. The text data processing method for a gas turbine power plant according to claim 6, characterized in that: The data analysis of the initial engineering management data set using the hybrid model includes: The text information of the initial engineering management dataset is rewritten as "archival text classification; file requirement information extracted from file content; file requirement processing results extracted from file content; manual evaluation of files". The rewritten text information is converted into a structured engineering management dataset using a professional vocabulary library and encoding rules. The encoding steps are: encoding the classification results using One-Hot encoding or TF-IDF encoding; encoding the file requirement information using a sequence labeling method; and binary encoding the file requirement processing results and manual evaluation results.
8. A system using the text data processing method for a gas turbine power plant according to any one of claims 1 to 7, characterized in that: include: The initial data set acquisition module is used to acquire and pre-process the text information of the gas turbine power plant engineering management, and generate the initial engineering management data set through data augmentation technology; The parameter tuning module is used to pre-train a group of language models simultaneously using open source datasets, and to tune the model parameters of the language models using the initial engineering management dataset; The hybrid model building module is used to splice the models after parameter adjustment to obtain a hybrid model; A data analysis module is used to perform data analysis on the initial engineering management dataset using a hybrid model.
9. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the text data processing method for a gas turbine power plant described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the text data processing method for a gas turbine power plant according to any one of claims 1 to 7.