Data set construction method for post-training of hydroelectric construction vertical class large language model
By using box plot algorithms to identify and process outliers, the problems of long-tail distribution and noise pollution in the training of large language models for hydropower construction were solved, a high-quality dataset was constructed, and the applicability and generation capability of the model in the field of hydropower construction were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- POWERCHINA BEIJING ENG CORP
- Filing Date
- 2026-03-31
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies for training large-scale language models in the field of hydropower construction suffer from problems such as long-tail distribution, noise pollution, and unbalanced data distribution. They have not designed long-tail scenario sample processing schemes for the field of hydropower construction, and lack effective methods for dataset distribution control and noise removal.
Outliers are identified and classified using a box plot algorithm. Data augmentation can enhance outliers and remove those that cannot be augmented. An optimized dataset is constructed, including calculating quartiles and interquartile ranges to identify outliers. Outlier types are then determined and processed based on business criticality and reasonableness.
It effectively reduces computing power consumption, quickly optimizes datasets, improves the model's adaptability and generation capabilities in hydropower construction scenarios, and enhances the model's accuracy and efficiency.
Smart Images
Figure CN122020183A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of vertical large language model training technology, specifically involving a method for constructing a dataset for post-training of a hydropower construction vertical large language model. Background Technology
[0002] With the rapid development of large language model technology, vertical large language models are increasingly being developed and implemented in the hydropower construction sector due to their ability to accurately adapt to the application needs of various industry-specific business scenarios. Post-training, as a crucial step in optimizing the professional adaptability of vertical large language models and improving their reasoning and generation capabilities in specific industry scenarios, directly determines the final application effect of the model and has become a core influencing factor in the development and implementation of vertical large language models in the hydropower construction field. Currently, the conventional technical approach in the domestic and international hydropower construction sector for post-training vertical large language models is to directly use actual scenario data collected from business operations as the post-training dataset, without addressing the core needs of model training by performing targeted distribution adjustments and noise screening on the dataset.
[0003] Relevant patent documents retrieved:
[0004] This document, published in China (CN120633797A) on September 12, 2025, discloses a training method for a large-scale vertical model in the field of building regulations. It constructs a dynamic legal knowledge base and a structured knowledge graph, and builds a dynamic data supply system with multiple types of building databases and an incremental knowledge pool to provide training data for the model and generate a large-scale model in the vertical field of building regulations.
[0005] The prior art represented by the aforementioned documents has at least the following unresolved technical problems or defects: (1) The problem of long-tail distribution and noise pollution in the training dataset of large vertical models has not been solved. The relevant evidence is that this paper only constructs a knowledge graph and an incremental data supply system, without any technical design for dataset distribution regulation and noise sample removal.
[0006] (2) No non-parametric detection method that does not depend on the normal distribution of data is provided. The relevant evidence is that the literature does not mention non-parametric algorithms such as box plots and quartiles, which cannot adapt to the actual characteristics of irregular distribution of vertical data.
[0007] (3) No long-tail scenario sample processing solution was designed for the field of hydropower construction. The relevant evidence is that the solution is only applicable to the field of building regulations and does not involve sample processing for long-tail scenarios such as rare faults and new technologies in hydropower construction.
[0008] In view of this, the present invention is hereby proposed. Summary of the Invention
[0009] To address the aforementioned technical problems in existing technologies, this invention provides a method for constructing a dataset for post-training a large language model for hydropower construction, thereby resolving issues such as the small number of long-tail samples, mixed noisy data, and unbalanced data distribution in large vertical model datasets.
[0010] To achieve the above objectives, the technical solution of the present invention is as follows: A method for constructing a dataset for training a large language model in the hydropower construction vertical category includes: S1. Determine the original dataset for training the vertical category large language model, and extract the sample frequencies of each category from the original dataset to form a sample frequency set. ; S2. Calculate the correlation statistics of the sample frequency set based on the box plot algorithm, and identify and separate outliers in the sample frequency set based on the statistics; S3. Classify and determine the outliers to obtain augmentable outliers and non-augmentable outliers; S4. Perform data augmentation on the sample categories corresponding to the augmentable outliers, and remove the sample categories corresponding to the non-augmentable outliers. S5. After processing, we obtain the optimized vertical category large language model and then the training dataset.
[0011] Further, in step S2, the relevant statistics include: The first quartile and the third quartile, wherein the first quartile is the 25th percentile of the sample frequency set, and the third quartile is the 75th percentile of the sample frequency set.
[0012] Furthermore, in step S2, the specific process of identifying outliers based on the aforementioned statistics is as follows: First, calculate the interquartile range between the first and third quartiles. The specific formula is as follows:
[0013] in, Interquartile range, It is the first quartile. It is the third quartile; Then satisfy or of It was identified as an outlier.
[0014] Furthermore, in step S3, the specific method for classifying and determining the outliers is as follows: If the sample category corresponding to the outlier is a business-critical long-tail scenario, and if a new sample that is semantically reasonable and technically compliant can be generated, it is determined to be an augmentable outlier; otherwise, it is determined to be a non-augmentable outlier.
[0015] Furthermore, in step S4, the requirement for performing data augmentation processing on the sample categories corresponding to the augmentable outliers is as follows: Increase the sample frequency of each sample category to The above yields the enhanced sample frequency. The specific expression is: .
[0016] Furthermore, the formula for calculating the enhanced sample frequency is as follows:
[0017] in, This represents the number of newly added samples; when hour, The minimum value is The specific calculation formula is as follows:
[0018] in, This is for rounding up.
[0019] Furthermore, in step S3, the sample category corresponding to the non-augmentable outlier is the sample category corresponding to the false category generated by data noise, acquisition error, non-target domain content, or log parsing error.
[0020] Furthermore, in step S4, the specific method for removing the sample categories corresponding to the non-augmentable outliers is as follows: Remove the sample categories corresponding to the non-augmentable outliers in the original dataset D. To remove from the dataset, the specific formula is:
[0021] in, To discard sample categories The subsequent dataset.
[0022] Furthermore, the vertical language model is a large language model in the field of hydropower construction business, and the key long-tail scenarios of the business include sample scenarios related to rare faults, new technology applications, or niche equipment in hydropower construction.
[0023] Furthermore, the sample frequency is the number of valid samples in each sample category in the original dataset, and the valid samples are sample data that match the training objective of the vertical large language model.
[0024] The beneficial effects of this invention are as follows: This invention uses the box plot algorithm to construct a large language model dataset, which has strong flexibility and does not rely on the assumption that the data in the dataset is normally distributed. At the same time, the box plot algorithm has low computational complexity and does not involve exponential calculations during the algorithm execution process, which can quickly achieve the optimization processing of the dataset and effectively reduce the consumption of computing power. Attached Figure Description
[0025] Figure 1 This is a flowchart illustrating the method for constructing a dataset for post-training a large language model for hydropower construction, as provided in an embodiment of the present invention. Detailed Implementation
[0026] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0027] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.
[0028] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.
[0029] Example See Figure 1 , Figure 1 This is a flowchart illustrating a method for constructing a dataset for post-training a large language model for hydropower construction, as proposed in this invention. The method employs a box plot algorithm to regulate dataset distribution, enhance long-tailed samples, and remove noisy samples. The box plot algorithm is a non-parametric outlier detection method based on quartiles, and does not rely on the assumption that the data follows a normal distribution. Specific steps may include: S1. Determine the original dataset for training the vertical category large language model, and extract the sample frequencies of each category from the original dataset to form a sample frequency set. ;in For sample category number, Indicates the first Number of samples per category; Among them, the vertical large language model is a large language model in the field of hydropower construction business. The key long-tail scenarios of the business include sample scenarios related to rare faults, new technology applications or niche equipment in hydropower construction. Rare faults specifically include large-scale energy storage fires, thermal runaway chain reactions, etc. New technology applications specifically include hydrogen energy storage, flow batteries, grid-type inverters, etc. Niche equipment specifically includes domestic flywheels, off-grid microgrid controllers, etc. The sample frequency is the number of effective samples in each sample category in the original dataset. The effective samples are sample data that match the training target of the vertical large language model.
[0030] S2. Calculate the correlation statistics of the sample frequency set based on the box plot algorithm, and identify and separate outliers in the sample frequency set based on the statistics; The relevant statistics include: the first quartile. and the third and fourth quartiles The first quartile The 25th percentile of the frequency set of the samples, i.e. The third quartile The 75th percentile of the frequency set of the samples, i.e. .
[0031] Furthermore, the specific process for identifying outliers based on the aforementioned statistics is as follows: First calculate the first quartile With the third quartile The interquartile range is calculated using the following formula:
[0032] in, Interquartile range, It is the first quartile. It is the third quartile; Then satisfy or of The outliers are identified and separated from the sample frequency set.
[0033] S3. Classify and determine the outliers to obtain augmentable outliers and non-augmentable outliers; The specific method for classifying and determining the outliers is as follows: If the sample category corresponding to the outlier is a business-critical long-tail scenario, and if a new sample that is semantically reasonable and technically compliant can be generated, it is determined to be an augmentable outlier; otherwise, it is determined to be a non-augmentable outlier.
[0034] The sample categories corresponding to outliers that cannot be augmented are those corresponding to spurious categories generated by data noise, collection errors, non-target domain content, or log parsing errors.
[0035] In addition, the requirements for performing data augmentation processing on the sample categories corresponding to the outliers that can be augmented are as follows: Increase the sample frequency of each sample category to The above yields the enhanced sample frequency. The specific expression is: .
[0036] S4. Perform data augmentation on the sample categories corresponding to the augmentable outliers, and remove the sample categories corresponding to the non-augmentable outliers. Specifically, for the sample category that can be augmented, the formula for calculating the frequency of the augmented sample is:
[0037] in, The number of new samples added for this sample category; when hour, The minimum value is The specific calculation formula is as follows:
[0038] in, This is for rounding up.
[0039] Furthermore, the specific method for removing the sample categories corresponding to the non-enhancing outliers is as follows: The sample categories corresponding to the non-augmentable outliers in the original dataset D are defined. To remove from the dataset, the specific formula is:
[0040] in, To discard sample categories The subsequent dataset.
[0041] S5. After processing, we obtain the optimized vertical category large language model and then the training dataset.
[0042] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for constructing a dataset for post-training a large language model in the hydropower construction vertical category, characterized in that, include: S1. Determine the original dataset for training the vertical category large language model, and extract the sample frequencies of each category from the original dataset to form a sample frequency set. ; S2. Calculate the relevant statistics of the sample frequency set based on the box plot algorithm, and identify and separate outliers in the sample frequency set based on the statistics; S3. Classify and determine the outliers to obtain augmentable outliers and non-augmentable outliers; S4. Perform data augmentation on the sample categories corresponding to the augmentable outliers, and remove the sample categories corresponding to the non-augmentable outliers. S5. After processing, we obtain the optimized vertical category large language model and then the training dataset.
2. The method for constructing a dataset for training a large language model for hydropower construction as described in claim 1, characterized in that, In step S2, the relevant statistics include: The first quartile and the third quartile, wherein the first quartile is the 25th percentile of the sample frequency set, and the third quartile is the 75th percentile of the sample frequency set.
3. The method for constructing a dataset for training a large language model for hydropower construction according to claim 2, characterized in that, In step S2, the specific process of identifying outliers based on the statistics is as follows: First, calculate the interquartile range between the first and third quartiles. The specific formula is as follows: in, Interquartile range, It is the first quartile. It is the third quartile; Then satisfy or of It was identified as an outlier.
4. The method for constructing a dataset for training a large language model for hydropower construction as described in claim 1, characterized in that, In step S3, the specific method for classifying and determining the outliers is as follows: If the sample category corresponding to the outlier is a business-critical long-tail scenario, and if a new sample that is semantically reasonable and technically compliant can be generated, it is determined to be an augmentable outlier; otherwise, it is determined to be a non-augmentable outlier.
5. The method for constructing a dataset for training a large language model for hydropower construction according to claim 4, characterized in that, In step S4, the requirements for performing data augmentation processing on the sample categories corresponding to the augmentable outliers are as follows: Increase the sample frequency of each sample category to The above yields the enhanced sample frequency. The specific expression is: .
6. The method for constructing a dataset for training a large language model for hydropower construction according to claim 5, characterized in that, The formula for calculating the frequency of the enhanced samples is as follows: in, This represents the number of newly added samples; when hour, The minimum value is The specific calculation formula is as follows: in, This is for rounding up.
7. The method for constructing a dataset for training a large language model for hydropower construction as described in claim 1, characterized in that, In step S3, the sample category corresponding to the non-augmentable outlier is the sample category corresponding to the spurious category generated by data noise, acquisition error, non-target domain content, or log parsing error.
8. The method for constructing a dataset for training a large language model for hydropower construction according to claim 7, characterized in that, In step S4, the specific method for removing the sample categories corresponding to the non-augmentable outliers is as follows: Remove the sample categories corresponding to the non-augmentable outliers from the original dataset D. To remove from the dataset, the specific formula is: in, To discard sample categories The subsequent dataset.
9. The method for constructing a dataset for training a large language model for hydropower construction according to claim 1, characterized in that, The vertical language model is a large language model in the field of hydropower construction business. The key long-tail scenarios of the business include sample scenarios related to rare faults, new technology applications, or niche equipment in hydropower construction.
10. The method for constructing a dataset for training a large language model for hydropower construction according to claim 1, characterized in that, The sample frequency refers to the number of valid samples in each sample category of the original dataset, and the valid samples are sample data that match the training objective of the vertical large language model.