A large model fine-tuning method, system, device and medium for power operation and maintenance

Abstract

The present invention provides a large model fine-tuning method, system, device and medium for electric power operation and maintenance, the method is to use each operation and maintenance data sample in the acquired power operation and maintenance data set to be processed as a sample of a problem to be optimized, extract data features of the problem sample to be optimized according to the operation and maintenance problem type identified by a preset keyword library and a first preset large language model to obtain a problem feature text, expand it according to a preset data type knowledge base and summarize it by the same type to obtain a problem fine-tuning training sample set, and divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset to fine-tune the second preset large language model to obtain an operation and maintenance problem large model, and merge all the operation and maintenance problem large models to obtain a power operation and maintenance vertical domain large model. The present invention can effectively improve the information utilization efficiency, adaptability and expression ability of model fine-tuning for complex power operation and maintenance data, thereby improving the efficiency and accuracy of fine-tuning training.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power operation and maintenance, and in particular to a large model fine-tuning method, system, equipment and medium for electric power operation and maintenance. Background Art

[0002] With the continuous development of the power operation and maintenance field, the demand for intelligent power equipment has increased rapidly, and power operation and maintenance data mining methods based on technologies such as graph neural networks and long short-term memory networks have emerged one after another. However, since the power operation and maintenance business involves various complex contents such as fault diagnosis, equipment maintenance, energy management, and power grid dispatching, it contains a large amount of unstructured sparse data and has the characteristics of complex data content, high noise, poor quality, and serious redundancy. As a result, the above data mining methods are not only inefficient in information processing when applied, but also difficult to achieve collaborative interaction in operation and maintenance. Based on this, large models are introduced into the field of power operation and maintenance because they can sort out complex unstructured data and achieve human-computer interaction and coordination.

[0003] Existing researchers only focus on the knowledge question and answer of power operation and maintenance based on the large model retrieval enhancement system, and introduce a long text embedding model to vectorize the basic power knowledge. However, although this technology can cope with the challenges brought by the large model illusion, it still has serious defects: 1) The field of power operation and maintenance is extremely professional, and the general large model lacks an accurate understanding of the professional knowledge of power information operation and maintenance, and cannot meet professional requirements in converting user instructions and generating intelligent answers; 2) The general large model lacks an alignment paradigm for the professional and personalized needs in the field of power operation and maintenance, and there are deviations in problem decomposition and logical chain transformation, which makes it difficult to land in actual scenarios. That is, the actual use of the existing large model in the field of power operation and maintenance is difficult to ensure information processing efficiency and processing quality while also meeting the requirements of professional knowledge interaction capabilities. Therefore, it is urgent to provide an efficient and reliable expert-level large model fine-tuning method suitable for power operation and maintenance big data analysis and processing from the perspective of data quality optimization. Summary of the invention

[0004] The purpose of the present invention is to provide a large model fine-tuning method for power operation and maintenance. The problem types of power operation and maintenance data are classified and sorted by category prompt words designed based on the instruction framework of professional knowledge of power information operation and maintenance, and the problem feature text corresponding to each power operation data is enhanced based on the addition of alignment data based on the knowledge base samples to obtain the corresponding problem fine-tuning training sample set, and then the QLoRA technology is introduced to perform large model fine-tuning training to construct a large model of power operation and maintenance vertical domain, which can accurately process the sparsity and unstructured attributes of the power operation and maintenance data, effectively improve the utilization efficiency of the effective information of the samples in the fine-tuning, and effectively improve the adaptability and expression ability of the model fine-tuning to the power operation and maintenance data, thereby improving the efficiency and accuracy of the fine-tuning training, and also taking into account the interactive ability of the large model in the professional field of power operation and maintenance, and providing strong support for multi-scenario applications in the field of power operation and maintenance.

[0005] In order to achieve the above objectives, it is necessary to provide a large model fine-tuning method, system, equipment and medium for power operation and maintenance in response to the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a large model fine-tuning method for power operation and maintenance, the method comprising the following steps:

[0007] Acquire a power operation and maintenance data set to be processed, and use each operation and maintenance data sample in the power operation and maintenance data set as a sample of the problem to be optimized; the power operation and maintenance data set to be processed includes a fault diagnosis sample and a load prediction sample;

[0008] Analyze each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type;

[0009] According to the operation and maintenance problem type and the first preset large language model, extract preset data features from each problem sample to be optimized to obtain a corresponding problem feature text, and expand the problem feature text according to a corresponding preset data type knowledge base to generate a corresponding problem fine-tuning training sample;

[0010] Aggregate problem fine-tuning training samples of the same operation and maintenance problem type to obtain a corresponding problem fine-tuning training sample set, and divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset;

[0011] According to the problem instruction fine-tuning sample subset and the problem increment fine-tuning sample subset, the second preset large language model is fine-tuned and trained based on the quantized low-rank adaptive fine-tuning technology to obtain the corresponding operation and maintenance problem large model, and all operation and maintenance problem large models are merged to obtain the power operation and maintenance vertical domain large model.

[0012] Furthermore, the preset keyword library includes a fault diagnosis keyword library and a load prediction keyword library; the step of analyzing each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type includes:

[0013] Performing vectorization calculation on the problem sample to be optimized by using a preset semantic vector model to obtain a corresponding vectorization processing result;

[0014] Performing vectorization processing on each preset keyword library through the preset semantic vector model to obtain a corresponding vectorized keyword library;

[0015] Calculating the cosine similarity between each keyword in each vectorized keyword library and the vectorization processing result to obtain a keyword similarity list corresponding to each vectorized keyword library;

[0016] Obtain the keyword corresponding to the maximum similarity value in the keyword similarity list of each vectorized keyword library as a candidate user question keyword;

[0017] The keyword with the largest similarity value among all candidate user question keywords is used as the user question keyword;

[0018] The operation and maintenance problem type is obtained according to the type of the preset keyword library corresponding to the user problem keyword.

[0019] Furthermore, when the operation and maintenance problem type is fault diagnosis, the problem feature text includes the fault type, fault feature and fault location; the step of expanding the problem feature text according to the corresponding preset data type knowledge base to generate the corresponding problem fine-tuning training sample includes:

[0020] According to the multi-dimensional similarity comprehensive evaluation method, the comprehensive similarity of the problem feature text and each case text in the preset data type knowledge base is calculated respectively to obtain the corresponding text comprehensive similarity;

[0021] When the comprehensive text similarity between the question feature text and the case text is greater than a preset similarity threshold, the case text is appended to the end of the question feature text to obtain a candidate question fine-tuning training sample;

[0022] The candidate question fine-tuning training sample is input into the first preset large language model for text comprehensibility and logical accuracy detection, and when the detection is qualified, the candidate question fine-tuning training sample is used as the question fine-tuning training sample.

[0023] Furthermore, when the operation and maintenance problem type is load prediction, the preset data features include prediction conditions and reference model knowledge; the reference model knowledge includes model type, data source, feature variables, target variables, prediction method, model evaluation index, prediction assumptions and external factors; the step of expanding the problem feature text according to the corresponding preset data type knowledge base to generate the corresponding problem fine-tuning training sample includes:

[0024] Preprocessing the prediction conditions and reference model knowledge in the question feature text to obtain corresponding prediction condition word lists and model knowledge word lists;

[0025] Obtaining the word frequency similarity and the Jaccard similarity coefficient of the prediction condition word list and the model knowledge word list, and performing weighted summation of the word frequency similarity and the Jaccard similarity coefficient to obtain the corresponding prediction condition influencing factor;

[0026] When the prediction condition influencing factor is greater than a preset credible threshold, a comprehensive similarity calculation is performed on the problem feature text and each case text in the preset data type knowledge base according to a multi-dimensional similarity comprehensive evaluation method to obtain a corresponding text comprehensive similarity;

[0027] When the comprehensive text similarity between the question feature text and the case text is greater than a preset similarity threshold, the case text is appended to the end of the question feature text to obtain a candidate question fine-tuning training sample;

[0028] The candidate question fine-tuning training sample is input into the first preset large language model for text comprehensibility and logical accuracy detection, and when the detection is qualified, the candidate question fine-tuning training sample is used as the question fine-tuning training sample.

[0029] Furthermore, the multi-dimensional similarity comprehensive evaluation method is to calculate the Jaccard similarity coefficient, the Dyss similarity coefficient and the N-tuple similarity value respectively, then calculate the similarity coefficient average value of the Jaccard similarity coefficient and the Dyss similarity coefficient, and perform weighted average of the similarity coefficient average value and the N-tuple similarity value to obtain the comprehensive similarity.

[0030] Furthermore, the step of dividing the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset comprises:

[0031] Respectively obtaining the text type of each question fine-tuning training sample in the question fine-tuning training sample set;

[0032] Classifying the question fine-tuning training samples whose text type is factual statement text as question increment fine-tuning samples, and classifying the question fine-tuning training samples whose text type is dialogue text as question instruction fine-tuning samples;

[0033] All the problem increment fine-tuning samples are aggregated to obtain the problem increment fine-tuning sample subset, and all the problem instruction fine-tuning samples are aggregated to obtain the problem instruction fine-tuning sample subset.

[0034] Furthermore, the to-be-processed power operation and maintenance data set also includes a general task sample; the preset keyword library also includes a general task keyword library; the method further includes:

[0035] When the operation and maintenance problem type of the problem sample to be optimized is the general task, the problem sample to be optimized is directly added to the corresponding problem fine-tuning training sample set.

[0036] In a second aspect, an embodiment of the present invention provides a large model fine-tuning system for power operation and maintenance, the system comprising:

[0037] A data acquisition module is used to obtain a power operation and maintenance data set to be processed, and use each operation and maintenance data sample in the power operation and maintenance data set as a sample of the problem to be optimized; the power operation and maintenance data set to be processed includes a fault diagnosis sample and a load prediction sample;

[0038] The type analysis module is used to analyze each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type;

[0039] A sample enhancement module is used to extract preset data features from each problem sample to be optimized according to the operation and maintenance problem type and the first preset large language model to obtain a corresponding problem feature text, and to expand the problem feature text according to a corresponding preset data type knowledge base to generate a corresponding problem fine-tuning training sample;

[0040] A data set partitioning module is used to aggregate problem fine-tuning training samples of the same operation and maintenance problem type to obtain a corresponding problem fine-tuning training sample set, and to divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset;

[0041] The model fine-tuning module is used to fine-tune the sample subset according to the problem instruction and the problem increment fine-tune the sample subset, fine-tune the second preset large language model based on the quantized low-rank adaptive fine-tuning technology, obtain the corresponding operation and maintenance problem large model, and merge all the operation and maintenance problem large models to obtain the power operation and maintenance vertical domain large model.

[0042] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0043] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0044] The present invention provides a large model fine-tuning method, system, computer equipment and storage medium for power operation and maintenance. The method is used to obtain a to-be-processed power operation and maintenance data set including fault diagnosis samples and load prediction samples, and each operation and maintenance data sample in the power operation and maintenance data set is used as a sample of a problem to be optimized. According to a preset keyword library, each problem sample to be optimized is analyzed to obtain a corresponding operation and maintenance problem type, and according to the operation and maintenance problem type and a first preset large language model, preset data features are extracted for each problem sample to be optimized to obtain a corresponding problem feature text, and after the problem feature text is expanded according to a corresponding preset data type knowledge base to generate a corresponding problem fine-tuning training sample, the problem fine-tuning training samples of the same operation and maintenance problem type are aggregated to obtain a corresponding problem fine-tuning training sample set, and the problem fine-tuning training sample set is divided into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset, and according to the problem instruction fine-tuning sample subset and the problem increment fine-tuning sample subset, the second preset large language model is fine-tuned and trained based on the quantized low-rank adaptive fine-tuning technology to obtain a corresponding operation and maintenance problem large model, and all operation and maintenance problem large models are merged to obtain a technical solution for the power operation and maintenance vertical domain large model. Compared with the existing technology, this large model fine-tuning method for power operation and maintenance can accurately process the sparsity and unstructured attributes of power operation and maintenance data, effectively improve the utilization efficiency of effective sample information in fine-tuning, and effectively improve the adaptability and expression ability of model fine-tuning to power operation and maintenance data, thereby improving the efficiency and accuracy of fine-tuning training, and taking into account the interactive capabilities of large models in the professional field of power operation and maintenance, providing strong support for multi-scenario applications in the field of power operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a large model fine-tuning method for power operation and maintenance in an embodiment of the present invention;

[0046] Figure 2 It is a schematic diagram of the process architecture of fine-tuning a large model for power operation and maintenance in an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of an application scenario of a large model fine-tuning method for power operation and maintenance in an embodiment of the present invention;

[0048] Figure 4 yes Figure 3 Schematic diagram of the Json format data structure of fault diagnosis data in the application scenario;

[0049] Figure 5is a structural schematic diagram of a large model fine-tuning system for power operation and maintenance in an embodiment of the present invention;

[0050] Figure 6 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In one embodiment, Figure 1 As shown, a large model fine-tuning method for power operation and maintenance is provided, comprising the following steps:

[0053] S11. Obtain a set of power operation and maintenance data to be processed, and use each operation and maintenance data sample in the power operation and maintenance data set as a sample of the problem to be optimized; the power operation and maintenance data set to be processed can be understood as power operation and maintenance user instructions or problem data collected in actual applications, including fault diagnosis samples and load prediction samples, and a corresponding database can be constructed based on the collected various types of power operation and maintenance data samples; among them, the fault diagnosis sample can be understood as a sample including information such as fault type, fault characteristics and fault location, and the load prediction sample can be understood as a sample including information such as prediction conditions and reference model knowledge, as shown in Table 1-2.

[0054] Table 1 Sample content examples of fault diagnosis training samples

[0055]

[0056] Table 2 Sample content examples of load prediction training samples

[0057]

[0058] Taking into account the characteristics of the actually collected power operation and maintenance data set to be processed, such as complex data content and including a large amount of unstructured sparse data, in order to facilitate the efficient and accurate understanding of power operation and maintenance professional knowledge during fine-tuning training of the large model, this embodiment preferably uses the collected power operation and maintenance data to fine-tune the large model. Before training, each operation and maintenance data sample in the power operation and maintenance data set to be processed is used as a sample of the problem to be optimized and optimized through a subsequent classification and sorting process.

[0059] S12. Analyze each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type; wherein, the preset keyword library can be understood as a prompt word database constructed in advance based on the representative keywords of various problem types according to the problem types that may be included in the power operation and maintenance data set; that is, the preset keyword library includes a fault diagnosis keyword library and a load prediction keyword library, wherein the fault diagnosis keyword library may include system failure, equipment failure, troubleshooting, error, crash, unable to start, shutdown, fault analysis, alarm and abnormality and other keywords; the load prediction keyword library may include load prediction, system load, CPU usage, memory occupancy, regression analysis, performance analysis, load trend, historical data, prediction conditions and prediction models and other keywords.

[0060] The operation and maintenance problem type in this embodiment can be understood as the most matching problem classification obtained by comparing the keywords in each problem sample to be optimized with the keywords in each preset keyword library; specifically, the step of analyzing each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type includes:

[0061] The problem sample to be optimized is vectorized by a preset semantic vector model to obtain a corresponding vectorization processing result; wherein the preset semantic vector model can adopt the bge-large-zh-v1.5 model, that is, the text of the problem sample to be optimized is vectorized by the bge-large-zh-v1.5 model. Map to a dimensional space, where is the dimension of the word vector, the default is 512, and the resulting high-dimensional vector is represented as As shown below:

[0062]

[0063] In the formula, is a 512-dimensional vector containing the textual semantic information of the problem sample to be optimized; each component Represents the different features of the problem samples to be optimized in the semantic space.

[0064] Each preset keyword library is vectorized by the preset semantic vector model to obtain a corresponding vectorized keyword library; wherein the vectorized keyword library can be understood as a database obtained by vectorizing each keyword in the preset keyword library by the bge-large-zh-v1.5 model, and storing the vectorized representations of all keywords in a data structure txt-base, which is expressed as:

[0065]

[0066] In the formula, Indicates the total number of keywords; For the corresponding Keywords The 512-dimensional vector representation is as follows:

[0067]

[0068] In the formula, Each component in the vector Indicates the characteristics of the keyword in the semantic space.

[0069] Calculate the cosine similarity between each keyword in each vectorized keyword library and the vectorized processing result to obtain a keyword similarity list corresponding to each vectorized keyword library; wherein, the keyword The cosine similarity of the vectorized processing result of the sample to be optimized is expressed as:

[0070]

[0071] In the formula,

[0072]

[0073]

[0074] in, express and The vector dot product of and They are and The norm of a vector.

[0075] The keyword corresponding to the maximum similarity value in the keyword similarity list of each vectorized keyword library is obtained as the candidate user question keyword; wherein the candidate user question keyword can be understood as the keyword with the highest similarity to the vectorization processing result of the optimization problem sample in the same vectorized keyword library.

[0076] The keyword with the largest similarity value among all candidate user question keywords is taken as the user question keyword; wherein the user question keyword can be understood as the keyword with the highest similarity to the vectorization processing result of the optimization problem sample in all vectorized keyword libraries.

[0077] The operation and maintenance problem type is obtained according to the type of the preset keyword library corresponding to the user problem keyword.

[0078] This embodiment uses preset problem type keywords for classification and identification of power operation and maintenance data samples to provide a reliable basis for subsequent targeted, efficient and accurate optimization processing based on different problem types and data characteristics of different problem types.

[0079] S13. According to the operation and maintenance problem type and the first preset large language model, the preset data features are extracted for each problem sample to be optimized to obtain the corresponding problem feature text, and the problem feature text is expanded according to the corresponding preset data type knowledge base to generate corresponding problem fine-tuning training samples; wherein, the first preset large language model can be selected according to actual application requirements, and this embodiment preferably adopts the Qwen2.5-14B-Instruct large model.

[0080] Since different types of operation and maintenance problems correspond to different key information, in order to ensure the reliability of fault diagnosis type data processing, this embodiment preferably sets that when the operation and maintenance problem type is fault diagnosis, the locally deployed Qwen2.5-14B-Instruct large model is used to extract the fault type, fault characteristics, fault location and other data in the corresponding problem sample to be optimized, and the extraction results are stored in a text data format, which is set as GZ-Text, that is, the required problem feature text is obtained. In order to ensure the reliability of subsequent fine-tuning training of the large language model based on fault diagnosis type data, this embodiment preferably searches for fault diagnosis case texts that are strongly related to the problem sample to be optimized from the preset data type knowledge base and performs data enhancement processing on them; specifically, the step of expanding the problem feature text according to the corresponding preset data type knowledge base to generate the corresponding problem fine-tuning training sample includes:

[0081] According to the multi-dimensional similarity comprehensive evaluation method, the comprehensive similarity of the problem feature text and each case text in the preset data type knowledge base is calculated respectively to obtain the corresponding text comprehensive similarity; wherein, the text comprehensive similarity can be understood as a similarity analysis result obtained by comprehensively evaluating the Jaccard similarity coefficient, the Dyss similarity coefficient and the N-tuple similarity value of the problem feature text and each case text in the preset data type knowledge base.

[0082] In practical applications, the application process of the multi-dimensional similarity comprehensive evaluation method can be understood as first calculating the Jaccard similarity coefficient, Dice similarity coefficient and N-gram similarity value of the two text objects respectively, then calculating the average similarity coefficient of the Jaccard similarity coefficient and the Dice similarity coefficient (Dice coefficient), and taking the weighted average of the similarity coefficient average and the N-gram similarity value to obtain the comprehensive similarity. The calculation process of the comprehensive similarity is as follows:

[0083] 1) Calculate the Jaccard similarity coefficient and the Dyss similarity coefficient and obtain the average coefficient:

[0084] Extract GZ-Text text and independent case text in the "fault history knowledge base" and convert them into word sets respectively and , use the following formula to calculate the Jaccard similarity coefficient:

[0085]

[0086] Where J represents the Jaccard similarity coefficient.

[0087] After obtaining the Jaccard similarity coefficient, the Dyss similarity coefficient is calculated using the same conditions:

[0088]

[0089] Where D represents the Dyss similarity coefficient.

[0090] Calculate the average coefficient G of the two coefficients:

[0091]

[0092] Where G is the average of Jaccard similarity coefficient and Dyss similarity coefficient.

[0093] 2) Calculate the N-tuple similarity value:

[0094] Choose an appropriate value of N (set it to ), split GZ-Text and case text into corresponding gather and gather , calculated using the following formula Similarity:

[0095]

[0096] in, Represents the N-tuple similarity value.

[0097] 3) and Weighted combination to form a comprehensive score :

[0098]

[0099] in, is the weight parameter, controlling and Similarity to text comprehensive similarity The impact ratio.

[0100] This multi-dimensional similarity comprehensive evaluation method is mainly used to find fault record samples and corresponding processing solutions that are highly similar to the problem samples to be optimized from the preset data type knowledge base, which are used as candidate content for subsequent data enhancement.

[0101] When the comprehensive text similarity between the problem feature text and the case text is greater than a preset similarity threshold, the case text is appended to the end of the problem feature text to obtain a candidate problem fine-tuning training sample; wherein, the preset similarity threshold can be determined according to actual application requirements and is not specifically limited here; when the comprehensive text similarity between the problem feature text and a case text is greater than the preset similarity threshold, it is considered that the case text has a high similarity with the problem sample to be optimized, and it can be added to the end of the problem feature text as a data enhancement candidate content and used as a candidate problem fine-tuning training sample.

[0102] The candidate question fine-tuning training sample is input into the first preset large language model for text comprehensibility and logical accuracy detection, and when the detection is qualified, the candidate question fine-tuning training sample is used as the question fine-tuning training sample; wherein, the text comprehensibility and logical accuracy detection can be understood as a detection and analysis process of the text of the candidate question fine-tuning training sample for language fluency, semantic relevance, content diversity, factual consistency, logical reasoning rationality and contextual consistency, etc., which can be implemented with reference to the existing relevant large language model technology, and will not be described in detail here. In practical applications, the candidate question fine-tuning training sample that passes the text comprehensibility and logical accuracy detection can be understood as qualified enhanced data processing and can be used for subsequent model fine-tuning training. If the detection fails, it is considered that the data enhancement is unreasonable and cannot be used as a subsequent training sample.

[0103] The multi-dimensional similarity comprehensive evaluation method designed in this embodiment based on the comprehensive analysis of Jaccard similarity, Dice coefficient and N-grams similarity uses Jaccard similarity to measure the overall coverage relationship of text content to obtain the global similarity between texts, highlighting the proportion of common information in the overall data, using Dice coefficient to capture the local strong correlation between texts, and using N-grams similarity to effectively capture the local order relationship and pattern similarity between texts with a lower degree of structure, thereby realizing multi-dimensional and multi-granular similarity analysis between texts, which can effectively capture the potential correlation between text contents, while combining the global similarity obtained based on the average value of Jaccard similarity coefficient and Dice similarity coefficient with the local similarity obtained based on N-grams similarity analysis, which can significantly improve the accuracy and adaptability of fault case matching, thereby ensuring the reliability of data enhancement processing. It should be noted that the comprehensive grading mechanism in the multi-dimensional similarity comprehensive evaluation method provided in this embodiment makes the similarity analysis results more interpretable, and can dynamically adjust the weights of each similarity for different application scenarios, which is suitable for rapid retrieval and precise positioning in large-scale knowledge bases.

[0104] At the same time, when the operation and maintenance problem type is load prediction, considering the characteristics of the load prediction data sample content shown in Table 2, it is preferred to set the preset data features to include prediction conditions and reference model knowledge, and the reference model knowledge includes model type, data source, feature variables, target variables, prediction methods, model evaluation indicators, prediction assumptions and external factors; the acquisition process of this type of problem feature text can be understood as extracting the prediction conditions and reference model knowledge and other data in the corresponding problem samples to be optimized through the locally deployed Qwen2.5-14B-Instruct large model, and storing the extraction results in text data format, setting it as FZ-Text, that is, obtaining the required problem feature text. In order to ensure the reliability of the subsequent fine-tuning training of the large language model based on the load prediction type data, this embodiment preferably searches for the load prediction case text that is strongly related to the problem sample to be optimized from the preset data type knowledge base and performs data enhancement processing on it. Specifically, the step of expanding the problem feature text according to the corresponding preset data type knowledge base to generate the corresponding problem fine-tuning training sample includes:

[0105] The prediction conditions and reference model knowledge in the problem feature text are preprocessed to obtain corresponding prediction condition word lists and model knowledge word lists; wherein the prediction conditions include the time information of the predicted load and other prediction conditions as shown in Table 2; the reference model knowledge includes the relevant information of the load equipment provided in the actual data as shown in Table 2; the corresponding prediction condition word list and model knowledge word list can be understood as two word lists obtained by removing stop words, punctuation marks and word segmentation processing from the text corresponding to the prediction conditions and reference model knowledge in the problem feature text. and The specific acquisition process can be referred to the relevant existing technology implementation, which will not be repeated here.

[0106] The word frequency similarity and Jaccard similarity coefficient of the prediction condition word list and the model knowledge word list are obtained, and the word frequency similarity and the Jaccard similarity coefficient are weighted and summed to obtain the corresponding prediction condition influencing factor; wherein the word frequency similarity can be understood as the word frequency vector obtained by statistically analyzing the occurrence frequency of each word in the prediction condition word list and the model knowledge word list, and then the co-occurrence calculated based on the two word frequency vectors is used to measure the matching degree of words in the two texts, that is, the overlap of the vocabulary between the "prediction condition" and the "reference model" (the sum of the number of occurrences of the intersection words of the two word lists), which is specifically expressed as:

[0107]

[0108] in, Indicates word frequency similarity; for each word in the intersection of the two lists of words. and Respectively represent words Frequency of terms in the “Prediction Condition” and “Reference Model Knowledge” word lists. It indicates the minimum value of word frequency, which is to prevent the word frequency from being too high, causing an imbalance in "co-occurrence".

[0109] The process of obtaining the Jaccard similarity coefficient in this embodiment is as follows:

[0110] The similarity between the "prediction condition word list" and the "reference model knowledge word list" is calculated using Jaccard similarity as follows:

[0111]

[0112] in, and The word sets representing "prediction conditions" and "reference model knowledge" respectively, and the obtained similarity is recorded as .

[0113] The corresponding prediction condition influencing factor can be obtained by weighted summing the word frequency similarity and Jaccard similarity coefficient obtained based on the above method, which is expressed as:

[0114]

[0115] In the formula, is the weight coefficient of word frequency similarity; It represents the prediction condition influencing factor. The larger the value is, the greater the influence of the prediction condition on the reference model knowledge is. That is, it can be used as data for subsequent model fine-tuning training. And the following method can be used to enhance the data sample to construct the corresponding prediction problem fine-tuning training sample set.

[0116] When the prediction condition influencing factor is greater than a preset credible threshold, according to the multi-dimensional similarity comprehensive evaluation method, the comprehensive similarity is calculated for the problem feature text and each case text in the preset data type knowledge base, respectively, to obtain the corresponding text comprehensive similarity; wherein, the preset credible threshold can be determined according to actual application requirements and is not specifically limited here; the method of performing comprehensive similarity calculation on the problem feature text whose prediction condition influencing factor exceeds the preset credible threshold to obtain the text comprehensive similarity can be implemented by referring to the relevant description in the analysis of the above-mentioned fault diagnosis sample, which will not be repeated here.

[0117] When the comprehensive text similarity between the problem feature text and the case text is greater than a preset similarity threshold, the case text is appended to the end of the problem feature text to obtain a candidate problem fine-tuning training sample; wherein, the process of obtaining the candidate problem fine-tuning training sample can be implemented by referring to the relevant description in the analysis of the above-mentioned fault diagnosis samples, and will not be repeated here.

[0118] The candidate question fine-tuning training sample is input into the first preset large language model for text comprehensibility and logical accuracy detection, and when the detection is qualified, the candidate question fine-tuning training sample is used as the question fine-tuning training sample; wherein, the process of obtaining the question fine-tuning training sample can also be implemented by referring to the relevant description in the analysis of the above-mentioned fault diagnosis samples, and will not be repeated here.

[0119] In order to facilitate the understanding of the above load prediction sample optimization process, the following is an exemplary description of the corresponding processing. If the text content of a load prediction sample is:

[0120] On December 1, 2024, I want to analyze the company's load forecast. Mainly based on historical data, I want to analyze the trend of the next week. We previously had an experimental record of the full-year data for 2023, which covered various historical data on CPU usage, network traffic, and storage read and write. But I'm not sure whether this data is still applicable to the current forecast scenario, because we recently replaced some hardware and the server configuration has been greatly adjusted;

[0121] In addition, I am concerned about whether this prediction can provide reliable performance estimates during future traffic peaks (especially from 10 am to 12 pm on weekdays). Previous experimental data was mainly about CPU load, but this time more detailed data analysis on network traffic and storage may be needed. Our most recent experiment was completed on October 10, 2024, and the data is recorded as follows:

[0122] Experiment name: October 2024 load experiment;

[0123] Peak CPU usage: 95%

[0124] Network traffic peak: 2Gbps;

[0125] Storage I / O peak: 500MB / s;

[0126] Experimental conditions: load stress test, duration 1 hour, using standard test tools

[0127] It needs to be clarified whether these data can be used as a new forecast basis? Is additional data needed? The forecast time range is from December 1 to December 7, 2024. I hope to get a specific analysis and conclusion.

[0128] Based on the above messy original data content, it is identified as a load prediction type based on a preset keyword library, and the first preset large language model is used to extract preset data features from the above content to obtain the corresponding question feature text as shown below:

[0129] Prediction conditions:

[0130] December 1, 2024 to December 7, 2024;

[0131] The CPU pressure is high and other tasks need to be performed at the same time;

[0132] Experimental record reference:

[0133] Experiment name: October 2024 load experiment;

[0134] Peak CPU usage: 95%

[0135] Network traffic peak: 2Gbps;

[0136] Storage I / O peak: 500MB / s;

[0137] Experimental conditions: load stress test, duration 1 hour, using standard test tools;

[0138] Experiment time: October 10, 2024;

[0139] Main prediction problems:

[0140] 1. Whether the forecasted load can provide reliable performance estimates from December 1 to December 7, 2024 (10:00 to 12:00 on weekdays).

[0141] 2. Whether the data in the experimental records are applicable to the prediction scenario, especially considering that the server configuration has been adjusted.

[0142] 3. Is it necessary to supplement more network traffic and store more detailed data?

[0143] By performing text preprocessing and similarity analysis on the prediction conditions and reference model knowledge in the above-obtained question feature text, the prediction condition influencing factor is obtained as follows: , assuming that the preset trust threshold is ,but >0.1, indicating that the prediction conditions in the above samples are strongly correlated with the reference model knowledge. They can be used as training samples for model fine-tuning, and data enhancement processing can also be performed based on them to construct more load prediction training samples.

[0144] Assume that a comprehensive similarity calculation and analysis is performed on the above-mentioned problem feature text and each case text in the corresponding load prediction data type knowledge base according to a multi-dimensional similarity comprehensive evaluation method, and an additional supplementary text is obtained, which is "Load prediction needs to be retested after hardware adjustment, and it is recommended to use the latest network traffic and storage I / O test records; peak load prediction needs to combine historical data and short-term dynamic adjustment models." Then, it is added to the above-mentioned problem feature text to form a candidate problem fine-tuning training sample; then the obtained candidate problem fine-tuning training sample is input into the first preset large language model for text comprehensibility and logical accuracy detection. If the detection is qualified, it is used as a problem fine-tuning training sample for constructing a subsequent problem fine-tuning training sample set.

[0145] This embodiment introduces the prediction condition influencing factors based on Jaccard similarity and word frequency co-occurrence analysis to identify the relationship between the data content in the load prediction sample, which can reliably determine whether the "prediction condition" in the data sample is strongly associated with the "reference model knowledge", and effectively avoid the problem of introducing training samples with weak logical correlation and poor data quality in model fine-tuning training due to ignoring the correlation between data when collecting data, thereby effectively ensuring the fine-tuning training effect of the subsequent large vertical domain model.

[0146] S14. Aggregate the problem fine-tuning training samples of the same operation and maintenance problem type to obtain the corresponding problem fine-tuning training sample set, and divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset; wherein the problem instruction fine-tuning sample subset can be understood as a sample set that uses natural language instructions to fine-tune the model; the problem increment fine-tuning sample subset can be understood as adding new fine-tuning training samples on the basis of the original training data to enhance the generalization ability of the model. Specifically, the step of dividing the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset includes:

[0147] The text type of each question fine-tuning training sample in the question fine-tuning training sample set is obtained respectively; wherein the text type can be understood as the content form of the text, including dialogue text and factual statement text. The corresponding dialogue text and factual statement text can refer to the following examples:

[0148] 1) Example of conversation sample data:

[0149] User: I need to re-evaluate the reliability of my load forecast based on our recent server configuration changes. Is the previous data still applicable?

[0150] Large model: Based on existing analysis, your previous experimental records (such as the load experiment in October 2024) have certain reference value in the current scenario, but due to adjustments to the server hardware, it is recommended to supplement the latest network traffic and storage I / O test records to improve prediction accuracy.

[0151] User: So the historical data from 2023 is completely useless?

[0152] Big model: The historical data from 2023 still has some reference significance, especially when analyzing long-term trends. However, due to changes in configuration and conditions, the weight of these data needs to be reduced, and it is recommended to combine them with recent experimental data for dynamic adjustment.

[0153] User: I am mainly concerned about the peak load in the coming week, especially from 10 am to 12 pm on weekdays. How should this data be processed?

[0154] Large model: For peak loads, it is recommended to use a short-term dynamic adjustment model, combined with the experimental records in October 2024 and supplemented by the latest test data, to ensure that the prediction results can accurately reflect the peak load conditions on weekdays.

[0155] 2) Incremental fine-tuning data sample:

[0156] Text 1: The load experiment records in October 2024 show that under the 1-hour load stress test conditions, the server CPU utilization peak reached 95%, the network traffic peak was 2Gbps, and the storage I / O peak was 500MB / s. The experimental conditions included the use of standard stress testing tools, and the experimental results reflect the performance of the server under high load to a certain extent.

[0157] Text 2: IMP calculations show that the reliability score of the experimental record is 0.141, which is higher than the set trust threshold of 0.1. Therefore, the experimental record can be used as the basic data for new prediction scenarios and combined with other data for further analysis.

[0158] Text 3: After hardware adjustments, load prediction requires new network traffic and storage I / O data for supplementary testing to reflect performance under the current configuration.

[0159] The question fine-tuning training samples whose text type is factual statement text are classified as question incremental fine-tuning samples, and the question fine-tuning training samples whose text type is dialogue text are classified as question instruction fine-tuning samples; wherein, the question incremental fine-tuning samples can be used to supplement new knowledge and improve the adaptability of the model to domain-specific situations; the question instruction fine-tuning samples are used to improve the model's ability to understand instructions in the field of power operation and maintenance.

[0160] All the problem increment fine-tuning samples are aggregated to obtain the problem increment fine-tuning sample subset, and all the problem instruction fine-tuning samples are aggregated to obtain the problem instruction fine-tuning sample subset.

[0161] This embodiment takes into account the need to emphasize special contexts and interactive relationships for power operation and maintenance services. The dialogue samples are closer to the characteristics of actual application scenarios, and the dialogue text is used as instruction fine-tuning data, which can help the model understand the user's intention more naturally when processing similar tasks; at the same time, the dialogue form can simulate different user needs, cover multiple rounds of interaction for complex problems, and guide users to gradually clarify the problem through dialogue, so that the model is better at disassembling and responding to multi-step tasks. In addition, this embodiment takes into account the characteristics of factual statement texts that directly present core information. It summarizes key points in a structured manner, which facilitates the large model to extract causal relationships related to specific tasks. It is suitable for power operation and maintenance fine-tuning to efficiently train models to identify and process complex data relationships, and can meet the high requirements of professional fine-tuning task backgrounds under low-cost conditions, effectively improving the professional scene performance of large models.

[0162] S15. According to the problem instruction fine-tuning sample subset and the problem increment fine-tuning sample subset, the second preset large language model is fine-tuned and trained based on the quantized low-rank adaptive fine-tuning technology to obtain the corresponding operation and maintenance problem large model, and all the operation and maintenance problem large models are merged to obtain the power operation and maintenance vertical domain large model; wherein, the second preset large language model can be selected according to actual application needs. This embodiment preferably adopts the Qwen2.5-7B-Instruct large model, and the corresponding quantized low-rank adaptive fine-tuning technology can be understood as the QLoRA fine-tuning technology. Under the background condition of performing 4-bits quantization, the locally deployed Qwen2.5-7B-Instruct large model can be fine-tuned and trained to reduce the model size and reduce memory usage. At the same time, Low-Rank Adaptation (LoRA) is used to optimize the model parameters to maintain the fine-tuning performance; specifically, the second preset large language model is fine-tuned and trained based on the quantized low-rank adaptive fine-tuning technology to obtain the corresponding operation and maintenance problem large model. The process can be implemented with reference to the existing QLoRA fine-tuning technology, which will not be repeated here.

[0163] This embodiment ensures the adaptability and efficiency of the model to power operation and maintenance scenarios through local deployment and fine-tuning of the large language model, and performs fine-tuning operations locally according to the adjusted data ratio, and can ensure that the optimized model can accurately and efficiently process prompt words and instruction data in the "power operation and maintenance field", providing professional support for actual operations in the power operation and maintenance field.

[0164] In addition, considering that there are general tasks other than fault diagnosis and load forecasting in actual power operation and maintenance, in order to ensure the extensive application scenarios of the power operation and maintenance vertical domain large model, in this embodiment, preferably, the to-be-processed power operation and maintenance data set also includes general task samples, and the corresponding preset keyword library also includes a general task keyword library, and the general task keywords may include inspection, query, request, operation and task data; the method also includes:

[0165] When the operation and maintenance problem type of the problem sample to be optimized is the general task, the problem sample to be optimized is directly added to the corresponding problem fine-tuning training sample set; among them, the problem samples to be optimized of the general task type are shown in Table 3. In this case, they are directly used as corresponding fine-tuning training samples.

[0166] Table 3 Examples of samples of problems to be optimized for general task types

[0167]

[0168] The embodiment of the present invention proposes Figure 2 The efficient fine-tuning method of the power operation and maintenance vertical domain big model suitable for sparse power operation and maintenance data shown in the figure takes the power operation and maintenance big data as the knowledge core, uses prompt words with classification identifiers to organize and classify the sparse power operation and maintenance data used for model fine-tuning, and combines the designed multi-dimensional similarity comprehensive evaluation method and prediction condition influencing factor analysis method for data enhancement processing. It can not only effectively meet the structured needs of power operation and maintenance fine-tuning training, ensure that effective information can be extracted and the structured utilization efficiency of data can be optimized during the fine-tuning process, and improve the accuracy of model fine-tuning training, but also make the data used for model training more suitable for model fine-tuning training, significantly improve the model's adaptability and expression ability for complex power operation and maintenance data, and can also achieve output alignment and fine-tuning practices through QLoRA fine-tuning technology, strengthen the professionalism of the big model in the vertical field of power operation and maintenance, ensure that the model is consistent and accurate when applying power operation and maintenance professional knowledge, and ultimately provide users with more professional and customized intelligent support, so that "sparse power operation and maintenance data" can achieve excellent performance in "big model vertical domain fine-tuning".

[0169] In order to verify the application effect of the present invention, Figure 3 The data response scenario shown is an example analysis of the overall implementation of the present invention. The present invention provides a large model efficient fine-tuning method suitable for sparse power operation and maintenance data, which can improve the efficiency of power information operation and maintenance processing and introduce a professional vertical domain large model human-computer interaction mode. The specific application steps are as follows:

[0170] 1) Data collection and command input:

[0171] This scenario sets up two entities, namely the data training center and the data processing center. The data training center is responsible for collecting and building three SQL databases, corresponding to the three parts of "fault diagnosis", "load prediction", and "general tasks (electricity knowledge)". Its data collection is shown in Table 4. In addition, the data training center and the data processing center will collect user instructions or questions at this stage, vectorize the instructions or questions with the classic Embedding model Word2Vec, and build a routing decision to call the corresponding database through the classification prompt word technology. According to different call signals, the routing can call the corresponding database data for the training set construction task.

[0172] Table 4 Data collection form

[0173]

[0174] 2) Construct problem fine-tuning training sample set:

[0175] Building a training set is a very important part. Data with clear reasoning ideas and well-aligned formats can greatly improve the quality of fine-tuning large models. The data training center will use the technology of the first goal of this invention in conjunction with the Json data format to unify the data structure. This example uses the prompt word construction process of fault diagnosis data as a standard to explain the construction process. Figure 4 shown.

[0176] In this example, the "function mapping strategy" is adopted as a whole to map key information into the Json file. The prompt word information is "power system fault diagnosis", which is used to guide the large model to perform fault diagnosis tasks. The reference data part contains two fault data, each of which includes four fields: fault type, fault characteristics, fault location, and historical sample reference. Among them, the fault type, fault characteristics, and fault location are used to describe the basic information of the fault, and the historical sample reference provides a reference case for handling this type of fault. Each case includes two fields: case number and handling measures, which are used to describe the specific handling method.

[0177] 3) Fine-tuning of large models:

[0178] After constructing the training data, the Data Training Center uses QLoRA technology to fine-tune the general large model. This technology can effectively fine-tune the large model by introducing a small number of trainable parameters while keeping the original parameters of the large model unchanged. During the fine-tuning process, the Data Training Center uses the AdamW algorithm, dynamic learning rate, and early stopping mechanism to improve the effect.

[0179] 4) Weight merging and reasoning test:

[0180] In this phase, the data training center will merge the weights of the fine-tuned large model to obtain a complete large model for the vertical domain of power information operation and maintenance. Then, the data processing center will use the test set to evaluate the merged large model to verify its level in the multi-task system. During the reasoning process, the large model will retrieve reference information from the corresponding database based on the user's instructions or questions, and reason with its own knowledge to generate answers or processing solutions that meet the specifications.

[0181] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0182] In one embodiment, Figure 5 As shown, a large model fine-tuning system for power operation and maintenance is provided, the system comprising:

[0183] Data acquisition module 1, used to obtain a power operation and maintenance data set to be processed, and use each operation and maintenance data sample in the power operation and maintenance data set as a sample of the problem to be optimized; the power operation and maintenance data set to be processed includes a fault diagnosis sample and a load prediction sample;

[0184] Type analysis module 2 is used to analyze each problem sample to be optimized according to a preset keyword library to obtain the corresponding operation and maintenance problem type;

[0185] The sample enhancement module 3 is used to extract preset data features from each problem sample to be optimized according to the operation and maintenance problem type and the first preset large language model, obtain corresponding problem feature text, and expand the problem feature text according to the corresponding preset data type knowledge base to generate corresponding problem fine-tuning training samples;

[0186] The data set division module 4 is used to aggregate the problem fine-tuning training samples of the same operation and maintenance problem type to obtain a corresponding problem fine-tuning training sample set, and divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset;

[0187] The model fine-tuning module 5 is used to fine-tune the sample subset according to the problem instruction and the problem increment fine-tune the sample subset, fine-tune the second preset large language model based on the quantized low-rank adaptive fine-tuning technology, obtain the corresponding operation and maintenance problem large model, and merge all the operation and maintenance problem large models to obtain the power operation and maintenance vertical domain large model.

[0188] For the specific definition of the large model fine-tuning system for power operation and maintenance, please refer to the definition of the large model fine-tuning method for power operation and maintenance mentioned above. The corresponding technical effects can also be equivalently obtained, which will not be repeated here. Each module in the above-mentioned large model fine-tuning system for power operation and maintenance can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0189] Figure 6 FIG. 1 shows an internal structure diagram of a computer device in an embodiment, and the computer device may specifically be a terminal or a server. Figure 6 As shown, the computer device includes a processor, a memory, a network interface, a display, a camera and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a large model fine-tuning method for power operation and maintenance is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0190] It can be understood by those skilled in the art that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computing device may include more or less components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0191] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0193] In summary, an embodiment of the present invention provides a large model fine-tuning method and system for power operation and maintenance. The large model fine-tuning method for power operation and maintenance realizes obtaining a to-be-processed power operation and maintenance data set including fault diagnosis samples and load prediction samples, and taking each operation and maintenance data sample in the power operation and maintenance data set as a sample of a problem to be optimized. According to a preset keyword library, each sample of the problem to be optimized is analyzed to obtain a corresponding operation and maintenance problem type, and according to the operation and maintenance problem type and a first preset large language model, preset data features are extracted for each sample of the problem to be optimized to obtain a corresponding problem feature text, and after the problem feature text is expanded according to a corresponding preset data type knowledge base to generate a corresponding problem fine-tuning training sample, the problem fine-tuning training samples of the same operation and maintenance problem type are aggregated to obtain a corresponding problem fine-tuning training sample set, and The problem fine-tuning training sample set is divided into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset, and according to the problem instruction fine-tuning sample subset and the problem increment fine-tuning sample subset, the second preset large language model is fine-tuned and trained based on the quantized low-rank adaptive fine-tuning technology to obtain the corresponding operation and maintenance problem large model, and all operation and maintenance problem large models are merged to obtain the power operation and maintenance vertical domain large model. The technical solution of this method can accurately process the sparsity and unstructured attributes of power operation and maintenance data, effectively improve the utilization efficiency of effective sample information in fine-tuning, and effectively improve the adaptability and expression ability of model fine-tuning to power operation and maintenance data, thereby improving the efficiency and accuracy of fine-tuning training, and can also take into account the interactive ability of the large model in the professional field of power operation and maintenance, and provide strong support for multi-scenario applications in the field of power operation and maintenance.

[0194] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] The above-mentioned embodiments only express several preferred implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principle of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the protection scope of the claims.

Claims

1. A large model fine-tuning method for power operation and maintenance, characterized in that: The method comprises the following steps: Acquire a power operation and maintenance data set to be processed, and use each operation and maintenance data sample in the power operation and maintenance data set as a sample of the problem to be optimized; the power operation and maintenance data set to be processed includes a fault diagnosis sample and a load prediction sample; Analyze each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type; According to the operation and maintenance problem type and the first preset large language model, extract preset data features from each problem sample to be optimized to obtain a corresponding problem feature text, and expand the problem feature text according to a corresponding preset data type knowledge base to generate a corresponding problem fine-tuning training sample; Aggregate problem fine-tuning training samples of the same operation and maintenance problem type to obtain a corresponding problem fine-tuning training sample set, and divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset; the problem instruction fine-tuning sample subset is a dialogue text set, and the problem increment fine-tuning sample subset is a fact statement text set; According to the problem instruction fine-tuning sample subset and the problem increment fine-tuning sample subset, fine-tune the second preset large language model based on the quantized low-rank adaptive fine-tuning technology to obtain the corresponding operation and maintenance problem large model, and merge all the operation and maintenance problem large models to obtain the power operation and maintenance vertical domain large model; Among them, when the operation and maintenance problem type is load prediction, the preset data features include prediction conditions and reference model knowledge; the reference model knowledge includes model type, data source, feature variables, target variables, prediction method, model evaluation index, prediction assumptions and external factors; the step of expanding the problem feature text according to the corresponding preset data type knowledge base to generate the corresponding problem fine-tuning training sample includes: Preprocessing the prediction conditions and reference model knowledge in the question feature text to obtain corresponding prediction condition word lists and model knowledge word lists; Obtaining the word frequency similarity and the Jaccard similarity coefficient of the prediction condition word list and the model knowledge word list, and performing weighted summation of the word frequency similarity and the Jaccard similarity coefficient to obtain the corresponding prediction condition influencing factor; When the prediction condition influencing factor is greater than a preset credible threshold, a comprehensive similarity calculation is performed on the problem feature text and each case text in the preset data type knowledge base according to a multi-dimensional similarity comprehensive evaluation method to obtain a corresponding text comprehensive similarity; When the comprehensive text similarity between the question feature text and the case text is greater than a preset similarity threshold, the case text is appended to the end of the question feature text to obtain a candidate question fine-tuning training sample; Inputting the candidate question fine-tuning training sample into the first preset large language model to detect text comprehensibility and logical accuracy, and when the detection is qualified, using the candidate question fine-tuning training sample as the question fine-tuning training sample; The multi-dimensional similarity comprehensive evaluation method is to calculate the Jaccard similarity coefficient, the Dyss similarity coefficient and the N-tuple similarity value respectively, then calculate the similarity coefficient average value of the Jaccard similarity coefficient and the Dyss similarity coefficient, and perform weighted average of the similarity coefficient average value and the N-tuple similarity value to obtain the comprehensive similarity.

2. The large model fine-tuning method for power operation and maintenance according to claim 1, characterized in that: The preset keyword library includes a fault diagnosis keyword library and a load prediction keyword library; The step of analyzing each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type includes: Performing vectorization calculation on the problem sample to be optimized by using a preset semantic vector model to obtain a corresponding vectorization processing result; Performing vectorization processing on each preset keyword library through the preset semantic vector model to obtain a corresponding vectorized keyword library; Calculating the cosine similarity between each keyword in each vectorized keyword library and the vectorization processing result to obtain a keyword similarity list corresponding to each vectorized keyword library; Obtain the keyword corresponding to the maximum similarity value in the keyword similarity list of each vectorized keyword library as a candidate user question keyword; The keyword with the largest similarity value among all candidate user question keywords is used as the user question keyword; The operation and maintenance problem type is obtained according to the type of the preset keyword library corresponding to the user problem keyword.

3. The large model fine-tuning method for power operation and maintenance according to claim 1, characterized in that: When the operation and maintenance problem type is fault diagnosis, the problem feature text includes the fault type, fault feature and fault location; The step of expanding the question feature text according to the corresponding preset data type knowledge base to generate corresponding question fine-tuning training samples includes: According to the multi-dimensional similarity comprehensive evaluation method, the comprehensive similarity of the problem feature text and each case text in the preset data type knowledge base is calculated respectively to obtain the corresponding text comprehensive similarity; When the comprehensive text similarity between the question feature text and the case text is greater than a preset similarity threshold, the case text is appended to the end of the question feature text to obtain a candidate question fine-tuning training sample; The candidate question fine-tuning training sample is input into the first preset large language model for text comprehensibility and logical accuracy detection, and when the detection is qualified, the candidate question fine-tuning training sample is used as the question fine-tuning training sample.

4. The large model fine-tuning method for power operation and maintenance according to claim 1, characterized in that: The step of dividing the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset comprises: Respectively obtaining the text type of each question fine-tuning training sample in the question fine-tuning training sample set; Classifying the question fine-tuning training samples whose text type is factual statement text as question increment fine-tuning samples, and classifying the question fine-tuning training samples whose text type is dialogue text as question instruction fine-tuning samples; All the problem increment fine-tuning samples are aggregated to obtain the problem increment fine-tuning sample subset, and all the problem instruction fine-tuning samples are aggregated to obtain the problem instruction fine-tuning sample subset.

5. The large model fine-tuning method for power operation and maintenance according to claim 1, characterized in that: The to-be-processed power operation and maintenance data set also includes a general task sample; the preset keyword library also includes a general task keyword library; the method also includes: When the operation and maintenance problem type of the problem sample to be optimized is the general task, the problem sample to be optimized is directly added to the corresponding problem fine-tuning training sample set.

6. A large model fine-tuning system for power operation and maintenance, characterized in that: The large model fine-tuning method for power operation and maintenance according to claim 1 is applied, and the system comprises: A data acquisition module is used to obtain a power operation and maintenance data set to be processed, and use each operation and maintenance data sample in the power operation and maintenance data set as a sample of the problem to be optimized; the power operation and maintenance data set to be processed includes a fault diagnosis sample and a load prediction sample; The type analysis module is used to analyze each problem sample to be optimized according to the preset keyword library to obtain the corresponding operation and maintenance problem type; A sample enhancement module is used to extract preset data features from each problem sample to be optimized according to the operation and maintenance problem type and the first preset large language model to obtain a corresponding problem feature text, and to expand the problem feature text according to a corresponding preset data type knowledge base to generate a corresponding problem fine-tuning training sample; A data set partitioning module is used to aggregate problem fine-tuning training samples of the same operation and maintenance problem type to obtain a corresponding problem fine-tuning training sample set, and to divide the problem fine-tuning training sample set into a problem instruction fine-tuning sample subset and a problem increment fine-tuning sample subset; The model fine-tuning module is used to fine-tune the sample subset according to the problem instruction and the problem increment fine-tune the sample subset, fine-tune the second preset large language model based on the quantized low-rank adaptive fine-tuning technology, obtain the corresponding operation and maintenance problem large model, and merge all the operation and maintenance problem large models to obtain the power operation and maintenance vertical domain large model.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images