Hydropower operation and maintenance large language model fine-tuning method, hydropower operation and maintenance method and device

Abstract

The application provides a fine-tuning method of a hydropower operation and maintenance large language model, a hydropower operation and maintenance method and device. The fine-tuning method of the hydropower operation and maintenance large language model comprises the following steps: obtaining a confidence degree generated by a confidence degree generation model for an operation and maintenance answer generated by an operation and maintenance answer generation model; judging the correctness of the operation and maintenance answer by using hydropower operation and maintenance knowledge to obtain a correctness judgment result; determining a reward value corresponding to the confidence degree according to a reward function constructed by using a logarithmic scoring rule and the correctness judgment result; and adjusting parameters of the confidence degree generation model based on the reward value. On the basis of decoupling the operation and maintenance answer generation and the confidence degree generation, the reward value of the confidence degree is determined according to the correctness judgment result of the operation and maintenance answer determined by using the hydropower operation and maintenance knowledge and the reward function constructed by using the logarithmic scoring rule, the parameters of the confidence degree generation model are adjusted, the confidence degree output by the trained confidence degree generation model is more accurate.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for fine-tuning a large language model for hydropower operation and maintenance, as well as a hydropower operation and maintenance method and apparatus. Background Technology

[0002] Hydropower operation and maintenance (O&M) is a core component ensuring the safe and stable operation of cascade hydropower stations, generating units, and power grid dispatching within a river basin. It encompasses a variety of high-value tasks, including equipment fault diagnosis, defect analysis, maintenance decision-making, emergency response, procedure interpretation, and data analysis. With the application of Large Language Models (LLM) in industrial O&M, functions such as natural language interaction, root cause analysis, O&M plan generation, and risk warning interpretation can be achieved, significantly improving O&M efficiency and decision-making standardization.

[0003] Currently, the training method for large language models is mainly supervised fine-tuning. This method focuses on expecting the model to output accurate answers. In the field of hydropower operation and maintenance, it is not only necessary for the model to give correct conclusions, but also to accurately express its own confidence level: give high confidence for certain conclusions, give low confidence for uncertain faults and suggest manual review.

[0004] However, current large language models generally lack confidence calibration capabilities, resulting in a high risk of model illusions and insufficient decision credibility. Summary of the Invention

[0005] This invention provides a fine-tuning method, a hydropower operation and maintenance method, and an apparatus for a large language model, which addresses the shortcomings of existing technologies where inaccurate confidence levels of model outputs in hydropower operation and maintenance scenarios lead to insufficient decision credibility. This invention improves the accuracy of the confidence levels output by the model, thereby enhancing the reliability of decision-making.

[0006] This invention provides a fine-tuning method for a large language model of hydropower operation and maintenance, which includes an operation and maintenance answer generation model and a confidence level generation model. The method includes: The confidence level of the confidence level generated by the confidence level generation model is obtained for the operation and maintenance answer generation model output by the operation and maintenance answer generation model. The operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels. The correctness of the operation and maintenance answers is judged using knowledge of hydropower operation and maintenance. The correctness judgment result is either correct, incorrect, or unknown. Based on the reward function constructed using the logarithmic scoring rule and the correctness judgment results, the reward value corresponding to the confidence level is determined; The parameters of the confidence generation model are adjusted based on the reward value.

[0007] According to the fine-tuning method of a large language model for hydropower operation and maintenance provided by the present invention, the reward value corresponding to the confidence level is determined based on the reward function constructed using a logarithmic scoring rule and the correctness judgment result, including: Using the correctness judgment result as the selection criterion, select the logarithmic scoring rule branch that matches the correctness judgment result from the reward function as the target branch; The reward value is obtained by determining the function value of the target branch based on the confidence level.

[0008] According to the present invention, a fine-tuning method for a large language model of hydropower operation and maintenance, using the correctness judgment result as the selection condition, selects the logarithmic scoring rule branch that matches the correctness judgment result from the reward function as the target branch, including: If the correctness judgment result is that the operation and maintenance answer is correct, the target branch is determined to be the first branch, and the input of the first branch is the maximum value between the confidence score and the preset minimum value; If the correctness judgment result is that the operation and maintenance answer is wrong, the target branch is determined to be the second branch. The input of the second branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value. If the correctness judgment result is that the correctness of the operation and maintenance answer is unknown, the target branch is determined to be the third branch. The input of the third branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value.

[0009] According to the present invention, a fine-tuning method for a large language model of hydropower operation and maintenance is provided, which obtains a reward value based on the function value of the target branch determined by the confidence level, including: Use the function value as the initial reward value; The initial reward value is normalized using the preset minimum reward extreme value and the preset maximum reward extreme value to obtain candidate reward values; The reward value is obtained based on the candidate reward value.

[0010] According to the present invention, a fine-tuning method for a large language model of hydropower operation and maintenance is provided, which obtains a reward value based on candidate reward values, including: Determine whether the operation and maintenance answer complies with the security procedures to obtain a compliance judgment result. The compliance judgment result is either that the operation and maintenance answer complies with the security procedures or does not comply with the security procedures. Select a security compliance reward value that matches the compliance assessment result; The reward value is obtained by weighted summation of the candidate reward values ​​and the security compliance reward values.

[0011] According to the present invention, a fine-tuning method for a large language model of hydropower operation and maintenance utilizes hydropower operation and maintenance knowledge to judge the correctness of operation and maintenance answers, and obtains the correctness judgment result, including: Based on the category of the operation and maintenance answers, determine whether there is corresponding water and electricity operation and maintenance knowledge. If such a relationship exists, the correctness of the operation and maintenance answer is determined based on the relationship between hydropower operation and maintenance knowledge and the operation and maintenance answer. If it does not exist, the result of determining the correctness of the operation and maintenance answer is "correctness unknown".

[0012] According to the present invention, a fine-tuning method for a large language model of hydropower operation and maintenance is provided. The hydropower operation and maintenance knowledge includes at least one of the following: standard fault labels for hydropower operation and maintenance, clauses of hydropower operation and maintenance procedures, and standard threshold ranges in the hydropower field. Based on the relationship between hydropower operation and maintenance knowledge and operation and maintenance answers, the correctness judgment result of the operation and maintenance answer is determined, including: If the category of the operation and maintenance answer is fault diagnosis, obtain the overlap between the operation and maintenance answer and the standard fault label of water and electricity operation and maintenance. If the overlap is greater than or equal to the preset overlap, the operation and maintenance answer is determined to be correct. If the category of the operation and maintenance answer is operation and maintenance, and it is determined that the operation and maintenance answer complies with the provisions of the hydropower operation and maintenance regulations, then the operation and maintenance answer is determined to be correct. If the category of the operation and maintenance answer is parameter analysis, and it is determined that the operation and maintenance answer is within the standard threshold range of the hydropower field, then the operation and maintenance answer is determined to be correct.

[0013] According to the present invention, a method for fine-tuning a large language model for hydropower operation and maintenance adjusts the parameters of a confidence generation model based on reward values, including: Calculate the near-end strategy optimization loss based on the reward value; The loss is optimized using a proximal strategy, and the parameters of the confidence generation model are adjusted.

[0014] According to the present invention, a fine-tuning method for a large language model of hydropower operation and maintenance is provided. The method is executed within the framework of a Markov decision process, wherein the Markov decision process is defined as a six-tuple, and the six-tuple consists of (state space, action space, state transition function, reward function, hydropower operation and maintenance knowledge set, and discount factor). The state space includes the hydropower operation and maintenance query data, operation and maintenance answers, and the generated confidence word sequence input to the operation and maintenance answer generation model; The action space includes actions that are the next confidence term selected by the confidence generation model; The state transition function is a deterministic transition function; The reward function is constructed using a logarithmic scoring rule; The hydropower operation and maintenance knowledge set includes an equipment fault database, an operation and maintenance procedure database, safety constraint thresholds, and unit operating parameters. Discount factors are used to control the importance of reward values.

[0015] This invention also provides a hydropower operation and maintenance method, comprising: Obtain target hydropower operation and maintenance query data; The target hydropower operation and maintenance query data is processed using the operation and maintenance answer generation model in the hydropower operation and maintenance big language model to obtain the target operation and maintenance answer; and the target confidence score is generated for the target operation and maintenance answer using the confidence score generation model in the hydropower operation and maintenance big language model. The hydropower operation and maintenance big language model is trained using any of the above-mentioned fine-tuning methods of the hydropower operation and maintenance big language model. Output the target operation and maintenance answer and the target confidence level.

[0016] This invention also provides a fine-tuning device for a large language model of hydropower operation and maintenance, which includes an operation and maintenance answer generation model and a confidence level generation model. The device includes the following modules: The confidence score acquisition module is used to acquire the confidence score generated by the confidence score generation model for the operation and maintenance answer generation model output by the operation and maintenance answer generation model. The operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels. The correctness judgment module is used to judge the correctness of operation and maintenance answers using hydropower operation and maintenance knowledge, and obtain the correctness judgment result, which is that the operation and maintenance answer is correct, incorrect, or the correctness is unknown; The reward value determination module is used to determine the reward value corresponding to the confidence level based on the reward function constructed using the logarithmic scoring rule and the correctness judgment result. The parameter adjustment module is used to adjust the parameters of the confidence generation model based on the reward value.

[0017] The present invention also provides a hydropower operation and maintenance device, comprising: The query data acquisition module is used to acquire target hydropower operation and maintenance query data; The operation and maintenance answer and confidence score acquisition module is used to process the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance big language model to obtain the target operation and maintenance answer; and to generate the target confidence score for the target operation and maintenance answer using the confidence score generation model in the hydropower operation and maintenance big language model. The hydropower operation and maintenance big language model is trained using any of the above-mentioned fine-tuning methods of the hydropower operation and maintenance big language model. The output module is used to output the target operation and maintenance answer and the target confidence level.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a fine-tuning method or a hydropower operation and maintenance method of any of the above-described large language model.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a fine-tuning method for a large language model of hydropower operation and maintenance or a hydropower operation and maintenance method as described above.

[0020] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements a fine-tuning method or a hydropower operation and maintenance method for a large language model as described above.

[0021] This invention provides a fine-tuning method, method, and apparatus for a large language model of hydropower operation and maintenance. It decouples the model's confidence expression from the generation of operation and maintenance answers and integrates them into a unified model. By combining objective hydropower operation and maintenance knowledge judgment with a mathematically rigorous logarithmic scoring rule, it fundamentally constrains the model's overconfidence problem. While retaining the original operation and maintenance question-and-answer capabilities, it achieves perfect confidence calibration, improves the accuracy of the confidence output by the confidence generation model after parameter fine-tuning, significantly reduces model decision illusions in high-risk scenarios, and enhances power safety compliance. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is one of the flowcharts illustrating the fine-tuning method for the large language model of hydropower operation and maintenance provided by this invention.

[0024] Figure 2 This is the second flowchart illustrating the fine-tuning method for the large language model of hydropower operation and maintenance provided by this invention.

[0025] Figure 3 This is a flowchart illustrating the hydropower operation and maintenance method provided by the present invention.

[0026] Figure 4 This is a schematic diagram of the structure of the fine-tuning device for the large language model of hydropower operation and maintenance provided by the present invention.

[0027] Figure 5 This is a schematic diagram of the hydropower operation and maintenance device provided by the present invention.

[0028] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] The following is combined with Figures 1 to 2 This invention describes a fine-tuning method for a large-scale language model of hydropower operation and maintenance. The invention decouples the generation of operation and maintenance answers and the generation of confidence scores within the large-scale language model. It utilizes the operation and maintenance answer generation model within the large-scale language model to generate operation and maintenance answers based on hydropower operation and maintenance query data, and utilizes the confidence score generation model within the large-scale language model to generate confidence scores for the operation and maintenance answers based on the query data and the answers themselves. Figure 1 This is one of the flowcharts illustrating the fine-tuning method for the large language model of hydropower operation and maintenance provided by this invention, such as... Figure 1 As shown, the method includes the following: Step 101: Obtain the confidence level generated by the confidence level generation model for the operation and maintenance answer output by the operation and maintenance answer generation model.

[0031] The operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels.

[0032] The hydropower operation and maintenance (O&M) query data samples can include descriptions of hydropower station equipment faults, procedural consultation questions, or status assessment instructions. O&M answer tags are the tags corresponding to the hydropower O&M query data samples; for example, O&M answer tags could be corresponding expert-level standard O&M guidance text. The O&M answer generation model has already learned from these samples and tags and possesses business question-and-answer capabilities.

[0033] Confidence is a numerical value used to quantify how confident a model is in the operational answers it generates. For example, it can be an integer from 0 to 10, or a decimal normalized to 0 to 1.

[0034] For example, in the fine-tuning training phase, a decoupled training approach is adopted: First, the parameters of the operation and maintenance answer generation model are fixed (i.e., the operation and maintenance answer generation model does not participate in subsequent gradient updates), and hydropower operation and maintenance query data is input to it to generate the corresponding operation and maintenance answer; then, the operation and maintenance answer and the hydropower operation and maintenance query data are used together as the input of the confidence generation model, which generates the corresponding confidence token sequence through autoregression, thereby obtaining the confidence score.

[0035] Step 102: Use hydropower operation and maintenance knowledge to judge the correctness of the operation and maintenance answer and obtain the correctness judgment result.

[0036] Hydropower operation and maintenance knowledge refers to the collection of knowledge in the field of hydropower, which may include, but is not limited to, equipment operating parameters, fault mechanisms, operation and maintenance procedures, safety thresholds, etc.

[0037] The correctness judgment result is categorized as "correct," "incorrect," or "correctness unknown" for the operation and maintenance (O&M) answer. The correctness judgment result is an objective evaluation of the O&M answer. It employs objective, hard logic based on hydropower O&M knowledge for evaluation. For example, if the description of the answer conforms to professional knowledge, it is considered correct; if it violates professional knowledge or contains factual errors (illusions), it is considered incorrect; for ambiguous faults or extremely rare operating conditions lacking knowledge base support, the correctness is determined to be unknown.

[0038] Step 103: Determine the reward value corresponding to the confidence level based on the reward function constructed using the logarithmic scoring rule and the correctness judgment result.

[0039] The logarithmic scoring rule is a mechanism that mathematically possesses the property of a "proper scoring rule." The reward function is an evaluation function used in reinforcement learning to guide the direction of model optimization.

[0040] Traditional large-scale models lack confidence calibration, making them prone to overconfidence (giving high confidence to incorrect answers), directly threatening the safety of hydropower operations. This invention introduces a logarithmic scoring rule to construct a reward function, achieving the following effects: high rewards are given to correct and highly confident outputs, while high penalties are imposed on incorrect but highly confident outputs. Different logical branches in the reward function are selected based on the correctness judgment result, calculating the reward value for the current confidence level.

[0041] Step 104: Adjust the parameters of the confidence generation model based on the reward value.

[0042] Within the reinforcement learning framework, the reward value calculated in step 103 is used as a feedback signal to update the network parameters of the confidence generation model through the backpropagation algorithm, in order to maximize the expected reward.

[0043] In the above scheme, the confidence level expression of the model is decoupled from the generation of operation and maintenance answers and integrated into a joint model. By combining objective judgment of hydropower operation and maintenance knowledge with a mathematically rigorous proven logarithmic scoring rule, the overconfidence problem of the model is constrained at its source. While retaining the original operation and maintenance question-and-answer capabilities, the confidence level is perfectly calibrated, improving the accuracy of the confidence level output by the confidence level generation model after parameter fine-tuning. This significantly reduces the model decision illusion in high-risk scenarios and enhances power safety compliance.

[0044] In one embodiment, knowledge of hydropower operation and maintenance is used to determine the correctness of the operation and maintenance answer, and the correctness determination result is obtained, including: Based on the category of the operation and maintenance answer, determine whether there is corresponding hydropower operation and maintenance knowledge; if so, determine the correctness judgment result of the operation and maintenance answer based on the relationship between hydropower operation and maintenance knowledge and the operation and maintenance answer; if not, determine the correctness judgment result of the operation and maintenance answer as unknown.

[0045] Because hydropower operation and maintenance tasks are complex, the types of operation and maintenance answers can be diverse, such as fault diagnosis, operation and maintenance operations, or parameter analysis.

[0046] During the judgment process, a knowledge base can be searched first. If the current scenario (e.g., a rare unit vibration phenomenon) has no matching item in existing hydropower operation and maintenance knowledge, it is directly judged as unknown. If a matching item exists, the correctness of the operation and maintenance answer is further determined based on the matching item. This judgment method will cause the reward function to automatically guide the model to output a low confidence level for this type of question, which is crucial in practical applications. It indicates to on-site operation and maintenance personnel that even the model is uncertain and manual intervention is required for verification. If a matching knowledge item is found, its correctness or incorrectness is determined based on logical comparison.

[0047] In one embodiment, the hydropower operation and maintenance knowledge includes at least one of the following: standard fault labels for hydropower operation and maintenance, clauses of hydropower operation and maintenance procedures, and standard threshold ranges in the hydropower field. Standard fault labels for hydropower operation and maintenance can indicate fault type, fault cause, or operation and maintenance methods for the fault, such as fault cause, fault performance data, and fault conclusions. The method for determining the correctness of the operation and maintenance answer based on the relationship between hydropower operation and maintenance knowledge and the operation and maintenance answer can include: If the category of the operation and maintenance answer is fault diagnosis, obtain the overlap between the operation and maintenance answer and the standard fault label of water and electricity operation and maintenance. If the overlap is greater than or equal to the preset overlap, the operation and maintenance answer is determined to be correct.

[0048] Among them, the overlap degree is calculated. The method can be referred to formula (1): Formula (1) in, Indicates the degree of overlap. Indicates accuracy. This represents recall rate. For example, when... If so, it is judged as correct and calibrated. If it is determined to be incorrect, then calibrate. In cases where the correctness of the operation and maintenance answer is unknown, no direct judgment is made. Instead, it triggers low-confidence guidance, such as encouraging the model to output low confidence for such scenarios through prompts or soft constraints.

[0049] If the maintenance answer is categorized as maintenance operation, and it is determined that the maintenance answer complies with the provisions of the hydropower maintenance regulations, then the maintenance answer is considered correct and marked. Similarly, if the category of the maintenance answer is maintenance operation, and it is determined that the maintenance answer does not comply with the provisions of the water and electricity maintenance regulations, then the maintenance answer is determined to be incorrect and marked. For example, the steps in a maintenance answer must fully cover the order and prohibitions of the water and electricity maintenance regulations to be considered correct. Similarly, if the correctness of a maintenance answer is determined to be unknown, it is not directly marked as correct. .

[0050] If the operation and maintenance answer falls under the category of parameter analysis, and it is determined that the answer is within the standard threshold range for the hydropower field, then the operation and maintenance answer is considered correct and calibrated. Similarly, if the operation and maintenance answer falls under the category of parameter analysis, and it is determined that the answer is outside the standard threshold range for the hydropower field, then the operation and maintenance answer is deemed incorrect and calibrated. For example, analyzing the temperature at a specific observation point requires extracting the numerical predictions or status descriptions from the maintenance and operation (O&M) answers and verifying whether they strictly fall within the standard threshold range for the hydropower equipment. Similarly, if the correctness assessment result of the O&M answer is "correctness unknown," it is not directly labeled. .

[0051] The above solution deeply integrates domain-specific knowledge of hydropower operation and maintenance across all dimensions, and designs differentiated and rigorous hard-judgment rules for different task types, so that the reward calculation is fully aligned with the real hydropower business logic, greatly improving the adaptability and professionalism of model fine-tuning in vertical industrial scenarios.

[0052] In one embodiment, the reward value corresponding to the confidence level is determined based on the reward function constructed using a logarithmic scoring rule and the correctness judgment result, including: Using the correctness judgment result as the selection criterion, the logarithmic scoring rule branch that matches the correctness judgment result is selected from the reward function as the target branch. In this embodiment, the reward function constructed based on logarithmic scoring is a piecewise function. By judging whether the current operation and maintenance answer is correct, incorrect, or unknown, the corresponding mathematical calculation branch (i.e., the target branch) is selected. For example, the reward function may include three branches: one branch corresponding to a correct operation and maintenance answer, one branch corresponding to an incorrect operation and maintenance answer, and one branch corresponding to an unknown correctness of the operation and maintenance answer.

[0053] In one embodiment, using the correctness judgment result as the selection criterion, the logarithmic scoring rule branch that matches the correctness judgment result is selected from the reward function as the target branch, including: If the correctness judgment result indicates that the operation and maintenance answer is correct, the target branch is determined to be the first branch, and the input of the first branch is the maximum value between the confidence score and the preset minimum value.

[0054] If the correctness judgment result is that the operation and maintenance answer is wrong, the target branch is determined to be the second branch. The input of the second branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value.

[0055] If the correctness judgment result is that the correctness of the operation and maintenance answer is unknown, the target branch is determined to be the third branch. The input of the third branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value.

[0056] For example, the reward function can be represented by formula (2): Formula (2) in, This indicates the answer from the operations and maintenance team. This represents the confidence level generated by the confidence generation model for this operation and maintenance answer. This indicates the result of the correctness judgment of the operation and maintenance answer. Indicates the preset minimum value. This represents the function value output by the reward function. The preset value in formula (1) is 1; in other embodiments, the preset value can be set to other values ​​as needed. The base of the logarithmic function can be e or other values, which are not limited here. The first difference is... The second difference is .

[0057] Considering that the correctness is unknown, we should also include... If the value is 0, then the true accuracy rate is... This will lead to contamination and calibration / alignment failure. Therefore, this solution distinguishes between the second and third branches, even when the correctness is unknown. Instead of setting the value to 0, a low-confidence prompt is triggered. This prompt, along with soft constraints, encourages the model to output low confidence for such scenarios. This has the following effect: if the correctness of the operational answer is unknown, and the model outputs low confidence, even if the answer is inaccurate, it will only incur a small penalty, conforming to the safe semantics of admitting ignorance. If the correctness of the operational answer is unknown, but the model still outputs high confidence, a severe penalty is imposed by the strict penalty mechanism of the logarithmic rule.

[0058] Then, the function value of the target branch is determined based on the confidence level, and the reward value is obtained.

[0059] For example, the confidence score can be substituted into the target branch for calculation, and the resulting function value can be directly used as the reward value. Alternatively, the function value can be used as the initial reward value and further processed to obtain the final reward value of the confidence generation model. Branched logarithmic scoring rules are designed for different correctness judgment results, ensuring the targeted nature of the reward signal. This accurately incentivizes the confidence generation model to output high confidence when correct and low confidence (expressing uncertainty) when incorrect or unknown.

[0060] In this embodiment, numerical pruning is performed by introducing a preset minimum value, which effectively reduces the problem of the logarithmic function being undefined at 0 / 1 during reinforcement learning training and ensures smooth convergence of the training process.

[0061] In one embodiment, the reward value is obtained by determining the function value of the target branch based on the confidence level, including: The function value is used as the initial reward value; the initial reward value is normalized using the preset minimum reward extreme value and the preset maximum reward extreme value to obtain the candidate reward value; the reward value is obtained based on the candidate reward value.

[0062] For example, the method for normalizing the initial reward value can refer to formula (3): Formula (3) in, Indicates the candidate reward value. This represents the initial reward value. This represents the maximum reward extreme value. This represents the minimum reward extreme value.

[0063] For example, the maximum reward extreme value and the minimum reward extreme value can be At that time, the maximum and minimum possible values ​​of the function output by the first branch of the reward function.

[0064] The purpose of normalization is to map initial reward values ​​that originally vary greatly (e.g., from negative numbers to zero) to a fixed standard range, such as [-1, 1].

[0065] In this embodiment, the initial logarithmic score reward is normalized to meet the stability requirements of reward value distribution in mainstream reinforcement learning algorithms (such as proximal policy optimization algorithms), thereby further improving the efficiency of model policy optimization.

[0066] The proof of the proposition is as follows: Let the true probability be... Expected reward Please refer to formula (4): Formula (4) right Taking the first derivative and setting it to 0, we get formula (5): Formula (5) The unique extreme point is found: .

[0067] The second derivative is less than 0, representing the global maximum. That is, the model outputs a confidence score only if the second derivative is less than 0. Equal to true accuracy At this point, the expected reward is maximized, meaning the model achieves perfect calibration.

[0068] The above-mentioned method of obtaining the reward value based on the candidate reward value can be: directly using the candidate reward value as the reward value; or, in one embodiment, in order to meet the principle of prioritizing the safety of hydropower operation and maintenance, a safety compliance reward item is added on the basis of the calibration reward. The above-mentioned method of obtaining the reward value based on the candidate reward value can be: judging whether the operation and maintenance answer complies with the safety procedures, obtaining a compliance judgment result, the compliance judgment result being whether the operation and maintenance answer complies with the safety procedures or not; selecting a safety compliance reward value that matches the compliance judgment result; and performing a weighted summation of the candidate reward value and the safety compliance reward value to obtain the reward value.

[0069] Hydropower operation and maintenance knowledge includes safety procedures. To determine whether an operation and maintenance answer complies with these procedures, a compliance assessment can be obtained by comparing the answer with the relevant safety procedures for the hydropower industry; alternatively, if the answer is classified as an operation and maintenance procedure, the correctness assessment result can be directly used as the compliance assessment result. These safety procedures can be clauses from hydropower operation and maintenance regulations or other clauses related to operation and maintenance safety.

[0070] The security compliance reward value is an auxiliary reward item specifically designed to determine whether the operation and maintenance answers themselves have crossed the security bottom line.

[0071] The above method of weighted summing of candidate reward values ​​and security compliance reward values ​​to obtain the reward value can be referred to formula (6): Formula (6) in, Indicates the calibration reward weight. . Indicates the weight of security rewards. ,and . Indicates the candidate reward value. This represents the reward value for safety and compliance.

[0072] The security compliance reward value is a positive number (e.g., 0.2) if the operation and maintenance answer complies with the security procedures, and a negative number (e.g., -1.0) if the operation and maintenance answer does not comply with the security procedures.

[0073] In this embodiment, a hybrid reward mechanism for safety constraints oriented towards hydropower operation and maintenance characteristics is integrated on top of the theoretical logarithmic scoring rule. This mechanism can further strengthen the model's safety bottom line awareness while calibrating the confidence level, and effectively reduce the generation of high-risk non-compliant operation and maintenance instructions.

[0074] In one embodiment, adjusting the parameters of the confidence generation model based on the reward value includes: calculating the near-end policy optimization loss based on the reward value; and adjusting the parameters of the confidence generation model using the near-end policy optimization loss.

[0075] For example, to prevent the model from crashing due to excessively large policy updates during reinforcement learning fine-tuning, this embodiment uses the PPO (Proximal Policy Optimization) algorithm to perform fine-tuning. For example, based on the reward value obtained in step 103, the advantage function is calculated, and then a proximal policy optimization loss function with a clipping mechanism is constructed.

[0076] Among them, the constructed near-end policy optimization loss Please refer to formula (7): Formula (7) in, This represents the ratio of the probabilities of the new and old strategies. Represents the dominance function. This represents the cutting factor. This represents the loss of the constructed near-end policy optimization.

[0077] in, The calculation method is as follows: Let the current strategy parameter be... (To be updated), Chapter The old policy parameters captured in the snapshot before PPO training are During the episode data sampling phase, using Interact with the environment, generate a token sequence, and record the probability of each step. During the strategy update phase, for the same Using the currently updating Recalculate forward , .

[0078] The dominance function can be determined using the generalized dominance estimation method based on the PPO standard. In the generalized dominance estimation method, the first... The immediate reward for each step is the reward value mentioned above. The state value function is estimated by the value network, and the discount factor takes the values ​​as follows.

[0079] When the ratio of the new and old strategies exceeds a set threshold (such as […]) The network parameters of the confidence generation model are then truncated when the expected calibration error (ECE) converges. Subsequently, the network parameters are precisely adjusted through backpropagation using gradient descent until the expected calibration error (ECE) converges.

[0080] In the above scheme, the advanced PPO proximal policy optimization algorithm is used to perform end-to-end reinforcement learning fine-tuning of large models. This can achieve efficient iteration of confidence policies while ensuring training stability, without relying on manual preference labeling, thus reducing the cost and subjective bias of fine-tuning.

[0081] In one embodiment, the fine-tuning method for the large language model of hydropower operation and maintenance provided by the present invention is executed within a Markov decision process framework, wherein the Markov decision process is defined as a six-tuple (state space... Action space State transition function reward function Hydropower Operation and Maintenance Knowledge Collection and discount factor ); The state space includes the hydropower operation and maintenance query data input to the operation and maintenance answer generation model, the operation and maintenance answers, and the generated confidence word sequence. For example, the state... It consists of hydropower operation and maintenance query data, operation and maintenance answers, and the generated confidence word token sequence, that is, , For user-input hydropower operation and maintenance query data, such as fault descriptions, procedure inquiries, operation instructions, or status assessments, etc. The operation and maintenance answer model has generated operation and maintenance answers, which can be in the form of text. This is the sequence of tokens with generated confidence scores. State The dynamic part only comes from ,and and The purpose of preserving the global condition in the state is to allow the policy to... You can see the complete query and answer at each step when generating confidence tokens.

[0082] The action space includes actions that are the next confidence term selected by the confidence generation model. For example, actions... This is the next confidence token selected by the model. The confidence token can be a confidence value (such as 0, 1, 2…10) or the terminator EOS. The terminator EOS indicates that confidence generation is complete and the current episode ends. The generated confidence token sequence is parsed to generate integer confidence scores for this operational answer. After the episode ends, the integer confidence scores are mapped to [0,1], thus obtaining the confidence score. .

[0083] The state transition function is a deterministic transition function. . Given the current state and action, the next state is uniquely determined.

[0084] The reward function is constructed using a logarithmic scoring rule. See above for details.

[0085] The hydropower operation and maintenance knowledge set includes an equipment fault database, an operation and maintenance procedure database, safety constraint thresholds, and unit operating parameters. The equipment fault database may include standard fault tags for hydropower operation and maintenance; the operation and maintenance procedure database may include hydropower operation and maintenance procedure clauses; the safety constraint thresholds may include safety threshold ranges for the hydropower field; and the unit operating parameters may include threshold ranges for various operating parameters. Both the safety constraint thresholds and the thresholds in the unit operating parameters can serve as standard threshold ranges for the hydropower field.

[0086] The discount factor is used to control the importance of the reward value. That is, To balance immediate calibration rewards with long-term decision reliability, this embodiment takes... =0.99.

[0087] In other words, in the state space, the current state is composed of the user's water and electricity operation and maintenance query data, the fixed operation and maintenance answers, and the gradually generated confidence word sequence.

[0088] In the action space, an action can be any number from "0" to "10" output by the model, or the generation of an end-of-sequence (EOS). Due to the autoregressive nature of large language models, given the current state and action, the next state is uniquely determined; therefore, the state transition function is a deterministic transition with a value of 1. The discount factor is typically a constant close to 1, such as 0.99, to balance the reliability of immediate calibration with the reliability of long-term sequence generation.

[0089] In this embodiment, the problem of generating answers and calibrating confidence in the large-scale hydropower operation and maintenance model is abstracted into a standard fully observed Markov decision process. The scattered reasoning and verification logic is encapsulated in a theoretically complete reinforcement learning fine-tuning framework, laying a solid mathematical model foundation for the final realization of theoretically guaranteed perfect calibration.

[0090] In one embodiment, the fine-tuning method for the large language model of hydropower operation and maintenance may include, for example: Figure 2 The following steps are shown: Step 201: Obtain the operation and maintenance answer generation model output by the operation and maintenance query data.

[0091] Operation and maintenance query data can include descriptions of hydropower station equipment faults, procedural consultation questions, or status assessment instructions.

[0092] Step 202: Use the operation and maintenance answer and transportation query data as input to the confidence generation model to obtain the confidence score for the operation and maintenance answer.

[0093] Step 203: Use hydropower operation and maintenance knowledge to judge the correctness of the operation and maintenance answer and obtain the correctness judgment result.

[0094] Step 204: Using the correctness judgment result as the selection condition, select the logarithmic scoring rule branch that matches the correctness judgment result from the reward function as the target branch; and use the function value of the target branch determined by the confidence level as the initial reward value.

[0095] Step 205: Normalize the initial reward value using the preset minimum reward extreme value and the preset maximum reward extreme value to obtain the candidate reward value.

[0096] Step 206: Select the security compliance reward value that matches the compliance judgment result of the operation and maintenance answer.

[0097] Step 207: The candidate reward values ​​and the security compliance reward values ​​are weighted and summed to obtain the reward value.

[0098] Step 208: Adjust the parameters of the confidence generation model based on the reward value.

[0099] At the theoretical level, this invention is the first to introduce a reinforcement learning method based on logarithmic scoring rules into the fine-tuning of large language models for hydropower operation and maintenance, realizing a mathematically rigorously provable confidence calibration mechanism. This fundamentally constrains the model's overconfidence problem and significantly reduces the risk of decision illusion. At the architectural level, it adopts a training mode that decouples answer generation and confidence optimization, thereby improving the model's confidence expression accuracy and uncertainty perception ability while fully preserving the model's original hydropower operation and maintenance business capabilities and question-answering performance. At the domain level, it deeply integrates professional knowledge from hydropower operation and maintenance procedures, equipment fault databases, and safety constraint thresholds. The knowledge construction domain-specific correctness judgment logic and safety reward mechanism make the model highly adaptable to the high-risk, high-safety, and high-reliability industrial application scenarios of hydropower. At the methodological level, based on the PPO algorithm, end-to-end fine-tuning is carried out directly with calibration reward as the optimization target, without relying on manual preference labeling, and has the advantages of low training cost, high optimization consistency, and strong scenario generalization. At the engineering level, an end-to-end device supporting the entire closed-loop process from training to inference is designed, which can simultaneously output operation and maintenance answers, calibration confidence and risk level, and can be directly embedded into the existing hydropower operation and maintenance business platform, with good engineering implementation and field applicability.

[0100] This invention has achieved significant and stable technical results in the practical application of the large-scale language model for hydropower operation and maintenance: the model's confidence calibration capability has been greatly improved, the expected calibration error (ECE) has been reduced, and it is close to the ideal perfect calibration level; at the same time, the proportion of high-confidence incorrect answers in the model has decreased significantly, thereby reducing high-risk illusion outputs and fully meeting the stringent safety operation requirements of the power industry. While optimizing calibration capabilities, the overall accuracy and basic business performance of the large-scale language model for hydropower operation and maintenance question answering remain stable, basically on par with the original model before fine-tuning, without sacrificing core business capabilities due to reinforcement learning training.

[0101] This invention also possesses outstanding generalization, engineering applicability, and safety compliance: even in small-sample scenarios such as unseen rare faults, boundary conditions, and extreme operating conditions, the model can still maintain stable and reliable confidence expression and calibration effects; the inference phase only adds a very small number of confidence-related tokens with no significant additional computational overhead, which can meet the needs of real-time response and online deployment in hydropower projects; the built-in safety constraint reward mechanism can effectively constrain the model's output behavior, avoid generating unauthorized operation and maintenance instructions that violate procedures and pose safety risks, and comprehensively improve the system's safety and compliance.

[0102] Please see Figure 3 The present invention provides a hydropower operation and maintenance method, which may include the following steps: Step 301: Obtain target hydropower operation and maintenance query data.

[0103] The target hydropower operation and maintenance query data can be the description of hydropower station equipment faults, procedure consultation questions, or status assessment instructions entered by the user.

[0104] Step 302: Process the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance big language model to obtain the target operation and maintenance answer; and generate the target confidence score for the target operation and maintenance answer using the confidence score generation model in the hydropower operation and maintenance big language model.

[0105] Among them, the hydropower operation and maintenance large language model is trained using any of the above-mentioned fine-tuning methods for hydropower operation and maintenance large language models.

[0106] Step 303: Output the target operation and maintenance answer and the target confidence level.

[0107] For example, the output format is "Answer: xxx | Confidence level: c / 10 |". The risk level can also be output as low / medium / high. Furthermore, a manual review reminder can be automatically triggered when the confidence level is less than or equal to 3, thus meeting water and electricity safety regulations.

[0108] For example, a field maintenance engineer inputs "Abnormal alarm for water leakage in the main shaft seal of Unit 3, please provide handling suggestions" into the system as the target hydropower maintenance query data. The hydropower maintenance big language model first calls its maintenance answer generation model to generate detailed emergency repair steps as the target maintenance answer; then, it calls the confidence generation model, which has been fine-tuned by the above reinforcement learning, to calculate a target confidence level (e.g., 8 / 10) based on the text.

[0109] Finally, the system front-end will present the combined output: "Answer: [Processing Suggestion Text] | Confidence: 8 / 10 | Risk Level: Medium". At this point, if the confidence level generated by the model is lower than a specific safety threshold (e.g., ≤3), a manual review reminder pop-up will be automatically triggered on the system front-end to block high-risk automated control commands.

[0110] The above scheme proposes a paradigm for hydropower operation and maintenance engineering that decouples and collaboratively outputs answer generation and confidence level generation. Thanks to rigorous fine-tuning using logarithmic scoring rules introduced in the early stages, the model can accurately output its uncertainty perception during the inference phase. This mechanism, which decouples decision recommendations from calibrated confidence levels and outputs them synchronously, not only supports risk-level management and effectively intercepts high-confidence illusion guidance under small sample or extreme conditions, but also fully meets the stringent safety and compliance requirements of the power system, demonstrating extremely high engineering implementation value.

[0111] The fine-tuning device for the large language model of hydropower operation and maintenance provided by this invention will be described below. The fine-tuning device for the large language model of hydropower operation and maintenance described below can be referred to in correspondence with the fine-tuning method for the large language model of hydropower operation and maintenance described above. Figure 4 As shown, the hydropower operation and maintenance large language model includes an operation and maintenance answer generation model and a confidence level generation model. The fine-tuning device 400 of the hydropower operation and maintenance large language model includes the following modules: The confidence level acquisition module 401 is used to acquire the confidence level generated by the confidence level generation model for the operation and maintenance answer generation model output by the operation and maintenance answer generation model. The operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels. The correctness judgment module 402 is used to judge the correctness of the operation and maintenance answer using hydropower operation and maintenance knowledge, and obtain the correctness judgment result, which is that the operation and maintenance answer is correct, incorrect, or the correctness is unknown. The reward value determination module 403 is used to determine the reward value corresponding to the confidence level based on the reward function constructed using the logarithmic scoring rule and the correctness judgment result; The parameter adjustment module 404 is used to adjust the parameters of the confidence generation model based on the reward value.

[0112] According to the fine-tuning device 400 of the large language model for hydropower operation and maintenance provided by the present invention, the reward value determination module 403 determines the reward value corresponding to the confidence level based on the reward function constructed using a logarithmic scoring rule and the correctness judgment result, including: Using the correctness judgment result as the selection criterion, select the logarithmic scoring rule branch that matches the correctness judgment result from the reward function as the target branch; The reward value is obtained by determining the function value of the target branch based on the confidence level.

[0113] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance includes a reward value determination module 403 that selects a logarithmic scoring rule branch that matches the correctness judgment result from the reward function as the target branch, using the correctness judgment result as the selection condition. If the correctness judgment result is that the operation and maintenance answer is correct, the target branch is determined to be the first branch, and the input of the first branch is the maximum value between the confidence score and the preset minimum value; If the correctness judgment result is that the operation and maintenance answer is wrong, the target branch is determined to be the second branch. The input of the second branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value. If the correctness judgment result is that the correctness of the operation and maintenance answer is unknown, the target branch is determined to be the third branch. The input of the third branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value.

[0114] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance includes a reward value determination module 403 that obtains a reward value based on the function value of the target branch determined by the confidence level, comprising: Use the function value as the initial reward value; The initial reward value is normalized using the preset minimum reward extreme value and the preset maximum reward extreme value to obtain candidate reward values; The reward value is obtained based on the candidate reward value.

[0115] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance includes a reward value determination module 403 that obtains a reward value based on candidate reward values, including: Determine whether the operation and maintenance answer complies with the security procedures to obtain a compliance judgment result. The compliance judgment result is either that the operation and maintenance answer complies with the security procedures or does not comply with the security procedures. Select a security compliance reward value that matches the compliance assessment result; The reward value is obtained by weighted summation of the candidate reward values ​​and the security compliance reward values.

[0116] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance includes a correctness judgment module 402 that uses hydropower operation and maintenance knowledge to judge the correctness of operation and maintenance answers and obtains a correctness judgment result, including: Based on the category of the operation and maintenance answers, determine whether there is corresponding water and electricity operation and maintenance knowledge. If such a relationship exists, the correctness of the operation and maintenance answer is determined based on the relationship between hydropower operation and maintenance knowledge and the operation and maintenance answer. If it does not exist, the result of determining the correctness of the operation and maintenance answer is "correctness unknown".

[0117] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance includes at least one of the following: hydropower operation and maintenance standard fault labels, hydropower operation and maintenance procedure clauses, and standard threshold ranges in the hydropower field. A correctness judgment module 402 determines the correctness judgment result of the operation and maintenance answer based on the relationship between the hydropower operation and maintenance knowledge and the operation and maintenance answer, including: If the category of the operation and maintenance answer is fault diagnosis, obtain the overlap between the operation and maintenance answer and the standard fault label of water and electricity operation and maintenance. If the overlap is greater than or equal to the preset overlap, the operation and maintenance answer is determined to be correct. If the category of the operation and maintenance answer is operation and maintenance, and it is determined that the operation and maintenance answer complies with the provisions of the hydropower operation and maintenance regulations, then the operation and maintenance answer is determined to be correct. If the category of the operation and maintenance answer is parameter analysis, and it is determined that the operation and maintenance answer is within the standard threshold range of the hydropower field, then the operation and maintenance answer is determined to be correct.

[0118] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance includes a parameter adjustment module 404 that adjusts the parameters of a confidence generation model based on reward values, comprising: Calculate the near-end strategy optimization loss based on the reward value; The loss is optimized using a proximal strategy, and the parameters of the confidence generation model are adjusted.

[0119] According to the present invention, a fine-tuning device 400 for a large language model of hydropower operation and maintenance is provided. The device 400 is executed under the framework of Markov decision process, wherein the Markov decision process is defined as a six-tuple, and the six-tuple is (state space, action space, state transition function, reward function, hydropower operation and maintenance knowledge set and discount factor). The state space includes the hydropower operation and maintenance query data, operation and maintenance answers, and the generated confidence word sequence input to the operation and maintenance answer generation model; The action space includes actions that are the next confidence term selected by the confidence generation model; The state transition function is a deterministic transition function; The reward function is constructed using a logarithmic scoring rule; The hydropower operation and maintenance knowledge set includes an equipment fault database, an operation and maintenance procedure database, safety constraint thresholds, and unit operating parameters. Discount factors are used to control the importance of reward values.

[0120] The hydropower operation and maintenance device provided by this invention is described below. The hydropower operation and maintenance device described below can be referred to in correspondence with the hydropower operation and maintenance method described above. For example... Figure 5 As shown, the hydropower operation and maintenance device 500 includes the following modules: The query data acquisition module 501 is used to acquire target hydropower operation and maintenance query data; The operation and maintenance answer and confidence level acquisition module 502 is used to process the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance big language model to obtain the target operation and maintenance answer; and to generate a target confidence level for the target operation and maintenance answer using the confidence level generation model in the hydropower operation and maintenance big language model. The hydropower operation and maintenance big language model is trained using any of the above-mentioned fine-tuning methods of the hydropower operation and maintenance big language model. Output module 503 is used to output the target operation and maintenance answer and the target confidence level.

[0121] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a fine-tuning method for the hydropower operation and maintenance large language model. This method includes: the hydropower operation and maintenance large language model includes an operation and maintenance answer generation model and a confidence generation model; the method includes: obtaining the confidence level generated by the confidence generation model for the operation and maintenance answer output by the operation and maintenance answer generation model, the operation and maintenance answer generation model being trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels; using hydropower operation and maintenance knowledge to judge the correctness of the operation and maintenance answer, obtaining a correctness judgment result, which is the operation and maintenance answer being correct, incorrect, or its correctness unknown; and applying a logarithmic scoring rule. The reward function and the correctness judgment result are used to determine the reward value corresponding to the confidence level; the parameters of the confidence level generation model are adjusted based on the reward value; or a hydropower operation and maintenance method is executed, which includes: obtaining target hydropower operation and maintenance query data; processing the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance large language model to obtain the target operation and maintenance answer; and generating a target confidence level for the target operation and maintenance answer using the confidence level generation model in the hydropower operation and maintenance large language model, which is trained using the above-mentioned fine-tuning method of the hydropower operation and maintenance large language model; and outputting the target operation and maintenance answer and the target confidence level.

[0122] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the fine-tuning method of the hydropower operation and maintenance large language model provided by the above methods. The method includes: the hydropower operation and maintenance large language model includes an operation and maintenance answer generation model and a confidence level generation model; the method includes: obtaining the confidence level generated by the confidence level generation model for the operation and maintenance answer output by the operation and maintenance answer generation model, wherein the operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels; using hydropower operation and maintenance knowledge to judge the correctness of the operation and maintenance answer, and obtaining the correctness judgment result. The accuracy judgment result is either correct, incorrect, or the accuracy is unknown for the operation and maintenance answer; based on the reward function constructed using the logarithmic scoring rule and the accuracy judgment result, the reward value corresponding to the confidence level is determined; the parameters of the confidence level generation model are adjusted based on the reward value; or the hydropower operation and maintenance method is executed, which includes: obtaining target hydropower operation and maintenance query data; processing the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance large language model to obtain the target operation and maintenance answer; and generating a target confidence level for the target operation and maintenance answer using the confidence level generation model in the hydropower operation and maintenance large language model, which is trained using the above-mentioned fine-tuning method of the hydropower operation and maintenance large language model; and outputting the target operation and maintenance answer and the target confidence level.

[0124] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a fine-tuning method for the large language model of hydropower operation and maintenance provided by the above methods. This method includes: the large language model of hydropower operation and maintenance includes an operation and maintenance answer generation model and a confidence level generation model; the method includes: obtaining the confidence level generated by the confidence level generation model for the operation and maintenance answer output by the operation and maintenance answer generation model, wherein the operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels; using hydropower operation and maintenance knowledge to judge the correctness of the operation and maintenance answer, and obtaining a correctness judgment result, wherein the correctness judgment result is that the operation and maintenance answer is correct or incorrect. The error or correctness is unknown; based on the reward function constructed using the logarithmic scoring rule and the correctness judgment result, determine the reward value corresponding to the confidence level; adjust the parameters of the confidence level generation model based on the reward value; or execute the hydropower operation and maintenance method, which includes: obtaining target hydropower operation and maintenance query data; processing the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance large language model to obtain the target operation and maintenance answer; and generating a target confidence level for the target operation and maintenance answer using the confidence level generation model in the hydropower operation and maintenance large language model, which is trained using the above-mentioned fine-tuning method of the hydropower operation and maintenance large language model; and outputting the target operation and maintenance answer and the target confidence level.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fine-tuning a large language model for hydropower operation and maintenance, characterized in that, The large language model for hydropower operation and maintenance includes an operation and maintenance answer generation model and a confidence level generation model. The method includes: Obtain the confidence level generated by the confidence generation model for the operation and maintenance answer output by the operation and maintenance answer generation model, wherein the operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels; The correctness of the operation and maintenance answer is judged using knowledge of hydropower operation and maintenance, and a correctness judgment result is obtained. The correctness judgment result is that the operation and maintenance answer is correct, incorrect, or its correctness is unknown. The reward value corresponding to the confidence level is determined based on the reward function constructed using the logarithmic scoring rule and the correctness judgment result. The parameters of the confidence generation model are adjusted based on the reward value.

2. The method according to claim 1, characterized in that, Based on the reward function constructed using a logarithmic scoring rule and the correctness judgment result, the reward value corresponding to the confidence level is determined, including: Using the correctness judgment result as the selection condition, select the logarithmic scoring rule branch that matches the correctness judgment result from the reward function as the target branch; The reward value is obtained by determining the function value of the target branch based on the confidence level.

3. The method according to claim 2, characterized in that, Using the correctness judgment result as the selection criterion, the logarithmic scoring rule branch that matches the correctness judgment result is selected from the reward function as the target branch, including: If the correctness judgment result is that the operation and maintenance answer is correct, the target branch is determined to be the first branch, and the input of the first branch is the maximum value between the confidence level and the preset minimum value; If the correctness judgment result is that the operation and maintenance answer is wrong, the target branch is determined to be the second branch. The input of the second branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value. If the correctness judgment result is that the correctness of the operation and maintenance answer is unknown, the target branch is determined to be the third branch. The input of the third branch is the maximum value between the first difference and the second difference. The first difference is the difference between the preset value and the confidence level, and the second difference is the difference between the preset value and the preset minimum value.

4. The method according to claim 2, characterized in that, The reward value is obtained by determining the function value of the target branch based on the confidence level, including: Use the function value as the initial reward value; The initial reward value is normalized using the preset minimum reward extreme value and the preset maximum reward extreme value to obtain candidate reward values; The reward value is obtained based on the candidate reward value.

5. The method according to claim 4, characterized in that, The reward value is obtained based on the candidate reward value, including: Determine whether the operation and maintenance answer complies with the security procedures to obtain a compliance judgment result, wherein the compliance judgment result is that the operation and maintenance answer complies with the security procedures or does not comply with the security procedures; Select a security compliance reward value that matches the compliance assessment result; The reward value is obtained by weighted summation of the candidate reward value and the security compliance reward value.

6. The method according to any one of claims 1 to 5, characterized in that, The correctness of the operation and maintenance answers is judged using knowledge of hydropower operation and maintenance, and the correctness judgment results are obtained, including: Based on the category of the operation and maintenance answer, determine whether there is corresponding water and electricity operation and maintenance knowledge. If such a relationship exists, the correctness of the operation and maintenance answer is determined based on the relationship between the hydropower operation and maintenance knowledge and the operation and maintenance answer. If it does not exist, the correctness judgment result of the operation and maintenance answer is determined to be unknown.

7. The method according to claim 6, characterized in that, The hydropower operation and maintenance knowledge includes at least one of the following: standard fault labels for hydropower operation and maintenance, clauses of hydropower operation and maintenance procedures, and standard threshold ranges in the hydropower field. Based on the relationship between the hydropower operation and maintenance knowledge and the operation and maintenance answer, the correctness judgment result of the operation and maintenance answer is determined, including: If the category of the operation and maintenance answer is fault diagnosis, obtain the overlap between the operation and maintenance answer and the standard fault label of water and electricity operation and maintenance, and if the overlap is greater than or equal to the preset overlap, determine that the operation and maintenance answer is correct; If the category of the operation and maintenance answer is operation and maintenance, and it is determined that the operation and maintenance answer conforms to the provisions of the hydropower operation and maintenance regulations, then the operation and maintenance answer is determined to be correct. If the category of the operation and maintenance answer is parameter analysis, and it is determined that the operation and maintenance answer is within the standard threshold range of the hydropower field, then the operation and maintenance answer is determined to be correct.

8. The method according to any one of claims 1 to 5, characterized in that, Based on the reward value, the confidence generation model is adjusted in terms of parameters, including: Calculate the near-end strategy optimization loss based on the reward value; The loss is optimized using the proximal strategy, and the parameters of the confidence generation model are adjusted.

9. The method according to any one of claims 1 to 5, characterized in that, The method is executed within a Markov decision process framework, wherein the Markov decision process is defined as a six-tuple, which consists of (state space, action space, state transition function, reward function, hydropower operation and maintenance knowledge set, and discount factor). The state space includes hydropower operation and maintenance query data input to the operation and maintenance answer generation model, the operation and maintenance answer, and the generated confidence word sequence. The action space includes actions that are the next confidence term selected by the confidence generation model; The state transition function is a deterministic transition function; The reward function is constructed using a logarithmic scoring rule; The hydropower operation and maintenance knowledge set includes an equipment fault database, an operation and maintenance procedure database, safety constraint thresholds, and unit operating parameters. Discount factors are used to control the importance of reward values.

10. A method for the operation and maintenance of hydropower, characterized in that, include: Obtain target hydropower operation and maintenance query data; The target hydropower operation and maintenance query data is processed using the operation and maintenance answer generation model in the hydropower operation and maintenance large language model to obtain the target operation and maintenance answer; and the target confidence score is generated for the target operation and maintenance answer using the confidence score generation model in the hydropower operation and maintenance large language model, wherein the hydropower operation and maintenance large language model is trained using the fine-tuning method of the hydropower operation and maintenance large language model as described in any one of claims 1 to 9; Output the target operation and maintenance answer and the target confidence level.

11. A fine-tuning device for a large language model of hydropower operation and maintenance, characterized in that, The large language model for hydropower operation and maintenance includes an operation and maintenance answer generation model and a confidence level generation model. The device includes: The confidence level acquisition module is used to acquire the confidence level generated by the confidence level generation model for the operation and maintenance answer generation model output by the operation and maintenance answer generation model. The operation and maintenance answer generation model is trained based on hydropower operation and maintenance query data samples and operation and maintenance answer labels. The correctness judgment module is used to use hydropower operation and maintenance knowledge to judge the correctness of the operation and maintenance answer and obtain the correctness judgment result, wherein the correctness judgment result is that the operation and maintenance answer is correct, incorrect, or the correctness is unknown; The reward value determination module is used to determine the reward value corresponding to the confidence level based on the reward function constructed using the logarithmic scoring rule and the correctness judgment result; The parameter adjustment module is used to adjust the parameters of the confidence generation model based on the reward value.

12. A hydropower operation and maintenance device, characterized in that, include: The query data acquisition module is used to acquire target hydropower operation and maintenance query data; The operation and maintenance answer and confidence level acquisition module is used to process the target hydropower operation and maintenance query data using the operation and maintenance answer generation model in the hydropower operation and maintenance large language model to obtain the target operation and maintenance answer; and to generate a target confidence level for the target operation and maintenance answer using the confidence level generation model in the hydropower operation and maintenance large language model, wherein the hydropower operation and maintenance large language model is trained using the fine-tuning method of the hydropower operation and maintenance large language model as described in any one of claims 1 to 9; The output module is used to output the target operation and maintenance answer and the target confidence level.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the fine-tuning method of the large language model for hydropower operation and maintenance as described in any one of claims 1 to 9, or the hydropower operation and maintenance method as described in claim 10.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fine-tuning method of the large language model for hydropower operation and maintenance as described in any one of claims 1 to 9, or the hydropower operation and maintenance method as described in claim 10.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the fine-tuning method of the large language model for hydropower operation and maintenance as described in any one of claims 1 to 9, or the hydropower operation and maintenance method as described in claim 10.

Images