Maintenance plan generation system, method and program product based on large language model
Through a multi-level agent system based on large language model, combined with reinforcement learning, the problems of information overload and poor adaptability in IETM are solved, and independent learning and highly adaptable maintenance plan generation are achieved, which significantly improves the accuracy of the maintenance plan and Adaptability.
Patent Information
- Application Number
- CN202411287838.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2025-01-17
- Estimated Expiration
- 2044-09-14
AI Technical Summary
The existing interactive electronic technical manual (IETM) has problems such as information overload, redundancy and slow update during the maintenance process, and it is difficult to adapt to complex environments and new tasks, and cannot meet the needs of intelligent maintenance.
Using a multi-level agent system based on large language models, through task allocators, related calculators, trainers, search policy networks and target search policy networks, combined with reinforcement learning, the ability to independently learn and adapt to the new environment can be achieved based on search Adjustments are made to the environment.
It realizes efficient maintenance plan generation, can independently plan and adapt in complex environments, provides efficient and dynamic interactive experience, has zero sample learning ability, adapts to new environments and new tasks, and significantly improves the accuracy of maintenance plan.
Smart Images

Figure CN118798859B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a maintenance plan generation system, method and program product based on a large language model, belonging to the technical field of artificial intelligence. Background Art
[0002] With the development of intelligent equipment, the complexity of equipment and working conditions continues to increase, and the workload of front-line maintenance personnel has become heavier, so the demand for electronic maintenance solutions in production activities is increasing. Traditional interactive electronic technical manuals (IETMs) are becoming increasingly inefficient and outdated. For example, on-site maintenance personnel provide a lot of experience maintenance information support during the maintenance process, but the information is usually overloaded, redundant, and updated slowly. In addition, when there are few sample maintenance needs, accurate retrieval in IETM requires high professionalism of maintenance personnel. Therefore, the existing IETM is not convenient for efficient management and customization of maintenance information.
[0003] To overcome the above problems, a Chinese invention patent application with publication number CN117522372A discloses a maintenance suggestion generation method and system for an automobile fault model based on deep learning, the method comprising: obtaining automobile fault description data, maintenance manual and expert database, performing preprocessing to obtain dialogue data about automobile fault questions and answers, and formatting the dialogue data to obtain formatted dialogue data; using automobile fault description, maintenance manual and expert database as nodes through relationship mapping, connecting the nodes to form a knowledge graph; performing multi-task fine-tuning training in vertical fields; obtaining automobile fault description input by the user, querying the knowledge graph to obtain professional knowledge as a prompt for the automobile fault description, and locating the automobile fault, while generating maintenance suggestions, presenting fault location results and maintenance suggestions; the system comprises: a data acquisition and processing module, a knowledge graph formation module, a model fine-tuning training module and a fault result output module; and realizing automobile fault location and maintenance suggestion generation.
[0004] However, the above methods and systems cannot learn autonomously, have poor ability to adapt to new environments and new tasks, lack the ability to plan autonomously, cannot adapt to the complex environment of the real retrieval environment, and cannot make corresponding adjustments based on changes and feedback in the retrieval environment, which makes it difficult to meet users' needs for intelligent maintenance and repair. Summary of the invention
[0005] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a maintenance plan generation system, method and computer program product based on a large language model, which can learn autonomously and has the ability to adapt to new environments and new tasks; has the ability of autonomous planning; can adapt to the complex environment in the real retrieval environment; and can make corresponding adjustments based on changes and feedback in the retrieval environment.
[0006] To achieve the above-mentioned purpose, the present invention provides a maintenance plan generation system based on a large language model, which includes a multi-level agent, wherein the first-level agent includes a task allocator, a correlation calculator, a first comparator and a trainer, wherein the task allocator is used to allocate a maintenance problem to each of the K second-level agents, the correlation calculator is used to calculate the correlation between the maintenance plan retrieved by each second-level agent and the allocated maintenance problem, the first comparator is used to compare the correlation with a threshold value, and provide the maintenance plan with a correlation greater than or equal to the threshold value to the trainer; the trainer calculates a first time variance target value using a set of maintenance plans retrieved by the second-level agents with a correlation greater than or equal to the threshold value, a set of retrieval strategies predicted by a target retrieval strategy network and a reward set, and broadcasts it to the K second-level agents; the second-level agent A k At least includes a retrieval strategy network and a target retrieval strategy network, wherein the retrieval strategy network is expressed as: , is the parameter of the retrieval strategy network; the target retrieval strategy network is expressed as: , is the parameter of the target detection strategy network; the retrieval strategy network is configured to search the database according to the maintenance problem assigned by the task allocator to generate a maintenance plan for the current time t of the maintenance problem , and the retrieval strategy at the current time t , Maintenance plan ,award Report to the first-level agent; the target retrieval strategy network makes a prediction to obtain the target retrieval strategy for the next time t+1 ), and the target search strategy , the maintenance plan retrieved at the next time t+1 Report to the first-level agent, k=1,2,…,K.
[0007] To achieve the above-mentioned object of the invention, the present invention also provides a maintenance plan generation method based on a large language model, which comprises the following steps:
[0008] Step 1: Assign maintenance problems to each of the K second-level agents through the task allocator of the first-level agent;
[0009] Step 2: Calculate the relevance between the maintenance solution retrieved by each secondary agent and the assigned maintenance problem through the relevance calculator of the primary agent; compare the relevance with the threshold through the first comparator, and provide the dimension modification solution with the relevance greater than or equal to the threshold to the trainer;
[0010] Step 3: The first-time variance target value is calculated by the trainer of the first-level agent using the maintenance solution set retrieved by the second-level agent with a relevance greater than or equal to the threshold, the target retrieval strategy network prediction retrieval strategy set and the reward set, and the first-time variance target value is broadcasted to K second-level agents;
[0011] Step 4: Through the secondary agent A k Search strategy network According to the maintenance problems assigned by the task allocator, the maintenance plan for the maintenance problem is generated by searching the database. , and the search strategy , Inspection Rewards Report to the first-level agent, k=1,2,…,K;
[0012] Step 5: Through the secondary agent A k Target retrieval strategy network Make a prediction to get the target retrieval strategy for the next time t+1 ), and the target search strategy , the maintenance plan retrieved at the next time t+1 Report to the first-level agent, Parameters of the target retrieval policy network, k=1,2,…,K.
[0013] To achieve the above object, the present invention further provides a computer program product, which includes computer program code, and the computer program code can be called by a processor to execute the above method.
[0014] Compared with the prior art, the maintenance plan generation system, method and computer program product based on a large language model provided by the present invention obtain maintenance plans for assigned maintenance problems by combining a large language model with reinforcement learning, learn through the interaction between the intelligent agent and the environment, do not need to rely on manually labeled data, and have the characteristics of autonomous learning, so that it can adapt to new environments and new tasks; can consider rewards to make current decisions, and has the ability of autonomous planning; can adapt to complex environments in real retrieval environments; can make corresponding adjustments according to changes and feedback in the retrieval environment, and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 It is a block diagram of the composition of the maintenance plan generation system based on the large language model provided by the first embodiment of the present invention.
[0016] Figure 2 It is a block diagram of the composition of the retrieval strategy network provided by the first embodiment of the present invention. DETAILED DESCRIPTION
[0017] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0018] First embodiment
[0019] Figure 1 is a block diagram of a maintenance plan generation system based on a large language model provided by a first embodiment of the present invention, such as Figure 1 As shown, the maintenance plan generation system based on the large language model provided by the first embodiment of the present invention includes a multi-level agent, wherein the first-level agent includes a task allocator, a correlation calculator, a first comparator and a trainer, wherein the task allocator is used to allocate a maintenance problem to each of the K second-level agents, the correlation calculator is used to calculate the correlation between the maintenance plan retrieved by each second-level agent and the allocated maintenance problem, the first comparator is used to compare the correlation with a threshold value, and provide the maintenance plan with a correlation greater than or equal to the threshold value to the trainer; the trainer calculates the first time variance target value using the maintenance plan set retrieved by the second-level agent with a correlation greater than or equal to the threshold value, the retrieval strategy set predicted by the target retrieval strategy network and the reward set, and broadcasts it to the K second-level agents; the second-level agent A k At least includes a retrieval strategy network and a target retrieval strategy network, wherein the retrieval strategy network is expressed as: , is the parameter of the retrieval strategy network; the target retrieval strategy network is expressed as: , is the parameter of the target detection strategy network; the retrieval strategy network is configured to search the database according to the maintenance problem assigned by the task allocator to generate a maintenance plan for the current time t of the maintenance problem , and the retrieval strategy at the current time t , Maintenance plan ,award Report to the first-level agent; the target retrieval strategy network makes a prediction to obtain the target retrieval strategy for the next time t+1 ), and the target search strategy , the maintenance plan retrieved at the next time t+1 Report to the first-level agent, k=1,2,…,K.
[0020] In the maintenance plan generation system based on the large language model provided in the first embodiment, the trainer includes a first value network, a first target value network and a first error calculator, wherein the action value function of the first value network is , is the set of maintenance solutions retrieved by K secondary agents at the current time t Each maintenance plan in is relevance calculated with the assigned maintenance problem using the relevance calculator. The set of maintenance plans retrieved with relevance greater than or equal to the threshold has the corresponding retrieval strategy set: , the corresponding reward set is: , are the parameters of the first value network, A is a secondary agent whose relevance is greater than or equal to the threshold m The current time t is based on the retrieval strategy The repair solutions retrieved, is the corresponding reward, m=1,…,M, M is a positive integer less than or equal to K; the action value function of the first target value network is , is the parameter of the action value function of the first target value network; the training process of the trainer includes:
[0021] The first value network makes a prediction to get the first value: = ; The first target value network predicts the first time variance target value :
[0022] ,
[0023] In the formula, a target retrieval strategy set predicted by the target retrieval strategy network for the retrieved maintenance solutions whose correlation with the assigned maintenance problem is greater than or equal to a threshold; ,γ is the adjustment coefficient; A is a secondary agent whose relevance is greater than or equal to the threshold m The next time t+1 is based on the target retrieval strategy Retrieved maintenance solutions;
[0024] The first error calculator is based on Calculate the first error and use the first error Update the first value network parameters for ,
[0025] In the formula, α is the adjustment coefficient, is the gradient function of w1;
[0026] First value network parameters Using first value network parameters Update the first target value network parameters for : , where is the adjustment factor.
[0027] In the maintenance plan generation system based on the large language model provided in the first embodiment, the second-level agent A k It also includes a second value network, a second target value network, a second comparator and a second error calculator. The action value function of the second value network is: , For the second-level agent A k The parameters of the second value network, the action value function of the second target value network is , For the second-level agent A k The second objective value network parameters, where
[0028] The second value network makes a prediction to obtain the secondary agent A k The second value of: ;
[0029] The second target value network makes a prediction to obtain the secondary agent A k Second time variance target value: ,
[0030] The second comparator is used to compare the first time variance target value and the second time variance target value, and provide the smaller time variance target value to the second error calculator; the second error calculator is based on the following formula: Calculate the second error δ2 and use the second error δ2 to update the parameters of the second value network for :
[0031] ,
[0032] In the formula, For The gradient function of
[0033] Second Value Network Utilization Update the parameters of the second target value network for :
[0034] .
[0035] In the maintenance plan generation system based on the large language model provided in the first embodiment, the second-level agent A k Let the retrieval strategy network make predictions to get the predicted retrieval strategy: ;
[0036] Update the retrieval strategy network parameters by for :
[0037] ,
[0038] In the formula, for The gradient function of for The gradient function of is the adjustment factor;
[0039] Search strategy network utilization Update the parameters of the target policy network for :
[0040] , γ is the adjustment coefficient.
[0041] Figure 2 is a block diagram of a search strategy network provided by the first embodiment of the present invention, such as Figure 2 As shown, the retrieval strategy network includes a low-rank adapter and an LLM model, wherein the low-rank adapter is configured to optimize the maintenance problem assigned by the task allocator and provide an LLM model, and the LLM model is configured to retrieve from the database according to the optimization task provided by the low-rank adapter to generate a maintenance plan for the task. The target retrieval strategy network has the same structure and will not be described again here. The low-rank adapter is, for example, a LoRA model.
[0042] The present invention retrieves maintenance solutions through multi-level agents, and can understand and execute instructions that are more complex than direct commands, thereby providing users with a more efficient and dynamic interactive experience. The maintenance solution that the first-level agent needs to retrieve is decomposed into multiple subtasks, and is retrieved by multiple two-dimensional agents respectively. The retrieved maintenance solutions are summarized by the first-level agent, and the value judgment is made on the retrieval results. The first time variance target value is distributed to multiple agents, and multiple agents compare their own second time variance target value with the first time variance target value, and take the smaller time variance target value to update the retrieval strategy and value network, so that the large language model can be trained better. At the same time, the technical solution of the first embodiment of the present invention can obtain the maintenance solution for the assigned maintenance problem, learn through the interaction between the agent and the environment, do not need to rely on manually labeled data, have the characteristics of autonomous learning, so that it can adapt to new environments and new tasks; can consider rewards to make current decisions, have the ability of autonomous planning; can adapt to the complex environment in the real retrieval environment; can make corresponding adjustments according to the changes and feedback of the retrieval environment, and has a strong beneficial effect of good adaptability.
[0043] The technical solution provided by the present invention significantly improves the accuracy of maintenance plan generation, and demonstrates the significant advantage of zero-sample learning capability when dealing with new scenarios that have never been encountered before. Even when faced with completely new maintenance problems, effective maintenance plans can be quickly generated.
[0044] Second embodiment
[0045] The second embodiment of the present invention provides a maintenance plan generation method based on a large language model, which includes the following steps:
[0046] Step 1: Assign maintenance problems to each of the K second-level agents through the task allocator of the first-level agent;
[0047] Step 2: Calculate the relevance between the maintenance solution retrieved by each secondary agent and the assigned maintenance problem through the relevance calculator of the primary agent; compare the relevance with the threshold through the first comparator, and provide the dimension modification solution with the relevance greater than or equal to the threshold to the trainer;
[0048] Step 3: The first-time variance target value is calculated by the trainer of the first-level agent using the maintenance solution set retrieved by the second-level agent with a relevance greater than or equal to the threshold, the target retrieval strategy network prediction retrieval strategy set and the reward set, and the first-time variance target value is broadcasted to K second-level agents;
[0049] Step 4: Through the secondary agent A k Search strategy network According to the maintenance problems assigned by the task allocator, the maintenance plan for the maintenance problem is generated by searching the database. , and the search strategy , Inspection Rewards Report to the first-level agent, k=1,2,…,K;
[0050] Step 5: Through the secondary agent A k Target retrieval strategy network Make a prediction to get the target retrieval strategy for the next time t+1 ), and the target search strategy , the maintenance plan retrieved at the next time t+1 Report to the first-level agent, Parameters of the target retrieval policy network, k=1,2,…,K.
[0051] The maintenance plan generation method based on the large language model provided in the second embodiment of the present invention also includes:
[0052] Step 6: Use the first value network of the trainer to make predictions and get the first value: = ,
[0053] In the formula, There are K secondary agents A k The maintenance plan set retrieved at the current time t Each maintenance plan in the above example uses a correlation calculator to perform correlation calculations with the assigned maintenance problem. The set of maintenance plans retrieved with correlations greater than or equal to the threshold has the following corresponding retrieval strategy set: , the corresponding reward set is: , are the parameters of the first value network, A is a secondary agent whose relevance is greater than or equal to the threshold m The current time t is based on the retrieval strategy The repair solutions retrieved, is the corresponding reward, m=1,…,M, M is a positive integer less than or equal to K;
[0054] Step 7: Use the first target value network of the trainer to make predictions to get the first time variance target value :
[0055] ,
[0056] In the formula, The target retrieval strategy set predicted by the target retrieval strategy network for the repair solutions retrieved whose correlation with the assigned repair problems is greater than or equal to the threshold, is the parameter of the first target value network; ; γ is the adjustment coefficient, A is a secondary agent whose relevance is greater than or equal to the threshold m The next time t+1 is based on the target retrieval strategy Retrieved maintenance solutions;
[0057] Step 8: Use the first error calculator of the trainer according to Calculate the first error δ1 and use the first error δ1 to update the parameters of the first value network for ,
[0058] In the formula, α is the adjustment coefficient, is the gradient function of w1;
[0059] Step 9: The first value network uses the updated parameters of the first value network Update the parameters of the first target value network for : .
[0060] The maintenance plan generation method based on the large language model provided in the second embodiment of the present invention also includes:
[0061] Step 10: Utilize Secondary Agent A k The second value network is used to predict the secondary agent A k The second value of: ;
[0062] Step 11: Using Secondary Agent A k The second target value network predicts the secondary agent A k Second time variance target value:
[0063] ;
[0064] Step 12: Utilize Secondary Agent A k The second comparator compares the first time variance target value and the second time variance target value, and provides the smaller time variance target value to the second error calculator; the second error calculator calculates the time variance target value according to the following formula: Calculate the second error δ2 and use the second error δ2 to update the parameters of the second value network for :
[0065] ,
[0066] In the formula, For The gradient function of
[0067] Step 13: Utilizing Secondary Agent A k The second value network benefits Update the parameters of the second target value network for :
[0068] .
[0069] The maintenance plan generation method based on the large language model provided in the second embodiment of the present invention also includes:
[0070] Step 14: Utilizing Secondary Agent A k The retrieval strategy network makes a prediction to obtain the retrieval strategy predicted at the current time t: ;
[0071] Step 15: Secondary Agent A k Update the retrieval strategy network parameters by for :
[0072] ,
[0073] In the formula, for The gradient function of for The gradient function of is the adjustment factor;
[0074] Step 16: Utilizing Secondary Agent A k Search strategy network utilization Update the parameters of the target policy network for :
[0075] , γ is the adjustment coefficient.
[0076] The beneficial effects of the second embodiment of the present invention are the same as those of the first embodiment, and will not be described again here.
[0077] Third embodiment
[0078] A third embodiment of the present invention provides a computer program product, which includes computer program codes. The computer program codes can be called by a processor to execute the method described in the second embodiment.
[0079] The first step in this invention is to extract relevant maintenance data from the maintenance process of the equipment. The data includes detailed historical maintenance records, maintenance procedure documents, and other relevant technical information. By comprehensively collecting these data, we ensure that we have a rich and accurate data foundation in the process of model training and optimization.
[0080] The collected maintenance data needs to be systematically organized to fit the data input format requirements of the Large Language Model (LLM). This process includes cleaning the data and converting it into a standardized machine-readable format.
[0081] The sorted data is divided into a training set and a test set. The training set is used to train the model, while the test set is used to evaluate the performance of the model. During the data division process, it is necessary to ensure that the data is representative so that the model can fully learn the semantic logical relationship between various maintenance objects and maintenance solutions.
[0082] Based on the specific needs and data characteristics of the maintenance field, the selection of the model needs to comprehensively consider the language, coverage and model architecture of the pre-training dataset to ensure that the selected LLM can effectively support the generation of maintenance plans and has sufficient scalability to adapt to the maintenance problems of different objects.
[0083] After determining the LLM, the pre-trained large language model (LLM) is fine-tuned through the mixed data proportional LoRA technology. LoRA optimizes the low-rank adaptation layer so that some weights are continuously adjusted by minimizing the loss function, thereby aligning to the vertical field.
[0084] The model performance is comprehensively evaluated through formulas, language models, and manual evaluation to ensure that it can be applied in actual maintenance scenarios. The test data includes known maintenance objects / schemes and unknown maintenance objects to fully test the generalization ability of LLM-R under normal sample and few-sample data input.
[0085] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. The meaning of "several" is one or more, unless otherwise clearly and specifically defined.
[0086] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A maintenance plan generation system based on a large language model, characterized in that: The invention comprises a multi-level intelligent agent, wherein the first-level intelligent agent comprises a task allocator, a correlation calculator, a first comparator and a trainer, wherein the task allocator is used to allocate a maintenance problem to each of the K second-level intelligent agents, the correlation calculator is used to calculate the correlation between the maintenance solution retrieved by each second-level intelligent agent and the allocated maintenance problem, the first comparator is used to compare the correlation with a threshold value, and provide the maintenance solution with a correlation greater than or equal to the threshold value to the trainer; the trainer calculates a first time variance target value using a set of maintenance solutions retrieved by the second-level intelligent agent with a correlation greater than or equal to the threshold value, a set of retrieval strategies predicted by a target retrieval strategy network and a reward set, and broadcasts the result to the K second-level intelligent agents; the second-level intelligent agent A k At least includes a retrieval strategy network and a target retrieval strategy network, wherein the retrieval strategy network is expressed as: , is the parameter of the retrieval strategy network; the target retrieval strategy network is expressed as: , is the parameter of the target detection strategy network; the retrieval strategy network is configured to search the database according to the maintenance problem assigned by the task allocator to generate a maintenance plan for the current time t of the maintenance problem , and the retrieval strategy at the current time t , Maintenance plan ,award Report to the first-level agent; the target retrieval strategy network makes a prediction to obtain the target retrieval strategy for the next time t+1 ), and the target search strategy , the maintenance plan retrieved at the next time t+1 Report to the first-level agent, k=1,2,…,K; The trainer includes a first value network, a first target value network and a first error calculator, wherein the action value function of the first value network is , is the set of maintenance solutions retrieved by K secondary agents at the current time t Each maintenance plan in is used to perform correlation calculation with the assigned maintenance problem using the correlation calculator. The set of maintenance plans with correlation greater than or equal to the threshold has the corresponding retrieval strategy set as follows: , the corresponding reward set is: , are the parameters of the first value network, A is a secondary agent whose relevance is greater than or equal to the threshold m The current time t is based on the retrieval strategy The repair solutions retrieved, is the corresponding reward, m=1,…,M, M is a positive integer less than or equal to K; the action value function of the first target value network is , is the parameter of the action value function of the first target value network; the training process of the trainer includes: The first value network makes a prediction to get the first value: = ; The first target value network predicts the first time variance target value : , In the formula, a target retrieval strategy set predicted by the target retrieval strategy network for the retrieved maintenance solutions whose correlation with the assigned maintenance problem is greater than or equal to a threshold; ,γ is the adjustment coefficient; A is a secondary agent whose relevance is greater than or equal to the threshold m The next time t+1 is based on the target retrieval strategy Retrieved maintenance solutions; The first error calculator is based on Calculate the first error and use the first error Update the first value network parameters for , In the formula, α is the adjustment coefficient, is the gradient function of w1; First Value Network Utilization Update the first target value network parameters for : , where is the adjustment factor.
2. The maintenance plan generation system based on a large language model according to claim 1 is characterized in that: Secondary Agent A k It also includes a second value network, a second target value network, a second comparator and a second error calculator. The action value function of the second value network is: , For the second-level agent A k The parameters of the second value network, the action value function of the second target value network is , For the second-level agent A k The second objective value network parameters, where The second value network makes a prediction to obtain the secondary agent A k The second value of: ; The second target value network makes a prediction to obtain the secondary agent A k Second time variance target value: , The second comparator is used to compare the first time variance target value and the second time variance target value, and provide the smaller time variance target value to the second error calculator; the second error calculator is based on the following formula: Calculate the second error δ2 and use the second error δ2 to update the parameters of the second value network for : , In the formula, For The gradient function of Second Value Network Utilization Update the parameters of the second target value network for : 。 3. The maintenance plan generation system based on a large language model according to claim 2 is characterized in that: Secondary Agent A k Let the retrieval strategy network make predictions to get the predicted retrieval strategy: ; Update the retrieval strategy network parameters by for : , In the formula, for The gradient function of for The gradient function of is the adjustment factor; Search strategy network utilization Update the parameters of the target policy network for : 。 4. The maintenance plan generation system based on a large language model according to claim 3 is characterized in that: Both the retrieval strategy network and the target retrieval strategy network include a low-rank adapter and an LLM model, wherein the low-rank adapter is configured to optimize the maintenance problem assigned by the task allocator and provide an LLM model, and the LLM model is configured to retrieve from the database according to the optimization task provided by the low-rank adapter to generate a maintenance plan for the task.
5. A maintenance plan generation method based on a large language model, characterized in that: The steps include: Step 1: Assign maintenance problems to each of the K second-level agents through the task allocator of the first-level agent; Step 2: Calculate the relevance between the maintenance solution retrieved by each secondary agent and the assigned maintenance problem through the relevance calculator of the primary agent; compare the relevance with the threshold through the first comparator, and provide the maintenance solution with a relevance greater than or equal to the threshold to the trainer; Step 3: The first-time variance target value is calculated by the trainer of the first-level agent using the maintenance solution set retrieved by the second-level agent with a relevance greater than or equal to the threshold, the target retrieval strategy network prediction retrieval strategy set and the reward set, and the first-time variance target value is broadcasted to K second-level agents; Step 4: Through the secondary agent A k Search strategy network According to the maintenance problems assigned by the task allocator, the maintenance plan for the maintenance problem is generated by searching the database. , and the search strategy , Inspection Rewards Report to the first-level agent, k=1,2,…,K; Step 5: Through the secondary agent A k Target retrieval strategy network Make a prediction to get the target retrieval strategy for the next time t+1 ), and the target search strategy , the maintenance plan retrieved at the next time t+1 Report to the first-level agent, Parameters of the target retrieval strategy network, k=1,2,…,K; Step 6: Use the first value network of the trainer to make predictions and get the first value: = , In the formula, There are K secondary agents A k The maintenance plan set retrieved at the current time t Each maintenance plan in the is used to perform correlation calculation with the assigned maintenance problem using the correlation calculator. The set of maintenance plans with correlation greater than or equal to the threshold has the corresponding retrieval strategy set as follows: , the corresponding reward set is: , are the parameters of the first value network, A is a secondary agent whose relevance is greater than or equal to the threshold m The current time t is based on the retrieval strategy The repair solutions retrieved, is the corresponding reward, m=1,…,M, M is a positive integer less than or equal to K; Step 7: Use the first target value network of the trainer to make predictions to get the first time variance target value : , In the formula, The target retrieval strategy set predicted by the target retrieval strategy network for the repair solutions retrieved whose correlation with the assigned repair problems is greater than or equal to the threshold, is the parameter of the first target value network; ; γ is the adjustment coefficient, A is a secondary agent whose relevance is greater than or equal to the threshold m The next time t+1 is based on the target retrieval strategy Retrieved maintenance solutions; Step 8: Use the first error calculator of the trainer according to Calculate the first error δ1 and use the first error δ1 to update the parameters of the first value network for , In the formula, a is the adjustment coefficient, is the gradient function of w1; Step 9: The first value network uses the updated parameters of the first value network Update the parameters of the first target value network for : , γ is the adjustment coefficient.
6. The maintenance plan generation method based on a large language model according to claim 5 is characterized in that: Also includes: Step 10: Utilize Secondary Agent A k The second value network is used to predict the secondary agent A k The second value of: ; Step 11: Using Secondary Agent A k The second target value network predicts the secondary agent A k Second time variance target value: ; Step 12: Utilize Secondary Agent A k The second comparator compares the first time variance target value and the second time variance target value, and provides the smaller time variance target value to the second error calculator; The second error calculator is based on the following formula: Calculate the second error δ2 and use the second error δ2 to update the parameters of the second value network for : , In the formula, For The gradient function of , α is the adjustment coefficient; Step 13: Utilizing Secondary Agent A k The second value network parameter Update the parameters of the second target value network for : 。 7. The maintenance plan generation method based on a large language model according to claim 6 is characterized in that: Also includes: Step 14: Utilizing Secondary Agent A k The retrieval strategy network makes a prediction to obtain the retrieval strategy predicted at the current time t: ; Step 15: Secondary Agent A k Update the retrieval strategy network parameters by for : , In the formula, for The gradient function of for The gradient function of is the adjustment factor; Step 16: Utilizing Secondary Agent A k Search strategy network utilization Update the parameters of the target policy network for : 。 8. A computer program product, characterized in that The method comprises a computer program code, and the computer program code can be called by a processor to execute the method according to any one of claims 5 to 7.
Citation Information
Patent Citations
Maintenance suggestion generation method and system of automobile fault model based on deep learning
CN117522372A
Fault diagnosis model self-learning method based on asynchronous parallel reinforcement learning
CN112801272A
Method for constructing equipment fault diagnosis and maintenance knowledge base based on large language model
CN116861189A