Machine learning operation and maintenance method based on large language model

By applying data preprocessing, advanced retrieval and human-in-loop mechanism of large language models on the MLOps platform, the user's intention understanding and automatic orchestration problems are solved, the operation and maintenance efficiency and intelligence level are improved, resource utilization is optimized and cost is reduced .

CN119003719APending Publication Date: 2024-11-22ZHEJIANG UNIV

Patent Information

Application Number
CN202411029888.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing MLOps platform has shortcomings in user interaction experience, intelligent operation guidance and advanced decision-making assistance, and is particularly difficult to deeply integrate the advanced capabilities of large language models, and cannot effectively understand user intentions, provide accurate help, and automatically orchestrate machine learning assembly lines.

Method used

Using machine learning operation and maintenance methods based on large language models, optimize machine learning operation and maintenance processes through data preprocessing, advanced retrieval strategies, task decomposition and processing capabilities, and human-in-loop mechanisms to achieve user intention understanding, precise help and automatic orchestration. . Specific steps include data cleaning, metadata extraction, document summary, vector storage, shard indexing, retrieval recall, request processing, feedback correction, and integration of processing results through the tool proxy system and the machine learning task proxy system.

Benefits of technology

It significantly improves operation and maintenance efficiency and intelligence level, optimizes the utilization of computing resources, reduces operation and maintenance costs, meets the needs of diversified projects, and opens up a new path for future machine learning operation and maintenance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003719A_ABST
    Figure CN119003719A_ABST
Patent Text Reader

Abstract

The invention discloses a machine learning operation and maintenance method based on a large language model, and the method comprises the steps: carrying out the data preprocessing of various asset documents on a machine learning operation and maintenance platform, including data cleaning, metadata extraction, document summarization, vector storage and fragment indexing; according to the user question, the dialogue history and the context, constructing a large language model and sending a query request to a matched knowledge base for targeted retrieval; the large language model analyzes user questions, identifies user intentions, directly answers simple questions, decomposes complex tasks into sub-questions, routes the sub-questions to the tool agent system or the machine learning task agent system for further processing, integrates processing results and constructs a final response; through a human real-time feedback mechanism, a reasoning process of a tool agent system or a machine learning task agent system is dynamically adjusted, and misjudgment of a large language model is corrected. The operation and maintenance efficiency and the intelligent level are improved by comprehensively optimizing the machine learning operation and maintenance process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning operation and maintenance technology, and in particular to a machine learning operation and maintenance method based on a large language model. Background Art

[0002] In the tide of the information age, the field of algorithm development and operation and maintenance is undergoing profound changes. With the exponential growth of data volume, the increasing complexity of system architecture, and the rapid iteration of business needs, the traditional operation and maintenance model faces severe challenges. Algorithm scientists and technical teams are particularly difficult in model deployment, resource scheduling, and service monitoring. This is not only due to the low efficiency of operation and maintenance due to insufficient software engineering knowledge, but also because the scarcity and high cost of high-performance computing resources, especially GPU resources, have severely restricted the speed of R&D and innovation capabilities.

[0003] In response to these pain points, MLOps (machine learning operation and maintenance) came into being and became a revolution in the field of information technology operation and maintenance. MLOps integrates big data processing, machine learning and automation technology, aiming to subvert the traditional operation and maintenance model, improve operation and maintenance efficiency, reduce operating costs, and ensure the high stability and reliability of the system.

[0004] Based on cloud native, virtualization and container technologies, the MLOps algorithm platform integrates leading open source tools and self-developed components to build a comprehensive and efficient algorithm development lifecycle management system. Through a dynamically scalable, automatically load-balanced microservice architecture, the platform provides a fully automated or semi-automated process from data processing, algorithm development to model deployment and service monitoring, which is particularly suitable for scenarios that pursue efficient algorithm implementation.

[0005] MLOps is suitable for algorithm projects of different sizes. For small-scale development, the platform simplifies the entire process from data processing to service monitoring; in large-scale team collaboration, it greatly reduces the management burden through automated training, packaging, and deployment functions. In addition, the platform's distributed training and elastic resource management mechanisms ensure the efficient use of key resources such as GPUs, avoiding idle resources and waste.

[0006] Although the concept and technical framework of MLOps have achieved initial results, there are still obvious shortcomings in practical applications, especially in user interaction experience, intelligent operation guidance, and advanced decision support. In particular, with the rapid development of large language model (LLM) technology, its outstanding performance in natural language processing, complex logical reasoning, and personalized response has provided unlimited imagination space for the further intelligent upgrade of the MLOps platform.

[0007] How to deeply integrate the advanced capabilities of LLM with the existing MLOps platform to create an intelligent assistant that can understand user intentions, provide accurate assistance, and even predict and automatically orchestrate machine learning pipelines has become a key issue that needs to be explored. Therefore, a machine learning operation and maintenance method based on a large language model is proposed to solve the above problems, which has important practical significance and broad application prospects. Summary of the invention

[0008] The present invention provides a machine learning operation and maintenance method based on a large language model. By comprehensively optimizing the machine learning operation and maintenance process, the operation and maintenance efficiency and intelligence level are improved, and a solution based on a large language model is provided for the field of machine learning operation and maintenance.

[0009] A machine learning operation and maintenance method based on a large language model includes the following steps:

[0010] In the preprocessing phase, data preprocessing is performed on various asset documents on the machine learning operation and maintenance platform, including data cleaning, metadata extraction, document summarization, vector storage, and shard indexing. Among them, various asset documents include dataset documents, model documents, and component documents.

[0011] In the retrieval and recall phase, based on user questions, conversation history, and context, the large language model builds and sends query requests to the matching knowledge base for targeted retrieval;

[0012] During the request processing phase, the large language model analyzes user questions, identifies user intent, directly answers simple questions, and decomposes complex tasks into sub-questions. The sub-questions are then routed to the tool agent system or machine learning task agent system for further processing, and the processing results are integrated to construct the final response.

[0013] During the feedback correction phase, the reasoning process of the tool agent system or the machine learning task agent system is dynamically adjusted through the real-time human feedback mechanism to correct the erroneous judgments of the large language model.

[0014] Furthermore, in the metadata extraction process, the document content is used as input, the metadata recognized and output by the large language model is formatted through a predefined template, and converted into structured information in JSON format and written into the platform asset configuration file; these steps are encapsulated into a call chain to realize the automated extraction and management of platform asset documents.

[0015] The document summary uses a recursive summary method to generate the expected summary content. The specific steps include:

[0016] The document is sharded according to the size of 512 tokens; the large language model reads each shard in sequence, combines the summary up to the previous shard with the original content of the current shard, and generates a summary up to the current shard to ensure the coherence and completeness of the summary content.

[0017] In the preprocessing stage, the small-to-big strategy is used to vectorize and index documents. The specific steps include:

[0018] Each type of asset document is sharded according to the size of 128, 256, 512, and 1024 tokens. Small shards of 128, 256, and 512 tokens are encoded to generate embedding vectors, which are stored in the vector database as indexes of the corresponding large shards of 1024 tokens. When the embedding vector of a small shard is retrieved in the vector database, the content of the large shard to which the small shard belongs is actually recalled to ensure the integrity of the recalled content and the retrieval efficiency.

[0019] In the retrieval and recall phase, the large language model builds and sends query requests to the matching knowledge base for targeted retrieval, including the following steps:

[0020] For asset detail questions, search in the vector database that stores asset document shards. Specifically: for questions routed to the vector database for retrieval, encode the retrieval question into an embedded vector, use cosine similarity to measure the distance between vectors, and apply the knn algorithm to search for the closest k shard vectors in the vector database; set a similarity threshold to filter low-similarity shards; recall the large shards corresponding to the filtered small shards. If multiple small shards correspond to the same large shard, the large shard is only recalled once; use the GTE-rerank rearrangement model to calculate the matching degree of all recalled shards and questions, and rearrange them according to the matching degree; add necessary meta information to the recalled shards, including the source document name and path of the shard, the associated asset name and asset type, and the task type;

[0021] For asset overview questions, the platform asset configuration file is searched. Specifically: For questions routed to the platform asset configuration file, the large language model calls the query API, controls the query scope and content output method through API parameters, and uses the API execution results as the recalled content.

[0022] During the request processing phase, the tool agent system uses a thought chain mechanism to perform routine tasks, including chart drawing and script writing. Through preset goals and callable tools, the large language model gradually infers until the final answer is obtained.

[0023] The machine learning task agent system summarizes user needs based on user questions and conversation history, and generates customized machine learning workflows.

[0024] The machine learning task agent system adopts the tool chain strategy to execute tasks according to the preset tool call sequence to achieve orderly processing of complex tasks;

[0025] When selecting components, the machine learning task agent system adopts the Plan-Action strategy, that is, the large language model first formulates an overall plan, and then gradually infers according to the plan steps to determine the optimal components for each link, so as to improve the accuracy and efficiency of component selection.

[0026] The machine learning task agent system adopts a recall-refinement strategy when selecting data sets, models, and components. The specific steps include: screening a preliminary set of asset candidates in the platform asset configuration file through metadata information; and selecting the most matching assets from the candidate set through a refinement algorithm to ensure the accuracy and efficiency of the selection.

[0027] The refined sorting algorithm uses an iterative screening mechanism to control the number of options for each selection of the large language model to no more than five, thereby reducing the inference cost of the large language model and improving the accuracy of the selection. The specific details of the refined sorting algorithm are as follows:

[0028] In the initialization phase, a queue is created and all elements of the initial list are placed in the queue. All elements refer to all data sets, models, and components, that is, various asset documents. A subset list is prepared to temporarily store some results. The queue is processed in a loop until the queue is empty. At the beginning of each loop, the length of the current queue is calculated. If there is only one element left in the queue, it is used as the final result and the loop ends. Otherwise, a threshold is calculated based on the length of the current queue to ensure that the number of subsets processed each time is not greater than 4. Each element in the current queue is traversed and placed in the subset list one by one. When the subset list reaches the set threshold or has been processed to the end of the queue, the large language model selects the optimal element from the subset list and adds it back to the queue to become the input for the next round of processing. When there is only one element left in the queue, it is returned as the final selection result. When selecting a data set, the result is the data set; when selecting a model, the result is the model; when selecting a component, the result is the component.

[0029] In the feedback correction phase, the reasoning process of the tool agent system or machine learning task agent system is dynamically adjusted through the real-time feedback mechanism, including the following steps:

[0030] Based on the classic thinking chain mechanism, the tool agent system incorporates the human-in-the-loop mechanism to improve the accuracy and flexibility of large language model decision-making. Specifically, in the original three steps of thinking, tool calling, and tool calling results of the large model (the classic React Agent mechanism in the field of large models), a human feedback link is embedded. After the thinking link and the tool calling result link, human experts are invited to review and give feedback to ensure the rationality and accuracy of the decision. Among them, after the thinking link, the tool agent system will present the reasoning process and expected actions of the large model to the user. After the tool calling result link, the tool agent system will present the execution result of the current action. The user confirms the decision by pressing Enter, clears the human feedback area, and advances to the next reasoning step. If the user enters n or N, the rollback mechanism is triggered, and the system will go back to the end point of the previous reasoning and restart this round of reasoning. If the user provides additional guidance or corrections, these feedbacks will be recorded and placed at the head of the prompt word of the current round of reasoning of the large model, giving it priority, and guiding the large model to reason in a direction that better meets user expectations.

[0031] Based on the toolchain strategy, the machine learning task agent system incorporates the human-in-the-loop mechanism to enhance the controllability and flexibility of long toolchain reasoning. Specifically, after each sub-toolchain completes reasoning, the system will present the reasoning result and invite the user to confirm or provide feedback. If the user confirms the decision by pressing Enter, it means that the reasoning result of the current sub-toolchain is recognized, and the system will continue to execute the next sub-toolchain to advance the overall task progress. If the user enters n or N, the rollback mechanism is triggered, and the system will backtrack to the end point of the reasoning of the previous sub-toolchain and restart the reasoning of the current sub-toolchain to ensure the correctness of the decision path and user satisfaction. If the user also provides additional guidance or corrective opinions, these feedbacks will be recorded and placed at the head of the prompt word of the current sub-toolchain, giving it priority, and guiding the toolchain to reason in a direction that is more in line with user expectations.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] This invention applies the large language model to the field of machine learning operation and maintenance. By integrating data preprocessing technology, advanced search strategies, task decomposition and processing capabilities, and human-in-the-loop mechanisms, the advanced capabilities of the large language model are deeply integrated with the existing MLOps platform, bringing a solution based on the large language model to the field of machine learning operation and maintenance. It can not only understand user intentions, provide accurate help, and even predict and automatically arrange machine learning pipelines, significantly improving operation and maintenance efficiency and intelligence, but also optimizing the use of computing resources, reducing operation and maintenance costs, meeting the needs of diverse projects, and opening up a new path for future machine learning operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1This is a flow chart of a machine learning operation and maintenance method based on a large language model according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of a large language model recursively summarizing a document in an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the "plan-action" strategy adopted by the machine learning task agent system in an embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of the "recall-fine sorting" strategy adopted by the machine learning task agent system in an embodiment of the present invention;

[0038] Figure 5 A flowchart of a tool agent system integrating a human-in-the-loop mechanism in an embodiment of the present invention;

[0039] Figure 6 This is a flow chart of integrating a human-in-the-loop mechanism into a machine learning task agent system in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0041] In view of the problem that traditional machine learning operation and maintenance platforms lack in-depth human-computer interaction and intelligent operation guidance, and cannot efficiently generate workflows and assist decision-making, an embodiment of the present invention provides a machine learning operation and maintenance method based on a large language model. By integrating data preprocessing technology, advanced retrieval strategies, task decomposition and processing capabilities, and a human-in-the-loop mechanism, the present invention aims to comprehensively optimize the machine learning operation and maintenance process, improve operation and maintenance efficiency and intelligence, and provide a solution based on a large language model for the field of machine learning operation and maintenance.

[0042] like Figure 1 As shown in the figure, a machine learning operation and maintenance method based on a large language model mainly includes the following steps:

[0043] Step 1: Perform data preprocessing on various assets (including but not limited to data sets, models, etc.) on the machine learning operation and maintenance platform, involving data cleaning, metadata extraction, document summarization, vector storage, and shard indexing.

[0044] Step 2: Based on user questions, conversation history, and context, a large model is constructed and a query request is sent to the relevant knowledge base to achieve accurate information retrieval.

[0045] Step 3: The large model analyzes user questions, identifies user intent, directly answers simple questions, breaks down complex tasks into sub-questions, and routes them to the tool agent system or machine learning task agent system for further processing, integrates the processing results, and constructs the final response.

[0046] Step 4: Through the real-time human feedback mechanism, dynamically adjust the reasoning process of the tool agent system or the machine learning task agent system, correct the erroneous judgments of the large model, integrate human experience, and continuously optimize the task performance of the large model.

[0047] In step 1, this embodiment adopts the "small-to-big" strategy to vectorize and index the text. The specific steps include:

[0048] The platform asset documents (such as datasets, models, etc.) are sharded according to the size of 128, 256, 512, and 1024 tokens; the small shards of size 128, 256, and 512 tokens are encoded to generate embedding vectors, which are stored in the vector database as indexes of the corresponding large shards of size 1024 tokens; when the embedding vector of a small shard is retrieved in the vector database, what is actually recalled is the content of the large shard to which the small shard belongs, so as to ensure the integrity of the recalled content and the retrieval efficiency.

[0049] In the metadata extraction process, the document content is used as input, the metadata recognized and output by the large model is formatted through a predefined template, and converted into structured information in JSON format and written into the platform asset configuration file; these steps are encapsulated into a call chain to achieve automated extraction and management of platform asset documents.

[0050] like Figure 2 As shown, in the embodiment, the document summary uses a recursive summarization method to generate the expected summary content, and the specific steps include:

[0051] The document is sharded according to the size of 512 tokens; the large model reads each shard in sequence, combines the summary up to the previous shard with the original content of the current shard, and generates a summary up to the current shard to ensure the coherence and completeness of the summary content.

[0052] Assume that F(D) represents the summary of document D by the large model, and split document D into n blocks, namely D1, D2, ...Dn; when summarizing D2, the previous summary (i.e. the summary of D1, denoted as O1) is also passed in, then: O2 = F(O1 + D2) = F(F(D1) + D2).

[0053] Finally, when summarizing the last block Dn, we have:

[0054] R n =F(O n-1+D n )R n =F(F(O n-2 +D n-1 )+D n )R n

[0055] =F(F(F(O n-3 +D n-2 )+D n-1 )+D n )...R n

[0056] =F(...F(F(F(F(O0+D1)+D2)+D3)...+D n-1 )+D n ) The document summary instructions for large models should follow the following paradigm:

[0057] """

[0058] You are an AI expert! The following content is from {title}: including the above summary and the current segment.

[0059] Based on the above summary and the current segment, please summarize {expected_description} in one sentence.

[0060] If the {expected_description} is already summarized in the above summary, please make full use of it!

[0061] hint:

[0062] {tips}

[0063] Summary of the above:

[0064] {summing_up}

[0065] Current content:

[0066] {content}

[0067] Please summarize {expected_description} in one sentence!

[0068] """

[0069] in,

[0070] expected_description indicates the expected summary content; tips is optional content, indicating tips to help the large model summarize. At the same time, we stipulate in the instruction: if the above summary already contains the expected content, the above summary should also be fully considered when summarizing the current shard content; this alleviates the situation where the effective information of the head shard is lost as the recursive depth increases.

[0071] In step 2, the retrieval and recall phase, the retrieval agent system uses advanced retrieval strategies to retrieve and recall content related to the user's question from the platform knowledge base. The specific steps include:

[0072] The big model reconstructs the user's question based on the conversation history and context. In the embodiment, if the user's question is directly used to retrieve relevant content in the knowledge base, the following problems may occur, affecting the final search effect: typos, grammatical errors, follow-up questions with missing context, etc. Therefore, the big model is used to reconstruct the user's question based on the conversation history and context.

[0073] In the embodiment, the machine learning operation and maintenance platform mainly includes two types of knowledge bases: a model and dataset document knowledge base, which contains the model and dataset overview, intended use, usage and other specific content, which has been stored in the vector database through the "small-to-big" sharding indexing strategy in step one; a platform configuration file knowledge base, which stores the basic information of all assets (datasets, models, components) of the machine learning operation and maintenance platform in json format.

[0074] In the embodiment, the big model selects the most matching knowledge base for targeted retrieval based on the question type and user intent; for questions about asset details, it searches in the vector database that stores asset document fragments; for questions about asset overviews, it searches in the platform asset configuration file.

[0075] More specifically, for questions routed to retrieval in the vector database, the retrieval questions are encoded into embedded vectors, the distance between vectors is measured using cosine similarity, and the knn algorithm is applied to search for the k closest shard vectors in the vector database; a similarity threshold is set to filter out low-similarity shards; the large shards corresponding to the filtered small shards are recalled, and if multiple small shards correspond to the same large shard, the large shard is only recalled once; the GTE-rerank rearrangement model is used to calculate the matching degree of all recalled shards with the questions, and they are rearranged according to the matching degree; necessary meta-information is added to the recalled shards, including the source document name and path of the shard, the associated asset name and asset type, task type, etc.

[0076] For issues routed to the platform asset configuration file, the big model calls the query API, controls the query scope (asset type, task type, etc.) and content output method (detailed, brief, etc.) through API parameters, and uses the API execution results as the recall content.

[0077] Step 3, the request processing stage, in addition to being able to directly answer the user's simple questions, the embodiment also has the ability to handle complex questions; the specific steps include: the big model decomposes and reconstructs the complex question into sub-questions; the big model routes the sub-questions to the corresponding proxy system for completion according to the sub-question type; the big model integrates the processing results of the proxy system, constructs the final response and returns it to the user.

[0078] The agent system includes a tool agent system and a machine learning agent system, where:

[0079] The tool agent system adopts the thinking chain (React) mechanism and is responsible for performing routine tasks such as chart drawing and script writing. Through preset goals and callable tools, the large model gradually reasons until the final answer is obtained.

[0080] The thought chain instructions of the tool agent system should follow the following paradigm:

[0081] """

[0082] Please answer the following questions to the best of your ability. You can use the following tools:

[0083] {tool_descs}

[0084] Please use the following format:

[0085] Question: The question you must answer

[0086] Think: You should always think about what you want to do

[0087] Action: The action to take, should be one of [{tool_names}]

[0088] Input: Input for action

[0089] Observation: The results of action

[0090] ...(this thought / action / input / observation can be repeated zero or many times)

[0091] Thinking: Now I know the final answer

[0092] Final answer: Final answer to the original question

[0093] start!

[0094] Question: {query}

[0095] """

[0096] Among them, tool_descs represents a tool description list, which is generally composed of `tool name: tool description: parameter description`; tool_names represents a tool name list; query represents a user request.

[0097] The machine learning task agent system summarizes user needs based on user questions and conversation history, and generates customized machine learning workflows.

[0098] More specifically, the machine learning task agent system adopts a tool chain (Chain) strategy to execute tasks according to the preset tool call sequence to achieve orderly processing of complex tasks; compared with the thinking chain (React), the Chain strategy emphasizes the order and consistency of task execution.

[0099] The machine learning task toolchain is composed of multiple sub-toolchains, namely: task identification toolchain, dataset selection toolchain, model selection toolchain, plan making toolchain, and component selection toolchain. Among them: the task identification toolchain is responsible for identifying the machine learning task type according to user needs and outputting it in json format; the dataset selection toolchain and the model selection toolchain are responsible for selecting the most matching dataset and model according to user needs and the identified machine learning task type, and outputting it in json format. The two sub-toolchains are executed in parallel; the plan making toolchain, as the pre-module of the component selection toolchain, is responsible for formulating macro-planning steps for subsequent component selection according to user needs and the obtained task type, model, and dataset, and outputting it in json format; the component selection toolchain is responsible for inputting user needs and the obtained task type, model, dataset, plan itself, and the selected components of the pre-steps into the large model in turn according to the order of the plan steps, and outputting the most matching components for the current step.

[0100] like Figure 3 As shown in the figure, the machine learning task agent system adopts the "Plan-Action" strategy when selecting components, that is, the big model first formulates the overall plan, and then gradually infers according to the plan steps to determine the optimal components for each link, so as to improve the accuracy and efficiency of component selection.

[0101] Compared with letting the big model directly output all the components of the workflow, the "Plan->Action" strategy allows the big model to infer the global optimal plan from a more macro perspective; then break it down into each step based on the plan: when selecting the optimal component for the current step, by providing the global plan and the selected components of the previous step in the prompt words, the component selection range of the current step can be constrained, thereby improving the accuracy of the big model in selecting components; while ensuring the optimal plan, the local optimality of the steps can be gradually achieved, so that the machine learning workflow generated by the big model is as close to the optimal solution as possible within the optional range.

[0102] like Figure 4 As shown in the figure, the machine learning task agent system adopts the "recall-fine sorting" strategy when selecting data sets, models, and components. The specific steps include:

[0103] Before selecting data sets, models, and components, first read the machine learning operation and maintenance platform configuration file of the corresponding asset type; filter the preliminary asset candidate set in the platform asset configuration file through metadata information; and then use the refined sorting algorithm to select the most matching asset from the candidate set to ensure the accuracy and efficiency of the selection.

[0104] More specifically, the fine sorting algorithm uses an iterative screening mechanism to control the selection of the large model to no more than five items each time, thereby reducing the inference cost of the large model and improving the accuracy of the selection. The code logic of the fine sorting algorithm is as follows:

[0105] In the initialization phase, create a queue and put all the elements of the initial list into the queue; prepare a subset list to temporarily store some results; process the queue in a loop until the queue is empty; at the beginning of each loop, calculate the length of the current queue. If there is only one element left in the queue, use it as the final result and end the loop; otherwise, calculate a threshold based on the length of the current queue to ensure that the number of subsets processed each time is no more than 4. Traverse each element in the current queue and put them into the subset list one by one. When the subset list reaches the set threshold or has been processed to the end of the queue, the large model selects the optimal element from the subset list and adds it back to the queue to become the input for the next round of processing; when there is only one element left in the queue, return it as the final selection result.

[0106] In step 4, the present invention incorporates a "human in the loop" mechanism to make the intermediate reasoning process of the big model visible to the user, and the user can provide real-time feedback on the intermediate reasoning process of the big model, thereby improving the accuracy and flexibility of the big model's decision-making.

[0107] like Figure 5 As shown in the figure, the tool agent system incorporates the "human-in-the-loop" mechanism based on the classic thinking chain (React) mechanism. The specific steps include:

[0108] The Human Feedback link is embedded in the original three steps of Thought, Action (tool call), and Observation (tool call result): after the Thought and Observation links, human experts are invited to review and provide feedback to ensure the rationality and accuracy of the decision; after the Thought link, the tool agent system will present the reasoning process and expected actions of the large model to the user; after the Observation link, the tool agent system will present the execution results of the current action.

[0109] The user can confirm the decision by pressing Enter, clear the Human Feedback area, and advance to the next reasoning step. If the user enters 'n' or 'N', the rollback mechanism is triggered, and the system will go back to the end point of the previous reasoning and restart the current round of reasoning. If the user provides additional guidance or corrections, these feedbacks will be recorded and placed at the head of the prompt word of the current round of reasoning of the large model, giving it priority, and guiding the large model to reason in a direction that better meets user expectations.

[0110] like Figure 6 As shown, in the embodiment, the machine learning task agent system incorporates a "human-in-the-loop" mechanism based on the tool chain strategy, and the specific steps include:

[0111] After each sub-toolchain completes reasoning, the system will present the reasoning results and invite the user to confirm or provide feedback; if the user confirms the decision by pressing Enter, it means that the reasoning results of the current sub-toolchain are recognized, and the system will continue to execute the next sub-toolchain to advance the overall task progress; if the user enters 'n' or 'N', the rollback mechanism will be triggered, and the system will backtrack to the end point of reasoning of the previous sub-toolchain and restart the reasoning of the current sub-toolchain to ensure the correctness of the decision path and user satisfaction; if the user also provides additional guidance or corrective opinions, these feedbacks will be recorded and placed at the head of the prompt word of the current sub-toolchain, giving it priority, and guiding the toolchain to reason in a direction that is more in line with user expectations.

[0112] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A machine learning operation and maintenance method based on a large language model, characterized in that: The following steps are involved: In the preprocessing phase, data preprocessing is performed on various asset documents on the machine learning operation and maintenance platform, including data cleaning, metadata extraction, document summarization, vector storage, and shard indexing. Among them, various asset documents include dataset documents, model documents, and component documents. In the retrieval and recall phase, based on user questions, conversation history, and context, the large language model builds and sends query requests to the matching knowledge base for targeted retrieval; During the request processing phase, the large language model analyzes user questions, identifies user intent, directly answers simple questions, and decomposes complex tasks into sub-questions. The sub-questions are then routed to the tool agent system or machine learning task agent system for further processing, and the processing results are integrated to construct the final response. During the feedback correction phase, the reasoning process of the tool agent system or the machine learning task agent system is dynamically adjusted through the real-time human feedback mechanism to correct the erroneous judgments of the large language model.

2. The machine learning operation and maintenance method based on a large language model according to claim 1, characterized in that: In the metadata extraction process, the document content is used as input, the metadata recognized and output by the large language model is formatted through a predefined template, and converted into structured information in JSON format and written into the platform asset configuration file; These steps are encapsulated into a call chain to achieve automated extraction and management of platform asset documents.

3. The machine learning operation and maintenance method based on a large language model according to claim 1, characterized in that: The document summary uses a recursive summary method to generate the expected summary content. The specific steps include: The document is sharded according to the size of 512 tokens; the large language model reads each shard in sequence, combines the summary up to the previous shard with the original content of the current shard, and generates a summary up to the current shard to ensure the coherence and completeness of the summary content.

4. The machine learning operation and maintenance method based on a large language model according to claim 1, characterized in that: In the preprocessing stage, the small-to-big strategy is used to vectorize and index documents. The specific steps include: Each type of asset document is sharded according to the size of 128, 256, 512, and 1024 tokens. Small shards of 128, 256, and 512 tokens are encoded to generate embedding vectors, which are stored in the vector database as indexes of the corresponding large shards of 1024 tokens. When the embedding vector of a small shard is retrieved in the vector database, the content of the large shard to which the small shard belongs is actually recalled to ensure the integrity of the recalled content and the retrieval efficiency.

5. The machine learning operation and maintenance method based on a large language model according to claim 4, characterized in that: In the retrieval and recall phase, the large language model builds and sends query requests to the matching knowledge base for targeted retrieval, including the following steps: For asset detail questions, search in the vector database that stores asset document shards. Specifically: for questions routed to the vector database for retrieval, encode the retrieval question into an embedded vector, use cosine similarity to measure the distance between vectors, and apply the knn algorithm to search for the closest k shard vectors in the vector database; set a similarity threshold to filter low-similarity shards; recall the large shards corresponding to the filtered small shards. If multiple small shards correspond to the same large shard, the large shard is only recalled once; use the GTE-rerank rearrangement model to calculate the matching degree of all recalled shards and questions, and rearrange them according to the matching degree; add necessary meta information to the recalled shards, including the source document name and path of the shard, the associated asset name and asset type, and the task type; For asset overview questions, the platform asset configuration file is searched. Specifically: For questions routed to the platform asset configuration file, the large language model calls the query API, controls the query scope and content output method through API parameters, and uses the API execution results as the recalled content.

6. The machine learning operation and maintenance method based on a large language model according to claim 1, characterized in that: During the request processing phase, the tool agent system uses a thought chain mechanism to perform routine tasks, including chart drawing and script writing. Through preset goals and callable tools, the large language model gradually infers until the final answer is obtained. The machine learning task agent system summarizes user needs based on user questions and conversation history, and generates customized machine learning workflows.

7. The machine learning operation and maintenance method based on a large language model according to claim 6, characterized in that: The machine learning task agent system adopts the tool chain strategy to execute tasks according to the preset tool call sequence to achieve orderly processing of complex tasks; When selecting components, the machine learning task agent system adopts the Plan-Action strategy, that is, the large language model first formulates an overall plan, and then gradually infers according to the plan steps to determine the optimal components for each link, so as to improve the accuracy and efficiency of component selection.

8. The machine learning operation and maintenance method based on a large language model according to claim 7, characterized in that: The machine learning task agent system adopts a recall-refinement strategy when selecting data sets, models, and components. The specific steps include: screening a preliminary set of asset candidates in the platform asset configuration file through metadata information; and selecting the most matching assets from the candidate set through a refinement algorithm to ensure the accuracy and efficiency of the selection.

9. The machine learning operation and maintenance method based on a large language model according to claim 8, characterized in that: The refined sorting algorithm uses an iterative screening mechanism to control the number of options for each selection of the large language model to no more than five, thereby reducing the inference cost of the large language model and improving the accuracy of the selection. The specific details of the refined sorting algorithm are as follows: In the initialization phase, a queue is created and all elements of the initial list are placed into the queue. All elements refer to all datasets, models, and components, that is, various asset documents. Prepare a subset list to temporarily store some results; The queue is processed in a loop until it is empty. At the beginning of each loop, the length of the current queue is calculated. If there is only one element left in the queue, it is used as the final result and the loop ends. Otherwise, a threshold is calculated based on the length of the current queue to ensure that the number of subsets processed each time is no more than 4. Each element in the current queue is traversed and placed in the subset list one by one. When the subset list reaches the set threshold or has been processed to the end of the queue, the large language model selects the optimal element from the subset list and adds it back to the queue to become the input for the next round of processing. When there is only one element left in the queue, it is returned as the final selection result.

10. The machine learning operation and maintenance method based on a large language model according to claim 7, characterized in that: In the feedback correction phase, the reasoning process of the tool agent system or machine learning task agent system is dynamically adjusted through the real-time feedback mechanism, including the following steps: Based on the classic thinking chain mechanism, the tool agent system incorporates the human-in-the-loop mechanism to improve the accuracy and flexibility of the large language model's decision-making. Specifically, in the original three steps of thinking, tool calling, and tool calling results of the large model, a human feedback link is embedded; after the thinking link and the tool calling result link, human experts are invited to review and give feedback to ensure the rationality and accuracy of the decision; among them, after the thinking link, the tool agent system will present the reasoning process and expected actions of the large model to the user; after the tool calling result link, the tool agent system will present the execution result of the current action; the user confirms the decision by pressing Enter, clears the human feedback area, and advances to the next reasoning step; if the user enters n or N, the rollback mechanism is triggered, and the system will go back to the end point of the previous reasoning and restart the current round of reasoning; if the user provides additional guidance or corrections, these feedbacks will be recorded and placed at the head of the prompt word of the current round of reasoning of the large model, giving it priority, and guiding the large model to reason in a direction that better meets the user's expectations; Based on the toolchain strategy, the machine learning task agent system incorporates the human-in-the-loop mechanism to enhance the controllability and flexibility of long toolchain reasoning. Specifically, after each sub-toolchain completes reasoning, the system will present the reasoning result and invite the user to confirm or provide feedback. If the user confirms the decision by pressing Enter, it means that the reasoning result of the current sub-toolchain is recognized, and the system will continue to execute the next sub-toolchain to advance the overall task progress. If the user enters n or N, the rollback mechanism is triggered, and the system will backtrack to the end point of the reasoning of the previous sub-toolchain and restart the reasoning of the current sub-toolchain to ensure the correctness of the decision path and user satisfaction. If the user also provides additional guidance or corrective opinions, these feedbacks will be recorded and placed at the head of the prompt word of the current sub-toolchain, giving it priority, and guiding the toolchain to reason in a direction that is more in line with user expectations.

Citation Information

Patent Citations

  • Answerer recommendation method based on topic model and machine learning

    CN107562836A

  • Intelligent data analysis method and device based on large language model

    CN117555986A

  • Search question-answering system and method based on large model and electronic equipment

    CN117708274A

  • Personalized traffic report dynamic generation method and device based on AI intelligent agent and medium

    CN117785965A

  • Knowledge base question-answering system based on large language model

    CN118193697A

Cited By

  • Method for improving private domain operation and maintenance knowledge retrieval quality based on RAG

    CN120277201A

  • Retrieval iteration method and device by means of generative large language model

    CN120448489A

  • Method for guiding generation of nuclear power maintenance regulation document based on multilevel thinking chain

    CN121351992A

  • Large language model back-end implementation system based on reasoning service

    CN121457543A

  • A large language model backend implementation system based on inference service

    CN121457543B