Reasoning optimization method and device for large model and medium

Abstract

The invention discloses a reasoning optimization method and device for a large model and a medium, and the method comprises the steps: determining an execution sequence of a plurality of to-be-processed tasks corresponding to to-be-processed cue words inputted by a user, and inputting the to-be-processed tasks into a plurality of task tools according to the execution sequence; determining a multi-dimensional score corresponding to the to-be-processed task in the plurality of task tools, and determining a target tool corresponding to the to-be-processed task and an optimal tool parameter corresponding to the target tool according to the multi-dimensional score, so as to fill the optimal tool parameter into the target tool; based on a pre-constructed knowledge graph, retrieving associated historical data of the cue word to be processed, and inputting the historical data as context information into the parameter-filled target tool; and based on a cue word distillation algorithm, extracting a core feature vector of a cue word to be processed, and dynamically adjusting a parameter weight in a task tool so as to realize reasoning optimization for a large model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an inference optimization method, device and medium for large models. Background Art

[0002] At present, since the landmark paper "Attention is all you need" came out in 2017, the field of Natural Language Processing (NLP) has entered a new era. This paper not only revolutionized people's understanding of sequence-to-sequence learning tasks, but also directly led to the rise of the Transformer architecture, laying a solid theoretical foundation for the leap of NLP technology. At the same time, the significant enhancement of computing power of companies around the world, especially the rapid development of cloud computing, high-performance computing and GPU technology, has provided strong hardware support for training and deploying large-scale language models. The deep integration of the two, like a dual-wheel drive, has accelerated the advancement of NLP technology from theoretical research to practical application, ushering in the golden age of natural language processing technology.

[0003] In this context, the GLM large language model series came into being and quickly became the focus of the industry. From the initial exploratory attempts to the mature application of the fourth-generation model, the GLM series has not only achieved an order of magnitude leap in the scale of model parameters, but more importantly, has made significant progress in multiple dimensions such as text understanding depth, language generation quality, and semantic reasoning accuracy. These advances are not only reflected in the improvement of academic evaluation indicators, but also in their ability to more effectively solve complex language processing problems in the real world, such as intelligent customer service, automatic summarization, machine translation, etc., greatly broadening the application boundaries of NLP technology.

[0004] As a successful example of the open source model, the Tongyi Qianwen Big Model has attracted the attention of developers around the world through open source code and community collaboration. Community members have carried out secondary development and optimization based on Tongyi Qianwen, which not only accelerated the iterative upgrade of model performance, but also promoted technological innovation and knowledge sharing, demonstrating the huge potential of the open source ecosystem in promoting technological progress. The Lingxi Big Model is a model of the deep integration of AIGC technology and ICT infrastructure. It provides highly customized solutions for specific industry needs, such as intelligent operation and maintenance, business process automation, etc. Through multi-level customization and modular design, it can flexibly respond to complex and changing business scenarios, effectively improving the operational efficiency and intelligence level of enterprises.

[0005] However, as the capabilities of large language models continue to increase, their application in the field of task planning and processing is also facing new challenges. First, faced with a wide range of tool libraries, how to efficiently and accurately select the most suitable tool for the current task has become an urgent problem to be solved. This requires the establishment of a comprehensive evaluation mechanism that comprehensively considers multi-dimensional information such as the tool's functions, performance, and user feedback to ensure the objectivity and efficiency of tool selection. Secondly, after the tool is selected, the precise configuration of parameters is also critical. Current technologies mostly rely on manual or simple rules, which are difficult to adapt to the needs of complex tasks, resulting in unreasonable parameter configuration and affecting system performance and operating efficiency. Furthermore, the effective use of historical data is also a major problem. Traditional methods have limitations in data retrieval, context matching, and understanding of complex problems, which limits the value of historical data in assisting decision-making. Finally, the contradiction between the complexity of rule description and reasoning efficiency has become increasingly prominent. How to optimize the rule description and improve the prediction accuracy and response speed of the model has become the key to improving the overall system performance. Summary of the invention

[0006] The embodiments of the present application provide a method, device and medium for reasoning optimization of large models to solve the above-mentioned technical problems.

[0007] On the one hand, the embodiment of the present application provides a method for optimizing reasoning for a large model, including: Determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user, so as to input the pending tasks into multiple task tools according to the execution order; Determine the multi-dimensional scores corresponding to the task to be processed in multiple task tools respectively, and determine the target tool corresponding to the task to be processed and the optimal tool parameters corresponding to the target tool according to the multi-dimensional scores, so as to fill the optimal tool parameters into the target tool; Based on the pre-built knowledge graph, retrieve the associated historical data of the prompt word to be processed, and input the historical data as context information into the target tool after parameter filling; Based on the prompt word distillation algorithm, the core feature vector of the prompt word to be processed is extracted, and the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for the large model.

[0008] In one implementation of the present application, determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user specifically includes: Receive a question input by a user, and determine a corresponding prompt word to be processed according to the question; Decomposing the prompt words to be processed to obtain a plurality of tasks to be processed, and determining dependencies between the plurality of tasks to be processed; According to the dependency relationship, an execution order among the multiple tasks to be processed is determined; wherein the execution order includes parallel execution and sequential execution.

[0009] In one implementation of the present application, determining the multi-dimensional scores corresponding to the to-be-processed tasks in the plurality of task tools respectively includes: Receiving the reasoning results of the multiple task tools for the tasks to be processed by using multi-threaded parallel processing technology; For each task tool, according to the task description of the task to be processed and the tool parameters of the task tool, and based on the thought chain technology, the complexity between the task to be processed and the task tool is calculated; In combination with the complexity, the multi-dimensional score corresponding to the task to be processed on the task tool is determined, and based on the complexity, the tool parameters of the task tool and the historical task execution data, the score threshold is dynamically adjusted to obtain the corresponding target score threshold.

[0010] In one implementation of the present application, determining the target tool corresponding to the task to be processed and the optimal tool parameters corresponding to the target tool according to the multi-dimensional score, so as to fill the optimal tool parameters into the target tool, specifically includes: Compare the multi-dimensional scores corresponding to the multiple task tools with the corresponding target score thresholds respectively, and determine the target tool corresponding to the task to be processed according to the comparison results; Based on a causal reasoning method, determining optimal tool parameters corresponding to the target tool, inputting the optimal tool parameters into the target tool, and obtaining an output result of the target tool; The output result is compared with the preset expected result, and the filled optimal tool parameters are corrected according to the erroneous output result.

[0011] In one implementation of the present application, before retrieving the associated historical data of the prompt word to be processed based on the pre-built knowledge graph, the method further includes: The structured information corresponding to the prompt word to be processed is used as a graph structure to construct a corresponding knowledge graph, and the node ID and description information corresponding to the prompt word to be processed are converted into a vector; The nodes in the graph structure are divided according to similarity through the K-nearest neighbor clustering algorithm, and the connectivity between the nodes in the graph structure is adjusted through the weakly connected component algorithm to obtain the target knowledge graph.

[0012] In one implementation of the present application, based on a pre-built knowledge graph, the associated historical data of the prompt word to be processed is retrieved, and the historical data is input as context information into the target tool after parameter filling, specifically including: According to the question input by the user and based on the target knowledge graph, a graph search is performed on the prompt words to be processed corresponding to the question; Calculate the similarity between the prompt word of the pending question and each node in the knowledge graph respectively, and determine the associated historical data in the knowledge graph with the highest similarity to the prompt word to be processed according to the similarity result; The associated historical data is used as context information corresponding to the prompt word to be processed, and the prompt word to be processed and the corresponding context information are input into the target tool after parameter filling.

[0013] In one implementation of the present application, based on a prompt word distillation algorithm, extracting the core feature vector of the prompt word to be processed specifically includes: By using natural language processing technology, the prompt words to be processed are segmented and POS tagged to identify keywords and phrases in the prompt words to be processed, and weights are assigned to the keywords and phrases based on an attention mechanism; The keywords and phrases after weight assignment are screened by using a prompt word distillation algorithm, and the screened keywords and phrases are fused to extract a core feature vector corresponding to the prompt word to be processed.

[0014] In one implementation of the present application, the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for a large model, specifically including: Calculating the correlation between the core feature vector and the current parameter weight in the task tool according to the core feature vector corresponding to the prompt word to be processed; Based on the correlation, determine the weight of the parameter to be adjusted, and determine the adjustment direction and adjustment range corresponding to the weight of the parameter to be adjusted; By using a gradient descent method and according to the adjustment direction and the adjustment amplitude, the weight of the parameter to be adjusted is iteratively updated until a preset convergence condition is reached; The performance indicators of the task tool are monitored in real time, and the adjustment direction and the adjustment range are optimized according to the monitoring results; wherein the performance indicators include accuracy and recall rate.

[0015] On the other hand, an embodiment of the present application further provides an inference optimization device for a large model, the device comprising: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an inference optimization method for a large model as described above.

[0016] On the other hand, an embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, which, when executed, implements a reasoning optimization method for a large model as described above.

[0017] The embodiments of the present application provide a method, device and medium for inference optimization of a large model, which at least have the following beneficial effects: By rationally planning the execution order of tasks, we can ensure that the logical coherence and dependencies between tasks are correctly handled, avoid task conflicts and waste of resources, and thus improve the efficiency and accuracy of overall task processing; through the multi-dimensional scoring mechanism, we can comprehensively evaluate the performance and applicability of task tools, so as to accurately select the target tool that best suits the current task, and combine the optimal tool parameters to further improve the accuracy and efficiency of task processing; using the rich semantic information and association relationships of the knowledge graph, we can quickly and accurately retrieve historical data related to the prompt words to be processed, and input them into the target tool as context information, which helps to improve the semantic understanding and accuracy of task processing, while reducing dependence on external data sources; through the prompt word distillation algorithm, we can efficiently extract the core feature vectors of the prompt words to be processed, and input them into the task tool as key information for task processing. At the same time, dynamically adjusting the parameter weights in the task tool can adapt to the needs of different tasks and data, realize reasoning optimization for large models, and improve the performance and accuracy of task processing. This dynamic adjustment mechanism also helps to reduce the risk of overfitting and underfitting of the model, and improve the generalization ability and stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a method for optimizing the reasoning of a large model provided in an embodiment of the present application; Figure 2 A schematic diagram of the GraphRAG architecture provided for an embodiment of the present application; Figure 3 A schematic diagram of the internal structure of an inference optimization device for large models provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0021] Figure 1 A flowchart of an inference optimization method for a large model provided in an embodiment of the present application.

[0022] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.

[0023] It should be noted that the server may be a single device or a system consisting of multiple devices, that is, a distributed server, and this application does not make any specific limitation on this.

[0024] like Figure 1 As shown, an inference optimization method for a large model provided in an embodiment of the present application includes: 101. Determine an execution order of a plurality of pending tasks corresponding to pending prompt words input by a user, so as to input the pending tasks into a plurality of task tools according to the execution order.

[0025] Specifically, in one embodiment of the present application, determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user specifically includes: Receive the question input by the user and determine the corresponding prompt word to be processed according to the question; Decomposing the prompt words to be processed to obtain multiple tasks to be processed, and determining the dependency relationship between the multiple tasks to be processed; According to the dependency relationship, the execution order between the multiple tasks to be processed is determined; wherein the execution order includes parallel execution and sequential execution.

[0026] In one embodiment, the optimization of task execution strategy is one of the key factors to improve the overall performance of the system. By decomposing the user input problem into multiple specific tasks to be processed, and adopting a parallel or sequential execution strategy based on the dependencies between tasks, efficient resource utilization can be achieved. In parallel execution, tasks of the same level are processed simultaneously, maximizing the concurrent processing capability of the system; and in the task sequence with dependencies, ensuring that the correct execution order is followed not only ensures the efficient operation of the system process, but also maintains the logical integrity of the operation. This comprehensive strategy not only improves the system response speed, but also further improves the operating efficiency of hardware resources.

[0027] In one embodiment, first, the system receives a question input by the user: "Please help me find the latest research results in the field of artificial intelligence, and summarize the innovations and application prospects therein." Based on this question, the system determines that the corresponding prompt words to be processed are "find the latest research results in the field of artificial intelligence" and "summarize the innovations and application prospects". Next, the system decomposes these two prompt words to be processed to obtain multiple tasks to be processed. For the first prompt word, it is decomposed into the following tasks: 1) search for relevant literature in the field of artificial intelligence; 2) screen the latest research results. For the second prompt word, it is decomposed into the following tasks: 1) analyze the innovations in the research results; 2) summarize the application prospects of the research results.

[0028] Then, the system determines the dependencies between these pending tasks. Task 1 (searching for relevant literature) is a prerequisite for Task 2 (screening the latest research results), so they need to be executed sequentially. Similarly, Task 3 (analyzing innovative points) and Task 4 (summarizing application prospects) both depend on the output of Task 2 (screening the latest research results), but they can be executed in parallel because analyzing innovative points and summarizing application prospects are two relatively independent processes. Finally, based on the dependencies, the system determines the execution order between these pending tasks. First, execute Task 1 (searching for relevant literature), then execute Task 2 (screening the latest research results), and then execute Task 3 (analyzing innovative points) and Task 4 (summarizing application prospects) in parallel. Through this execution order, the system can efficiently process the questions input by users and give accurate answers.

[0029] 102. Determine the multi-dimensional scores corresponding to the tasks to be processed in multiple task tools, and determine the target tool corresponding to the tasks to be processed and the optimal tool parameters corresponding to the target tool based on the multi-dimensional scores, so as to fill the optimal tool parameters into the target tool.

[0030] Specifically, in one embodiment of the present application, determining the multi-dimensional scores corresponding to the tasks to be processed in multiple task tools includes: Receive the reasoning results of multiple task tools for the tasks to be processed through multi-threaded parallel processing technology; For each task tool, according to the task description of the task to be processed and the tool parameters of the task tool, and based on the thought chain technology, the complexity between the task to be processed and the task tool is calculated; Combined with the complexity, the multi-dimensional score corresponding to the task to be processed on the task tool is determined, and based on the complexity, tool parameters of the task tool and historical task execution data, the score threshold is dynamically adjusted to obtain the corresponding target score threshold.

[0031] In one embodiment, after the tasks to be processed are identified, in order to ensure the efficiency and accuracy of task execution, multi-threaded parallel processing technology is used to analyze the phrases of the decomposed tasks to be processed in detail, and combined with the characteristic description of the task tool, each candidate task tool is scored in multiple dimensions. The scoring mechanism combines the detailed information of the task description with the characteristic parameters of the tool, and deeply understands the complex relationship between the task and the tool through the thinking chain technology, so as to provide more accurate evaluation results. The scoring indicators include but are not limited to the functional matching degree of the tool, performance indicators, user evaluation, and relevance to the current task requirements.

[0032] Based on this scoring, a "depth-first search" strategy is introduced, which enables access to and integration of detailed information and tool selection results of the previous level of tasks when processing the next level of tasks. This mechanism significantly improves the contextual relevance of the decision-making process, making tool selection more accurate and intelligent, thereby optimizing the strategy and efficiency of task execution. In addition, the concept of dynamic scoring thresholds is introduced. Based on the complexity of the task, the performance of the tool, and past task execution data, the scoring criteria are intelligently adjusted to ensure that the most suitable tool can be selected in different scenarios. At each evaluation stage, the system provides users with detailed debugging information to help users optimize task descriptions or adjust tool selection strategies to further improve the efficiency and accuracy of task execution.

[0033] In one embodiment, a multi-threaded parallel processing mechanism is first started to simultaneously receive reasoning results from multiple task tools for specific tasks to be processed. These task tools may include but are not limited to natural language processing models, image recognition algorithms, data analysis software, etc. Each tool analyzes and reasons from different angles for the tasks to be processed based on its expertise. Taking the task to be processed as "analyze market trends and predict sales in the next quarter" as an example, the system receives results from three task tools at the same time: Tool A (market analysis model), Tool B (time series prediction model) and Tool C (consumer behavior analysis model). Each tool independently completes the task and returns the reasoning results based on its built-in algorithm and parameters.

[0034] Next, the system uses the thought chain technique to calculate the complexity of each task tool based on the task description of the task to be processed (i.e., "analyze market trends and predict sales for the next quarter") and the tool parameters of the task tool (such as model complexity, training data size, algorithm type, etc.). The thought chain technique is a method that simulates the human thinking process, which can help the system understand the logical relationship and difficulty level between tasks and tools. For example, the system may evaluate the computational complexity of tool A in parsing market dynamics, the data dependency of tool B in predicting time series, and the feature extraction difficulty of tool C in analyzing consumer behavior. Based on the calculated complexity, the system assigns a multi-dimensional score to each task tool on the task to be processed. This score takes into account the difficulty of the task, the performance of the tool, and the degree of match between the two. For example, if tool A performs well in processing market trend analysis, but its complexity is also relatively high, then its score may reflect this "high risk and high return" characteristic.

[0035] Finally, the system dynamically adjusts the scoring threshold based on the complexity, tool parameters of the task tool, and historical task execution data to obtain the corresponding target scoring threshold. The historical task execution data provides a record of the performance of different tools on different tasks, which is critical for adjusting the threshold. For example, if historical data shows that tool B has a consistently high prediction accuracy on similar tasks, the system may set a higher scoring threshold for it to reflect its superior performance. On the contrary, if tool C performs unstably on similar tasks, the system may set a lower threshold for it to reduce potential risks.

[0036] In one embodiment of the present application, according to the multi-dimensional scoring, the target tool corresponding to the task to be processed and the optimal tool parameters corresponding to the target tool are determined, so as to fill the optimal tool parameters into the target tool, specifically including: Compare the multi-dimensional scores corresponding to the multiple task tools with the corresponding target score thresholds respectively, and determine the target tool corresponding to the task to be processed according to the comparison results; Based on the causal reasoning method, the optimal tool parameters corresponding to the target tool are determined, so as to input the optimal tool parameters into the target tool and obtain the output result of the target tool; Compare the output results with the preset expected results, and modify the optimal tool parameters based on the erroneous output results.

[0037] In one embodiment, in response to the challenge of automatic filling of tool parameters, we implemented a method that adopts a causal reasoning (CoT) approach, which significantly enhanced the large model's ability to identify key information and effectively reduced the deviations that may occur during the content filling process. Faced with the problem that limiting the output format may lead to a decrease in model reasoning and context understanding, we introduced a multi-level CoT strategy to maintain the consistency of the model output process and protect its performance from format restrictions. In practice, we noticed that the model performed poorly when dealing with specific problems, so we introduced a verification step. During the verification process, by comparing the model output results with the expected results, the erroneous output results were corrected, thereby greatly improving the accuracy of parameter filling, and then improving the overall performance of the large model. This integrated method has achieved a breakthrough in the field of automatic filling of tool parameters, improving the intelligence level and application efficiency of the system.

[0038] 103. Based on the pre-built knowledge graph, retrieve the associated historical data of the prompt word to be processed, and input the historical data as context information into the target tool after the parameters are filled.

[0039] In one embodiment of the present application, before retrieving the associated historical data of the prompt word to be processed based on the pre-built knowledge graph, the method further includes: The structured information corresponding to the prompt word to be processed is used as a graph structure to construct the corresponding knowledge graph, and the node ID and description information corresponding to the prompt word to be processed are converted into vectors; The nodes in the graph structure are divided according to similarity through the K-nearest neighbor clustering algorithm, and the connectivity between the nodes in the graph structure is adjusted through the weakly connected component algorithm to obtain the target knowledge graph.

[0040] In one embodiment, in order to make full use of historical data in complex tasks, a historical data retrieval technology based on GraphRAG is introduced. By building a knowledge graph, the structured information of the document is represented as a graph structure and stored in the database. Emdedding is used to convert the id and description information of the node into a vector, which further improves the retrieval efficiency. Through the K nearest neighbor clustering algorithm, the system can divide the nodes in the graph database according to similarity, and use the weakly connected component algorithm to enhance the connectivity between nodes to build a highly reliable knowledge graph.

[0041] In one embodiment of the present application, based on the pre-built knowledge graph, the associated historical data of the prompt word to be processed is retrieved, and the historical data is input into the target tool after parameter filling as context information, specifically including: According to the question input by the user and based on the target knowledge graph, a graph search is performed on the prompt words to be processed corresponding to the question; Calculate the similarity between the prompt word of the question to be processed and each node in the knowledge graph respectively, and determine the associated historical data in the knowledge graph with the highest similarity to the prompt word to be processed based on the similarity results; The associated historical data is used as context information corresponding to the prompt word to be processed, and the prompt word to be processed and the corresponding context information are input into the target tool after the parameters are filled.

[0042] In one embodiment, when a user queries, relevant historical data can be quickly retrieved through a similarity matching algorithm and provided to the large model as context information to assist the decision-making process. This mechanism significantly improves the model's ability to solve similar but unseen complex problems.

[0043] Figure 2 This is a schematic diagram of the GraphRAG architecture provided in the embodiment of this application. Figure 2 As shown, this application receives questions input by users and inputs the questions into the GenAI application to perform graph retrieval on the questions, determine the corresponding associated historical data based on the pre-built knowledge graph, and input the associated historical data into the target tool as context information. At the same time, the node ID and description information corresponding to the prompt word to be processed are converted into vectors, so that the knowledge graph and vector are output as the documents and content corresponding to the user input question, and the corresponding complete answer is obtained through the large model. The complete answer is returned to the corresponding user through the GenAI application.

[0044] 104. Based on the prompt word distillation algorithm, the core feature vector of the prompt word to be processed is extracted, and the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for large models.

[0045] Specifically, in one embodiment of the present application, based on the prompt word distillation algorithm, extracting the core feature vector of the prompt word to be processed specifically includes: Through natural language processing technology, the prompt words to be processed are segmented and POS tagged to identify the keywords and phrases in the prompt words to be processed, and the keywords and phrases are weighted based on the attention mechanism; The weighted keywords and phrases are screened through the prompt word distillation algorithm, and the screened keywords and phrases are fused to extract the core feature vector corresponding to the prompt word to be processed.

[0046] In one embodiment, in the face of the problem that there are complex rule descriptions in the prompt words to be processed in the large language model, which leads to low efficiency of understanding and reasoning of the large language model and slow response, a new prompt word distillation method (Prompt Distillation) is proposed. This technology builds a teacher-student model pair based on convolutional network learning, realizes the effective compression of complex rule descriptions, and then constructs a lightweight representation, which significantly improves the reasoning speed. Unlike the problem of large storage parameters of traditional models, the prompt word distillation method only needs to save the compressed representation of complex rule descriptions, thereby greatly reducing the disk space and time cost required for model switching. This process not only optimizes resource utilization efficiency, but also realizes efficient processing of rule descriptions without sacrificing model performance.

[0047] In one embodiment, the server receives a prompt word to be processed: "Looking for popular science books suitable for children to read". First, through natural language processing technology, the system performs word segmentation and part-of-speech tagging on the prompt word. The word segmentation process splits the prompt word into independent words, such as "find", "suitable", "children", "reading", "of", "popular science", and "books". Part-of-speech tagging assigns a part-of-speech tag to each word, such as "find" as a verb, "children" as a noun, etc. Next, the system identifies the keywords and phrases in the prompt word based on the results of word segmentation and part-of-speech tagging. In this example, the keywords may include "find", "suitable", "children", "popular science", and "books", while the phrases may be "suitable for children to read", "popular science books", etc. These keywords and phrases are crucial to understanding the intention and focus of the prompt word.

[0048] The system then introduces an attention mechanism to assign weights to the identified keywords and phrases. The attention mechanism can assign different weights to words based on their importance and relevance in the prompt word. For example, in this prompt word, "children" and "popular science books" may be regarded as more important words and therefore be assigned higher weights. Subsequently, the system uses the prompt word distillation algorithm to filter the weighted keywords and phrases. The purpose of the distillation algorithm is to select the most representative words from a large number of words to simplify the subsequent processing process. In this example, the system may filter out keywords such as "children" and "popular science books", while ignoring some minor words such as "of" and "reading".

[0049] Finally, the system fuses the filtered keywords and phrases to extract the core feature vector corresponding to the prompt word to be processed. The fusion process may use methods such as vector concatenation and weighted average to convert the filtered words into a fixed-dimensional vector that contains the main information and features of the prompt word. This core feature vector can be used in subsequent tasks such as text processing, information retrieval, and recommendation systems to support more accurate and efficient processing.

[0050] In one embodiment of the present application, the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for a large model, specifically including: According to the core feature vector corresponding to the prompt word to be processed, the correlation between the core feature vector and the current parameter weight in the task tool is calculated; Based on the correlation, the weight of the parameter to be adjusted is determined, and the adjustment direction and adjustment range corresponding to the weight of the parameter to be adjusted are determined; By using the gradient descent method, the weight of the parameter to be adjusted is iteratively updated according to the adjustment direction and adjustment amplitude until the preset convergence condition is reached; The performance indicators of the task tools are monitored in real time, and the adjustment direction and range are optimized according to the monitoring results; among them, the performance indicators include accuracy and recall rate.

[0051] In one embodiment, the method combines the dynamic weight adjustment mechanism with JS divergence, which can optimize the model parameters in real time according to the feedback of prediction accuracy, ensure the adaptability of the model when processing new rule descriptions, and improve the system's reasoning efficiency and ability to process complex rules. This strategy not only solves the efficiency of traditional methods in complex rule processing, but also improves the overall performance of the system when facing complex rule descriptions.

[0052] In one embodiment, the parameter weights in the task tool are adjusted based on this core feature vector to optimize the performance of the task tool. First, the system calculates the correlation between the core feature vector and the current parameter weights in the task tool. The correlation calculation may use methods such as cosine similarity, Pearson correlation coefficient, etc. to measure the linear relationship or similarity between the core feature vector and the parameter weights. Through this calculation, the system can understand the degree of match between the current parameter weights and the prompt word to be processed.

[0053] Based on the calculated correlation, the system determines the parameter weights to be adjusted, and determines the adjustment direction and adjustment range corresponding to these parameter weights. For example, if a parameter weight has a low correlation with the core feature vector, the system may decide to increase the value of the weight to improve its match with the core feature vector. On the contrary, if the correlation is high but exceeds a certain threshold, the system may decide to reduce the value of the weight to avoid problems caused by overfitting or excessive weights.

[0054] Next, the system iteratively updates the weights of the parameters to be adjusted based on the determined adjustment direction and adjustment amplitude through the gradient descent method. The gradient descent method is a commonly used optimization algorithm that calculates the gradient of the loss function and updates the parameters in the opposite direction of the gradient to gradually reduce the value of the loss function. In this process, the system will continuously adjust the parameter weights according to the preset learning rate and the rules for controlling parameter updates until the preset convergence conditions are reached. The convergence conditions may be that the value of the loss function is less than a certain threshold, or the amplitude of the parameter update is less than a certain value, etc.

[0055] During the parameter weight update process, the system also monitors the performance indicators of the task tool in real time. Performance indicators may include accuracy, recall, etc., which are used to measure the performance of the task tool when processing the prompt words to be processed. By monitoring these indicators, the system can promptly understand the impact of parameter weight adjustment on the performance of the task tool, and optimize the adjustment direction and adjustment range based on the monitoring results. For example, if it is found that a certain adjustment direction leads to a decrease in accuracy, the system may adjust the direction or reduce the adjustment range in time to avoid performance deterioration.

[0056] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, the embodiment of this application also provides an inference optimization device for a large model, whose structure is as follows: Figure 2 shown.

[0057] Figure 2 The internal structure diagram of a reasoning optimization device for a large model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the device includes: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to: Determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user, so as to input the pending tasks into multiple task tools according to the execution order; Determine the multi-dimensional scores corresponding to the tasks to be processed in the multiple task tools respectively, and determine the target tool corresponding to the tasks to be processed and the optimal tool parameters corresponding to the target tool according to the multi-dimensional scores, so as to fill the optimal tool parameters into the target tool; Based on the pre-built knowledge graph, retrieve the historical data associated with the prompt word to be processed, and input the historical data as context information into the target tool after parameter filling; Based on the prompt word distillation algorithm, the core feature vector of the prompt word to be processed is extracted, and the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for large models.

[0058] The present application also provides a non-volatile computer storage medium storing computer executable instructions. When the computer executable instructions are executed, they can: Determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user, so as to input the pending tasks into multiple task tools according to the execution order; Determine the multi-dimensional scores corresponding to the tasks to be processed in the multiple task tools respectively, and determine the target tool corresponding to the tasks to be processed and the optimal tool parameters corresponding to the target tool according to the multi-dimensional scores, so as to fill the optimal tool parameters into the target tool; Based on the pre-built knowledge graph, retrieve the historical data associated with the prompt word to be processed, and input the historical data as context information into the target tool after parameter filling; Based on the prompt word distillation algorithm, the core feature vector of the prompt word to be processed is extracted, and the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for large models.

[0059] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0060] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0062] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0063] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0066] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0067] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0068] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0069] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0070] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for optimizing reasoning for a large model, characterized in that: The method comprises: Determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user, so as to input the pending tasks into multiple task tools according to the execution order; Determine the multi-dimensional scores corresponding to the task to be processed in multiple task tools respectively, and determine the target tool corresponding to the task to be processed and the optimal tool parameters corresponding to the target tool according to the multi-dimensional scores, so as to fill the optimal tool parameters into the target tool; Based on the pre-built knowledge graph, retrieve the associated historical data of the prompt word to be processed, and input the historical data as context information into the target tool after parameter filling; Based on the prompt word distillation algorithm, the core feature vector of the prompt word to be processed is extracted, and the parameter weights in the task tool are dynamically adjusted to achieve reasoning optimization for the large model.

2. The inference optimization method for a large model according to claim 1, characterized in that: Determining the execution order of multiple pending tasks corresponding to the pending prompt words input by the user, specifically includes: Receive a question input by a user, and determine a corresponding prompt word to be processed according to the question; Decomposing the prompt words to be processed to obtain a plurality of tasks to be processed, and determining dependencies between the plurality of tasks to be processed; According to the dependency relationship, an execution order among the multiple tasks to be processed is determined; wherein the execution order includes parallel execution and sequential execution.

3. The inference optimization method for a large model according to claim 1, characterized in that: Determining the multi-dimensional scores corresponding to the to-be-processed tasks in the plurality of task tools, specifically including: Receiving the reasoning results of the multiple task tools for the tasks to be processed by using multi-threaded parallel processing technology; For each task tool, according to the task description of the task to be processed and the tool parameters of the task tool, and based on the thought chain technology, the complexity between the task to be processed and the task tool is calculated; In combination with the complexity, the multi-dimensional score corresponding to the task to be processed on the task tool is determined, and based on the complexity, the tool parameters of the task tool and the historical task execution data, the score threshold is dynamically adjusted to obtain the corresponding target score threshold.

4. The inference optimization method for a large model according to claim 1, characterized in that: Determining the target tool corresponding to the task to be processed and the optimal tool parameters corresponding to the target tool according to the multi-dimensional score, so as to fill the optimal tool parameters into the target tool, specifically includes: Compare the multi-dimensional scores corresponding to the multiple task tools with the corresponding target score thresholds respectively, and determine the target tool corresponding to the task to be processed according to the comparison results; Based on a causal reasoning method, determining optimal tool parameters corresponding to the target tool, inputting the optimal tool parameters into the target tool, and obtaining an output result of the target tool; The output result is compared with the preset expected result, and the filled optimal tool parameters are corrected according to the erroneous output result.

5. The inference optimization method for a large model according to claim 1, characterized in that: Before retrieving the associated historical data of the prompt word to be processed based on the pre-built knowledge graph, the method further includes: The structured information corresponding to the prompt word to be processed is used as a graph structure to construct a corresponding knowledge graph, and the node ID and description information corresponding to the prompt word to be processed are converted into a vector; The nodes in the graph structure are divided according to similarity through the K-nearest neighbor clustering algorithm, and the connectivity between the nodes in the graph structure is adjusted through the weakly connected component algorithm to obtain the target knowledge graph.

6. The inference optimization method for a large model according to claim 5, characterized in that: Based on the pre-built knowledge graph, the associated historical data of the prompt word to be processed is retrieved, and the historical data is input into the target tool after parameter filling as context information, specifically including: According to the question input by the user and based on the target knowledge graph, a graph search is performed on the prompt words to be processed corresponding to the question; Calculate the similarity between the prompt word of the pending question and each node in the knowledge graph respectively, and determine the associated historical data in the knowledge graph with the highest similarity to the prompt word to be processed according to the similarity result; The associated historical data is used as context information corresponding to the prompt word to be processed, and the prompt word to be processed and the corresponding context information are input into the target tool after parameter filling.

7. The inference optimization method for a large model according to claim 1, characterized in that: Based on the prompt word distillation algorithm, the core feature vector of the prompt word to be processed is extracted, specifically including: By using natural language processing technology, the prompt words to be processed are segmented and POS tagged to identify keywords and phrases in the prompt words to be processed, and weights are assigned to the keywords and phrases based on an attention mechanism; The keywords and phrases after weight assignment are screened by using a prompt word distillation algorithm, and the screened keywords and phrases are fused to extract a core feature vector corresponding to the prompt word to be processed.

8. The inference optimization method for a large model according to claim 1, characterized in that: Dynamically adjust the parameter weights in the task tool to achieve inference optimization for large models, including: Calculating the correlation between the core feature vector and the current parameter weight in the task tool according to the core feature vector corresponding to the prompt word to be processed; Based on the correlation, determine the weight of the parameter to be adjusted, and determine the adjustment direction and adjustment range corresponding to the weight of the parameter to be adjusted; By using a gradient descent method and according to the adjustment direction and the adjustment amplitude, the weight of the parameter to be adjusted is iteratively updated until a preset convergence condition is reached; The performance indicators of the task tool are monitored in real time, and the adjustment direction and the adjustment range are optimized according to the monitoring results; wherein the performance indicators include accuracy and recall rate.

9. A reasoning optimization device for a large model, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an inference optimization method for a large model as described in any one of claims 1-8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed, an inference optimization method for a large model as described in any one of claims 1 to 8 is implemented.

Images