Method and device for implementing model inference search based on intelligent computing center

Abstract

This application provides a method and apparatus for model inference search based on the computing power of an intelligent computing center. The method includes: inputting a user's request into a large language model to obtain multiple sub-steps and sending them to an executor for execution; scoring the sub-steps based on the model; selecting the sub-step with the highest score; inputting the sub-step and the request into the model to obtain multiple new sub-steps and sending them to the executor for execution; scoring the multiple new sub-steps based on the large language model and the score of the highest-scoring sub-step, and obtaining a score result corresponding to each new sub-step; repeating the steps of generating sub-steps, sending them to the executor for execution, and scoring until the score result of the new sub-step is greater than a preset threshold, or until the solution formed by the highest-scoring sub-step is determined to be valid and is taken as the target solution. Thus, the efficiency and quality of solution generation can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computing infrastructure technology, and in particular to a method and apparatus for realizing model inference search based on computing power of an intelligent computing center. Background Technology

[0002] With the development of artificial intelligence and computing power technologies, the concept of intelligent computing centers has emerged. An "intelligent computing center" refers to a facility that utilizes large-scale heterogeneous computing resources, including general-purpose and intelligent computing power, to provide the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models). Intelligent computing centers encompass facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0003] "Large language model" refers to a large-scale language model (LLM), which is a language model with a large number of parameters. It is designed to understand and generate human language. It is trained on a large amount of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.

[0004] The Scaling Law describes the power-law relationship between model performance and model parameters, data volume, and computational resources. Research shows that as resources increase, the improvement in model performance exhibits diminishing returns. This means that when the model size is small, increasing resources can significantly improve performance; however, in large-scale models, the impact of increasing resources on performance gradually weakens. Therefore, the Scaling Law not only reveals the balance between resource investment and performance improvement but also provides researchers with a theoretical framework for effective performance optimization under conditions of limited data and computing power.

[0005] As internet data resources dwindle, the traditional Scaling Law paradigm has shifted, moving from a pre-training-centric model to one focused on inference-based expansion. This new paradigm emphasizes the integration of inference models, synthetic data generation, and reinforcement learning (RL) technologies, indicating that inference will play an increasingly important role in future AI (Artificial Intelligence) research. This shift not only reflects the need for more efficient data utilization but also drives deeper exploration of inference services.

[0006] Against this backdrop, OpenAI's o1 model, released on September 13, 2024, leverages reinforcement learning techniques to push the limits of LLM inference, pioneering a new paradigm for inference extension. The o1 model particularly emphasizes the computational demands of the training and inference phases, proposing two novel post-RL training scaling laws: "train-time compute" and "test-time compute." By demonstrating the significant performance improvement of the model as reinforcement learning time and inference thinking time increase, the o1 model provides strong empirical support for the importance of inference services.

[0007] Under the new paradigm, computational demands have not decreased; in fact, they have increased significantly due to the introduction of the inference model. Traditionally, the computational demands for pre-training and post-training are in a 1:1 ratio, but in the O1 model framework, the computational demands during the inference phase account for a larger proportion. Components such as the generator, reward model, policy model, and validator generate a large amount of "berry training data" (inference path data), significantly increasing the overall computational demands. During inference, the computational load of forward propagation increases significantly, while the proportion of gradient calculations and weight adjustments decreases, which supports the potential of distributed large clusters.

[0008] In conclusion, with the development of AI systems, computational overhead will increasingly focus on inference services, rather than just pre-training computation. Therefore, effectively utilizing the computing resources of intelligent computing centers to support inference with large language models and improving the quality of solutions generated by these models in complex scenarios has become a pressing technical challenge. Summary of the Invention

[0009] This application provides a method and apparatus for model inference search based on the computing power of an intelligent computing center, in order to solve the technical problem of how to effectively utilize the computing power resources of an intelligent computing center to support the inference of large language models and improve the quality of solutions generated by large language models under complex scenarios.

[0010] To solve the above-mentioned technical problems, this application is implemented as follows:

[0011] In a first aspect, this application provides a method for model inference search based on the computing power of an intelligent computing center, the method comprising:

[0012] Step S1: Obtain the user's input request, input the request into the large language model, and obtain multiple sub-steps output by the large language model. The large language model is used to take the request as the root node of the Monte Carlo tree and, based on the thinking chain technology, reason to obtain the multiple sub-steps that are equivalent to the leaf nodes at the same level of the Monte Carlo tree.

[0013] Step S2: Send the multiple sub-steps into the executor for execution, and score the multiple sub-steps based on the large language model to obtain the score result corresponding to each sub-step;

[0014] Step S3: Select the sub-step with the highest score based on the score results, and input the sub-step with the highest score and the request into the large language model to obtain multiple new sub-steps output by the large language model. The large language model is used to infer the multiple new sub-steps that are equivalent to the leaf nodes of the next level of the Monte Carlo tree based on the request, the sub-step with the highest score, and the thinking chain technique. The leaf nodes of the next level are the leaf nodes of the next level extended based on the sub-step with the highest score.

[0015] Step S4: Send the multiple new sub-steps into the executor for execution, and score the multiple new sub-steps based on the large language model and the score of the sub-step with the highest score, to obtain the score result corresponding to each new sub-step.

[0016] Step S5: Repeat steps S3 and S4 until the score of the new sub-step is greater than a preset threshold. Then record the solution formed by the sub-step with the highest score in the leaf nodes of each level; or, until the solution formed by the sub-step with the highest score in the leaf nodes of each level is determined to be valid by the large language model based on the request and the large language model.

[0017] Step S6: Use the solution described above as the target solution for the request.

[0018] Optionally, the sub-step includes: the large language model inferring the calling method and the input parameters corresponding to the calling method based on the request and the thought chain technology.

[0019] Optionally, the invocation method includes at least one of the following: a base invocation function, an invocation method generated by the large language model based on the base invocation function and the request.

[0020] Optionally, step S2 includes:

[0021] Step S21: Send the multiple sub-steps into the executor for execution and obtain the execution result, which includes any one of the following: execution successful, execution failed;

[0022] Step S22: If the execution result is execution failure, then based on the large language model, obtain the first score result corresponding to each failed sub-step, and the first score result is 0 points;

[0023] Step S23: If the execution result is successful, then based on the large language model, obtain the second score result corresponding to each successful sub-step, and the second score result is a positive number.

[0024] Optionally, step S6 includes:

[0025] Step S61: Input the solution into the actuator for verification;

[0026] Step S62: If the verification is successful, the solution is taken as the target solution of the request.

[0027] Secondly, embodiments of this application provide an apparatus for realizing model inference search based on the computing power of an intelligent computing center, the apparatus comprising:

[0028] The acquisition module is used to execute step S1: acquire the user input request, input the request into the large language model, and obtain multiple sub-steps output by the large language model. The large language model is used to take the request as the root node of the Monte Carlo tree and, based on the thinking chain technology, reason to obtain the multiple sub-steps that are equivalent to the leaf nodes at the same level of the Monte Carlo tree.

[0029] The execution module is used to execute step S2: send the multiple sub-steps into the executor for execution, and score the multiple sub-steps based on the large language model to obtain the score result corresponding to each sub-step.

[0030] For executing step S3: Select the sub-step with the highest score based on the score result, and input the sub-step with the highest score and the request into the large language model to obtain multiple new sub-steps output by the large language model. The large language model is used to reason about the multiple new sub-steps that are equivalent to the leaf nodes of the next level of the Monte Carlo tree based on the request, the sub-step with the highest score and the thinking chain technique. The leaf nodes of the next level are the leaf nodes of the next level extended based on the sub-step with the highest score.

[0031] For executing step S4: send the multiple new sub-steps into the executor for execution, and score the multiple new sub-steps based on the large language model and the score of the sub-step with the highest score, to obtain the score result corresponding to each new sub-step;

[0032] To execute step S5: Repeat steps S3 and S4 until the score of the new sub-step is greater than a preset threshold, then record the solution formed by the sub-step with the highest score in the leaf nodes of each level; or, until the solution formed by the sub-step with the highest score in the leaf nodes of each level is determined to be valid by the large language model based on the request and the large language model.

[0033] To perform step S6: Use the solution as the target solution for the request.

[0034] Optionally, the sub-step includes: the large language model inferring the calling method and the input parameters corresponding to the calling method based on the request and the thought chain technology.

[0035] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the steps of a method for model inference search based on the computing power of an intelligent computing center as described in the first aspect.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a method for model inference search based on the computing power of an intelligent computing center as described in the first aspect.

[0037] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of a method for model inference search based on the computing power of an intelligent computing center as described in the first aspect.

[0038] The method provided in this application combines the reasoning capabilities of a large language model with Monte Carlo tree search and thought chain techniques, which can fully utilize the advantages of each. For example, the large language model can generate solutions that describe natural language, while the Monte Carlo tree, as the skeleton of the search, can expand the solution space of the original large language model and solve reasoning problems of complex issues, and the thought chain technique can provide support for logical reasoning.

[0039] Specifically, the Monte Carlo tree search algorithm allows for extensive exploration of the solution space. By randomly sampling and evaluating different decision paths, this algorithm generates multiple possible solutions, thus avoiding getting trapped in local optima. Meanwhile, the thought chain technology emphasizes solving complex problems through step-by-step reasoning. Combining the reasoning capabilities of a large language model, it generates detailed sub-steps at each stage, which are then analyzed and evaluated. This step-by-step optimization method ensures that each sub-step is carefully considered, thereby improving the quality of the final solution. Furthermore, in the dynamic generation, execution, and evaluation loop, the generated sub-steps can be continuously adjusted and optimized based on the execution results and feedback information. By evaluating the effectiveness of each sub-step in real time, low-quality options can be quickly identified and eliminated, thereby improving the overall quality of the solution.

[0040] The intelligent computing center provides powerful computing resources, enabling the Monte Carlo Tree Search algorithm to perform efficient random sampling and evaluation in large-scale state spaces. Simultaneously, the reasoning processes of the thought chain technology and large language models can be accelerated within the parallel computing environment provided by the intelligent computing center. Furthermore, the intelligent computing center can dynamically adjust the allocation of computing resources based on task complexity and real-time requirements. For example, more computing power can be allocated for extensive exploration during the Monte Carlo Tree Search phase, while resources can be concentrated for in-depth analysis during the thought chain reasoning phase.

[0041] In summary, by combining the computing resources of the intelligent computing center with Monte Carlo tree search algorithm, thought chain technology, and reasoning of large language models, high-quality solutions can be obtained in complex scenarios. Moreover, the dynamic generation, execution, and evaluation loop mechanism not only optimizes the efficiency of solution generation but also improves the quality of the solution. Attached Figure Description

[0042] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0043] Figure 1 A flowchart illustrating a method for model inference search based on the computing power of an intelligent computing center, provided as an embodiment of this application;

[0044] Figure 2 A flowchart illustrating a method for model inference search based on the computing power of an intelligent computing center, provided in this application embodiment;

[0045] Figure 3A structural block diagram of a device for realizing model inference search based on the computing power of an intelligent computing center, provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The following is a brief explanation of the technical terms used in this application.

[0049] The “computing power” mentioned in this application refers to the ability of computer equipment or computing / data centers to process information. It is the ability of computer hardware and software to work together to perform a certain computing requirement. It is the computing power to achieve the target output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, and mainly provides services to society through computing power infrastructure.

[0050] The "computational power" (CP) described in this application refers to the ability of a data center server to process data and output results. It is a comprehensive indicator for measuring the computing power of a data center, encompassing general computing power, supercomputing power, and intelligent computing power. The commonly used unit of measurement is floating-point operations per second (FLOPS, 1 EFLOPS = 10^18 FLOPS), with a higher value indicating stronger overall computing power. It is estimated that 1 EFLOPS is approximately the computing power output of 5 Tianhe-2A supercomputers, 500,000 mainstream server CPUs, or 2 million mainstream laptops. The calculation formula is: CP = General CP + Intelligent CP + Super CP.

[0051] The "Network Power" (NP) described in this application refers to the data transmission capability of computing facilities, encompassing comprehensive capabilities including network architecture, network bandwidth, transmission latency, and intelligent management and scheduling. Network power involves network transmission within and between data centers and is a comprehensive indicator of network transmission scheduling capabilities. In this embodiment, network power is expressed as memory bandwidth.

[0052] The "Storage Power" (SP) described in this application refers to the comprehensive capabilities of a data center in four aspects: data storage capacity, performance, security and reliability, and green and low-carbon operation. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and internal storage devices in servers. The commonly used unit of measurement for storage capacity is exabytes (EB, 1EB = 2^60 bytes), the commonly used unit of measurement for performance is the number of read / write operations per second per unit capacity (IOPS / TB), and the disaster recovery ratio is an important indicator of security and reliability.

[0053] The “computing infrastructure” described in this application is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission, and application of information, and exhibits characteristics such as diversity and ubiquity, intelligence and agility, security and reliability, and green and low carbon emissions.

[0054] The “computing power” mentioned in this application includes general computing power, intelligent computing power, and supercomputing power.

[0055] The “general-purpose computing power” described in this application refers to the computing capabilities provided by servers based on CPU (Central Processing Unit) chips, used to support basic general-purpose computing such as cloud computing and edge computing.

[0056] The "intelligent computing power" described in this application refers to a computing platform deployed on a large scale based on dedicated chips such as GPUs (Graphics Processing Units), FPGAs (Field Programmable Gate Arrays), and ASICs (Application Specific Integrated Circuits) for various innovative artificial intelligence applications, such as natural language processing and machine vision.

[0057] The “supercomputing power” described in this application mainly refers to the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for computing in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0058] The "intelligent computing center" described in this application refers to a facility that utilizes large-scale heterogeneous computing resources, including general-purpose computing power (CPU: Central Processing Unit) and intelligent computing power (GPU: Graphics Processing Unit, FPGA: Field Programmable Gate Array, ASIC: Application Specific Integrated Circuit, etc.), primarily providing the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models). The intelligent computing center encompasses facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0059] The “computing resources” mentioned in this application refer to the technologies and facilities required for the development of the digital society that have the ability to compute, transmit, store and apply information, including but not limited to computing resources such as CPU (Central Processing Unit) and GPU (Graphics Processing Unit), network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guaranteeing resources such as wind, fire, water and electricity.

[0060] The “large language model” mentioned in this application refers to a large-scale language model, which is a language model with a large number of parameters. It is designed to understand and generate human language, and is trained with a large amount of text data. It can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.

[0061] The "Key Technologies for Post-training" described in this application are divided into the following aspects:

[0062] "Self-play Reinforcement Learning (Self-play RL)": Self-play is a reinforcement learning framework in which an agent collects training data by playing against itself multiple times. This method can effectively reduce dependence on complex environments. The idea of ​​self-play originated from AlphaGo Zero, which generated data through self-play and used Monte Carlo Tree Search (MCTS) to guide each step of the decision. This method is also applicable to other tasks requiring policy planning.

[0063] "Chain of Thought (CoT)": The Chain of Thought is key to reasoning on complex problems in large models. In 2022, Wei et al. added a text of "step-by-step reasoning" to the dataset to stimulate the thinking ability of large models. This method allows models to break down multi-step reasoning problems into intermediate steps, thereby improving the accuracy and interpretability of the model when handling complex problems. By simulating the human thought process when solving problems, the Chain of Thought technique enables large language models to analyze problems step by step and generate intermediate reasoning steps, ultimately arriving at the accurate answer. This method not only improves the performance of models in mathematical, common sense, and symbolic reasoning tasks, but also has a significant impact on model size, with larger models benefiting more. In addition, the Chain of Thought prompt makes large language models more interpretable and provides opportunities to debug reasoning path errors.

[0064] Monte Carlo Tree Search (MCS) is a heuristic search algorithm used in certain decision-making processes. Its core idea is to put resources on branches that are more worthwhile to search, that is, to concentrate computing power on more valuable places. In response to the inference scaling law caused by the O1 model, it uses search methods to allow LLM to explore a larger solution space and find a complete solution. Essentially, it uses the CoT method to optimize the path of LLM to sample the correct solution.

[0065] Each cycle of a Monte Carlo tree search consists of four steps:

[0066] Selection: Starting from the root node R, continuously select child nodes down to the leaf node L;

[0067] Expansion: Unless the game ends at L due to the win or loss of either side, create one or more child nodes and select one of them, C.

[0068] Simulation: Starting from node C, the game is played using a random strategy, also known as playout or rollout;

[0069] Backpropagation: Use the results of a random game to update the node information on the path from C to R.

[0070] Figure 1 This application illustrates a method for model inference search based on the computing power of an intelligent computing center, according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0071] Step S1: Obtain the user's input request, input the request into the large language model, and obtain multiple sub-steps output by the large language model;

[0072] Among them, the large language model is used to take the request as the root node of the Monte Carlo tree, and based on the thinking chain technology, it infers multiple sub-steps that are equivalent to the leaf nodes at the same level of the Monte Carlo tree.

[0073] Step S2: Send multiple sub-steps into the executor for execution, and score each sub-step based on the large language model to obtain the score result corresponding to each sub-step.

[0074] Step S3: Select the sub-step with the highest score based on the score results, and input the sub-step with the highest score and the request into the large language model to obtain multiple new sub-steps output by the large language model;

[0075] Among them, the large language model is used to reason about multiple new sub-steps based on the request, the highest-scoring sub-step, and the thought chain technique, which are equivalent to the leaf nodes of the next level of the Monte Carlo tree. The leaf nodes of the next level are the leaf nodes of the next level extended based on the highest-scoring sub-step.

[0076] Step S4: Send multiple new sub-steps into the executor for execution, and score each new sub-step based on the large language model and the score of the sub-step with the highest score, to obtain the score result corresponding to each new sub-step.

[0077] Step S5: Repeat steps S3 and S4 until the score of the new sub-step is greater than the preset threshold. Then record the solution formed by the sub-step with the highest score in the leaf nodes of each level; or, until the solution formed by the sub-step with the highest score in the leaf nodes of each level is determined to be valid by the large language model based on the request and the large language model.

[0078] Step S6: Set the solution as the target solution of the request.

[0079] In step S1, the user-input query can be a question, task, or requirement, accurately conveying the user's intent to a large language model (such as GPT). The large language model uses the request as the root node of a Monte Carlo tree, reasoning through multiple sub-steps using a thought chain technique. These sub-steps are equivalent to leaf nodes at the same level of the Monte Carlo tree, representing multiple preliminary actions that can be taken from the current request. The sub-steps include: a calling method inferred by the large language model based on the request and the thought chain technique, and the corresponding input parameters. The calling method includes at least one of the following: a base calling function, and a calling method generated by the large language model based on the base calling function and the request. That is, when the defined base calling function cannot handle the situation, the large language model can adaptively generate code based on the base calling function and embed it into the workflow to ensure the complete execution of the solution.

[0080] In step S2, the multiple sub-steps generated in step S1 can be input into the executor. The executor is responsible for actually executing these sub-steps. It can execute the sub-steps by calling external services, running algorithms, or performing other operations. The large language model can score each sub-step, obtaining a score result for each sub-step. Therefore, by executing the sub-steps and scoring them, the actual effect of each sub-step can be evaluated, providing a basis for selecting the optimal path in the next step.

[0081] It's important to note that, unlike typical mathematical and coding domains, each sub-step in solution generation outputs a standard JSON dictionary containing the methods to be called (e.g., functions to be called), input parameters, etc., and uses an external executor to execute the called methods, thus providing a complete solution. For example, if a user's request is to develop a climbing detection algorithm to automatically detect and issue real-time alerts regarding whether individuals entering a detection area have their hands raised and feet off the ground exceeding a set threshold, ensuring the safety of smart parks and real estate buildings, then sub-step 1 could be as follows:

[0082] Step 1: {

[0083] "current_step":"Uses a pose estimation function to process the video, detecting key points of people in the video, especially the positions of their hands and feet, to provide basic data for subsequent climb recognition."

[0084] "function":"pose",

[0085] "input":{

[0086] "video_path":"Enter the video path"

[0087] }

[0088] In one possible implementation, such as Figure 2 As shown, step S2 includes:

[0089] Step S21: Send the multiple sub-steps into the executor for execution and obtain the execution result, which includes any one of the following: execution successful or execution failed;

[0090] Step S22: If the execution result is execution failure, then based on the large language model, obtain the first score result corresponding to each failed sub-step, and the first score result is 0 points;

[0091] Step S23: If the execution result is successful, then based on the large language model, obtain the second score result corresponding to each successful sub-step. The second score result is a positive number.

[0092] Therefore, by setting different scoring results, the performance of each sub-step can be clearly evaluated, and the best-performing sub-step can be selected to participate in the generation of the solution and serve as the basis for the generation of subsequent sub-steps, thereby further improving the quality of the solution.

[0093] In step S3, combining the Monte Carlo tree search algorithm and the thought chain technique, the large language model can reason about multiple new sub-steps, equivalent to the leaf nodes of the next level of the Monte Carlo tree, based on the request, the highest-scoring sub-step, and the thought chain technique. The leaf nodes of the next level are extensions of the highest-scoring sub-step. Specifically, after obtaining the score of each sub-step, the highest-scoring sub-step is selected to ensure it is the most promising. This highest-scoring sub-step, along with the original request, is input into the large language model to generate new sub-steps, which are equivalent to the leaf nodes of the next level of the Monte Carlo tree. This process, based on the thought chain technique, ensures that the generated new sub-steps are logically sound and request-related extensions. Thus, the large language model can progressively delve deeper into reasoning and explore the possibilities of the solution, ensuring that each step is an extension of existing successful steps to improve the quality of the solution.

[0094] In step S4, multiple new sub-steps are sent to the executor for execution. Based on the large language model and the score of the highest-scoring sub-step, each new sub-step is scored, resulting in a score for each new sub-step. In other words, the multiple new sub-steps generated in step S3 are input into the executor for execution and scored again. This scoring differs from the previous one in that it incorporates the score of the previously highest-scoring sub-step to further evaluate the value of the newly generated sub-steps.

[0095] In step S5, steps S3 and S4 are repeated until the score of the new sub-step exceeds a preset threshold. Then, the solution formed by the highest-scoring sub-step in each level's leaf nodes is recorded; or, based on the request and the large language model, the large language model determines that the solution formed by the highest-scoring sub-step in each level's leaf nodes is valid. In other words, steps S3 and S4 are repeated to form a loop until the score of the new sub-step exceeds the preset threshold, or when the large language model confirms that the currently generated solution is valid. This process ensures that the solution is optimized through continuous iteration, gradually approaching the optimal result.

[0096] Step S6 may further include: Step S61: Input the solution into the executor for verification; Step S62: If verification is successful, use the solution as the target solution requested. Thus, after obtaining the solution, it can be input into the executor for unified verification, ensuring successful execution of the solution. If verification fails, detailed feedback information about the failure can be collected from the executor, such as error messages, execution logs, performance metrics, etc., and the specific reasons for the failure can be analyzed. Based on the analysis results, the original solution can be adjusted as necessary and then verified again. Furthermore, during the adjustment of the solution, steps S1 to S6 can be re-executed, and the overall verification of the solution can be performed again until verification is successful and the target solution is obtained.

[0097] In summary, the method provided in this application combines the reasoning capabilities of a large language model with Monte Carlo tree search and thought chain techniques, fully leveraging their respective advantages. For example, the large language model can generate solutions described in natural language, while the Monte Carlo tree, as the skeleton of the search, can expand the solution space of the original large language model and solve reasoning problems of complex issues, and the thought chain technique can provide support for logical reasoning.

[0098] Specifically, the Monte Carlo tree search algorithm allows for extensive exploration of the solution space. By randomly sampling and evaluating different decision paths, this algorithm generates multiple possible solutions, thus avoiding getting trapped in local optima. Meanwhile, the thought chain technology emphasizes solving complex problems through step-by-step reasoning. Combining the reasoning capabilities of a large language model, it generates detailed sub-steps at each stage, which are then analyzed and evaluated. This step-by-step optimization method ensures that each sub-step is carefully considered, thereby improving the quality of the final solution. Furthermore, in the dynamic generation, execution, and evaluation loop, the generated sub-steps can be continuously adjusted and optimized based on the execution results and feedback information. By evaluating the effectiveness of each sub-step in real time, low-quality options can be quickly identified and eliminated, thereby improving the overall quality of the solution.

[0099] The intelligent computing center provides powerful computing resources, enabling the Monte Carlo Tree Search algorithm to perform efficient random sampling and evaluation in large-scale state spaces. Simultaneously, the reasoning processes of the thought chain technology and large language models can be accelerated within the parallel computing environment provided by the intelligent computing center. Furthermore, the intelligent computing center can dynamically adjust the allocation of computing resources based on task complexity and real-time requirements. For example, more computing power can be allocated for extensive exploration during the Monte Carlo Tree Search phase, while resources can be concentrated for in-depth analysis during the thought chain reasoning phase.

[0100] In summary, by combining the computing resources of the intelligent computing center with Monte Carlo tree search algorithm, thought chain technology, and reasoning of large language models, high-quality solutions can be obtained in complex scenarios. Moreover, the dynamic generation, execution, and evaluation loop mechanism not only optimizes the efficiency of solution generation but also improves the quality of the solution.

[0101] Furthermore, based on the method provided in this application, each search generates usable data, which can support subsequent fine-tuning of the base model, such as SFT (Supervised Fine-Tuning) and the reward model. Both positive and negative examples play a crucial role. Positive examples help enhance the model's understanding of the target task, while negative examples help the model identify and avoid errors, thereby improving overall performance. By rationally utilizing synthetic data, the model can be continuously optimized, making it perform better in practical applications.

[0102] The method for model inference search based on the computing power of an intelligent computing center, as provided in the embodiments of this application, will now be explained from the perspective of specific application scenarios and code.

[0103] Taking Titan (a platform built by an agent) as an example, the root node of the Monte Carlo tree is the user's query (equivalent to the request mentioned above); the leaf nodes below it are each different step (equivalent to a sub-step), and a step is just one step in solving the query; and the number of tree branches (i.e. the number of expansions of sub-steps at the same level) can be predefined. For example, if branch=3, then for the same query, three different steps will be generated, but at the same level; simulation selects one of the steps and uses the root node (query) to the selected step as the existing solution. Based on this, simulation continues to search for solution steps downward; backtracking changes the value of all nodes on this path when a simulation step is reached or a final solution is found.

[0104] It's important to note that "value" is a crucial concept in the search process, especially in reward models within reinforcement learning. It's used to evaluate the impact of each choice on the final solution, generating a reward signal, typically a decimal between 0 and 1. These reward signals are accumulated to guide subsequent search directions and support the selection of the final solution.

[0105] Against the backdrop of the above introduction, this application focuses on the CV (Computer Vision) scenario and introduces a method for model inference search based on the computing power of an intelligent computing center, as shown in the embodiments of this application.

[0106] The query aims to develop a climbing recognition algorithm to automatically detect and issue real-time alerts to whether individuals entering the detection area have their hands raised and feet off the ground exceeding a set threshold, thereby ensuring the safety of smart parks and building real estate.

[0107] At this point, the existing steps are empty, and the large language model will provide guidance based on the query, that is, based on the current relevant information, what method should be called next. For example, the standardized guidance might be: First, the "pose function" needs to be used to estimate the pose of the video, detecting key points of people in the video, especially the positions of their hands and feet, which is the foundation for climbing recognition.

[0108] Then, the large language model expands the nodes based on opinions and context. For example, it can generate three next steps, one of which might be:

[0109] Step 1: {

[0110] "current_step":"Uses a pose estimation function to process the video, detecting key points of people in the video, especially the positions of their hands and feet, to provide basic data for subsequent climb recognition."

[0111] "function":"pose",

[0112] "input":{

[0113] "video_path":"Enter the video path"

[0114] }

[0115] It should be noted that the large language model returns information about the called method. After obtaining the called method and input parameters, it can be sent to the executor for execution, and the execution result can be obtained, which is either successful or unsuccessful. After obtaining the execution result of the step, the large language model will determine the reward score of the step based on the current query and the execution result of the step, that is, how much help this step can provide to the final solution. It should also be noted that after generating existing steps, the large language model will determine the reward score of the current step in the scoring based on the current query, the execution result of the step, and the reward scores of existing steps.

[0116] Furthermore, after obtaining the reward score, a basis can be provided for selecting the next node (sub-step, step), that is, selecting the next node to be expanded based on the size of the reward. For example: in the above three next steps, the reward score of step 1 is 0.3, while the scores of the other two next steps are 0.2 and 0.1, then the node can be expanded based on step 1, and this process can be repeated until the score of the final step is greater than the preset threshold, or the large language model determines that the solution is effective, then the search ends.

[0117] For example:

[0118] The final search results are as follows:

[0119]

[0120]

[0121] In this way, any query submitted by the user can be processed, and through multiple steps of searching, a solution that can ultimately be successfully executed can be found.

[0122] In summary, by combining the computing resources of the intelligent computing center with Monte Carlo tree search algorithm, thought chain technology, and reasoning of large language models, high-quality solutions can be obtained in complex scenarios. Moreover, the dynamic generation, execution, and evaluation loop mechanism not only optimizes the efficiency of solution generation but also improves the quality of the solution.

[0123] Figure 3 This invention illustrates an apparatus for implementing model inference search based on the computing power of an intelligent computing center, according to an embodiment of this application. The apparatus 30 includes:

[0124] The acquisition module 301 is used to execute step S1: acquire the user input request, input the request into the large language model, and obtain multiple sub-steps output by the large language model. The large language model is used to take the request as the root node of the Monte Carlo tree and, based on the thinking chain technology, reason to obtain multiple sub-steps equivalent to the leaf nodes at the same level of the Monte Carlo tree.

[0125] The execution module 302 is used to execute step S2: send multiple sub-steps into the executor for execution, and score the multiple sub-steps based on the large language model to obtain the score result corresponding to each sub-step.

[0126] To execute step S3: Select the sub-step with the highest score based on the score results, and input the sub-step with the highest score and the request into the large language model to obtain multiple new sub-steps output by the large language model. The large language model is used to reason based on the request, the sub-step with the highest score and the thinking chain technique to obtain multiple new sub-steps that are equivalent to the leaf nodes of the next level of the Monte Carlo tree. The leaf nodes of the next level are the leaf nodes of the next level extended based on the sub-step with the highest score.

[0127] For executing step S4: send multiple new sub-steps into the executor for execution, and score each new sub-step based on the large language model and the score of the sub-step with the highest score, to obtain the score result corresponding to each new sub-step.

[0128] To execute step S5: Repeat steps S3 and S4 until the score of the new sub-step is greater than the preset threshold, then record the solution formed by the sub-step with the highest score in the leaf nodes of each level; or, until the solution formed by the sub-step with the highest score in the leaf nodes of each level is determined to be valid by the large language model based on the request and the large language model.

[0129] To perform step S6: Set the solution as the target solution of the request.

[0130] In one possible implementation, the sub-steps include: the invocation method and the corresponding input parameters inferred by the large language model based on request and thought chain techniques.

[0131] In one possible implementation, the invocation method includes at least one of the following: a base invocation function, an invocation method generated by the large language model based on the base invocation function and the request.

[0132] In one possible implementation, the execution module 302 is also used to execute step S21: sending multiple sub-steps into the executor for execution and obtaining execution results, the execution results including any one of the following: execution successful, execution failed;

[0133] Execution step S22: If the execution result is execution failure, then based on the large language model, obtain the first score result corresponding to each failed sub-step, and the first score result is 0 points;

[0134] Execution step S23: If the execution result is successful, then based on the large language model, obtain the second score result corresponding to each successful sub-step. The second score result is a positive number.

[0135] In one possible implementation, the execution module 302 is also used to perform step S61: inputting the solution into the executor for verification;

[0136] Execute step S62: If the verification is successful, the solution will be used as the target solution of the request.

[0137] In summary, by combining the computing resources of the intelligent computing center with Monte Carlo tree search algorithm, thought chain technology, and reasoning of large language models, high-quality solutions can be obtained in complex scenarios. Moreover, the dynamic generation, execution, and evaluation loop mechanism not only optimizes the efficiency of solution generation but also improves the quality of the solution.

[0138] Please refer to Figure 4 This application also provides an electronic device 40, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the computer program is executed by the processor 401, it implements the various processes of the above-described method embodiment for realizing model inference search based on the computing power of an intelligent computing center, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0139] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described method embodiment for model inference search based on the computing power of an intelligent computing center, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0140] This application also provides a computer program product, including computer instructions. When executed by a processor, these computer instructions implement the various processes of the above-described method embodiment for realizing model inference search based on the computing power of an intelligent computing center, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0141] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0143] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for model inference search based on the computing power of an intelligent computing center, characterized in that, The method includes: Step S1: Obtain the user's input request, input the request into the large language model, and obtain multiple sub-steps output by the large language model. The large language model is used to take the request as the root node of the Monte Carlo tree and, based on the thinking chain technology, reason to obtain the multiple sub-steps that are equivalent to the leaf nodes at the same level of the Monte Carlo tree. Step S2: Send the multiple sub-steps into the executor for execution, and score the multiple sub-steps based on the large language model to obtain the score result corresponding to each sub-step; Step S3: Select the sub-step with the highest score based on the score results, and input the sub-step with the highest score and the request into the large language model to obtain multiple new sub-steps output by the large language model. The large language model is used to infer the multiple new sub-steps that are equivalent to the leaf nodes of the next level of the Monte Carlo tree based on the request, the sub-step with the highest score, and the thinking chain technique. The leaf nodes of the next level are the leaf nodes of the next level extended based on the sub-step with the highest score. Step S4: Send the multiple new sub-steps into the executor for execution, and score the multiple new sub-steps based on the large language model and the score of the sub-step with the highest score, to obtain the score result corresponding to each new sub-step. Step S5: Repeat steps S3 and S4 until the score of the new sub-step is greater than a preset threshold. Then record the solution formed by the sub-step with the highest score in the leaf nodes of each level; or, until the solution formed by the sub-step with the highest score in the leaf nodes of each level is determined to be valid by the large language model based on the request and the large language model. Step S6: Use the solution described above as the target solution for the request.

2. The method according to claim 1, characterized in that, The sub-step includes: the invocation method and the input parameters corresponding to the invocation method inferred by the large language model based on the request and the mind chain technology.

3. The method according to claim 2, characterized in that, The invocation method includes at least one of the following: a base invocation function, and an invocation method generated by the large language model based on the base invocation function and the request.

4. The method according to claim 1, characterized in that, Step S2 includes: Step S21: Send the multiple sub-steps into the executor for execution and obtain the execution result, which includes any one of the following: execution successful, execution failed; Step S22: If the execution result is execution failure, then based on the large language model, obtain the first score result corresponding to each failed sub-step, and the first score result is 0 points; Step S23: If the execution result is successful, then based on the large language model, obtain the second score result corresponding to each successful sub-step, and the second score result is a positive number.

5. The method according to any one of claims 1-4, characterized in that, Step S6 includes: Step S61: Input the solution into the actuator for verification; Step S62: If the verification is successful, the solution is taken as the target solution of the request.

6. A device for realizing model inference search based on the computing power of an intelligent computing center, characterized in that, The device includes: The acquisition module is used to execute step S1: acquire the user input request, input the request into the large language model, and obtain multiple sub-steps output by the large language model. The large language model is used to take the request as the root node of the Monte Carlo tree and, based on the thinking chain technology, reason to obtain the multiple sub-steps that are equivalent to the leaf nodes at the same level of the Monte Carlo tree. The execution module is used to execute step S2: send the multiple sub-steps into the executor for execution, and score the multiple sub-steps based on the large language model to obtain the score result corresponding to each sub-step. For executing step S3: Select the sub-step with the highest score based on the score result, and input the sub-step with the highest score and the request into the large language model to obtain multiple new sub-steps output by the large language model. The large language model is used to reason about the multiple new sub-steps that are equivalent to the leaf nodes of the next level of the Monte Carlo tree based on the request, the sub-step with the highest score and the thinking chain technique. The leaf nodes of the next level are the leaf nodes of the next level extended based on the sub-step with the highest score. For executing step S4: send the multiple new sub-steps into the executor for execution, and score the multiple new sub-steps based on the large language model and the score of the sub-step with the highest score, to obtain the score result corresponding to each new sub-step; To execute step S5: Repeat steps S3 and S4 until the score of the new sub-step is greater than a preset threshold, then record the solution formed by the sub-step with the highest score in the leaf nodes of each level; or, until the solution formed by the sub-step with the highest score in the leaf nodes of each level is determined to be valid by the large language model based on the request and the large language model. To perform step S6: Use the solution as the target solution for the request.

7. The apparatus according to claim 6, characterized in that, The sub-step includes: the invocation method and the input parameters corresponding to the invocation method inferred by the large language model based on the request and the mind chain technology.

8. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method for model inference search based on the computing power of an intelligent computing center as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for model inference search based on the computing power of an intelligent computing center as described in any one of claims 1-5.

10. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the method for model inference search based on the computing power of an intelligent computing center as described in any one of claims 1-5.

Images