A user problem based processing method and communication device and storage medium

Abstract

Embodiments of the present application disclose a user question-based processing method and a communication device and a storage medium, which are used to improve inference efficiency of a model and reduce consumption of GPU resources. The user question-based processing method comprises: performing inference on a first model according to inference data to obtain a first reply result, wherein the inference data comprises a first user question; performing iterative inference on the first model according to the inference data and the first reply result to obtain a second reply result; and in a case where a similarity between the first reply result and the second reply result is greater than or equal to a threshold value, outputting the second reply result or the first reply result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and specifically to a user-question-based processing method, communication device, and storage medium. Background Technology

[0002] Test-time scaling is a method to improve the performance of large language models. Current test-time scaling methods include introducing slow thinking (long thought chains), chain-based iterative algorithms, or tree search-based algorithms. However, large language models lack the ability to adaptively select the degree of scaling when faced with a wide variety of problems. For example, current slow-thinking models tend to overthink when faced with relatively simple questions, requiring more reasoning resources to obtain the answer, thus excessively consuming graphics processing unit (GPU) resources. Summary of the Invention

[0003] This application provides a user-question-based processing method, communication device, and storage medium to improve the inference efficiency of the model and reduce the consumption of GPU resources.

[0004] To address the aforementioned technical problems, this application provides the following technical solutions:

[0005] In a first aspect, embodiments of this application provide a user-question-based processing method. This method can be executed by a communication device, or by a component of the communication device, such as a processor, chip, or chip system. It can also be implemented by a logic module or software capable of performing all or part of the functions of the communication device. For example, the communication device can be a computing device. The user-question-based processing method provided in the first aspect includes:

[0006] The first model is inferred based on the inference data to obtain the first response result, wherein the inference data includes the first user question;

[0007] The first model is iteratively reasoned based on the inference data and the first response result to obtain the second response result;

[0008] If the similarity between the first response result and the second response result is greater than or equal to a threshold, output the second response result or the first response result.

[0009] In the above scheme, when the similarity between the first response result and the second response result is greater than or equal to a threshold, the inference accuracy of the first model already meets the inference performance requirements of the first model. At this point, the inference of the first model can be terminated, and the second response result or the first response result can be directly output. In the embodiments of this application, the first model can adaptively perform model inference and iterative inference, and output the second response result or the first response result when the similarity between the first response result and the second response result is greater than or equal to a threshold. For the first model, through adaptive iterative inference, the inference efficiency of the model is improved. Outputting the second response result or the first response result when the similarity between the two response results is greater than or equal to a threshold allows the first model to quickly meet the stopping condition and reduces the consumption of graphics processor resources of the communication device.

[0010] In one possible implementation of the first aspect, the iterative reasoning of the first model based on the inference data and the first response result to obtain the second response result includes: optimizing the inference data and the first response result according to the user rounds to obtain first optimized data; inputting the first optimized data into the first model, and performing inference through the first model to obtain the second response result. In the above scheme, the first response result is added after the inference data according to the user rounds to optimize the inference data, thereby obtaining the first optimized data. Then, the first optimized data is input into the first model, and performing inference through the first model to obtain the second response result. Since the first optimized data is obtained by optimizing the inference data and the first response result, two rounds of data can be added to the inference of the first model.

[0011] In one possible implementation of the first aspect, the similarity between the first response result and the second response result being greater than or equal to a threshold includes: the first response result and the second response result being identical; or, the first response result and the second response result being consistent. In the above scheme, the stopping condition of the first model may include the similarity between the first response result and the second response result being greater than or equal to a threshold. The similarity between the first response result and the second response result can be measured by a threshold, for example, a threshold of 100%, in which case the first response result and the second response result are required to be identical. Alternatively, the threshold may be a percentage; as long as the similarity between the first response result and the second response result exceeds this percentage, it can be determined that the stopping condition of the first model is met, that is, the first response result and the second response result are consistent. The percentage corresponding to this threshold is not limited.

[0012] In one possible implementation of the first aspect, the method further includes: extracting a first answer from the first response result according to an extraction rule, wherein the first answer is identified in the first response result by a first symbol indicated by the extraction rule; extracting a second answer from the second response result according to the extraction rule, wherein the second answer is identified in the second response result by a second symbol indicated by the extraction rule; the similarity between the first response result and the second response result being greater than or equal to a threshold includes: the similarity between the first answer and the second answer being greater than or equal to the threshold. In the above scheme, the determination of whether the two response results are consistent can be made through the similarity between the first answer and the second answer.

[0013] In one possible implementation of the first aspect, the method further includes: determining, through the first model, that the similarity between the first response result and the second response result is greater than or equal to a threshold. In the above scheme, determining that the similarity between the first response result and the second response result is greater than or equal to the threshold through the first model allows the first model to automatically determine whether the similarity between the two response results is greater than or equal to the threshold, simplifying the difficulty of identifying the similarity between the two response results and enhancing the generalizability of the first model.

[0014] In one possible implementation of the first aspect, the method further includes: performing model-aware iterative training on a first model based on training data to obtain a trained first model, wherein the training data includes a second user question; and performing inference on the first model based on inference data, which includes performing inference on the trained first model based on the inference data. In the above scheme, performing model-aware iterative training on the first model enables self-optimization through multiple rounds, improving the accuracy of the trained first model. Furthermore, by employing model-aware iterative training on the first model, the first model can achieve self-optimization through multiple rounds, resulting in less consumption of the graphics processing unit (GPU) resources of the communication device, thus reducing the occupancy of GPU resources.

[0015] In one possible implementation of the first aspect, the iterative training of the first model based on training data includes: inputting a second user question from the training data into the first model, training the first model to obtain a trained first model; and using the trained first model as a base model, inputting the second user question from the training data into the trained first model, and performing iterative training using the trained first model. In the above scheme, using the trained first model as a base model and repeatedly performing iterative training of the model can fully stimulate model performance.

[0016] In one possible implementation of the first aspect, the training data further includes: the standard answer corresponding to the second user question; the iterative training of the first model based on the training data includes: inputting the second user question from the training data into the first model, performing K training iterations through the first model to obtain K training results, where K is a positive integer; and performing iterative training of the model based on the training data if at least one of the K training results conforms to the standard answer. In the above scheme, the K training results obtained from K training iterations are filtered, and only training data that conforms to the standard answer can be used for iterative training of the model, thus achieving data filtering for model perception, reducing the size of the training data, and reducing the consumption of graphics processing unit resources.

[0017] In one possible implementation of the first aspect, the iterative training of the first model based on training data further includes: stopping the iterative training of the model based on training data if none of the K training results meet the standard answer. In this scheme, the K training results obtained from K training iterations are filtered, and only training data that meets the standard answer can be used for iterative training of the model. If none of the K training results meet the standard answer, the iterative training of the model based on training data is stopped. Therefore, data filtering for model perception can be achieved, reducing the size of the training data and reducing the consumption of graphics processing unit resources.

[0018] In one possible implementation of the first aspect, the model-aware iterative training of the first model based on training data further includes: updating the training data based on the K training results to obtain updated training data; and performing supervised fine-tuning of the first model based on the updated training data to obtain a trained first model. In the above scheme, supervised fine-tuning of the first model based on updated training data yields a trained first model. In this embodiment, using updated training data for supervised fine-tuning of the first model allows it to better adapt to new tasks or datasets, thereby improving model performance. Compared to training a new model from scratch, fine-tuning a pre-trained model requires less time and computational resources. Furthermore, by combining a pre-trained model with task-specific data, the model can better generalize to new, unseen data, thereby improving its generalization ability.

[0019] In one possible implementation of the first aspect, the training data further includes: the standard answer corresponding to the second user question; the iterative training of the first model based on the training data includes: inputting the second user question from the training data into the first model, training the first model to obtain a first training result; obtaining second optimized data based on the first training result and the standard answer; inputting the second optimized data into the first model, and iteratively training the first model. In the above scheme, since the second optimized data is obtained by optimizing the first training result and the standard answer, two rounds of data can be added to the iterative training of the first model, thereby improving the training efficiency of the first model.

[0020] Secondly, embodiments of this application provide a user-question-based processing method. This method can be executed by a communication device, or by a component of the communication device, such as a processor, chip, or chip system, or by a logic module or software capable of implementing all or part of the functions of the communication device. For example, the communication device can be a computing device. The user-question-based processing method provided in the second aspect includes:

[0021] The first model is iteratively trained using model-aware methods based on training data to obtain a trained first model, wherein the training data includes a second user question; the trained first model is iteratively inferred based on inference data to obtain a first response result and a second response result; if the similarity between the first response result and the second response result is greater than or equal to a threshold, the second response result or the first response result is output.

[0022] In one possible implementation of the second aspect, the iterative training of the first model based on training data includes: inputting a second user question from the training data into the first model, training the first model to obtain a trained first model; and using the trained first model as a base model, inputting the second user question from the training data into the trained first model, and iteratively training the trained first model.

[0023] In one possible implementation of the second aspect, the training data further includes: the standard answer corresponding to the second user question; the iterative training of the first model based on the training data includes: inputting the second user question from the training data into the first model, performing K training iterations through the first model to obtain K training results, where K is a positive integer; and performing iterative training of the model based on the training data if at least one of the K training results matches the standard answer.

[0024] In one possible implementation of the second aspect, the iterative training of the first model based on the training data further includes: stopping the iterative training of the model based on the training data if none of the K training results meet the standard answer.

[0025] In one possible implementation of the second aspect, the step of performing model-aware iterative training on the first model based on the training data further includes: updating the training data based on the K training results to obtain updated training data; and performing supervised fine-tuning on the first model based on the updated training data to obtain the trained first model.

[0026] In one possible implementation of the second aspect, the training data further includes: the standard answer corresponding to the second user question; the iterative training of the first model based on the training data includes: inputting the second user question from the training data into the first model, training the first model to obtain a first training result; obtaining second optimized data based on the first training result and the standard answer; inputting the second optimized data into the first model, and iteratively training the first model.

[0027] Thirdly, embodiments of this application also provide a communication device, including:

[0028] The reasoning module is used to reason about the first model based on reasoning data to obtain a first response result, wherein the reasoning data includes a first user question;

[0029] An iterative reasoning module is used to perform iterative reasoning on the first model based on the reasoning data and the first response result to obtain a second response result;

[0030] The output module is used to output either the second response result or the first response result if the similarity between the first response result and the second response result is greater than or equal to a threshold.

[0031] In a third aspect of this application, the constituent modules of the communication device may also perform the steps described in the first aspect and various possible implementations, as detailed in the foregoing description of the first aspect and various possible implementations.

[0032] Fourthly, embodiments of this application also provide a communication device, including:

[0033] An iterative training module is used to perform model-aware iterative training on the first model based on training data to obtain the trained first model, wherein the training data includes the second user question.

[0034] The iterative inference module is used to perform iterative inference on the trained first model based on inference data, and obtain the first response result and the second response result respectively.

[0035] The output module is used to output either the second response result or the first response result if the similarity between the first response result and the second response result is greater than or equal to a threshold.

[0036] In the fourth aspect of this application, the constituent modules of the communication device may also perform the steps described in the second aspect and various possible implementations, as detailed in the foregoing description of the second aspect and various possible implementations.

[0037] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first or second aspect above.

[0038] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the first or second aspect above.

[0039] In a sixth aspect, embodiments of this application provide a communication device, which may include entities such as computing devices, terminal devices, network devices, or chips. The communication device includes: a processor and a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, causing the communication device to perform the method as described in any one of the first or second aspects above.

[0040] In a seventh aspect, this application provides a chip system including a processor for supporting a communication device in implementing the functions involved in the first or second aspect described above, such as transmitting or processing data and / or information involved in the methods described above. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the communication device. This chip system may be composed of chips or may include chips and other discrete devices.

[0041] Eighthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the method in the first aspect or any possible implementation of the first aspect.

[0042] The eighth aspect and any implementation thereof correspond to any implementation thereof of the first aspect and any implementation thereof, respectively. The technical effects corresponding to the eighth aspect and any implementation thereof are similar to those corresponding to the second aspect and any implementation thereof, and will not be repeated here. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of a data center architecture provided in an embodiment of this application;

[0044] Figure 2a This is an exemplary system diagram provided in an embodiment of this application;

[0045] Figure 2b This is another exemplary system diagram provided in the embodiments of this application;

[0046] Figure 3 This is a schematic diagram of an embodiment of a user problem processing method provided in this application;

[0047] Figure 4 This is a schematic diagram of another user problem-based processing method provided in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the architecture of a model-aware iterative training process provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the framework of an adaptive self-iterative reasoning process provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the composition structure of a communication device provided in an embodiment of this application;

[0051] Figure 8 This is a schematic diagram of the composition structure of a communication device provided in an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0053] Figure 10 This is a schematic diagram of a computing device cluster provided in an embodiment of this application;

[0054] Figure 11 This is a schematic diagram of a computing device cluster provided in an embodiment of this application. Detailed Implementation

[0055] This application provides a user-question-based processing method, communication device, and storage medium to improve the inference efficiency of the model and reduce the consumption of GPU resources.

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0057] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0058] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not expressly listed or inherent to those processes, methods, products, or apparatus.

[0059] In the field of deep learning, especially during supervised fine-tuning (SFT) of large language models (LLMs), the models that need to be saved at different training stages exhibit significant performance differences and instability across various sub-capabilities, thus reducing the performance of models saved at different training stages. Meanwhile, LLM model fusion aims to combine multiple pre-trained LLMs into a new model to achieve better overall performance, offering advantages such as high efficiency and low cost.

[0060] First, let me explain the terminology that may be involved in the embodiments of this application:

[0061] (1) Infrastructure

[0062] Infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This communication occurs through sensors; computing power is provided by intelligent chips, such as hardware acceleration chips (CPUs, NPUs, GPUs, ASICs, or FPGAs); the basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, which is then provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0063] (2) Data

[0064] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0065] (3) Data processing

[0066] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0067] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0068] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0069] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0070] (4) General ability

[0071] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0072] (5) Smart Products and Industry Applications

[0073] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent terminals, intelligent transportation, intelligent healthcare, autonomous driving, and safe cities.

[0074] This application relates to applications of deep learning. To better understand the solutions in this application, the relevant terms and concepts that may be involved in this application are introduced below. It should be understood that the explanation of the relevant concepts may be limited due to the specific circumstances of this application, but it does not mean that this application is limited to only that specific situation. The specific circumstances of different embodiments may also differ, and no specific limitation is made here.

[0075] Large language models (LLMs) are a class of deep learning-based artificial intelligence models designed to process and generate natural language text. Trained on large-scale text datasets, LLMs can understand and generate text similar to human language, performing various natural language processing tasks including text generation, translation, and sentiment analysis. The core technical architecture of LLMs is the Transformer model, a deep learning model based on a self-attention mechanism. The Transformer architecture captures dependencies in the sequence by calculating the attention score of each element to other elements, thus achieving efficient processing of natural language. LLMs are widely used in natural language processing tasks, including question answering, summarization, and sentiment analysis.

[0076] Large vision-language model (LVLM): This is a model that combines vision and language processing capabilities to handle various types of tasks, including image description, question answering, and content creation. By integrating a large vision-language model (LVLM), it improves the processing performance of multimodal tasks, enabling it to understand image / audio content and generate relevant text descriptions, or generate corresponding images / audio based on text descriptions.

[0077] Supervised fine-tuning (SFT) involves training a pre-trained model using labeled data to adjust its parameters, making it better suited for a specific task. The main purpose of this process is to improve the model's performance on specific tasks, such as text classification, sentiment analysis, and translation.

[0078] Neural network components: These refer to the basic units and structures that make up a neural network. These components work together to enable the neural network to learn and process complex data.

[0079] A multilayer perceptron (MLP) is a type of feedforward neural network. An MLP consists of an input layer, multiple hidden layers, and an output layer, and is one of the fundamental models in deep learning.

[0080] Semantic consistency: Semantic consistency mainly refers to the mutual consistency of meaning among the words or phrases in a sentence or text. It requires that the concepts, facts, and arguments within the text remain consistent and avoid semantic contradictions, ambiguities, and confusion. The semantic consistency of a model typically refers to the model producing semantically consistent results given synonymous inputs.

[0081] A prompt is a text or signal that guides the model to generate a specific output.

[0082] Intermediate feature vectors (IMVs) are feature representations generated in intermediate layers of a neural network when processing input data. These IMVs are extracted from the original input data by the neural network through a series of transformations and calculations. They capture the key attributes and patterns of the input data and are used for subsequent classification, regression, or other tasks. Intermediate IMVs play a crucial role in deep learning. They not only provide the model with rich feature representations but also enable the model to learn complex patterns and structures in the input data. By adjusting and optimizing the parameters and structure of the neural network, more expressive and discriminative intermediate IMVs can be generated, thereby improving the model's performance and accuracy.

[0083] (6) Test-time Scaling

[0084] Test-time scaling is a way to improve the performance of large language models, while training-time scaling extends training time by increasing the model size and data. Test-time scaling improves performance differently from training-time scaling. After the model is trained, it improves performance by extending the time spent on testing.

[0085] Agent devices are intelligent agents that can autonomously complete tasks and interact with users. They can be played by large language models by introducing prompts from different systems.

[0086] Hints are input to a large language model; for example, hints can be natural language strings.

[0087] Prompt engineering is a technique that improves the performance of large language models by finding more suitable prompts.

[0088] Model-aware is a method that determines specific operations based on the trained base model.

[0089] The background of this application's embodiments stems primarily from the limitations of current test-time scaling methods. Current test-time scaling methods, including those incorporating slow thinking (long thought chains), chain-based iterative algorithms, or tree-search-based algorithms, while performing well in handling complex problems, suffer from excessive consumption of graphics processing unit (GPU) resources. These models lack the ability to adaptively select the scaling degree when faced with a wide variety of problems. For example, current slow-thinking models such as OpenAIO1 and DeepSeek R1 exhibit overthinking even with relatively simple questions (such as "What is 2+3?"), requiring considerable inference time to arrive at the answer.

[0090] Furthermore, current scaling methods perform poorly in agent scenarios, and there is no scaling method that performs well in both general and agent scenarios.

[0091] In one current approach, slow-thinking techniques, exemplified by OpenAIO1 / O3 and DeepSeek R1 models, are an effective method for test-time scaling. This approach improves the model's accuracy when faced with complex reasoning problems by allowing it to engage in long thought processes before providing an answer.

[0092] It suffers from at least the following drawbacks: It relies on extensive training to enable the model to master slow-thinking techniques, which is time-consuming. The process of constructing the corresponding training data is complex, relying on distillation or cueing engineering of larger slow-thinking models to assemble the slow-thinking data. It cannot adaptively select the length of the thinking process, leading to overthinking issues.

[0093] Another approach involves implementing test-time scaling using tree search algorithms. Representative algorithms include Beam Search and Monte Carlo Tree Search (MCTS). Specifically, this approach introduces tree search, generating multiple steps for each user's question. These generated steps serve as nodes in a tree, and each node is evaluated or scored. The appropriate node is selected, and the tree is continuously expanded until the problem is solved. Utilizing the randomness of model inference, the model's potential can be continuously expanded, improving performance through increased inference overhead. However, because it relies on an additional verifier to evaluate or score each node, the final performance is highly sensitive to the choice of verifier and the training method, requiring high generalization ability from the verifier. Furthermore, it cannot adaptively select the scaling method based on the problem difficulty; parameters such as tree width and depth are pre-set hyperparameters.

[0094] Another approach currently employs chain-iterative algorithms for test-time expansion. Representative algorithms include Best-of-N (BFN) and Sequential Revisions. Unlike tree-search-based methods, this type of algorithm expands only in a single direction. For example, unidirectional expansion can include lateral expansion, allowing the model to independently answer the same question multiple times and then select the most suitable answer. It can also include vertical expansion, where the model continuously improves its responses to the same question until a suitable answer is obtained. This approach also leverages the randomness of model inference, requiring fewer pre-set hyperparameters and a simpler implementation compared to tree-search-based methods. It can also utilize majority voting without requiring validation. However, since answer selection is still necessary, it relies on validation or majority voting. Majority voting, however, can only select the final answer and cannot select suitable process responses. Furthermore, it cannot adaptively select the expansion method based on question difficulty, meaning it cannot use the same method for questions of all difficulty.

[0095] This application addresses the problem that current Test-time Scaling methods cannot adaptively adjust the scaling degree according to the problem, ensuring performance improvement of scaling while balancing inference efficiency.

[0096] This application proposes an adaptive test-time scaling method. Based on a self-iterative framework, it introduces a model-aware iterative training process and an adaptive self-refinement mechanism to optimize the balance between model performance and efficiency across various scenarios. By incorporating stopping condition learning into model training, the model learns to identify situations where early stopping is possible during iterations. This allows for different scaling levels for different problems, while also enabling the selection of appropriate responses without relying on validation. Therefore, this application can optimize the model's inference performance when solving various problems while maintaining inference efficiency.

[0097] The user-question-based processing method provided in this application can be executed on a communication device, such as a server, or on a terminal device. The terminal device can be a mobile phone with text, voice, and image processing capabilities, a tablet personal computer (TPC), a media player, a smart TV, a laptop computer (LC), a personal digital assistant (PDA), a personal computer (PC), a camera, a camcorder, a smartwatch, a wearable device (WD), or an autonomous vehicle, etc. This application does not limit the scope of the method.

[0098] The user-question-based processing method provided in this application can be applied to a computing device cluster, which may include one or more computing devices.

[0099] The type of computing device is not limited here. For example, any computing device can be a terminal device, a cloud server, a container, or a virtual machine, etc.

[0100] This computing device cluster can provide a data processing platform to implement the user-problem-based processing method of this application embodiment. Specifically, a data processing platform can be provided in the terminal device and in the cloud server. Based on the respective data processing platforms of the terminal device or the cloud server, consistency and computational accuracy between the terminal side and the cloud side can be achieved, while also making full use of the differences in computing power between different devices. The specific form of this computing device cluster and the corresponding data processing platform is not limited here.

[0101] In one example, the cluster of computing devices can be used to implement a cloud management platform; in other words, the data processing platform of this application embodiment can be implemented through a cloud management platform.

[0102] A cloud management platform is used to manage the infrastructure that provides cloud services. It can provide computing, networking, and storage capabilities based on hardware and software resources. For example, a cloud management platform may include one or more data centers to provide cloud resources through one or more data centers.

[0103] The following is combined Figure 1 The diagram shown illustrates an architecture for a data center.

[0104] Figure 1 In this data center, the cloud management platform communicates with one or more servers (such as...) via the data center's internal network. Figure 1 Servers 1 and 2 interact with each other. The server comprises a hardware layer and a software layer. The hardware layer includes the server's hardware configuration, such as peripheral component interconnect (PCI) devices (e.g., network cards, graphics processing units, GPUs, offloading cards, etc.) that can be plugged into peripheral component interconnect (PCI) or peripheral component interconnect express (PCIe) slots. The software layer includes the operating system installed and running on the server (the operating system relative to the virtual machine can be called the host operating system). The host operating system contains a virtual machine manager (also called a hypervisor), whose role is to implement computational virtualization, network virtualization, and storage virtualization of the virtual machine, and to manage the virtual machine. A virtual machine (virtual machine) refers to a complete computer system simulated by software, possessing full hardware system functionality, and running in a completely isolated environment. Figure 1 In the system architecture shown, a data center contains multiple servers, which can be used to run virtual machines. The specifications of the virtual machines can be the same or different. These virtual machines can also be called Elastic Compute Service (ECS), Elastic Instances, etc., and different cloud service providers may use different names.

[0105] In one example of the embodiments of this application, the cloud management platform can be a public cloud platform. In this case, cloud service providers such as individuals or software developers with cloud resource development capabilities can provide cloud services to users. Users obtain cloud services through the Internet, but do not own cloud computing resources.

[0106] Specifically, in Figure 1 In the example shown, the cloud management platform can provide access interfaces (such as user interfaces or application programming interfaces, APIs). Users of the cloud management platform and cloud service providers can operate the client to remotely access the access interface to register cloud accounts and passwords on the cloud management platform. After the cloud management platform successfully authenticates the cloud accounts and passwords, they can log in to the cloud management platform to create, manage, log in to, and operate virtual machines in the cloud data center, thereby performing corresponding tasks through the cloud resources of the cloud management platform.

[0107] For example, such as Figure 2a In the example shown, the data processing platform that implements the user problem-based processing method in the embodiments of this application can be provided to the user in the form of a cloud service.

[0108] When data processing tasks require cross-device storage of graph data, some enterprises, organizations, or individuals can purchase cloud services that include the data processing platform. They can then use the cloud resources of the cloud management platform to perform the relevant data processing tasks and obtain the processing results of the input data from the cloud management platform.

[0109] This data processing platform can be implemented based on cloud resources such as one or more cloud data centers in the cloud management platform.

[0110] This data processing platform can be provided to users as an independent cloud service, and it is also compatible with computing frameworks such as TensorFlow and PyTorch. In other words, it can be used as an operator in computing frameworks such as TensorFlow and PyTorch, providing functions such as preprocessing, computation, feature extraction and / or other data processing for real-time and offline data.

[0111] Of course, the cloud management platform can also be other types of cloud management platforms, and this application embodiment does not limit this.

[0112] In another example, such as Figure 2bAs shown, the data processing platform can be provided to the user in the form of a software product or a functional service within a software product. The software product can be deployed on the user's terminal device or on the enterprise user's server cluster. The user can purchase a software license to obtain the right to use the software product and, within the license period, implement the user problem-based processing method of this application embodiment through the software product.

[0113] The following section uses the field of intelligent question answering as an example to illustrate the practical application of a data processing platform that implements this user question-based processing method.

[0114] In one exemplary application scenario, a user might want to transmit their question to a cloud server for subsequent processing, such as model inference. The user can train and infer the model using computing frameworks like TensorFlow and PyTorch, thereby outputting a response. This computing framework can be deployed on client devices or on the cloud server. After receiving the user's question, the cloud management platform utilizes cloud resources to provide model training and inference.

[0115] It is understood that the above example is only an exemplary introduction to one application scenario. The user problem-based processing method of this application embodiment can be applied to multiple fields, and is not limited here.

[0116] like Figure 3 As shown in the figure, a user question-based processing method provided in this application embodiment can realize an adaptive iterative reasoning approach and may include the following steps:

[0117] 301. Based on the inference data, infer the first model to obtain the first response result, wherein the inference data includes the first user question.

[0118] In this embodiment, the user can provide a first user query through an agent. The agent can then input this first user query as inference data into a first model. This first model can be a model trained using a machine learning algorithm, such as an LLM (Limited Learning Model). This embodiment does not limit the training method or the type of the first model. For example, the user query can involve various scenarios, such as an agent scenario where the model selects a tool from a candidate toolset. Alternatively, the user query can involve data inference scenarios, such as asking for mathematical calculations, which the model then solves. In this embodiment, the inference of the first model requires the use of the graphics processing unit (GPU) resources of the communication device, i.e., the graphics card resources of the communication device.

[0119] For example, a user question could be "Help me book two plane tickets from Shanghai to Beijing tomorrow," or "Help me book a hotel in Shanghai from March 20th to 22nd." The candidate toolset for this user question could include: flight_booking(), hotel_booking(), and get_weather(), etc. The first model can combine the user question with inference from the candidate toolset to obtain the first response result.

[0120] In this embodiment, the inference data input to the first model includes not only the first user question but also at least one of the following: context information, intermediate inference data, external knowledge data, test-time enhancement data, and external tool data. Context information may include user dialogue history, previous user interaction states, or task-related background information. Intermediate inference data is the thought chain or other intermediate labeled sequence generated by the model, used to decompose complex problems and deduce answers step by step. External knowledge data is external knowledge bases, databases, or structured data obtained through techniques such as enhanced generation, used to supplement the model's internal knowledge. Fine-tuning and alignment data is domain-specific data introduced during supervised fine-tuning or reinforcement learning. Test-time enhancement data may include dynamically adjusted prompt templates, batch samples, or scaling techniques, used to improve inference efficiency or generation quality. External tool data is the result returned by calling external tools.

[0121] In this embodiment, the composition of the reasoning data is not limited. Any reasoning data used to support the first model in generating answers through logical reasoning can be used as the reasoning data input to the first model. No limitation is made here.

[0122] In this embodiment, after inputting reasoning data into the first model, the first model can reason about a first user question. Through the reasoning process of the first model, a first response result can be output, which can also be called the first answer. In different application scenarios, the first response result output by the first model can be determined according to the scenario. For example, if the first user question is "What is 2+3?", then the first response result output by the first model can be the result obtained by calculating 2+3.

[0123] 302. Based on the reasoning data and the first response result, iteratively reason the first model to obtain the second response result.

[0124] In this embodiment, the first model can obtain a first response result after inference based on the inference data. The first model can also perform adaptive iterative inference, that is, the first model input obtained from the initial inference is re-inputted into the first model. Iterative inference can be performed on the first model based on the inference data and the first response result. Since the first model can automatically continue inference using the inference data and the first response result after outputting the first response result, adaptive iterative inference of the first model can be achieved, outputting either a second response result or the first response result. The number of iterations for the first model is not limited; for example, it can be performed once or multiple times. The second response result can be obtained through one iteration or multiple iterations; this is not limited here. In this embodiment, the number of iterations for the first model is not limited; for example, it can be performed once or multiple times. Therefore, the second response result can include the response result obtained through one iteration or multiple iterations.

[0125] In some embodiments of this application, step 302, as described above, iteratively infers the first model based on the inference data and the first response result to obtain the second response result, including:

[0126] A1. Optimize the inference data and the first response result based on the user round to obtain the first optimized data;

[0127] A2. Input the first optimized data into the first model, and use the first model to perform inference to obtain the second response result.

[0128] After obtaining the first response result through model inference, user turns are added to optimize the first model. Multi-turn dialogues are introduced during the first model's inference process. These multi-turn dialogues refer to the interaction between the first model and the user through continuous, multi-round information exchange to complete complex tasks. Through these multi-turn dialogues, the agent can understand the context of the inference data input by the user turns and maintain coherent tracking of core tasks and key details. Multi-turn dialogues can be divided into user turns and assistant turns. The user turns present their needs, and the agent uses the first model to infer and answer the user's questions. The first response result is added after the inference data according to the number of user turns to optimize the inference data, resulting in first optimized data. This first optimized data is then input into the first model for inference to obtain the second response result. Since the first optimized data is obtained by optimizing the inference data and the first response result, two rounds of data can be added to the first model's inference process.

[0129] In some embodiments of this application, the similarity between the first response result and the second response result is greater than or equal to a threshold, including:

[0130] The results of the first and second replies are the same;

[0131] or,

[0132] The results of the first and second replies are consistent.

[0133] The stopping condition for the first model can include a similarity between the first and second response results greater than or equal to a threshold. This similarity can be measured by a threshold, for example, a threshold of 100%, which requires the first and second response results to be identical. Alternatively, the threshold can be a percentage; if the similarity between the first and second response results exceeds this percentage, the stopping condition for the first model is satisfied, meaning the first and second response results are identical. The percentage corresponding to this threshold is not specified.

[0134] In some embodiments of this application, in addition to performing the aforementioned method steps, the user-question-based processing method further includes:

[0135] B1. Extract the first answer from the first response result according to the extraction rules. The first answer is identified in the first response result by the first symbol indicated by the extraction rules.

[0136] B2. Extract the second answer from the second response result according to the extraction rules. The second answer is identified in the second response result by the second symbol indicated by the extraction rules.

[0137] B3. The similarity between the first response and the second response is greater than or equal to the threshold, including: the similarity between the first answer and the second answer is greater than or equal to the threshold.

[0138] Extraction rules are used to extract answers from responses. For example, extraction rules can include identifying answers in responses using symbols. Therefore, for the first and second responses, answers are extracted according to the extraction rules. Specifically, the first answer is extracted using the first symbol indicated by the extraction rules, and the second answer is extracted using the second symbol indicated by the extraction rules. The similarity between the two responses can be determined by the similarity between the first and second answers. For example, these extraction rules can be simply called rules, and they can be Python regular expressions, etc. The specific rules used and the extracted answers depend on the actual data.

[0139] For example, based on rules, it is possible to determine whether the first and second responses are consistent. In model responses, the final answer is often placed at the end and wrapped in a special symbol. The final answer can be extracted using this rule, and the two extracted answers can be compared.

[0140] In other embodiments of this application, in addition to performing the aforementioned method steps, the user-question-based processing method further includes:

[0141] C1. The similarity between the first response result and the second response result is determined to be greater than or equal to the threshold by the first model.

[0142] In this process, through iterative reasoning of the first model, after obtaining the first and second response results, the first model can determine whether the similarity between the first and second response results is greater than or equal to a threshold. In other words, the first model can answer whether the similarity between the two response results is greater than or equal to the threshold, simplifying the difficulty of identifying the similarity between the two response results. The first model has stronger generalization ability.

[0143] For example, the method uses a first model to determine whether two responses are consistent, based on a similarity threshold of 7 or 8. This determination is far less difficult than the difficulty of scoring answers using verifiers in other extended algorithms, and it also has stronger generalization capabilities. For instance, other extended algorithms directly score an answer, such as 1-10, which can introduce errors. In contrast, the method in this embodiment greatly simplifies the model's reasoning process by determining whether two responses are consistent. For example, the question is: What is 9 to the power of 3? The answer is: Let's calculate step by step… The answer is 729. The difficulty of the first model directly determining whether this answer is correct is much simpler than scoring the answer itself. For example, the model can determine whether the following two answers are consistent: Answer 1: The answer is 729, Answer 2: 9 to the power of 3 is 729.

[0144] 303. If the similarity between the first response result and the second response result is greater than or equal to the threshold, output the second response result or the first response result.

[0145] In this embodiment of the application, after the first model performs iterative reasoning to obtain the second response result, it can be determined whether the first response result and the second response result output by the first model satisfy the stopping condition. The content of the stopping condition is not limited. For example, the stopping condition of the first model is pre-configured. After obtaining two response results, it can be directly determined whether the two response results satisfy the stopping condition. If the stopping condition is satisfied, the reasoning of the first model can be terminated, and the most recently obtained response result can be output.

[0146] In this embodiment, to simplify the reasoning process of the first model and reduce the consumption of reasoning resources, the stopping condition can be set to the similarity between the first response result and the second response result being greater than or equal to a threshold. This similarity can also mean that the first response result and the second response result are consistent. The similarity between the first response result and the second response result can be achieved using a cosine similarity algorithm or obtained through other similarity algorithms.

[0147] In this embodiment, when the similarity between the first response result and the second response result is greater than or equal to a threshold, it indicates that the inference accuracy of the first model has met the inference performance requirements of the first model. At this point, the inference of the first model can be terminated, and the second response result or the first response result can be directly output. In this embodiment, the first model can adaptively perform model inference and iterative inference, and output the second response result or the first response result when the similarity between the first response result and the second response result is greater than or equal to a threshold. For the first model, adaptive iterative inference improves the inference efficiency of the model. Outputting the second response result or the first response result when the similarity between the two response results is greater than or equal to a threshold allows the first model to quickly meet the stopping condition and reduces the consumption of graphics processor resources of the communication device.

[0148] like Figure 4 As shown in the embodiments of this application, a method for processing user questions is also provided, characterized in that the method includes:

[0149] 401. Perform iterative training of the first model with model awareness based on the training data to obtain the trained first model. The training data includes the second user question.

[0150] In this embodiment, the user can provide a second user query through the agent. The agent can then use the second user query as training data to input into the initial first model. The first model can be a model trained by a machine learning algorithm, such as an LLM. The training method and type of the first model are not limited in this embodiment.

[0151] In this embodiment, a second user question from the training data is input into a first model, and the first model undergoes iterative training with model awareness to obtain a trained first model. Model awareness refers to the process of adjusting the parameters of the first model using training data to enable it to correctly classify training samples. Specifically, model awareness in machine learning, particularly for binary classification problems, involves adjusting the model's weights and biases through iterative optimization algorithms to minimize the number of misclassified samples. In this embodiment, the iterative training of the first model can be performed without limitation on the number of iterations; for example, it can be performed once or multiple times. This iterative training with model awareness in this embodiment allows for self-optimization of the first model over multiple rounds, improving the accuracy of the trained first model. Furthermore, by employing iterative training with model awareness, the first model can achieve self-optimization over multiple rounds, reducing the consumption of graphics processing unit (GPU) resources on the communication device and minimizing GPU resource usage.

[0152] In some embodiments of this application, optionally, in addition to performing the aforementioned step 401, the communication device may also perform the following steps 402 and 403.

[0153] 402. Based on the inference data, perform iterative inference on the trained first model to obtain the first response result and the second response result respectively.

[0154] 403. If the similarity between the first response result and the second response result is greater than or equal to the threshold, output the second response result or the first response result.

[0155] For explanations of 402 and 403 mentioned above, please refer to the aforementioned... Figure 3 Steps 301 to 303 shown are not repeated here.

[0156] Specifically, the first model is iteratively trained using model-aware methods based on training data, including a second user question. Through this iterative training, the trained first model is output. Then, the trained first model is iteratively inferred using inference data to obtain a first response result and a second response result.

[0157] In some embodiments of this application, step 401 performs model-aware iterative training on the first model based on training data, including:

[0158] D1. Input the second user question from the training data into the first model, train the first model, and obtain the trained first model;

[0159] D2. Using the first trained model as the base model, input the second user question from the training data into the first trained model, and perform iterative training through the first trained model.

[0160] In this embodiment, after inputting the second user question from the training data into the first model, the first model can reason about the second user question. Through the reasoning process of the first model, it can output a second response result or a first response result, which can also be called a second answer. The second response result output by the first model can be determined according to different application scenarios. The model is trained based on the obtained second response result to obtain a trained first model. Using the trained first model as a base model, the second user question from the training data is input into the trained first model, and iterative training is performed through the trained first model. In this embodiment, using the trained first model as a base model and repeatedly executing the iterative training of the model can fully stimulate the model performance.

[0161] In some embodiments of this application, the training data further includes: the standard answer to the second user question;

[0162] Step 401 involves iterative training of the first model using model-aware training data, including:

[0163] E1. Input the second user question from the training data into the first model, and train the first model K times to obtain K training results, where K is a positive integer;

[0164] E2. If at least one of the K training results matches the standard answer, perform iterative training of the model-aware system based on the training data.

[0165] The user-provided training data includes not only the second user question but also the corresponding standard answer. The standard answer can be specifically configured for different models and user questions. After the second user question is input into the training data, the first model can reason about it. Through this reasoning process, the first model outputs training results. Multiple training iterations are performed based on these results, yielding one training result per iteration. With K training iterations, K training results are obtained. If at least one of these K results matches the standard answer, iterative training for model perception is performed. Since the K training results from the K iterations can be filtered, only those matching the standard answer can be used for iterative training of model perception. This enables data filtering for model perception, reducing the size of the training data and minimizing the consumption of GPU resources.

[0166] Furthermore, in some other embodiments of this application, step 401, which involves performing model-aware iterative training on the first model based on the training data, further includes:

[0167] E3. If none of the K training results meet the standard answer, stop iterative training of the model based on the training data.

[0168] With K training iterations, K training results can be obtained. If at least one of the K training results matches the standard answer, iterative training of model perception can be performed based on the training data. Since the K training results obtained from the K training iterations can be filtered, only the training data that matches the standard answer can be used for iterative training of model perception. If none of the K training results match the standard answer, iterative training of model perception based on the training data is stopped. Therefore, data filtering for model perception can be achieved, reducing the size of the training data and reducing the consumption of graphics processing unit resources.

[0169] In some embodiments of this application, step 401, which involves iterative training of the first model using model-aware training data, further includes:

[0170] F1. Update the training data based on K training results to obtain the updated training data;

[0171] F2. Perform supervised fine-tuning of the first model based on the updated training data to obtain the trained first model.

[0172] In this embodiment, K training iterations yield K training results. The training data is then updated based on these K results to obtain updated training data. Updating the training data can involve removing invalid data from the training data based on the K results, thereby reducing the size of the training data set. After obtaining the updated training data, the first model is supervised fine-tuned using this updated data to obtain the trained first model. In this embodiment, the updated training data is used for supervised fine-tuning of the first model, allowing it to better adapt to new tasks or datasets, thus improving model performance. Compared to training a new model from scratch, fine-tuning a pre-trained model requires less time and computational resources. Furthermore, by combining the pre-trained model with task-specific data, the model can better generalize to new, unseen data, thereby improving its generalization ability.

[0173] In some embodiments of this application, the training data further includes: the standard answer to the second user question;

[0174] Step 401 involves iterative training of the first model using model-aware training data, including:

[0175] G1. Input the second user question from the training data into the first model, train the model, and obtain the first training result;

[0176] G2. Obtain second optimized data based on the first training results and the standard answer;

[0177] G3. Input the second optimized data into the first model and perform iterative training through the first model.

[0178] After obtaining the first training result through model training, user turns are added to optimize the first model. Multi-turn dialogues are introduced during the training process of the first model. These multi-turn dialogues refer to the interaction between the first model and the user through continuous information exchange to complete complex tasks. Through these multi-turn dialogues, the agent can perform contextual understanding of the inference data input by the user turns and maintain the ability to coherently track core tasks and key details. These multi-turn dialogues can be divided into user turns and assistant turns. A user turn is one turn in the multi-turn dialogue, where the user raises a request, and the agent uses the first model to infer and answer the user's question. A standard answer is added to the first training result according to the number of user turns to optimize the training data, resulting in second optimized data. This second optimized data is then input into the first model for iterative training. Since the second optimized data is obtained by optimizing the first training result and the standard answer, it constitutes two rounds of data added to the iterative training of the first model, thereby improving the training efficiency of the first model.

[0179] As illustrated by the foregoing embodiments, model-aware iterative training of the first model enables self-optimization through multiple rounds, improving the accuracy of the trained first model. Furthermore, by employing model-aware iterative training, the first model can achieve self-optimization through multiple rounds, consuming less GPU resources in the communication device and reducing GPU resource usage. When the similarity between the first and second response results is greater than or equal to a threshold, it indicates that the inference accuracy of the first model meets the inference performance requirements. At this point, inference of the first model can be terminated, and either the second or first response result can be directly output. In this embodiment, the first model can adaptively perform model inference and iterative inference, and output the second or first response result when the similarity between the first and second response results is greater than or equal to a threshold. For the first model, adaptive iterative inference improves inference efficiency, and outputting the second or first response result when the similarity between two response results is greater than or equal to a threshold allows the first model to quickly meet the stopping condition, reducing the consumption of GPU resources in the communication device.

[0180] To facilitate a better understanding and implementation of the above-described solutions in the embodiments of this application, specific examples of corresponding application scenarios are provided below.

[0181] This application proposes an adaptive test-time scaling method based on a self-iterative framework. It introduces a model-aware iterative training process and an adaptive self-refinement mechanism to optimize the balance between model performance and efficiency across various scenarios. By incorporating stopping condition learning into model training, the model learns to identify situations where early stopping is possible during iterations. This allows for different scaling levels for different problems, while simultaneously selecting appropriate responses without relying on additional validation processes.

[0182] In the system architecture applicable to the embodiments of this application, it can be embedded in the inference system of a deep learning model. The inference system may include a data preprocessing module, an adaptive iterative inference module, a stopping strategy module, and a result post-processing module, wherein...

[0183] The data preprocessing module is used to standardize and normalize the input data;

[0184] The adaptive iterative inference module and the stopping strategy module are used to dynamically adjust the number of inference iterations and the stopping time based on the real-time performance feedback of the model.

[0185] The post-processing module is used to decode and parse the output of the model.

[0186] The embodiments of this application can be widely applied to scenarios requiring real-time response and high accuracy, such as intelligent tutoring, intelligent customer service, and medical diagnosis. For example, in intelligent tutoring scenarios, the embodiments of this application can improve the model's accuracy in handling complex questions while maintaining a fast response speed for simple questions. Similarly, in intelligent customer service scenarios, it can improve the model's understanding of user questions and the quality of its responses. Furthermore, in medical diagnosis scenarios, it can improve the model's accuracy in identifying lesions and its diagnostic efficiency.

[0187] This application provides an adaptive test-time scaling method applicable to various products based on large language models, such as agent assistants. It can provide corresponding functionality through software update packages or as a cloud service, allowing users to utilize computing resources on demand. The adaptive test-time scaling method provided in this application can be deployed as a standalone software product on a user's server; it can also be used as a cloud service; or it can be an open-source project, providing a basic version while offering advanced features and services.

[0188] In this embodiment of the application, the method of inference time change during model use is determined by analysis of the application programming interface (API), and adaptive stopping strategies can also be adopted.

[0189] like Figure 5 The diagram illustrates the iterative training process of model awareness. Embodiments of this application can be used for self-iterative improvement in proxy scenarios, which may include tool invocation. For example, a tool invocation scenario primarily involves calling external tools to solve a user's actual problem. The input is typically a user query and a set of candidate tools T. The model needs to analyze the user query, determine which candidate tools can solve the problem, and output the corresponding invocation statement.

[0190] To achieve adaptive self-refinement, the process consists of two phases: a training phase and an inference phase.

[0191] During the training phase, such as Figure 5 As shown, the iterative training of the model-aware system includes the following three processes:

[0192] First, data filtering for model awareness is performed. Data is filtered using the pass@k method, obtaining user questions q, candidate tool sets T, and ground truth labeled data A from the dataset. For each training data point, the model is made to answer k times, and the training data point is only used if the model outputs a result close to the standard answer at least once.

[0193] Then, self-optimization data construction is performed. Based on the model's first response, the output response A' is obtained, and the direct data A' is approximately equal to A. After the model's response, user turns are added to form a multi-turn dialogue. After adding user turns aimed at optimizing the model, a standard answer is added next, forming two rounds of data that are added to the model for training. In the dialogue of the user turns, the user can make requests for improvement, and the output response A' is obtained, resulting in improved data A' approximately equal to A. At the same time, data that is the same in both responses is retained, so that the model learns that when the response is good enough, no further changes are needed. Here, a good enough response means that the model's output response result is consistent with the standard answer.

[0194] Finally, iterative training is performed. Using the trained model as the base model, the above data filtering and self-optimization data construction are repeated to fully stimulate the model's performance, allowing the model to make its own judgments. The model's response will include the selected tools and the parameters entered.

[0195] like Figure 6The diagram illustrates an adaptive self-iterative inference process. Iterative inference is introduced into the model's inference phase, allowing the model to continuously optimize based on the previous response until the two responses are consistent, at which point the inference stops, and the last response is taken as the final response.

[0196] As illustrated by the foregoing examples, the embodiments of this application provide a self-optimizing adaptive iterative inference method, which effectively improves the model's performance on complex problems while maintaining inference efficiency on relatively simple problems. Furthermore, filtering the training data enables model-aware training, facilitating the selection of suitable training data. Employing self-optimizing data construction and learning stopping conditions allows the model to learn self-optimization and when to stop. Iterative training fully unleashes the model's potential and effectively utilizes existing data.

[0197] The method provided in this application can be applied to chain-based expansion methods, achieving adaptive expansion by introducing stopping conditions that the model can automatically recognize. It can also be introduced into tree-search-based expansion methods or even slow-thinking-based expansion methods by redesigning different stopping conditions and relying on training. For example, slow-thinking data containing certain special textual minimal units (tokens) can be constructed, allowing the model to learn to stop thinking and start outputting an answer when it encounters that special token.

[0198] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0199] To facilitate better implementation of the above-described solutions in the embodiments of this application, related apparatus for implementing the above-described solutions is also provided below.

[0200] Please see Figure 7 As shown in the embodiment of this application, a communication device 700 may include:

[0201] The reasoning module 701 is used to reason about the first model based on reasoning data to obtain a first response result, wherein the reasoning data includes a first user question;

[0202] The iterative reasoning module 702 is used to perform iterative reasoning on the first model based on the reasoning data and the first response result to obtain a second response result;

[0203] The output module 703 is used to output the second response result or the first response result when the similarity between the first response result and the second response result is greater than or equal to a threshold.

[0204] Please see Figure 8 As shown in the embodiment of this application, a communication device 800 may include:

[0205] The iterative training module 801 is used to perform model-aware iterative training on the first model based on the training data to obtain the trained first model, wherein the training data includes the second user question.

[0206] The iterative inference module 802 is used to perform iterative inference on the trained first model based on inference data, and obtain a first response result and a second response result respectively.

[0207] The output module 803 is used to output the second response result or the first response result when the similarity between the first response result and the second response result is greater than or equal to a threshold.

[0208] This application also provides a computing device 130. For example... Figure 9 As shown, the computing device 130 includes a bus 132, a processor 134, a memory 136, and a communication interface 138. The processor 134, the memory 136, and the communication interface 138 communicate with each other via the bus 132. The computing device 130 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 130.

[0209] Bus 132 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 9 The bus 134 is represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 134 may include a path for transmitting information between various components of the computing device 130 (e.g., memory 136, processor 134, communication interface 138).

[0210] The processor 134 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0211] Memory 136 may include volatile memory, such as random access memory (RAM). Processor 134 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0212] The memory 136 stores executable program code, and the processor 134 executes the executable program code to implement the functions of the aforementioned acquisition module and training module, thereby realizing the layout method of the circuit board applied to the computing device cluster in the above embodiment. That is, the memory 136 stores instructions for executing the layout method of the circuit board applied to the computing device cluster in the above embodiment.

[0213] The communication interface 138 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 130 and other devices or communication networks.

[0214] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0215] like Figure 10 As shown, the computing device cluster includes at least one computing device 130. The memory 136 in one or more computing devices 130 in the computing device cluster may store the same instructions for executing the layout method of the circuit board.

[0216] In some possible implementations, the memory 136 of one or more computing devices 130 in the computing device cluster may also store partial instructions for executing the layout method of the circuit board. In other words, a combination of one or more computing devices 130 can jointly execute instructions for executing the layout method of the circuit board.

[0217] It should be noted that the memory 136 in different computing devices 130 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the circuit board layout method. That is, the instructions stored in the memory 136 of different computing devices 130 can implement one or more functions of the acquisition module.

[0218] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN), a local area network (LAN), or similar. Figure 10 One possible implementation is shown. For example... Figure 10 As shown, two computing devices 130A and 130B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 136 in computing device 130A may store instructions for executing the functions of the first processing module. Simultaneously, the memory 136 in computing device 130B may store instructions for executing the functions of the second processing module. Alternatively, the memory 136 in computing device 130A may store instructions for executing some functions of the second processing module, while the memory 136 in computing device 130B may store instructions for executing another part of the functions of the second processing module, and so on.

[0219] It should be understood that Figure 11 The functions of the computing device 130A shown can also be performed by multiple computing devices 130. Similarly, the functions of the computing device 130B can also be performed by multiple computing devices 130.

[0220] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 9 and Figure 10 The connection method of the computing device cluster. The difference is that the memory 136 of one or more computing devices 130 in the computing device cluster can store the same instructions for executing the layout method of the circuit board.

[0221] In some possible implementations, the memory 136 of one or more computing devices 130 in the computing device cluster may also store partial instructions for executing the layout method of the circuit board. In other words, a combination of one or more computing devices 130 can jointly execute instructions for executing the layout method of the circuit board.

[0222] It should be noted that the memory 136 in different computing devices 130 within the computing device cluster can store different instructions for executing some functions of the circuit board layout method. That is, the instructions stored in the memory 136 of different computing devices 130 can implement one or more functions of the processing module.

[0223] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a circuit board layout method.

[0224] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform a circuit board layout method.

[0225] This application also provides a chip system including a processor for implementing the steps performed by the aforementioned computing device cluster. In one possible design, the chip system may further include a memory for storing necessary program instructions and data. This chip system may be composed of chips or may include chips and other discrete devices.

[0226] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0227] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0228] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0229] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0230] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A user-question-based processing method, characterized in that, The method includes: The first model is inferred based on the inference data to obtain the first response result, wherein the inference data includes the first user question; The first model is iteratively reasoned based on the inference data and the first response result to obtain the second response result; If the similarity between the first response result and the second response result is greater than or equal to a threshold, output the second response result or the first response result.

2. The method according to claim 1, characterized in that, The step of iteratively reasoning the first model based on the reasoning data and the first response result to obtain the second response result includes: The inference data and the first response result are optimized according to the user rounds to obtain the first optimized data; The first optimized data is input into the first model, and reasoning is performed through the first model to obtain the second response result.

3. The method according to any one of claims 1 to 2, characterized in that, The similarity between the first response result and the second response result is greater than or equal to a threshold, including: The first response result is the same as the second response result; or, The first response result is consistent with the second response result.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The first answer is extracted from the first response result according to the extraction rules, and the first answer is identified in the first response result by the first symbol indicated by the extraction rules. The second answer is extracted from the second response result according to the extraction rules, and the second answer is identified in the second response result by the second symbol indicated by the extraction rules. The similarity between the first response result and the second response result is greater than or equal to a threshold, including: the similarity between the first answer and the second answer is greater than or equal to the threshold.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The first model determines that the similarity between the first response result and the second response result is greater than or equal to a threshold.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The first model is iteratively trained using model-aware methods based on the training data to obtain the trained first model, wherein the training data includes the second user question. The inference process based on the inference data for the first model includes: The trained first model is inferred based on the inference data.

7. The method according to claim 6, characterized in that, The step of performing model-aware iterative training on the first model based on training data includes: The second user question from the training data is input into the first model, and the first model is trained to obtain the trained first model. Using the first trained model as the base model, the second user question from the training data is input into the first trained model, and iterative training is performed through the first trained model.

8. The method according to claim 6 or 7, characterized in that, The training data also includes: the standard answer to the second user question; The step of performing model-aware iterative training on the first model based on training data includes: The second user question from the training data is input into the first model, and the first model is trained K times to obtain K training results, where K is a positive integer; If at least one of the K training results matches the standard answer, iterative training of the model-aware system is performed based on the training data.

9. The method according to claim 8, characterized in that, The iterative training of the first model based on the training data with model awareness further includes: If none of the K training results meet the standard answer, stop iterative training of the model-aware system based on the training data.

10. The method according to claim 8 or 9, characterized in that, The iterative training of the first model based on the training data with model awareness further includes: The training data is updated based on the K training results to obtain the updated training data; The first model is then fine-tuned under supervision based on the updated training data to obtain the trained first model.

11. The method according to any one of claims 6 to 10, characterized in that, The training data also includes: the standard answer to the second user question; The step of performing model-aware iterative training on the first model based on training data includes: The second user question from the training data is input into the first model, and the first model is used for training to obtain the first training result; The second optimized data is obtained based on the first training result and the standard answer; The second optimized data is input into the first model, and iterative training is performed through the first model.

12. A user-question-based processing method, characterized in that, The method includes: The first model is iteratively trained using model-aware methods based on the training data to obtain the trained first model, wherein the training data includes the second user question. Based on the inference data, the trained first model is iteratively inferred to obtain the first response result and the second response result respectively. If the similarity between the first response result and the second response result is greater than or equal to a threshold, output the second response result or the first response result.

13. The method according to claim 12, characterized in that, The step of performing model-aware iterative training on the first model based on training data includes: The second user question from the training data is input into the first model, and the first model is trained to obtain the trained first model. Using the first trained model as the base model, the second user question from the training data is input into the first trained model, and iterative training is performed through the first trained model.

14. The method according to claim 12 or 13, characterized in that, The training data also includes: the standard answer to the second user question; The step of performing model-aware iterative training on the first model based on training data includes: The second user question from the training data is input into the first model, and the first model is trained K times to obtain K training results, where K is a positive integer; If at least one of the K training results matches the standard answer, iterative training of the model-aware system is performed based on the training data.

15. The method according to claim 14, characterized in that, The iterative training of the first model based on the training data with model awareness further includes: If none of the K training results meet the standard answer, stop iterative training of the model-aware system based on the training data.

16. The method according to claim 14 or 15, characterized in that, The iterative training of the first model based on the training data with model awareness further includes: The training data is updated based on the K training results to obtain the updated training data; The first model is then fine-tuned under supervision based on the updated training data to obtain the trained first model.

17. The method according to any one of claims 12 to 16, characterized in that, The training data also includes: the standard answer to the second user question; The step of performing model-aware iterative training on the first model based on training data includes: The second user question from the training data is input into the first model, and the first model is used for training to obtain the first training result; The second optimized data is obtained based on the first training result and the standard answer; The second optimized data is input into the first model, and iterative training is performed through the first model.

18. A communication device, characterized in that, The device includes: The reasoning module is used to reason about the first model based on reasoning data to obtain a first response result, wherein the reasoning data includes a first user question; An iterative reasoning module is used to perform iterative reasoning on the first model based on the reasoning data and the first response result to obtain a second response result; The output module is used to output either the second response result or the first response result if the similarity between the first response result and the second response result is greater than or equal to a threshold.

19. A communication device, characterized in that, The device includes: An iterative training module is used to perform model-aware iterative training on the first model based on training data to obtain the trained first model, wherein the training data includes the second user question. The iterative inference module is used to perform iterative inference on the trained first model based on inference data, and obtain the first response result and the second response result respectively. The output module is used to output either the second response result or the first response result if the similarity between the first response result and the second response result is greater than or equal to a threshold.

20. A communication device comprising a processor and a memory coupled to the processor, the processor being configured to perform the method according to any one of claims 1 to 17.

21. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 17.

22. A computer program product comprising instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 17.

23. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the method of any one of claims 1 to 17.

Images