Method, apparatus, device and storage medium for information processing

Abstract

According to embodiments of this disclosure, a method, apparatus, device, and storage medium for information processing are provided. The method includes: receiving a processing request, the processing request indicating a problem to be solved; and providing response content generated by a model based on the processing request, the response content including an output answer to the problem and multiple parts corresponding to multiple steps in solving the problem, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment. Based on this approach, the model in the embodiments of this disclosure can generate executable source code content during the process of solving a problem through multiple steps, and complete intermediate inference tasks by executing the source code content, thereby improving the model's ability to handle precise tasks.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatuses, devices, and storage media for information processing. Background Technology

[0002] With the development of computer technology, some language models can apply Chain of Thought (CoT) technology for complex reasoning. The core idea of ​​CoT is to allow the model to demonstrate the reasoning process step by step, like a human, rather than directly outputting the final answer. By explicitly generating intermediate reasoning steps, the model can more accurately solve problems that require multi-step logical deduction. However, applying CoT limits the model's ability to handle precise tasks. Summary of the Invention

[0003] In a first aspect of this disclosure, an information processing method is provided. The method includes: receiving a processing request, the processing request indicating a problem to be solved; and providing response content generated by a model based on the processing request, the response content including an output answer to the problem and multiple parts corresponding to multiple steps in solving the problem, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

[0004] In a second aspect of this disclosure, an apparatus for information processing is provided. The apparatus includes: a receiving module configured to receive a processing request, the processing request indicating a problem to be solved; and a providing module configured to provide response content generated by a model based on the processing request, the response content including an output answer to the problem and multiple parts corresponding to multiple steps in solving the problem, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method according to a first aspect of this disclosure.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar embedded elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A first flowchart of an information processing procedure according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A second flowchart illustrating an information processing procedure according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic structural block diagram of an apparatus for information processing according to certain embodiments of the present disclosure is shown;

[0014] Figure 5 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] The embodiments of this disclosure may involve user data, data acquisition, and / or use. These aspects all comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the user should be informed of the data or data type, scope of use, and usage scenarios that may be involved, and their authorization should be obtained, in accordance with relevant laws and regulations and through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0019] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal data beyond that required for basic functions will not affect the user's use of basic functions.

[0020] In language models, Chain of Thought (CoT) techniques are commonly used for complex reasoning, such as solving mathematical word problems or logic problems step-by-step through natural language. However, this method relies entirely on natural language as the intermediate form of reasoning, thus limiting the model's ability to handle precise tasks. Alternatively, language models can also use Program of Thought (PoT) for reasoning. PoT is a technique that allows the model to solve problems by generating executable program code. It leverages the precision of programming languages ​​to transform complex calculations or logic problems into code, obtaining accurate results by executing the code, thereby avoiding errors in pure mathematical or symbolic reasoning inherent in language models. However, PoT typically lacks a generalized reasoning structure, resulting in lower interpretability.

[0021] Embodiments of this disclosure propose an information processing scheme. According to the scheme, a processing request is received, indicating a problem to be solved; and response content generated by a model based on the processing request is provided. The response content includes an output answer to the problem and multiple parts corresponding to multiple steps in solving the problem, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

[0022] The model in the embodiments of this disclosure can generate executable source code during the problem-solving process through multiple steps, and complete intermediate inference tasks by executing the source code, thereby improving the model's ability to handle precise tasks. Based on this approach, the embodiments of this disclosure can combine the interpretability of thought chains with the computational accuracy of program-assisted thinking, thereby improving the model's problem-solving performance.

[0023] Example Environment

[0024] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include electronic device 110. In some embodiments, electronic device 110 may receive a processing request indicating a problem to be solved; and provide response content generated by the model based on the processing request, the response content including the output answer to the problem and multiple parts corresponding to multiple steps in solving the problem, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment. Model 120 may be deployed on electronic device 110, or on other devices, which will not be elaborated here.

[0025] In some embodiments, the electronic device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 may also support any type of interface for the target user (such as "wearable" circuitry).

[0026] Electronic device 110 can also be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Electronic device 110 may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in cloud environments, and so on.

[0027] It should be understood that the structure and function of the various embedded elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0028] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0029] Example process

[0030] Figure 2 A flowchart of an information processing procedure 200 according to some embodiments of the present disclosure is shown. Procedure 200 can be implemented at electronic device 110. Reference is made below. Figure 1 Describe the process 200.

[0031] In box 210, electronic device 110 can receive processing requests.

[0032] In some embodiments, the processing request indicates the problem to be solved. For example, the processing request may include "calculate 13 × 15".

[0033] In box 220, electronic device 110 can provide response content generated by the model based on the processing request.

[0034] In the embodiments of this disclosure, to improve model performance, a Code Interpreter-Chain of Thought (CI-CoT) technique can be constructed by combining thought chains and program-assisted thought. This CI-CoT technique is a hybrid enhancement technique that combines the interpretability of natural language reasoning with the computational accuracy of program code execution. Applying CI-CoT allows for the generation of executable code through step-by-step reasoning, and the dynamic verification of results using a code interpreter, thereby improving the model's accuracy in mathematical computation, symbolic reasoning, and structured tasks.

[0035] In some embodiments, to enable the model to automatically combine reasoning with executable code, a modular, step-by-step system deployment scheme can be applied, specifically implemented through a plug-in mechanism. The response content generated by the model includes the output answer to the question and multiple parts corresponding to the multiple steps in answering the question, with a one-to-one correspondence between the multiple steps and the multiple parts. The multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

[0036] In some embodiments, the multiple steps are multiple forward reasoning steps for solving a problem. A forward reasoning step is a single reasoning unit that, in the process of solving a problem, starts from known input data or intermediate results and gradually derives new intermediate results or final conclusions according to predefined rules, model structures, or logical relationships. Each forward reasoning step is an independent logical unit, and its output serves as the input for the next step, until the final goal is reached. Furthermore, the multiple forward reasoning steps do not include a verification step.

[0037] For example, the processing request includes "calculate 13 × 15", see reference. Figure 3 The middle frame is 310. The mathematical logic for solving the problem is as follows:

[0038] The first step is:

[0039] Divided into 13 × (10 + 5).

[0040] 13 × 10 = 130, 13 × 5 = 65

[0041] The second step is:

[0042] Add the results together.

[0043] 130 + 65 = 195

[0044] In the embodiments of this disclosure, in order to improve the accuracy of the model output, the source code content is applied to perform the calculation in the second step, referring to... Figure 3 For box 320, the reasoning logic for solving the problem using multiple steps can be as follows:

[0045] The first step is:

[0046] Divided into 13 × (10 + 5).

[0047] 13 × 10 = 130, 13 × 5 = 65

[0048] The second step is:

[0049] Python

[0050] result = 130 + 65

[0051] print(result)

[0052] ```

[0053] The content included in the first step corresponds to the first part of the natural language content, and the content included in the second step corresponds to the second part of the source code content. Next, the code can be run in a pre-configured runtime environment to obtain the execution results, which are then fed back to the model so that the model can learn and correct itself based on the feedback.

[0054] In some embodiments, upon detecting a code termination trigger flag <code>< / code> The model continues to generate content until it encounters an end-of-sequence (EOS) marker. This ensures that the model can output complete code when generating code snippets, while explicitly ending the generation process by applying the end-of-sequence marker.

[0055] In some embodiments, the model also has a self-correcting function, and the electronic device 110 can generate first source code content corresponding to the target step from the model. Then, in response to an execution error of the first source code content in the runtime environment, the model is provided with indication information indicating that the first source code content has executed incorrectly. Next, the model regenerates second source code corresponding to the target step, wherein the second part includes both the first and second source code content.

[0056] For example, the model can generate the first source code corresponding to the target step, and then run the first source code in a pre-configured runtime environment. If the first source code encounters an error during execution in the runtime environment, it can provide feedback to the model indicating the error. Through this feedback, the model can learn that the previously generated first source code contains an error and needs correction. Then, the model can regenerate the second source code corresponding to the target step. Furthermore, the process of the model automatically correcting the code can be observed on the user's side. For example, the user can see a piece of code generated, then an error reported, and then another piece of code generated.

[0057] In this way, the model acquires error detection and self-correction capabilities, enabling it to identify flaws in its reasoning process and continuously iterate and improve its methods while solving complex problems. Furthermore, this dynamic feedback mechanism greatly enhances the stability of the reinforcement learning training process.

[0058] In some embodiments, the electronic device 110 can acquire streaming content generated by the model based on a processing request. Then, in response to detecting a preset code identifier from the streaming content, source code content is extracted from the streaming content based on the code identifier. Subsequently, the source code content is executed in a pre-configured runtime environment to obtain an execution result.

[0059] In some embodiments, the execution result is provided to the model based on the following process: providing the model with the execution result and a preset result identifier. For example, a code snippet from the most recent output can be extracted using regular expressions, sent to a remote sandbox, and executed within the sandbox environment. After the code execution is complete, the sandbox output is populated. <interpreter>< / interpreter> The system retrieves the execution results from the tags and feeds them back to the model, allowing it to learn and correct itself. For example, it can first attempt to connect to a remote sandbox for code execution; if the remote connection fails, it automatically switches to the local code interpreter to ensure stability and efficiency in the execution process.

[0060] In some embodiments, to accelerate model inference and avoid redundant computation, the electronic device 110 may save the model's context information in response to the execution of source code content in the runtime environment. Then, based on the execution result of the source code, the saved context information is updated for subsequent model processing. For example, during code execution, the current context is saved, and the execution result returned by the sandbox is appended to the context in real time, then the model's key-value cache is updated to accelerate subsequent inference.

[0061] Additionally, to reduce memory overhead during execution, all previous key-value pairs are cached each time a code termination trigger is detected, and the key-value cache is only computed for feedback from the code interpreter. This will significantly reduce the key-value caching required for each deployment.

[0062] In some embodiments, reference Figure 3 Box 340 allows for imitation learning through supervised fine-tuning to enhance the tool's ensemble reasoning capabilities. Electronic device 110 acquires a first sample question from the sample set. It then provides the first sample question to the first model to determine if solving it requires generating source code content. Next, in response to the first model's output indicating that solving the first sample question requires generating source code content, the sample question is added to the first sample data. Based on the first sample data, a first training process is then performed on the model.

[0063] Supervised fine-tuning (SFT) is the process of further training a pre-trained language model using a dataset with explicit labels or supervisory signals. This method guides the model to perform better on specific tasks or domains by introducing manually labeled data. Imitation learning is a method of training a model by observing expert behavior. Its core idea is to teach the model how to imitate the behavior of experts on specific tasks, thereby achieving expert-level performance.

[0064] For example, to ensure sample quality, a rigorous screening process can be pre-executed using the first model. This allows the first model to assess whether solving the first sample problem requires generating source code, i.e., to evaluate the applicability of the tool to the first sample problem. If the first model determines that solving the first sample problem requires generating source code, it means that the tool can be applied to significantly improve the solution process for that first sample problem and increase computational accuracy. Then, the first sample problem is added to the first sample data. Finally, based on the first sample data, the first training process is performed on the model.

[0065] In some embodiments, to enhance the model's ability to determine when it is appropriate to use computational tools, the corresponding content can also be verified. The electronic device 110 can, in response to the output of the first model indicating that solving the first sample problem requires generating source code content, obtain the first response content generated by the second model based on the first sample problem, the first response content including code snippets. Then, in response to the first response content meeting preset conditions, the first sample problem is added to the first sample data. This allows for the acquisition of high-quality sample data.

[0066] In some embodiments, the preset conditions include at least one of the following: the code format of the code snippet meets preset specifications, the code snippet runs successfully in the runtime environment, and the answer included in the first response content is consistent with the answer indicated by the first sample data. That is, if the first model determines that solving the first sample problem requires generating source code content, then the second model generates the first response content based on the first sample problem. If the first response content meets the above preset conditions, then the first sample problem is considered a qualified sample and is added to the first sample data.

[0067] Specifically, two-stage validation can be performed on the response content. For example, refer to... Figure 3 Box 330 allows for the initial format validation of the first response content, verifying whether its format meets the requirements. Format validation improves readability while ensuring syntactic consistency, enabling effective detection of the computational tool's invocation trigger conditions in subsequent reinforcement learning stages. Next, the first response content's answer is validated; if the validation result shows the answer is correct, the first sample question is added to the first sample data. This two-stage validation provides a reliable initial setup for subsequent tool-based reinforcement learning.

[0068] In some embodiments, to eliminate unnecessary or interfering information during loss calculation, thereby enabling the model to focus more effectively on the primary task and improve training efficiency and final performance, the training loss of the first training process can be made independent of the execution results of the code snippets. For example, since the inference process includes code interpretation or inference steps generated by a code interpreter, to ensure that the model primarily focuses on the final code generation or problem-solving task, rather than relying too much on interpretive output, the feedback output of the code interpreter can be masked in the loss calculation. This helps the model focus more on the core task, thereby improving overall performance.

[0069] In some embodiments, during training, preset prompts can be pre-set. These prompts are pre-designed frameworks or formats used to organize and guide input data so that the model can better understand and process it. For example, preset prompts might include "Solve the following problem step by step. You can now optionally write executable Python code to enhance the inference process. This Python code will be executed in an external sandbox, and its output (wrapped in...") <interpreter> output_str< / interpreter> The information within the tag will be returned to aid reasoning and help you arrive at the final answer. The Python code should be a complete script, including the necessary import statements. The model can then generate the corresponding response based on this preset prompt.

[0070] In some embodiments, reference Figure 3 Box 350 further enhances model performance through reinforcement learning. The electronic device can perform a second training process on the model based on second sample data; this second training process is associated with reinforcement learning. Reinforcement learning is an important branch of machine learning that focuses on how an agent takes a series of actions in an environment to maximize cumulative rewards. Reinforcement learning learns through trial and error; the agent continuously adjusts its strategy as it interacts with the environment to obtain better long-term returns.

[0071] In some embodiments, the electronic device 110 can provide the model with a second sample question from the second sample data to generate a second response content, the second response content including a first answer to the second sample question. Then, based on a comparison between the first answer and the second answer indicated by the second sample data, reward information for reinforcement learning is determined to execute a second training process. For example, during reinforcement learning, the reward function can be used as the primary training signal to determine the optimization direction of reinforcement learning. For questions with deterministic answers, the model can be required to provide the answer in a specified format, such as within a \box, to evaluate the accuracy of the answer.

[0072] In some embodiments, the reward information is also associated with whether the code snippet included in the second response content successfully runs in the runtime environment. For example, after each sandbox invocation, a lower reward is assigned as a penalty for failed code snippet execution based on the running status of the code snippet, thereby providing a more accurate reward value for the overall training process.

[0073] In some embodiments, neural reward models are data-trained models used to replace hand-designed reward functions. However, neural reward models are prone to reward hacking, where an agent learns to exploit vulnerabilities or biases in the reward model to obtain high rewards without actually completing the target task. Furthermore, reward mechanisms are used in reinforcement learning to guide models to learn specific behaviors. If a model's reward mechanism is primarily based on the performance of the code interpreter, the model will tend to optimize tasks related to code interpretation. Therefore, embodiments of this disclosure avoid using neural reward models and code interpreter feedback to prevent the aforementioned problems. A simple reward mechanism can promote more diverse problem-solving behaviors.

[0074] In some embodiments, asynchronous code sandboxes can also be used to accelerate the reinforcement learning training process. The sandbox container acts as a worker node in the resource pool, independently pulling tasks based on its current capabilities, thus establishing an efficient load balancing mechanism. This distributed asynchronous method shortens the time for collecting experience data by allowing worker nodes to interact with the environment in parallel, thereby accelerating the training speed of reinforcement learning. It prevents slower worker nodes from becoming bottlenecks, ensuring optimal resource utilization and maintaining a stable processing speed during training.

[0075] The embodiments of this disclosure can first construct high-quality samples using a first model through imitation learning to ensure detailed and validated mathematical logic. Then, reinforcement learning is applied to guide the model to use external computational tools optimally, thereby enhancing tool integration. A sandboxed code interpreter is employed to provide immediate feedback on computational accuracy. This approach significantly improves the model's ability to perform complex mathematical reasoning using tools. Furthermore, to effectively integrate mathematical logic and programming-based methods, a dedicated reinforcement learning pipeline for tool usage is introduced, enhancing the model's ability to appropriately select and apply tools during reasoning, thereby improving the overall performance of the model.

[0076] Example devices and equipment

[0077] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 4 A schematic structural block diagram of an apparatus 400 for information processing according to certain embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in the electronic device 110 discussed above. The various modules / components in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0078] like Figure 4 As shown, the device 400 includes a receiving module 410 configured to receive a processing request indicating a problem to be solved; and a providing module 420 configured to provide response content generated by the model based on the processing request, the response content including the output answer to the problem and multiple parts corresponding to multiple steps in solving the problem, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

[0079] In some embodiments, the second part is generated based on the following process: generating first source code content corresponding to the target step by the model; providing the model with indication information of the execution error of the first source code content in the runtime environment in response to an execution error of the first source code content; and regenerating the second source code corresponding to the target step by the model, wherein the second part includes the first source code content and the second source code content.

[0080] In some embodiments, the execution result is generated based on the following process: obtaining streaming content generated by the model based on the processing request; in response to detecting a preset code identifier from the streaming content, extracting source code content from the streaming content based on the code identifier; and executing the source code content in a pre-configured runtime environment to obtain the execution result.

[0081] In some embodiments, the apparatus 400 is further configured to save the model's context information in response to the execution of the source code content in the runtime environment; and to update the saved context information based on the execution result of the source code for subsequent processing of the model.

[0082] In some embodiments, the execution result is provided to the model based on the following process: providing the model with the execution result and a preset result identifier.

[0083] In some embodiments, the model is trained based on the following process: obtaining a first sample problem from a sample set; providing the first sample problem to a first model to determine whether solving the first sample problem requires generating source code content; and adding the sample problem to the first sample data in response to the output of the first model indicating that solving the first sample problem requires generating source code content; and performing a first training process on the model based on the first sample data.

[0084] In some embodiments, adding a sample problem to the first sample data includes: in response to an output result of a first model indicating that solving the first sample problem requires generating source code content, obtaining first response content generated by a second model based on the first sample problem, the first response content including code snippets; and in response to the first response content meeting preset conditions, adding the first sample problem to the first sample data.

[0085] In some embodiments, the preset conditions include at least one of the following: the code format of the code snippet meets the preset specifications; the code snippet runs successfully in the runtime environment; and the answer included in the first response content is consistent with the answer indicated by the first sample data.

[0086] In some embodiments, the training loss of the first training process is independent of the execution result of the code snippet.

[0087] In some embodiments, the model is trained based on the following process: a second training process is performed on the model based on second sample data, the second training process being associated with reinforcement learning.

[0088] In some embodiments, performing a second training process on the model based on second sample data includes: providing the model with a second sample question from the second sample data to generate a second response content, the second response content including a first answer to the second sample question; and determining reward information for reinforcement learning based on a comparison of the first answer and the second answer indicated by the second sample data to perform the second training process.

[0089] In some embodiments, the reward information is also associated with whether the code snippet included in the second response content successfully runs in the runtime environment.

[0090] In some embodiments, the multiple steps are multiple forward reasoning steps for answering a question.

[0091] The units included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 400 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0092] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to achieve Figure 1 The electronic device 110 shown.

[0093] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0094] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store data and / or data (e.g., training data for training) and can be accessed within electronic device 500.

[0095] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0096] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0097] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0098] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0099] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0100] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable information processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable information processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable information processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0101] Computer-readable program instructions can be loaded onto a computer, other programmable information processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable information processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable information processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0103] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. An information processing method, comprising: Receive a processing request, the processing request indicating the problem to be solved; as well as Provide response content generated by the model based on the processing request, the response content including the output answer to the question and multiple parts corresponding to multiple steps to answer the question, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

2. The method of claim 1, wherein the second part is generated based on the following process: The model generates the first source code content corresponding to the target step; In response to an execution error in the first source code content in the runtime environment, the model is provided with indication information indicating that the first source code content has executed the error. as well as The model is used to regenerate a second source code corresponding to the target step, wherein the second part includes the content of the first source code and the content of the second source code.

3. The method according to claim 1, wherein the execution result is generated based on the following process: Obtain the streaming content generated by the model based on the processing request; In response to detecting a preset code identifier from the streaming content, the source code content is extracted from the streaming content based on the code identifier; and In the pre-configured runtime environment, the source code content is executed to obtain the execution result.

4. The method according to claim 3, further comprising: In response to the execution of the source code content in the runtime environment, the context information of the model is saved; as well as Based on the execution result of the source code, the saved context information is updated for subsequent processing of the model.

5. The method of claim 1, wherein the execution result is provided to the model based on the following process: The model is provided with the execution result and a preset result identifier.

6. The method of claim 1, wherein the model is trained based on the following process: The problem is to obtain the first sample in the sample set. The first sample problem is provided to the first model to determine whether solving the first sample problem requires generating source code content; as well as In response to the output of the first model indicating that the solution to the first sample problem requires the generation of source code content, the first sample problem is added to the first sample data; as well as Based on the first sample data, the model is subjected to a first training process.

7. The method of claim 6, wherein adding the sample question to the first sample data comprises: In response to the output of the first model indicating that the solution to the first sample problem requires the generation of source code content, the second model generates a first response content based on the first sample problem, the first response content including code snippets; as well as In response to the first response content meeting the preset conditions, the first sample question is added to the first sample data.

8. The method according to claim 7, wherein the preset condition includes at least one of the following: The code format of the code snippet meets the preset specifications; The code snippet ran successfully in the runtime environment; The answer included in the first response content is consistent with the answer indicated by the first sample data.

9. The method of claim 6, wherein the training loss of the first training process is independent of the execution result of the code snippet.

10. The method of claim 1, wherein the model is trained based on the following process: Based on the second sample data, a second training process is performed on the model, which is associated with reinforcement learning.

11. The method of claim 10, wherein performing a second training process on the model based on the second sample data comprises: The model is provided with a second sample question from the second sample data to generate a second response content, the second response content including a first answer to the second sample question; Based on the comparison between the first answer and the second answer indicated by the second sample data, the reward information for the reinforcement learning is determined to execute the second training process.

12. The method of claim 11, wherein the reward information is further associated with whether the code snippet included in the second response content successfully runs in the runtime environment.

13. The method of claim 1, wherein the plurality of steps are plurality of forward reasoning steps for answering the problem.

14. An apparatus for information processing, comprising: The receiving module is configured to receive a processing request, which indicates a problem to be solved. as well as A providing module is configured to provide response content generated by the model based on the processing request, the response content including an output answer to the question and multiple parts corresponding to multiple steps in answering the question, wherein the multiple parts include at least a first part corresponding to natural language content and a second part corresponding to source code content, and the output answer is generated based on the execution result of the source code content in a pre-configured runtime environment.

15. An electronic device comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

17. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 13.

Images