A data processing method, related device, equipment and storage medium

Abstract

The application discloses a data processing method, related devices, equipment and a storage medium, and is used in the field of computer technology. The method comprises the following steps: firstly, an environment access maintenance intelligent agent generates a change description document according to a task requirement text and an environment requirement document, so that the change influence range is traceable and transferable. Then, an algorithm development and optimization intelligent agent generates an adaptation description document according to the change description document, so that the downstream feature processing and reward shaping logic can automatically follow the upstream change to adjust synchronously. Finally, a data link verification intelligent agent generates a data link verification result according to the change description document and the adaptation description document, and automatically performs a full-link consistency check after the change and the adaptation are completed, so as to ensure that each environment change undergoes the three stages of recording change, synchronous adaptation and verification confirmation, and to make the reinforcement learning intelligent agent always train on correct data, thereby avoiding waste of algorithm resource.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a data processing method, related apparatus, device, and storage medium. Background Technology

[0002] Reinforcement learning is a machine learning paradigm that optimizes decisions by collecting data through interaction with the environment. Its complete development process covers multiple technical aspects, including game environment backend integration, communication protocol adaptation, observation data decoding, reward function design, training framework scheduling, policy network training, and training effect diagnosis. These aspects are closely interdependent in terms of data; changes in any aspect may trigger a chain reaction of adjustments to multiple downstream components.

[0003] In existing reinforcement learning development practices, the aforementioned stages are handled manually by engineers with different responsibilities. Due to the dispersed information and long communication chains between these stages, when the game's backend environment changes, the change information needs to be passed from the backend engineers to the developers of the decoding, feature processing, and reward shaping stages, who then each complete the synchronous modifications. The entire propagation process relies entirely on manual coordination and lacks standardized verification and confirmation mechanisms.

[0004] This development model, which relies on manual changes, makes it difficult to fully identify the actual impact of environmental changes. In real-world projects, it's common for downstream components to be overlooked, leading to misaligned encoding / decoding fields, mismatched observation dimensions, or biased reward calculations. While these data errors don't cause program crashes, they force reinforcement learning agents to be trained on erroneous data, wasting computational resources. Therefore, a new method is urgently needed to address these issues. Summary of the Invention

[0005] This application provides a data processing method, related apparatus, device, and storage medium, which solves the problem of silent data errors caused by the omission of downstream component modifications due to manual information transmission after environmental changes in reinforcement learning development.

[0006] This application provides a data processing method, including: Obtain the task requirement text for the target task; When the target task is used to instruct the modification of the reinforcement learning environment of the target business, obtain the environment requirements document, which is used to characterize the configuration information of the reinforcement learning environment of the target business. The environment access and maintenance agent is invoked so that it can generate a change description document based on the task requirement text and the environment requirement document. The environment access and maintenance agent is an agent with environment configuration responsibility built based on a large language model. The change description document is used to describe the changes that occur after the reinforcement learning environment is modified. The algorithm development and optimization agent is invoked so that it can generate an adaptation description document based on the change description document. The algorithm development and optimization agent is an agent built on a large language model with the responsibility of algorithm business configuration. The adaptation description document is used to describe the content generated after modifying the reinforcement learning algorithm. The data link verification agent is invoked to generate data link verification results based on the change description document and the adaptation description document. The data link verification agent is an agent with correctness verification responsibility built based on a large language model.

[0007] This application also provides a data processing apparatus, comprising: The acquisition unit is used to acquire the task requirement text of the target task. The acquisition unit is also used to acquire an environment requirement document when the target task indicates that the reinforcement learning environment of the target service should be modified, wherein the environment requirement document is used to characterize the configuration information of the reinforcement learning environment of the target service. The processing unit is used to call the environment access and maintenance agent so that the environment access and maintenance agent can generate a change description document based on the task requirement text and the environment requirement document. The environment access and maintenance agent is an agent with environment configuration responsibility built based on a large language model. The change description document is used to describe the changes that occur after the reinforcement learning environment is modified. The processing unit is also used to call the algorithm development and optimization agent so that the algorithm development and optimization agent can generate an adaptation description document based on the change description document. The algorithm development and optimization agent is an agent built on a large language model with the responsibility of algorithm business configuration. The adaptation description document is used to describe the content generated after the reinforcement learning algorithm is modified. The processing unit is also used to invoke the data link verification agent, so that the data link verification agent can generate data link verification results based on the change description document and the adaptation description document. The data link verification agent is an agent with correctness verification responsibility built based on a large language model.

[0008] In another aspect, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods described above.

[0009] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described above.

[0010] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods described above.

[0011] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: This application provides a data processing method that transforms the propagation of environmental changes from manual coordination to a closed-loop process executed automatically by the system through a mandatory serial collaboration mechanism of three dedicated intelligent agents. First, the environment access and maintenance agent generates a change description document based on the task requirement text and environment requirement document. This explicitly records the changes, previously scattered across the individual experience of backend engineers, in a structured document format, making the impact of the changes traceable and transferable. Next, the algorithm development and optimization agent generates an adaptation description document based on the change description document. This allows downstream feature processing and reward shaping logic to automatically adjust synchronously with upstream changes, eliminating reliance on manual hierarchical notifications and individual modifications, thus preventing the possibility of missing any downstream components. Finally, the data link verification agent generates data link verification results based on the change description document and the adaptation description document. After the changes and adaptations are completed, it automatically performs a full-link consistency check, enabling errors such as encoding / decoding misalignment and dimension mismatch to be proactively detected before training begins, rather than passively identified through abnormal indicators after multiple training rounds. This ensures that every environmental change goes through the three stages of recording changes, synchronizing and adapting, and verifying and confirming, so that the reinforcement learning agent is always trained on the correct data, thereby avoiding the waste of computing resources caused by silent data errors. Attached Figure Description

[0012] Figure 1 The architecture diagram of the data processing method provided in this application embodiment applied to a reinforcement learning training scenario in a new game environment; Figure 2 A flowchart illustrating the data processing method provided in the embodiments of this application; Figure 3 An architecture diagram of the data processing method provided in the embodiments of this application; Figure 4 A timing diagram illustrating the modification process of the reinforcement learning environment provided in the embodiments of this application; Figure 5 A schematic diagram illustrating the environmental access and maintenance agent processing procedure provided in this application embodiment; Figure 6 A schematic diagram illustrating the algorithm development and optimization process of the intelligent agent provided in the embodiments of this application; Figure 7 A schematic diagram illustrating the data link verification agent processing procedure provided in this application embodiment; Figure 8 A schematic diagram illustrating the strategy learning and debugging agent provided in this embodiment of the application testing a reinforcement learning environment; Figure 9 A timing diagram of the environmental data anomaly verification process provided in the embodiments of this application; Figure 10 A schematic diagram illustrating the testing of a reinforcement learning algorithm by a policy learning debugging agent provided in an embodiment of this application; Figure 11 A timing diagram of the anomaly verification process of the reinforcement learning algorithm provided in the embodiments of this application; Figure 12 This is a schematic diagram illustrating the joint testing of a reinforcement learning environment and algorithm by a policy learning debugging agent provided in an embodiment of this application. Figure 13 A timing diagram of the data link anomaly verification process provided in the embodiments of this application; Figure 14 A schematic diagram illustrating the framework development and maintenance agent processing procedure provided in this application embodiment; Figure 15 A schematic diagram illustrating the strategy learning and debugging agent processing procedure provided in this application embodiment; Figure 16 The reinforcement learning agent system provided in the embodiments of this application; Figure 17 This is a structural diagram of the reinforcement learning agent system provided in the embodiments of this application; Figure 18 A schematic diagram illustrating the behavioral reward deduction method provided in this application embodiment; Figure 19 This is a structural diagram of the data processing apparatus provided in an embodiment of this application. Detailed Implementation

[0013] This application provides a data processing method that, through the forced serial collaboration of an environment access maintenance agent, an algorithm development and optimization agent, and a data link verification agent, automatically intercepts silent data errors caused by environment changes before training begins, thus avoiding the waste of computing power caused by the reinforcement learning agent training on erroneous data.

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0017] It should be noted that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0018] Before introducing the specific methods of this application, let's first explain the terminology used in the data processing methods of this application: (1) Reinforcement Learning (RL) is a machine learning paradigm that optimizes decision-making by collecting data through interaction with the environment. The agent iteratively updates its strategy based on observations and reward signals from the environment. In this application, reinforcement learning is the training method adopted by the business system to be developed. The collaborative processing of the environment access and maintenance agent, the algorithm development and tuning agent, and the data link verification agent all revolve around the modification and verification of the reinforcement learning environment.

[0019] (2) Large Language Model (LLM): A large-scale pre-trained language model based on the Transformer architecture, possessing capabilities such as natural language understanding, code generation, and logical reasoning. In this application, the large language model is the core of reasoning and decision-making for the environment access and maintenance agent, the algorithm development and optimization agent, the data link verification agent, the policy learning and debugging agent, and the framework development and maintenance agent. Each agent is a software entity built based on the large language model.

[0020] (3) An agent is a software entity with specific responsibilities and capabilities built based on a large language model. It can perceive input, perform reasoning, output results, and collaborate with other agents through standardized interfaces. In this application, the agents include an environment access and maintenance agent, an algorithm development and optimization agent, a data link verification agent, a policy learning and debugging agent, and a framework development and maintenance agent. Each agent has a single responsibility boundary and performs tasks in the order determined by the linkage protocol.

[0021] (4) The environment requirements document (env-spec document) is the sole source of project-specific information. It contains complete information such as an environment overview, observation space, action space, reward signals, communication protocols, file structure, and training configuration. All agents obtain business information from this document rather than through hard-coding. In this application, the environment requirements document is used to characterize the configuration information of the reinforcement learning environment for the target business. The environment access and maintenance agent modifies the reinforcement learning environment based on the task requirement text and the environment requirements document.

[0022] (5) Change description document, also known as change list, is a structured document output by the environment access maintenance agent after executing backend code and synchronously changing the decoding end. It records the content of the changes and the information of the affected observation fields. In this application, the change description document is used to describe the changes that occur after modifying the reinforcement learning environment, and serves as the input basis for the algorithm development and tuning agent to perform adaptation.

[0023] (6) Adaptation description document, also known as adaptation list, is a structured document output after the algorithm development and tuning agent synchronizes the feature adaptation of the observation builder, the logical adaptation of the reward shaper, and the dimension definition file. In this application, the adaptation description document is used to describe the content generated after modifying the reinforcement learning algorithm and serves as one of the input bases for the data link verification agent to perform full-link verification.

[0024] (7) Data link verification result, also known as data link health report, is the result output by the data link verification agent after verifying at least one link of the data conversion link. In this application, the data link verification result is used to characterize whether the verification is successful, and the environmental change process is determined to be over when the verification is successful.

[0025] (8) Data transformation link, which is the complete data flow path from backend environment encoding, network transmission, Python decoding, observation builder feature processing, reward shaping by the reward shaper to algorithm training. In this application, the data link verification agent verifies at least one link of the data transformation link. The verification links include encoding / decoding field alignment, observation data value range verification, reward value verification, data dimension consistency check, sample pool data integrity check, and training signal health check.

[0026] (9) Reward Shaping: A reward shaping method based on potential function, which constructs dense guiding signals without changing the optimal policy. In this application, the algorithm development and tuning agent performs the logical adaptation of the reward shaper, and the policy learning and debugging agent diagnoses and verifies anomalies in the reward function.

[0027] (10) Behavior reward deduction: This method uses behavior as the analysis unit to simulate and deduce the total reward of the agent in each frame under different behavior choices, and verifies whether the reward function leads to the expected optimal behavior ranking. In this application, the policy learning debugging agent discovers anomalies in the reward function based on the behavior reward deduction method when verifying the algorithm training log.

[0028] (11) Linkage protocol: a standardized collaborative process that must be executed among multiple agents after an environmental change, ensuring that the impact of the change is fully propagated and verified, and preventing silent data errors. In this application, the linkage protocol is manifested as the environmental access maintenance agent, the algorithm development and optimization agent, and the data link verification agent being executed sequentially in the order of first changing, then adapting, and finally verifying. This protocol cannot be skipped.

[0029] (12) A three-layer decoupled architecture is used to divide the reinforcement learning system into a framework layer, an algorithm layer, and a business layer. There is zero cross-dependency between the three layers, and interaction is achieved through Opaque Buffer and callback injection. In this application, the framework development and maintenance agent is responsible for maintaining the framework layer code. When the framework interface changes, it notifies the environment access maintenance agent and the algorithm development and tuning agent to perform synchronous adaptation.

[0030] (13) Human-in-the-Loop: A security mechanism that requires outputting a change description document and waiting for manual confirmation before any changes are made to the framework layer, to prevent automated operations from disrupting the stability of the training framework. In this application, after generating the framework interface change request information, the framework development and maintenance agent must respond with a confirmation operation before modifying the reinforcement learning framework interface.

[0031] (14) Environment access and maintenance agent: An agent built based on a large language model, responsible for environment configuration, and responsible for connecting the game environment backend to the reinforcement learning training framework according to standard specifications. In this application, the environment access and maintenance agent is used to modify the reinforcement learning environment and generate a change description document according to the task requirement text and environment requirement document, and to perform corresponding modifications when receiving the environment verification result of the policy learning debugging agent or the framework interface change information of the framework development and maintenance agent.

[0032] (15) An algorithm development and tuning agent, which is an agent built based on a large language model and has the responsibility of algorithm business configuration. It is responsible for algorithm implementation, reward shaping design and environmental change adaptation. In this application, the algorithm development and tuning agent is used to modify the reinforcement learning algorithm according to the change description document and generate an adaptation description document, and to perform corresponding modifications when it receives the algorithm verification result of the policy learning and debugging agent or the framework interface change information of the framework development and maintenance agent.

[0033] (16) A data link verification agent, which is an agent built based on a large language model and has the responsibility of correctness verification, is responsible for systematically verifying each link of the data conversion link. In this application, the data link verification agent is used to verify at least one link of the data conversion link according to the change description document and the adaptation description document and generate a data link verification result, and to verify the corresponding link of the data link when it receives the link verification result of the policy learning and debugging agent.

[0034] (17) A policy learning debugging agent, which is an agent built based on a large language model and has the responsibility of policy training diagnosis and analysis. It is responsible for discovering efficiency problems and diagnosing policy learning anomalies from training logs. In this application, the policy learning debugging agent is used to verify the environment, algorithm, data link or framework interface according to the training logs, and to trigger the corresponding agent to perform modification or verification when the verification result indicates an anomaly.

[0035] (18) A framework development and maintenance agent is an agent built based on a large language model, responsible for maintaining the framework code, ensuring the quality, performance optimization, and functional evolution of the framework layer code. In this application, the framework development and maintenance agent is used to generate framework interface change request information when the framework interface changes, execute modifications and generate framework interface change information after confirming the operation, and notify the environment access maintenance agent and the algorithm development and tuning agent to perform synchronous adaptation.

[0036] (19) Task requirement text, which is a natural language description of the user's requirements input through the interactive orchestration layer. In this application, the task requirement text is used to indicate the content of the target task. When the target task is used to indicate the modification of the reinforcement learning environment of the target business, it triggers a serial collaborative process of the environment access maintenance agent, the algorithm development and optimization agent, and the data link verification agent.

[0037] (20) Framework interface change request information, corresponding to the change description document, is the implementation scheme output by the framework development and maintenance agent when the framework interface changes. In this application, the framework interface change request information is used to characterize the implementation scheme of the reinforcement learning framework interface change, and the framework development and maintenance agent can only execute the modification after confirmation.

[0038] (21) Framework interface change information: The change result output by the framework development and maintenance agent after confirming the operation and executing the framework interface modification. In this application, the framework interface change information is used to characterize the changes in the modified reinforcement learning framework interface and serves as the basis for the environment access maintenance agent and the algorithm development and tuning agent to perform synchronous adaptation.

[0039] Before introducing the specific methods of this application, we will first provide an illustrative example of the application scenarios for the data processing methods in this application. It should be understood that the following application scenarios are merely examples and are not limited to these.

[0040] (i) The method provided in this application can be applied in scenarios where reinforcement learning training is introduced into a new game environment.

[0041] When a user needs to integrate a new game into the reinforcement learning training platform, the user inputs task requirement text through the interaction orchestration layer, such as "integrate a new MOBA game environment". After obtaining the task requirement text, the interaction orchestration layer identifies the target task as instructing the modification of the reinforcement learning environment for the target business, and then obtains the corresponding environment requirement document for the game. This environment requirement document records configuration information such as the game's backend communication protocol, observation space, action space, and reward signal.

[0042] The environment access and maintenance agent is invoked. Based on the task requirements text and environment requirements document, the agent automatically generates adapter code, observation decoding logic, and action encoding logic, connecting the game backend to the training framework according to standard specifications, and generating a change description document. This change description document details the changes involved in this access, including the communication protocol type, observation field list, and field dimensions.

[0043] After the change description document is generated, a linkage process is automatically triggered, invoking the algorithm development and optimization agent. Based on the observation dimensions and action space information recorded in the change description document, the algorithm development and optimization agent constructs a policy network decoupled from the game logic, and designs dense guiding signals based on the backend reward signals. Upon completion, it generates an adaptation description document, recording the adaptation content of feature processing logic and reward shaping logic.

[0044] Subsequently, the data link verification agent is automatically invoked. Based on the change description document and adaptation description document, the data link verification agent verifies each stage of the data transformation link, including whether the encoding / decoding fields are aligned, whether the observed data value range is within the expected range, whether the reward value is correct, and whether the dimension definitions are consistent, generating a data link verification result. When the data link verification result confirms successful verification, the environment access process is considered complete, and the user can immediately start training.

[0045] In the scenario of integrating a reinforcement learning training platform into a new game environment, the method provided in this application transforms the traditional serial workflow, which requires sequential notifications and individual modifications under manual methods, into a standardized closed-loop process executed automatically by three intelligent agents. The environment integration and maintenance agent automatically generates adapter code and outputs a change description document based on the task requirement text and environment requirement document, thus consolidating the integration work, originally scattered across individual engineers' experience, into a structured document. The algorithm development and optimization agent automatically constructs the policy network and adapts the reward function after receiving the change description document, achieving synchronous integration between the algorithm and environment sides without manual intervention. The data link verification agent automatically performs full-link verification after adaptation, outputting a health report after confirming that the encoding / decoding, value range, and dimension all match. This three-stage collaboration reduces the integration time from several days under manual methods to several hours, ensuring that downstream adaptation and verification stages are not overlooked.

[0046] (ii) The method provided in this application can be applied in scenarios where silent data errors are automatically intercepted after environmental changes.

[0047] When the game's backend environment changes—for example, if backend developers modify the data type of an observation field in the communication protocol or add a new observation field—the user inputs a task requirement text through the interactive orchestration layer, instructing that the reinforcement learning environment for that target business be modified. After obtaining the task requirement text, the corresponding environment requirement document is retrieved, which records the complete configuration information of the current environment.

[0048] The environment access and maintenance agent is invoked. Based on the task requirements text and environment requirements document, the environment access and maintenance agent synchronously modifies the backend code, communication encoding, and Python decoding end, and generates a change description document. This change description document records details of the changes, including field names, field type changes, and newly added field information.

[0049] However, in traditional manual collaboration models, after backend changes are completed, developers on the downstream feature processing or reward shaping sides may not be aware of the changes in a timely manner. This can lead to the decoding logic of the corresponding fields in the observation builder not being updated synchronously, or the dimension constants in the dimension definition file not being adjusted accordingly. This oversight will not cause the program to crash, but it can cause misalignment or numerical deviation in the observation data at the corresponding field positions.

[0050] In this application, a linkage process is automatically triggered after the change description document is generated, invoking the algorithm development and optimization agent. Based on the changes recorded in the change description document, the algorithm development and optimization agent checks and synchronously modifies the feature processing logic of the observation builder, the shaping logic of the reward shaper, and the dimension definition file, generating an adaptation description document.

[0051] Subsequently, the data link verification agent is automatically invoked. Based on the change description document and adaptation description document, the data link verification agent performs systematic verification on each stage of the data transformation link. In the encoding / decoding field alignment stage, the data link verification agent compares the number, order, and type of fields on the backend encoding end with those on the Python decoding end to ensure consistency. In the observation data range verification stage, the data link verification agent checks whether the normalized values ​​of each observation field are within the expected range. If a field has an abnormal value due to asynchronous decoding logic, it will be detected at this stage. In the data dimension consistency check stage, the data link verification agent compares whether the dimension constants declared in the dimension definition file match the dimensions actually output by the observation builder. If any problem is found in any of the above verification stages, the data link verification result will indicate that the verification failed and will output the specific error location and reason to the user. This automatically intercepts silent data errors such as encoding / decoding misalignment and dimension mismatch caused by environmental changes before training starts, eliminating the need to passively investigate through abnormal indicators after multiple training rounds.

[0052] In scenarios involving automatic data error interception, the method provided in this application shifts quality assurance after environmental changes from post-event manual checks to automatic interception before training begins. The environment access and maintenance agent generates a change description document after executing the change, explicitly recording the field names, type changes, and new information involved, making the impact of the change traceable. The algorithm development and tuning agent synchronously modifies the observation builder, reward shaper, and dimension definition files according to this document and outputs an adaptation description document, ensuring that each affected component is covered. The data link verification agent performs systematic verification of encoding / decoding field alignment, observation range, and dimension consistency before training begins. If field misalignment or dimension mismatch is found, the specific error location is directly reported. This mechanism enables silent data errors, which were previously only passively discovered through abnormal indicators after multiple training rounds, to be automatically detected before training begins, fundamentally avoiding the waste of computing power caused by training with erroneous data.

[0053] (iii) The method provided in the embodiments of this application can be applied in the scenario of training and diagnosis automation.

[0054] After users complete environment integration and algorithm adaptation and start training, the training process generates a large amount of training log data. In the traditional manual diagnostic mode, when the training win rate stagnates or exhibits abnormal fluctuations, engineers need to manually check multiple indicators such as policy entropy, value loss, and win rate curves to troubleshoot the root cause of the problem one by one. Regarding the core issue of whether the reward function design is reasonable, traditional methods can only examine whether the values ​​of each reward component are within the expected range vertically, and cannot discover deeper issues such as cross-stage reward inversion after component combination.

[0055] After a user inputs a training and diagnostic request, the environment access and maintenance agent generates a change description document, the algorithm development and optimization agent generates an adaptation description document, and the modified reinforcement learning algorithm is trained. Then, the policy learning and debugging agent is invoked. The policy learning and debugging agent automatically acquires the algorithm training logs and analyzes them from five dimensions: policy iteration efficiency, sample production efficiency, sample usage efficiency, reward function analysis, and the rationality of algorithm hyperparameters.

[0056] In the reward function analysis dimension, the policy learning debugging agent performs behavioral reward inference. It automatically extracts the potential function structure from the reward shaping source code and identifies all discrete state transition points. Using behavior as the analysis unit, the policy learning debugging agent defines key behaviors, such as team battle charges, retreats, and lane advances, and divides them into different stages. It constructs an inference table for quantitative calculation, simulating the total reward obtained per frame for the agent when choosing different behaviors in each stage. By listing the rewards of all behaviors across all stages in the same table for cross-stage comparison, the policy learning debugging agent calculates the cross-stage advantage ratio to check whether the optimal behavior reward in the target stage truly outweighs the optimal behavior reward in non-target stages. If it is found that the overall reward of a certain behavior in a non-target stage is higher than the reward of the behavior in the target stage, it is determined that there is a cross-stage reward inversion.

[0057] Simultaneously, the policy learning and debugging agent also checks whether the potential function is close to saturation in the high-value range, which could lead to a small increase in the potential function from further behavioral improvements, while the dense reward compensation is insufficient, causing the agent to stagnate at a high level. It audits the actual reward direction of the dense reward, verifying whether the subsidized action set aligns with the design intent. Figure 1 This is to prevent unintended behavior from being subsidized by insufficiently directional rewards. The above diagnostic results are summarized into a diagnostic report, which clearly points out the specific defects in the existing reward function design and directions for improvement.

[0058] Subsequently, the collaborative process is automatically triggered, invoking the algorithm development and tuning agent. Based on the reward function anomalies indicated in the algorithm verification results output by the policy learning and debugging agent, the algorithm development and tuning agent adjusts the weight coefficients of the reward shaper, such as reducing the distance reward weight in non-target stages and adding effective behavioral reward components to the target stage. After completion, it re-outputs the modified adaptation description document. This entire process allows users to obtain professional-level diagnostic conclusions and remediation solutions without needing a deep understanding of the meaning of various metrics in the training logs or mastering the analytical methodology of behavioral reward derivation. In particular, it can detect cross-stage reward inversion issues that traditional vertical component review cannot detect.

[0059] In the context of automated training diagnosis, the method provided in this application transforms training effect diagnosis from manual troubleshooting relying on individual engineer experience into an automated closed-loop process driven by a policy learning and debugging agent. The policy learning and debugging agent automatically reads training logs and performs a five-dimensional systematic analysis. Particularly in the reward function analysis dimension, it employs a behavioral reward extrapolation method, using behavior as the analysis unit to simulate and extrapolate the total reward for each frame under different behavior choices. Through cross-stage parallel comparison and quantitative advantage ratio calculation, it can discover cross-stage reward inversion problems that traditional vertical component review cannot detect, as well as deep-seated defects such as insufficient compensation for dense rewards due to high-level saturation of the potential function and directional deviations in dense rewards. After diagnosis, the policy learning and debugging agent automatically triggers an algorithm development and optimization agent to perform targeted repairs to the reward function. This transforms the extensive work that previously required engineers to repeatedly compare log metrics, troubleshoot reward components, and manually adjust parameters into an automated closed-loop analysis, diagnosis, and repair process, with the depth and accuracy of the diagnosis exceeding the capabilities of traditional manual methods.

[0060] (iv) The method provided in the embodiments of this application can be applied in scenarios where cross-game environments can be reused with one click.

[0061] When a team completes the development of a reinforcement learning agent for a MOBA game and needs to transfer the same training to another shooting or racing game, the traditional approach requires engineers to delve into the framework layer code and modify the dimensions, feature processing logic, communication protocol adaptation, and other game-specific parts one by one. Because the observation space, action space, and communication protocols of different games differ significantly, this task is equivalent to re-executing the environment integration process and requires developers familiar with the original framework code structure.

[0062] Users only need to provide the environment requirement document corresponding to the new game. This document describes the communication protocol type, observation field list and dimensions, action space definition, backend reward signal type, and other configuration information for the new game. After obtaining the task requirement text, the environment requirement document for the new game is retrieved, and the environment access and maintenance agent is invoked. Based on the task requirement text and the environment requirement document, the environment access and maintenance agent automatically parses the new game's communication protocol, generates adapter code and encoding / decoding logic, and outputs a change description document. Subsequently, a linked process is automatically triggered, invoking the algorithm development and optimization agent. Based on the new game's observation dimensions and action space information recorded in the change description document, the algorithm development and optimization agent constructs a policy network adapted to the new game and designs reward shaping logic based on the new game's backend reward signal, generating an adaptation description document. Finally, the data link verification agent is invoked to perform full-link verification on the encoding / decoding alignment, observation range, reward value, and dimensional consistency in the new game environment, generating data link verification results. Once the verification is confirmed to be successful, the new environment can be put into training.

[0063] The environment access and maintenance agent, algorithm development and optimization agent, and data link verification agent did not modify their own code. They only generated new change description documents and adaptation description documents based on the environment requirement document corresponding to the new game. This is because all business information of each agent is obtained from the environment requirement document rather than hard-coded in the code. The environment requirement document, as the sole source of project-specific information, completely externalizes the differences between different game environments, and the agents themselves maintain universality and cross-project reusability.

[0064] In scenarios where cross-game environments can be reused with a single click, the method provided in this application transforms the work of migrating reinforcement learning training across environments from modifying the framework and algorithm layer code to providing a new environment requirements document. The three-layer decoupled architecture completely separates the framework layer code from the specific environment configuration, and the environment requirements document mechanism ensures that the agent's business processing logic no longer depends on hard-coded game-specific information. When migrating to a new game environment, the user only needs to replace the environment requirements document. The environment access and maintenance agent, the algorithm development and optimization agent, and the data link verification agent can then complete environment adaptation, algorithm adaptation, and link verification respectively based on the new document. The agents themselves do not require any modifications, thus improving the efficiency of cross-environment reuse from several days of code-level modification and debugging in the traditional model to only a few hours of document-level configuration replacement.

[0065] Please see Figure 1 , Figure 1 This diagram illustrates the architecture of the data processing method provided in this application, applied to a reinforcement learning training scenario in a new game environment. The scenario includes a user's terminal 110, a network 120, and a server 130.

[0066] Terminal 110 includes a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen provides a human-computer interaction interface to receive user-input task request text for human-computer interaction operations. The processor generates interaction instructions in response to the aforementioned human-computer interaction operations and sends the interaction instructions to the server.

[0067] The terminal 110 involved in this application includes, but is not limited to, mobile phones, tablets, laptops, desktop computers, smart voice interaction devices, virtual reality devices, smart home appliances, vehicle terminals, aircraft, etc.

[0068] Run client 140 on terminal 110. Client 140 is deployed on terminal 110. Client 140 can run on terminal 110 through a browser, or as a standalone application (APP) or mini-program, etc.

[0069] Network 120 uses standard communication technologies and / or protocols, typically the Internet, but can also be any network, including but not limited to Bluetooth, local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, private network, or any combination of virtual private network. In some embodiments, custom or dedicated data communication technologies may be used to replace or supplement the aforementioned data communication technologies.

[0070] Server 130 includes a processor. Server 130 in this application can be a standalone physical server, a server cluster or distributed system consisting of at least one physical server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence (AI) platforms.

[0071] Specifically: The user inputs the task requirement text through the terminal. The terminal then sends the task requirement text to the server via the network.

[0072] For example, in step S101, server 130 obtains the task requirement text of the target task; For example, in step S102, when the target task is used to indicate the modification of the reinforcement learning environment of the target service, the server 130 obtains the environment requirement document, wherein the environment requirement document is used to characterize the configuration information of the reinforcement learning environment of the target service. For example, in step S103, server 130 calls the environment access and maintenance agent so that the environment access and maintenance agent generates a change description document based on the task requirement text and environment requirement document. The environment access and maintenance agent is an agent with environment configuration responsibility built based on a large language model. The change description document is used to describe the changes that occur after the reinforcement learning environment is modified. For example, in step S104, server 130 calls the algorithm development and optimization agent so that the algorithm development and optimization agent generates an adaptation description document based on the change description document. The algorithm development and optimization agent is an agent built on a large language model with the responsibility of algorithm business configuration. The adaptation description document is used to describe the content generated after modifying the reinforcement learning algorithm. For example, in step S105, server 130 calls the data link verification agent so that the data link verification agent generates data link verification results based on the change description document and the adaptation description document. The data link verification agent is an agent with correctness verification responsibility built based on a large language model.

[0073] The method provided in this application achieves rapid access to a new game environment through forced serial collaboration of three agents. The environment access and maintenance agent automatically generates adapter code and outputs a change description document based on the task requirement text and environment requirement document, thus documenting the access process in a structured document format. The algorithm development and optimization agent automatically constructs the policy network and adapts the reward function after receiving the change description document, achieving synchronous integration between the algorithm and the environment. The data link verification agent automatically performs full-link verification after adaptation, outputting a health report after confirming that the encoding / decoding, value range, and dimension all match. This three-stage collaboration reduces the access time to a few hours, and downstream adaptation and verification stages are automatically covered.

[0074] Please see Figure 2 , Figure 2 A flowchart of a data processing method is shown. It should be noted that the data processing method provided in this application embodiment can be applied to a server, and this application embodiment does not impose any limitations. The method includes: S201. Obtain the task requirement text for the target task.

[0075] Understandably, task requirement text refers to the natural language description of user needs input through the interactive orchestration layer, indicating the specific content of the target task. For example, task requirement text could be "Access a new real-time strategy game environment," "Add a hero skill cooldown time field to environment observation," "Diagnose the reason why training win rate is not increasing," or "Modify the reward weight of turret attack power." This text expresses the type and specific content of the operation the user expects to complete in natural language.

[0076] First, the system receives a natural language description of the user's desired task. This text describes the target task the user expects to complete. This text serves as the trigger point for the entire processing flow. Obtaining the task requirement text eliminates the need for users to understand the responsibilities and calling rules of the underlying agents; they only need to express their development or diagnostic intentions in natural language, thus lowering the barrier to entry. Simultaneously, the task requirement text serves as the sole basis for subsequent intent analysis and agent scheduling, ensuring a unified and traceable source of requirements throughout the entire process.

[0077] Users do not need to master the professional knowledge of each stage of reinforcement learning development; they can drive the subsequent multi-agent collaborative process using natural language.

[0078] S202. When the target task indicates that the reinforcement learning environment for the target service needs to be modified, obtain the environment requirements document. The environment requirements document is used to characterize the configuration information of the reinforcement learning environment for the target service.

[0079] Understandably, the environment requirements document is the only source of project-specific information. It contains complete configuration information for the reinforcement learning environment of the target business, including the observation space, action space, reward signal, communication protocol, file structure, and training configuration. All agents obtain business information from this document rather than hardcoding it in the code.

[0080] After identifying the target task as instructing modifications to the reinforcement learning environment of the target business, the corresponding environment requirements document is obtained. This document records complete configuration information of the current environment, including the backend communication protocol type, the dimensions and types of observation fields, the definition method of the action space, and the composition of the backend reward signal. As the sole source of business information, the environment requirements document ensures that subsequent agents do not need to parse the environment configuration from the code when performing modifications; instead, they can directly read the standardized description from the document, ensuring that all agents collaborate based on the same information source.

[0081] By externalizing the environment configuration information into standardized documents, the processing logic of the intelligent agent is decoupled from the specific environment, laying the foundation for cross-environment reuse.

[0082] S203. Invoke the environment access and maintenance agent to generate a change description document based on the task requirement text and environment requirement document. The environment access and maintenance agent is an agent built on a large language model with environment configuration responsibilities. The change description document describes the changes made after modifying the reinforcement learning environment.

[0083] Understandably, the environment access and maintenance agent is an agent built on a large language model with environment configuration responsibilities. It is responsible for implementing the adapter for the new game environment and maintaining backend code changes, and is the initiator of environment change linkages. Its core responsibilities include environment access and environment maintenance. Environment access involves connecting the new game environment to the reinforcement learning training framework according to standard specifications, while environment maintenance involves executing backend environment code modification requirements and triggering downstream linkages upon completion. This agent possesses three core capabilities: First, it is language-independent; the environment backend can be in different languages ​​such as Go, C++, C#, Python, or Rust, with specific information obtained from the environment requirements document. Second, it supports multiple protocols, covering four communication modes: Hypertext Transfer Protocol (HTTP), WebSocket JSON, WebSocket Binary, and Stream. Third, it initiates linkages; after a change is completed, it automatically triggers the algorithm development and optimization agent for adaptation and the data link verification agent for verification. The new environment access process includes six steps: understanding the backend protocol, designing the observation space, implementing the adapter, implementing the observation builder (obs_builder), implementing the environment bridge (env_bridge), and end-to-end verification. The environment maintenance and change process includes four steps: change impact assessment, executing backend changes, triggering downstream linkage, and confirming deployment. After the environment change is completed, the agent sends a standardized linkage notification to the algorithm development and tuning agent. The linkage notification includes information such as the change content, environment language, involved files, changes in communication protocols, newly added or deleted observation fields, and changes in value ranges. The agent's quality red lines include four requirements: encoding / decoding must be aligned with the backend field-by-field and byte-by-byte; all default values ​​must be commented; the two adapter script files must be kept synchronized; and downstream linkage must be triggered after an environment change without skipping any steps.

[0084] The Change Description Document (CDV) is a structured document output by the environment access and maintenance agent after executing backend code and synchronously changing the decoding end. It describes the changes made to the reinforcement learning environment, recording the changes and affected observation field information. The CDV includes information such as the changes, environment language, involved files, changes to communication protocols, additions to the `obs` field, deletions of the `obs` field, and changes to the value range.

[0085] After the environment access and maintenance agent is invoked, it reads the change requirements described in the task requirement text and the current environment configuration recorded in the environment requirement document, identifying the backend code files, communication encoding logic, and decoding code that need to be modified. Following a standardized environment change process, the agent sequentially modifies the corresponding parts of the backend code, updates the encoding method of the communication protocol, and synchronously adjusts the field parsing logic of the decoding end. After modification, the agent compares the environment before and after modification, extracts the differences, and generates a change description document. This document details the list of files involved in the change, whether the communication protocol has changed, the names of newly added or deleted observation fields, and changes in field value ranges.

[0086] By transforming the previously scattered modification actions manually performed by backend engineers into a standardized execution process driven by documents, and by making the changes explicit in the output change description documents, a structured source of change information is provided to downstream agents.

[0087] S204. Invoke the algorithm development and tuning agent to generate an adaptation description document based on the change description document. The algorithm development and tuning agent is an agent built on a large language model with algorithm business configuration responsibilities. The adaptation description document describes the content generated after modifying the reinforcement learning algorithm.

[0088] Understandably, the algorithm development and tuning agent is an agent built on a large language model with algorithm business configuration responsibilities. It is responsible for implementing algorithms such as Proximal Policy Optimization (PPO), Soft Actor Commentator (SAC), and Deep Q-Network (DQN), as well as hyperparameter tuning, reward shaping design, and training scheme orchestration. Its core capabilities include four aspects: First, algorithm implementation, responsible for encoding the trainer, network structure, and callback interfaces, and integrating them into the training scheme; second, reward shaping, analyzing backend reward signals, designing potential functions, and performing numerical verification; third, environment change adaptation, performing feature adaptation of the observation builder (obs_builder), shaping logic adaptation of the reward shaper, and dimension synchronization of the dimension definition file (dims.py) according to the change description document; and fourth, training analysis, analyzing training logs to diagnose policy entropy, win rate (WR) curves, and value loss. The agent follows four core principles: zero game dependency at the algorithm layer, meaning the algorithm directory does not reference business layer code, and dimensional information is injected through constructor parameters; opaque batch interface, where the update method of the near-end policy optimization agent receives opaque batch data; alignment of shaping signals with observations, using only information observable by the agent for reward shaping; and prioritizing numerical stability, including non-numerical checks, gradient pruning, and advantage normalization.

[0089] The adaptation description document is a structured document output after synchronizing the feature adaptation of the algorithm development and tuning agent, the logical adaptation of the reward shaper, and the dimension definition file. It is used to describe the content generated after modifying the reinforcement learning algorithm.

[0090] After the algorithm development and tuning agent is invoked, it reads the changes recorded in the change description document. If the change involves adding or modifying observation fields, the agent synchronously adjusts the feature extraction logic for the corresponding fields in the observation builder. If the change involves changes to the reward signal, the agent synchronously adjusts the structure and weight coefficients of the potential function in the reward shaper. If the change causes a change in data dimensions, the agent synchronously updates the dimension constants in the dimension definition file. After all adaptations are completed, the agent compares the algorithm code before and after the modification, extracts the differences, and generates an adaptation description document that records the changes to the feature processing logic, the reward shaping logic, and the dimension definitions.

[0091] By developing and optimizing algorithms, intelligent agents can be automatically triggered and executed, eliminating the need for personnel to manually notify algorithm developers and thus eliminating the possibility of downstream components being missed in synchronization.

[0092] S205. Invoke the data link verification agent to generate data link verification results based on the change description document and the adaptation description document. The data link verification agent is an agent built based on a large language model and responsible for correctness verification.

[0093] Understandably, the data link verification agent is an agent built on a large language model with the responsibility of correctness verification. It is responsible for discovering defects before they cause training performance loss. Through systematic static analysis and runtime verification, it examines every data transformation step from backend encoding to near-end policy optimization (PPO) training. This agent follows six core principles: no trust in default values ​​(every operation with a default value must verify its validity); binary and data exchange format equivalence (each field in binary encoding must have a corresponding field in the data exchange format); value range protection (all network inputs must be within the expected range); non-zero shaping verification (each reward shaping component must be non-zero in typical scenarios); cross-file consistency (the same field must have consistent name and type across the encoder, decoder, observer builder, and reward shaper); and log evidence priority (runtime data must be obtained first when diagnosing problems, without guesswork). The agent employs a seven-layer, step-by-step standard operating procedure (SOP): Layer 1: binary encoding / decoding alignment; Layer 2: game information (game_info) field integrity; Layer 3: observer builder value range verification; Layer 4: reward shaping numerical verification; Layer 5: dimension definition consistency; Layer 6: sample pool data integrity; and Layer 7: near-end policy optimization training signal health. Enhanced verification is performed after environmental changes, including six steps: confirming the change range, full re-verification of encoding / decoding consistency, feature processing correctness, reward processing correctness, full re-verification of dimension consistency, and outputting a verification report.

[0094] The data link verification result is the output of the data link verification agent after verifying at least one link in the data conversion link, and is used to characterize whether the verification is successful.

[0095] After the data link verification agent is invoked, it reads the change description document and the adaptation description document. Following preset data transformation link verification rules, the agent checks the consistency of each step layer by layer. Regarding encoding / decoding field alignment, it compares the number, order, and type of fields on the backend encoding end with those on the decoding end. Regarding the observation data value range, it checks whether the normalized values ​​of each observation field are within the expected range. Regarding reward values, it constructs typical scenarios to verify whether each reward component is non-zero and in the correct direction. Regarding data dimension consistency, it compares whether the dimension constants declared in the dimension definition file match the dimensions actually output by the observation builder. When all verification steps pass, a confirmed data link verification result is generated, and the environment change process is considered complete.

[0096] Perform a full verification of the data link before training begins, so that silent data errors such as encoding / decoding misalignment and dimension mismatch are proactively detected and intercepted before they cause training waste.

[0097] For example, please refer to Figure 3 , Figure 3 The architecture diagram of the method in this application is shown. When a user needs to connect a new game to the reinforcement learning training platform, the user inputs task requirement text through the interaction orchestration layer, such as "connect to a new MOBA game environment". After obtaining the task requirement text, the interaction orchestration layer identifies the target task as indicating the modification of the reinforcement learning environment for the target business, and then obtains the environment requirement document corresponding to the game. This environment requirement document records configuration information such as the communication protocol, observation space, action space, and reward signal of the game backend.

[0098] The environment access and maintenance agent is invoked. Based on the task requirements text and environment requirements document, the agent automatically generates adapter code, observation decoding logic, and action encoding logic, connecting the game backend to the training framework according to standard specifications, and generating a change description document. This change description document details the changes involved in this access, including the communication protocol type, observation field list, and field dimensions.

[0099] After the change description document is generated, a linkage process is automatically triggered, invoking the algorithm development and optimization agent. Based on the observation dimensions and action space information recorded in the change description document, the algorithm development and optimization agent constructs a policy network decoupled from the game logic, and designs dense guiding signals based on the backend reward signals. Upon completion, it generates an adaptation description document, recording the adaptation content of feature processing logic and reward shaping logic.

[0100] Subsequently, the data link verification agent is automatically invoked. Based on the change description document and adaptation description document, the data link verification agent verifies each stage of the data transformation link, including whether the encoding / decoding fields are aligned, whether the observed data value range is within the expected range, whether the reward value is correct, and whether the dimension definitions are consistent, generating a data link verification result. When the data link verification result confirms successful verification, the environment access process is considered complete, and the user can immediately start training.

[0101] For examples, please refer to Table 1 for examples of the impact of environmental changes. Change types include adding binary fields, modifying field types or value ranges, adding levels or items, adding item types, and modifying physical parameters. Adding binary fields refers to adding new observation fields to the data frames encoded in the environment backend. Modifying field types or value ranges refers to changing the data type or numerical range of existing observation fields. Adding levels or items refers to adding new game content to the game environment. Adding item types refers to adding entirely new types of items. Modifying physical parameters refers to adjusting the physical attribute values ​​in the game environment.

[0102] The affected components include the environment backend encoder, decoder, observation builder, reward shaper, dimension definition file, and whether retraining is required. The environment backend encoder is responsible for encoding observation data into binary data frames. The decoder is responsible for parsing the binary data frames into an observation array. The observation builder is responsible for extracting features from the observation array. The reward shaper is responsible for constructing a dense guiding signal based on the potential function. The dimension definition file is responsible for declaring the constants for the observation and action dimensions.

[0103] Taking the addition of a binary field as an example, both the environment backend encoder and decoder need to be modified synchronously, and the observation builder, reward shaper, and dimension definition file may be affected. Taking the modification of a field type or value range as an example, the environment backend encoder, decoder, and observation builder need to be modified, the reward shaper may be affected, and the dimension definition file remains unchanged. Taking the addition of a level or item as an example, the environment backend encoder and decoder remain unchanged, the observation builder needs to check slot allocation, the reward shaper needs to check perception logic, and the dimension definition file may be affected. Taking the addition of an item type as an example, the environment backend encoder, decoder, observation builder, reward shaper, and dimension definition file all need to be modified, and the modified reinforcement learning algorithm needs to be retrained. Taking the modification of physical parameters as an example, the environment backend encoder and decoder remain unchanged, the observation builder needs to check the value range, the reward shaper needs to check numerical correctness, and the dimension definition file remains unchanged.

[0104] The algorithm development and optimization agent matches the impact range of the corresponding components in the matrix based on the change types recorded in the change description document, and only performs adaptation on the components that need to be modified. The data link verification agent selects the corresponding verification stage to perform full-link verification based on the adaptation content recorded in the adaptation description document.

[0105] Table 1

[0106] For easier understanding, please refer to Figure 4 , Figure 4 This diagram illustrates the sequence of the reinforcement learning environment modification process provided in this application. The interactive orchestration layer receives the task requirement text input by the user and sends it to the orchestration agent. As shown in step S411, the orchestration agent performs intent analysis on the task requirement text to obtain intent analysis information. If the intent analysis information indicates that the target task is to modify the reinforcement learning environment of the target service, as shown in step S412, the orchestration agent sends the task requirement text and environment requirement document to the environment access and maintenance agent. As shown in step S413, the environment access and maintenance agent performs synchronous changes to the backend code and decoding end based on the task requirement text and environment requirement document, generating a change description document. As shown in step S414, the environment access and maintenance agent sends the change description document to the algorithm development and optimization agent. As shown in step S415, the algorithm development and optimization agent performs adaptation of the observer builder, reward shaper, and dimension definition file, generating an adaptation description document. As shown in step S416, the algorithm development and optimization agent passes the adaptation description document to the data link verification agent. As in step S417, the environment access maintenance agent also passes the change description document to the data link verification agent. As in step S418, the data link verification agent performs full-link verification on encoding / decoding consistency, observation range, reward value, and dimension definition, generates data link verification results, and determines that the environment change process ends when the verification passes.

[0107] The method provided in this application transforms the post-environment change processing flow from a manually sequential notification and modification mode to a forced serial collaboration mode where three intelligent agents automatically execute the changes. The environment access and maintenance agent executes changes based on the task requirement text and environment requirement document, outputting a change description document to ensure the changes are stored in a structured form. The algorithm development and tuning agent, upon receiving the change description document, automatically synchronizes and adapts the observation builder, reward shaper, and dimension definition file, outputting an adaptation description document to ensure complete coverage of each affected component. The data link verification agent, upon receiving the change description document and adaptation description document, automatically performs encoding / decoding alignment, value range verification, numerical verification, and dimension consistency checks, outputting verification results to proactively detect silent data errors before training begins. These three stages are executed sequentially and cannot be skipped, fundamentally eliminating the possibility of missed modifications to downstream components in a manual mode and avoiding wasted computational power due to training with erroneous data.

[0108] In an optional embodiment of the data processing method provided in the above embodiments of this application, an environment access and maintenance agent is invoked to generate a change description document based on the task requirement text and the environment requirement document, including: The environment access and maintenance agent is invoked to modify the reinforcement learning environment based on the task requirement text and environment requirement document, and to generate a change description document.

[0109] Understandably, a reinforcement learning environment is a problem model that outputs observation and reward information based on its own state and action inputs, including a state observation mechanism, a state transition mechanism, and a reward mechanism. The state observation mechanism determines the form of the state and observations and the relationship between them; the state transition mechanism determines how the input action causes changes in the environment's state; and the reward mechanism determines the environment's numerical evaluation of the action. In this application, the reinforcement learning environment is a virtual or real-world scenario in which the reinforcement learning agent interacts and learns, including the definitions of an observation space, an action space, and a reward signal, used to provide the environmental feedback required for the agent's training.

[0110] After invoking the environment access and maintenance agent, the agent reads the change requirements described in the task requirement text and the current environment configuration recorded in the environment requirement document, identifying the backend code files and communication encoding logic that need to be modified. Following a standardized environment change process, the agent sequentially modifies the corresponding parts of the backend code, updates the encoding method of the communication protocol, and synchronously adjusts the field parsing logic at the decoding end. After modification, the agent compares the environment state before and after modification, extracts the differences, and generates a change description document. This document details the list of files involved in the change, whether the communication protocol has changed, the names of newly added or deleted observation fields, and changes in field value ranges. Using the environment access and maintenance agent as the execution entry point for environment changes transforms backend environment modification work from relying on manual, item-by-item operations to standardized, document-driven automatic execution by the agent. The output change description document provides a structured record of the changes, offering a clear source of change information to downstream agents and eliminating potential omissions that might occur with manual information transmission.

[0111] The method provided in this application uses the environment access and maintenance agent as the entry point for environmental changes. The task requirement text provides change instructions, and the environment requirement document provides the current configuration baseline. The agent completes the modification based on both and outputs a change description document, which transforms the change content from implicit experience into a structured document. This ensures that the change information is completely recorded and can be transmitted to downstream agents, thus avoiding the problem of information loss or transmission errors in the manual mode from the source.

[0112] In an optional embodiment of the data processing method provided in the above embodiments of this application, an environment access and maintenance agent is invoked to modify the reinforcement learning environment according to the task requirement text and the environment requirement document, and to generate a change description document, including: Input the task requirements text and environment requirements document into the environment access and maintenance agent; The environment access and maintenance agent is invoked so that it can modify the reinforcement learning environment according to the task requirement text and environment requirement document, and generate the modified reinforcement learning environment. Based on the differences between the reinforcement learning environment before and after the modification, a change description document is generated.

[0113] Understandably, the difference information refers to the differences between the reinforcement learning environment before and after the modification, describing the specific changes involved in this modification and comparing the states before and after the modification.

[0114] First, the task requirement text and environment requirement document are input into the environment access and maintenance agent. The task requirement text provides the specific instructions for this modification, and the environment requirement document provides the current environment's configuration baseline. Together, they constitute the agent's complete input. After invoking the environment access and maintenance agent, the agent parses the change requirements in the task requirement text, reads the current configuration, the backend code files to be modified, and the communication encoding logic in the environment requirement document, executes the modification operation, and generates the modified reinforcement learning environment. Since the environment states before and after the modification have been recorded, the difference information is extracted by comparing the two and a change description document is generated. Limiting the input to two structured information sources—the task requirement text and the environment requirement document—provides the agent's modification behavior with clear basis and boundaries, avoiding modification deviations caused by inconsistent information sources. Generating a change description document based on the difference information before and after the modification transforms the change content from implicit operations to explicit records, making it easier for downstream agents to directly locate the scope of the change.

[0115] For easier understanding, please refer to Figure 5 , Figure 5 The diagram illustrates the processing procedure of the environment access and maintenance agent provided in this application. The task requirement text and environment requirement document are input into the environment access and maintenance agent; the agent is invoked to modify the reinforcement learning environment based on the task requirement text and environment requirement document, generating a modified reinforcement learning environment; and a change description document is generated based on the differences between the original and modified reinforcement learning environments.

[0116] The method provided in this application limits the input of the environment access and maintenance agent to task requirement text and environment requirement document, so that modification operations have a unified information source and clear execution boundaries. The agent performs modifications based on the two structured documents and records the differences before and after the modifications, making the changes explicit in the form of difference information. The generated change description document fully describes the state comparison before and after the changes, and downstream agents can accurately locate the scope of changes based on this, and complete the adaptation without tracing back the entire code.

[0117] In an optional embodiment of the data processing method provided in the above embodiments of this application, an algorithm development and tuning agent is invoked to generate an adaptation description document based on the change description document, including: Call the algorithm development and optimization agent so that the agent can modify the reinforcement learning algorithm for the target business based on the change description document and generate an adaptation description document.

[0118] Understandably, the reinforcement learning algorithm for the target business is used to train the policy network and related components of the reinforcement learning agent, including network structure definition, reward shaping logic, and dimension definition, and is used to iteratively update the agent's decision-making policy based on environmental feedback. The adaptation description document is a structured document output after synchronizing the feature adaptation of the algorithm development and tuning agent's execution observation builder, the logic adaptation of the reward shaper, and the dimension definition files. It describes the content generated after modifying the reinforcement learning algorithm.

[0119] After invoking the algorithm development and tuning agent, the agent reads the changes recorded in the change description document and identifies the impact range of the environmental change on the algorithm. If the change description document records a new observation field, the agent adds corresponding feature extraction logic to the observation builder, incorporating the new field into the observation vector. If the change description document records a change in the field value range, the agent adjusts the normalization parameters of the corresponding dimension in the observation builder. If the change causes a change in the observation dimension, the agent synchronously updates the dimension constants in the dimension definition file. After completing all adaptations, the agent compares the algorithm code before and after modification, extracts the difference information, and generates an adaptation description document. The algorithm development and tuning agent uses the change description document as the sole input, ensuring that the adaptation range on the algorithm side precisely corresponds to the change range on the environment side, avoiding omissions or over-modifications that might occur if algorithm developers manually determine the adaptation range in manual mode. After adaptation is complete, the adaptation description document is output, providing a complete record of the adaptation content for the next stage of verification.

[0120] The method provided in this application limits the input of the algorithm development and tuning agent to a change description document, ensuring that the algorithm-side adaptation is strictly performed based on the actual content of the environmental changes, and the adaptation range precisely corresponds to the change range. The agent automatically completes the synchronous modification of the observer builder, reward shaper, and dimension definition file, transforming the adaptation work that originally required manual item-by-item judgment and manual modification into a document-driven standardized execution, thus eliminating the possibility of downstream components being missed in synchronization.

[0121] In an optional embodiment of the data processing method provided in the above embodiments of this application, an algorithm development and optimization agent is invoked to modify the reinforcement learning algorithm of the target service according to the change description document, and generate an adaptation description document, including: Input the change description document into the algorithm development and tuning agent; Call the algorithm development and optimization agent so that the algorithm development and optimization agent can modify the reinforcement learning algorithm of the target business according to the change description document and generate the modified reinforcement learning algorithm. The modified reinforcement learning algorithm is adapted to the modified reinforcement learning environment. Based on the differences between the original and modified reinforcement learning algorithms, an adaptation description document is generated.

[0122] Understandably, the modified reinforcement learning algorithm is generated by the algorithm development and tuning agent based on the change description document, modifying the original reinforcement learning algorithm. Its observer builder, reward shaper, and dimension definitions are all matched to the modified reinforcement learning environment. The original reinforcement learning algorithm, which was in an adapted state before the environment change, is used to compare with the modified algorithm to extract differences.

[0123] First, the change description document is input into the algorithm development and tuning agent. This document records the specific changes made to the environment, providing a unique basis for algorithm adaptation. After invoking the algorithm development and tuning agent, it parses the changes recorded in the change description document and identifies the components that need adaptation. If the change involves adding observation fields, the agent adds feature extraction logic for the corresponding dimension to the observation builder. If the change involves changes to the reward signal, the agent adjusts the structure or weight coefficients of the potential function in the reward shaper. If the change involves changes in dimensions, the agent updates the constants in the dimension definition file. After modification, a modified reinforcement learning algorithm is generated, maintaining a correspondence with the modified reinforcement learning environment in terms of observation dimensions, action space, and reward signal. Next, the original and modified reinforcement learning algorithms are compared, and the differences between them are extracted. Based on this difference information, an adaptation description document is generated. Using the change description document as the sole input for algorithm adaptation provides a clear execution boundary for the adaptation operation. The adaptation relationship between the modified algorithm and the environment is explicitly defined, avoiding the misalignment issues that may occur during manual adaptation. An adaptation description document is generated based on the differences between the algorithm before and after the modification, making the adaptation content traceable and facilitating the identification of the specific scope of adaptation changes in subsequent verification stages.

[0124] For easier understanding, please refer to Figure 6 , Figure 6 The diagram illustrates the processing flow of the algorithm development and optimization agent. A change description document is input into the algorithm development and optimization agent; the agent is invoked to modify the reinforcement learning algorithm for the target business based on the change description document, generating a modified reinforcement learning algorithm adapted to the modified reinforcement learning environment; and an adaptation description document is generated based on the differences between the original and modified reinforcement learning algorithms.

[0125] The method provided in this application limits the input of the algorithm development and tuning agent to a change description document, ensuring that algorithm adaptation strictly follows the changes in the environment. The agent automatically modifies the observer builder, reward shaper, and dimension definition file synchronously, generating a modified reinforcement learning algorithm that corresponds to the environment. By comparing the algorithm states before and after modification to extract difference information and generate an adaptation description document, the adaptation content changes from implicit modification to explicit recording. Subsequent verification can then accurately locate the adaptation range without having to backtrack all the code.

[0126] In an optional embodiment of the data processing method provided in the above embodiments of this application, a data link verification agent is invoked to generate a data link verification result based on the change description document and the adaptation description document, including: The data link verification agent is invoked to verify the modified reinforcement learning environment and the modified reinforcement learning algorithm based on the change description document and the adaptation description document, and to generate data link verification results.

[0127] Understandably, the modified reinforcement learning environment is generated by the environment access and maintenance agent after modifying the original reinforcement learning environment according to the task requirements text and environment requirements document. Its observation space, action space, or communication protocol have been adjusted according to the change requirements. The modified reinforcement learning algorithm is generated by the algorithm development and tuning agent after modifying the original reinforcement learning algorithm according to the change description document. Its observation builder, reward shaper, and dimension definition are all matched with the modified reinforcement learning environment. The data link verification result is the output of the data link verification agent after verifying at least one link in the data transformation link, used to characterize whether the verification passed.

[0128] After invoking the data link verification agent, the agent reads the change description document and the adaptation description document to obtain the specific content of the environment-side changes and the algorithm-side adaptations. Following preset verification rules, the agent checks the consistency between the modified reinforcement learning environment and the modified reinforcement learning algorithm one by one. Regarding encoding / decoding alignment, it compares whether the number, order, and type of fields on the backend encoder are consistent with those on the decoder. Regarding the observation range, it checks whether the values ​​of each dimension output by the observation builder are within the expected range. Regarding reward values, it constructs typical scenarios to verify that each reward component is non-zero and in the correct direction. Regarding dimensional consistency, it compares whether the constants in the dimension definition file match the dimensions actually output by the observation builder. When all checks pass, a data link verification result confirming successful verification is generated. The change description document defines the expected goals of the changes, and the adaptation description document records the actual adaptation content. These two documents together constitute the verification benchmark, enabling the verification process to accurately determine whether there are inconsistencies between the environment-side changes and the algorithm-side adaptations. Validation is performed before training begins, so that silent data errors such as encoding / decoding misalignment and dimension mismatch are proactively detected before they cause training waste.

[0129] The method provided in this application uses a data link verification agent as the final step in a three-stage collaborative process, automatically performing full-link verification after environmental changes and algorithm adaptation are completed. The change description document provides the expected goals of the change, while the adaptation description document provides the actual adaptation content. These two documents together constitute the verification benchmark, enabling the verification process to accurately determine whether there are misalignments between the environment and the algorithm. Verification is completed before training begins, intercepting silent data errors before they consume computational resources, thus avoiding the waste of training with erroneous data.

[0130] In an optional embodiment of the data processing method provided in the above embodiments of this application, a data link verification agent is invoked to verify the modified reinforcement learning environment and the modified reinforcement learning algorithm based on the change description document and the adaptation description document, and to generate a data link verification result, including: Input the change description document and the adaptation description document into the data link verification agent; The data link verification agent is invoked to verify at least one link in the data transformation link based on the change description document and the adaptation description document, and to generate a data link verification result. The data transformation link is a data flow path from the environment backend encoding to the algorithm training, and at least one link includes at least one of the following: encoding / decoding field alignment link, observation data value range verification link, reward value verification link, data dimension consistency check link, sample pool data integrity check link, and training signal health check link.

[0131] Understandingly, the data transformation chain describes the complete data flow path from backend environment encoding, network transmission, observer builder feature processing, reward shaping by the reward shaper, to algorithm training. It describes the sequential processing stages data undergoes in a reinforcement learning training system. The encoding / decoding field alignment step verifies consistency in the number, order, and type of fields between the binary encoder and decoder, ensuring no field misalignment or loss occurs during encoding and decoding. The observation data range verification step checks whether the values ​​of each dimension output by the observer builder are within the expected range, identifying data out-of-bounds errors caused by decoding logic errors or normalization parameter errors. The reward value verification step constructs typical scenarios to verify whether each reward component is non-zero and in the correct direction, ensuring the reward shaping logic generates the intended guiding signal. The data dimension consistency check step compares the dimension constants declared in the dimension definition file with the actual output dimensions of the observer builder, identifying data dimension mismatches caused by missed updates to the dimension definition file. The sample pool data integrity check is a verification step that examines indicators such as sample pool capacity, version latency, and the proportion of expired samples to ensure the integrity and timeliness of training data. The training signal health check is a verification step that analyzes the trends of training signals such as policy entropy, value loss, and approximate KL divergence to determine whether the training process is in a healthy state.

[0132] First, the change description document and the adaptation description document are input into the data link verification agent. These two documents respectively provide the expected goals of the environment-side changes and the actual content of the algorithm-side adaptation. After invoking the data link verification agent, it performs verification on at least one step of the data transformation link based on the information recorded in the two documents. In the encoding / decoding field alignment step, the number, order, and type of fields at the backend encoding end are compared to those at the decoding end to ensure consistency. In the observation data value range verification step, the normalized values ​​of each observation field are checked to ensure they are within the expected range. In the reward value verification step, typical scenarios are constructed to verify the numerical and directional correctness of each reward component. In the data dimension consistency check step, the constants in the dimension definition file are compared to the actual output dimensions of the observation builder to ensure they match. In the sample pool data integrity check step, the sample pool capacity, version latency, and proportion of expired samples are checked to ensure they are within healthy ranges. In the training signal health check step, the policy entropy value is analyzed to determine if it is too low, if the value loss has converged, and if the approximate KL divergence is within a reasonable range. After verification, the data link verification results are generated. The verification scope covers the entire data flow path from backend encoding to algorithm training. The corresponding verification stage can be selected according to the type of change, so that data link problems under different change scenarios can be systematically detected before training starts, avoiding the omission of potential data errors due to incomplete verification.

[0133] The data link verification agent adopts a 7-layer step-by-step scanning SOP to systematically verify each data conversion step from backend encoding to PPO training, as shown in Table 2. Table 2 shows the 7-layer data link verification methodology. The data link verification agent adopts a 7-layer step-by-step scanning SOP to systematically verify each data conversion step from backend encoding to PPO training. Specifically, this includes: Layer 1 checking Binary encoding / decoding alignment, verifying by comparing the number, order, and type of fields one by one; Layer 2 checking the integrity of the game_info field, verifying phase mapping and consistency between prep_end and game_end; Layer 3 checking the value range of obs_builder, verifying by checking the normalized value range of each field, any deviation from expectations indicates a bug; Layer 4 checking the Reward-Shaping numerical verification, verifying by constructing typical scenarios to verify that each component is non-zero and in the correct direction; Layer 5 checking the consistency of dims, verifying by comparing the constants in dims.py with the actual output dimensions of obs_builder; Layer 6 checking the integrity of the sample pool data, verifying by checking the health of pool_size, ver_lag, and stale metrics; and Layer 7 checking the health of the training signal, verifying by trend analysis of entropy / v_loss / approx_kl.

[0134] Table 2

[0135] For easier understanding, please refer to Figure 7 , Figure 7 A schematic diagram illustrating the data link verification agent's processing flow is shown. The change description document and adaptation description document are input into the data link verification agent. The data link verification agent is invoked to verify at least one link in the data transformation link based on the change description document and the adaptation description document, and to generate a data link verification result. The data transformation link is a data flow path from the environment backend encoding to the algorithm training, and at least one link includes at least one of the following: encoding / decoding field alignment link, observation data value range verification link, reward value verification link, data dimension consistency check link, sample pool data integrity check link, and training signal health check link.

[0136] The method provided in this application limits the verification scope of the data link verification agent to at least one link in the data transformation link, allowing the verification operation to flexibly select the corresponding verification link according to the specific type of environmental change. The encoding / decoding field alignment link ensures data consistency at both encoding and decoding ends; the observation data value range verification link intercepts data out-of-bounds errors caused by decoding errors; the reward value verification link ensures the reward signal conforms to the design intent; the data dimension consistency check link prevents mismatch between dimension definition and actual output; the sample pool data integrity check link ensures the timeliness of training data; and the training signal health check link judges the stability of the training process. Each link covers the complete path from data generation to data consumption, enabling all data errors in the entire link after environmental changes to be systematically detected before training begins.

[0137] In an optional embodiment of the data processing method provided in the above embodiments of this application, after generating the change description document, the method further includes: Train the modified reinforcement learning environment and generate an environment training log; The policy learning debugging agent is invoked so that it can verify the modified reinforcement learning environment based on the environment training log and generate environment verification results. The policy learning debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the environment verification result indicates that the environment-side data is abnormal, the environment access and maintenance agent is invoked so that the environment access and maintenance agent can modify the modified reinforcement learning environment according to the environment verification result.

[0138] It is understandable that the environment training log is the runtime record data generated during the training of the modified reinforcement learning environment, used to record information such as observed values, reward signals and policy outputs during the training process.

[0139] The policy learning debugging agent is an agent built on a large language model that has the responsibility of policy training diagnosis and analysis. It is responsible for discovering efficiency problems from training logs, diagnosing policy learning anomalies, recording historical experience, and comparing the effects of different training batches.

[0140] This intelligent agent adheres to four core principles. The rigorous attribution principle requires distinguishing between observed phenomena and causal attributions, and refraining from making causal inferences when comparison conditions differ. The data-driven principle requires all conclusions to be supported by log data. The business-agnostic principle requires that the analytical framework and methodology be general, with business characteristics derived from documentation. The historical retrospective principle requires key findings to be recorded in the training history documentation. The quantification-oriented principle requires using numerical metrics to quantify the problem as much as possible.

[0141] The agent is analyzed from five dimensions. Dimension one is policy iteration efficiency, analyzing trends in win rate, policy entropy, policy loss, value loss, and approximate KL divergence. Dimension two is sample production efficiency, analyzing sample throughput, game completion rate, and environment step latency. Dimension three is sample utilization efficiency, analyzing sample version latency, number of expired samples, maximum reuse count, and sample pool capacity. Dimension four is reward function analysis and boundary gradient calculation, which is the most distinctive analytical dimension of this agent. Dimension five is the rationality of algorithm hyperparameters, analyzing learning rate, entropy coefficient, pruning parameters, generalization advantage estimation parameters, and weight synchronization interval.

[0142] In the reward function analysis dimension, the policy learning debugs the agent's behavior reward extrapolation method. It automatically extracts the potential function structure from the reward shaper source code, identifies all discrete state switching points, and uses behavior as the analysis unit rather than reward components. The agent defines key behaviors at different stages, constructs an extrapolation table for quantitative calculation, and simulates the total reward obtained per frame when the agent selects different behaviors at each stage. By listing all behavior rewards across all stages in the same table for cross-stage comparison, the cross-stage advantage ratio is calculated to check whether the optimal behavior reward in the target stage truly outweighs the optimal behavior reward in non-target stages. If it is found that the overall reward of a certain behavior in a non-target stage is higher than that in the target stage, it is determined that there is a cross-stage reward inversion. The agent also checks whether the potential function is close to saturation in the high-value range, resulting in a small potential function increment from further behavior improvement and insufficient dense reward compensation. Simultaneously, it audits the actual reward direction of dense rewards to verify whether the subsidized action set is consistent with the design intent. Figure 1 To.

[0143] Upon detecting an anomalies, the policy learning and debugging agent triggers the corresponding agent to perform repairs based on the problem type. When an anomaly is detected in the environment-side data, the environment access and maintenance agent is triggered to investigate and repair it. When an anomaly is detected in the reward function or hyperparameters, the algorithm development and tuning agent is triggered to make modifications. When a silent error is suspected in the data link, the data link verification agent is triggered to perform full-link verification. When a performance issue is detected at the framework layer, the framework development and maintenance agent is triggered to investigate. After repairs are completed, the policy learning and debugging agent can continue to verify the repaired training logs, forming a closed loop of diagnosis, repair, and re-verification.

[0144] The policy learning and debugging agent employs a behavior reward inference method, using behavior as the analysis unit to simulate and infer the total reward for each frame under different behavior choices, verifying whether the reward function guides the expected optimal behavior ranking. This method specifically includes the following sub-functions: First, potential function structure extraction: automatically extracting all potential function components from the reward shaper source code, including weights, ranges, and dependent variables, and identifying all discrete state transition points, i.e., the locations where the potential function is discontinuous due to conditional branches. Second, boundary state transition gradient calculation: calculating the difference in potential function before and after each discrete state transition point, judging whether the sign, direction, and magnitude of the difference are reasonable, and identifying a cliff where the absolute value of the potential function difference exceeds 30% of the total potential function. Third, continuous component gradient analysis: sampling and calculating the derivative of the potential function with respect to the state variable in key intervals, checking whether the gradient direction is correct, and whether there are flat regions and reversal points. The fourth function is behavior reward extrapolation, which is the core function. It simulates and extrapolates the total reward of the agent in each frame under different behavior choices, using behavior as the analysis unit. Specifically, this includes automatically deriving the behavior dimensions of each stage from the reward shaping source code and establishing a mapping between behavior and reward, calculating the value of the potential function difference term in the potential function-based reward shaping of each frame, and checking whether the actual behavior reward ranking matches the design intention. Figure 1 To detect reward inversion, calculate the cross-stage advantage ratio between the optimal behavior in the target stage and the highest competitive behavior in non-target stages. A ratio greater than or equal to 2 is considered healthy. Check whether the action set of dense reward actual subsidies is consistent with the design intent. Figure 1 The process includes: 1) Completing the reward directionality audit; 2) Multi-component interaction analysis to check whether the gradients of multiple reward components cancel each other out under certain state combinations; 3) Overfitting trap detection to automatically identify six traps: line-of-sight cliff, radius bottleneck, exploration saturation, direction lock, gradient vacuum, and reverse gradient; 4) Shaping signal alignment check to ensure that the reward shaping signal uses only information observable by the agent; and 5) Comparing the final reward with the shaping magnitude to ensure that the magnitude relationship between sparse final rewards and dense shaping rewards is reasonable.

[0145] The behavioral reward extrapolation method sets six inviolable red lines: The first red line is to analyze behavior as the unit of analysis, meaning that regardless of the rationality of each reward component, only whether the total reward of the desired behavior overwhelmingly surpasses that of competing behaviors. Violation of this red line results in the omission of cross-stage reward inversions after component combinations. The second red line is cross-stage parallel comparison, meaning that behavioral rewards for all stages must be listed in the same table for comparison. Violation of this red line results in the inability to detect cross-stage reward inversions. The third red line is a cross-stage advantage ratio greater than or equal to two, meaning that the ratio of the optimal behavioral reward in the target stage to the optimal behavioral reward in non-target stages is not less than two. Violation of this red line results in the agent lacking the motivation to switch to the target stage. The fourth red line is total reward rather than single component, meaning that all activated reward components must be summed for each behavior. Violation of this red line results in the neglect of insufficient dense reward compensation when reward shaping based on the potential function reaches high saturation. The fifth red line is quantitative extrapolation, meaning that specific numerical calculations are used instead of rough estimates. Violation of this red line results in the masking of problems where reward magnitudes are similar but the order is incorrect. The sixth red line is directional auditing, which means checking whether the actual set of actions for dense reward subsidies is consistent with the design intent. Figure 1 Therefore, the consequence of violating this rule is that it provides undirected rewards and subsidies for undesirable behaviors.

[0146] Please see Figure 18 , Figure 18 A schematic diagram of the behavioral reward extrapolation method is shown. This method is based on the reward shaping theory of potential functions. Its core concept is that the backend only returns simple sparse signals, i.e., victory, defeat, or zero, and all shaping operations are completed on the algorithm side. The formula for calculating the shaping reward is: the shaped reward equals the original reward plus a discount factor multiplied by the next state potential function value, minus the current state potential function value. This formula constructs a dense guiding signal while ensuring the optimal policy remains unchanged.

[0147] This design methodology follows six principles. The first principle is that the shaping signal must be aligned with the agent's observable information, using only the agent's observable information for shaping. The second principle is smooth decay when information is lost, avoiding a sharp drop in the potential function that would cause drastic fluctuations in the reward signal. The third principle is non-linear distance guidance, where nearby incentives should be stronger than distant ones, reflecting the diminishing marginal effect of distance. The fourth principle is that all weights are read from configuration files, not hardcoded into the code. The fifth principle is numerical verification, constructing typical scenarios to verify that the potential function is non-zero and its direction is correct. The sixth principle is checking the continuity of the potential function during state transitions to avoid reward cliffs caused by discrete state transitions.

[0148] The behavioral reward extrapolation method uses "behavior" as the unit of analysis (rather than components) to simulate and extrapolate the reward for each frame under different behavior choices, verifying whether the reward function leads to the expected optimal behavior ranking. Specifically, it includes the following key points: using behavior as the unit of analysis (whether different components are reasonable, and whether the expected behavioral reward satisfies competing behaviors); cross-stage parallel comparison (behavioral rewards for all stages must be in one table); cross-stage advantage ratio R ≥ 2.0 (optimal in the target stage / optimal in non-target stages ≥ 2.0); total reward rather than components (all activated reward components must be summed for each behavior); quantitative extrapolation (using specific numerical calculations, not approximate values); directional auditing (checking the action set for dense rewards that actually yields the reward).

[0149] The environment verification result is the output of the policy learning debugging agent after verifying the modified reinforcement learning environment based on the environment training logs. It is used to indicate whether there are any data anomalies on the environment side. Environment-side data anomalies refer to data problems related to environment configuration, such as encoding / decoding errors, out-of-bounds observation values, or missing fields, as reflected in the environment training logs.

[0150] After starting training on the modified reinforcement learning environment, an environment training log is generated, recording the observation data, reward signal, and action output for each frame. The policy learning and debugging agent is then invoked. This agent reads the environment training log, analyzes whether the value distribution of the observation data is within the expected range, checks for cases where certain observation fields are consistently zero or consistently out of bounds, and determines whether the distribution of the reward signal is consistent with the environment design. Figure 1 After analysis, an environment verification result is generated. If the environment verification result shows anomalies such as out-of-bounds observation values ​​or missing fields, the environment access and maintenance agent is invoked. This agent locates and modifies the corresponding part of the reinforcement learning environment based on the abnormal fields and anomaly types indicated in the environment verification result. The policy learning and debugging agent transforms training log analysis from manual inspection to automatic verification, enabling the detection of anomalies in environment-side data early in training. Upon detection of anomalies, the environment access and maintenance agent is automatically triggered to perform repairs, forming a closed loop from diagnosis to repair, preventing abnormal data from continuously affecting training.

[0151] For easier understanding, please refer to Figure 8 , Figure 8 This diagram illustrates how a policy learning debugging agent tests a reinforcement learning environment. The modified reinforcement learning environment is trained, generating an environment training log. The policy learning debugging agent is then invoked to verify the modified reinforcement learning environment based on the environment training log, generating an environment verification result. If the environment verification result indicates abnormal data on the environment side, the environment access maintenance agent is invoked to modify the modified reinforcement learning environment based on the environment verification result.

[0152] For easier understanding, please refer to Figure 9 , Figure 9 A timing diagram of the environmental data anomaly verification process provided in an embodiment of this application is shown. Steps S911 to S918 are... Figure 4 Steps S411 to S418 are the same and will not be repeated here. For example, in step S920, the policy learning debugging agent verifies the modified reinforcement learning environment and generates an environment verification result. If the environment verification result indicates abnormal data on the environment side, as in step S921, the policy learning debugging agent sends the environment verification abnormality result to the environment access maintenance agent. As in step S922, the environment access maintenance agent modifies the modified reinforcement learning environment according to the environment verification result.

[0153] The method provided in this application introduces a policy learning and debugging agent into the collaborative process after environmental changes, transforming the analysis of training logs from relying on manual investigation to automatic verification by the agent. By analyzing the distribution of observed data values ​​and reward signals, the policy learning and debugging agent can detect anomalies in environmental data early in the training process, preventing anomalous data from being amplified over long training periods. Upon detection of anomalies, the environment access and maintenance agent is automatically triggered to perform repairs, forming a closed-loop process from problem discovery to cause identification and repair verification, thus shortening the time interval from the occurrence of anomalies to the completion of repairs.

[0154] In an optional embodiment of the data processing method provided in the above embodiments of this application, after generating the adaptation description document, the method further includes: Train the modified reinforcement learning algorithm and generate an algorithm training log. The policy learning debugging agent is invoked so that it can verify the modified reinforcement learning algorithm based on the algorithm training log and generate the algorithm verification result. The policy learning debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the algorithm verification result indicates that the reward function or hyperparameters are abnormal, the algorithm development and tuning agent is invoked so that the algorithm development and tuning agent can modify the reinforcement learning algorithm according to the algorithm verification result.

[0155] Understandably, the algorithm training log is the runtime record data generated during the training of the modified reinforcement learning algorithm. It records information such as policy entropy, value loss, win rate curve, and reward signal distribution during the training process. The algorithm verification result is the output of the policy learning debugging agent after verifying the modified reinforcement learning algorithm based on the algorithm training log. It is used to indicate whether there are any abnormalities in the reward function or hyperparameters on the algorithm side. Abnormalities in the reward function refer to problems such as reward inversion in the potential function constructed by the reward shaper during cross-stage comparisons, insufficient compensation for dense rewards due to high-level saturation of the potential function, or inconsistencies between the directionality of dense rewards and the design intent. Abnormalities in hyperparameters refer to the deviation of algorithm hyperparameters such as learning rate, entropy coefficient, pruning parameter, or generalization advantage estimation parameter from a reasonable range, resulting in slow policy convergence or premature entry into local optima.

[0156] After training the modified reinforcement learning algorithm, an algorithm training log is generated, recording metrics such as policy entropy, value loss, win rate, approximate KL divergence, and reward signal distribution for each iteration. The policy learning debugging agent is invoked, which reads the algorithm training log and performs verification across five dimensions: policy iteration efficiency, sample production efficiency, sample utilization efficiency, reward function analysis, and algorithm hyperparameter rationality. In the reward function analysis dimension, the policy learning debugging agent performs behavioral reward deduction, extracting the potential function structure from the reward shaper source code. Using behavior as the analysis unit, it simulates and deduces the total reward for each frame under different behavior choices, checking for reward inversion through cross-stage parallel comparison and quantitative advantage ratio calculation. In the hyperparameter rationality dimension, it checks whether the policy entropy prematurely approaches zero, whether the value loss continuously oscillates, and whether the win rate curve stagnates. Algorithm verification results are generated after analysis. If the algorithm verification results indicate reward function or hyperparameter anomalies, the algorithm development tuning agent is invoked. This agent adjusts the weight coefficients of the reward shaper or modifies the hyperparameter configuration based on the anomaly type and specific location indicated in the verification results. The policy learning and debugging agent transforms training log analysis from manual inspection to automatic verification, enabling the early detection of reward function design flaws and hyperparameter setting issues. Upon anomaly detection, it automatically triggers the algorithm development and tuning agent to execute modifications, forming a closed loop from diagnosis to repair. This ensures that the reward function and hyperparameters are optimized to a reasonable state early in training, avoiding prolonged training under incorrect reward guidance or unreasonable hyperparameters.

[0157] For easier understanding, please refer to Figure 10 , Figure 10This diagram illustrates how a policy learning debugging agent tests a reinforcement learning algorithm. The modified reinforcement learning algorithm is trained, generating an algorithm training log. The policy learning debugging agent is then invoked to verify the modified algorithm based on the training log, generating a verification result. This agent, built on a large language model, is responsible for policy training diagnosis and analysis. If the verification result indicates an anomaly in the reward function or hyperparameters, the algorithm development and tuning agent is invoked to modify the modified reinforcement learning algorithm based on the verification result.

[0158] For easier understanding, please refer to Figure 11 , Figure 11 This diagram illustrates a timing diagram of the anomaly verification process for a reinforcement learning algorithm provided in an embodiment of this application. Steps S1111 to S1118 are... Figure 4 Steps S411 to S418 are the same and will not be repeated here. For example, in step S1120, the policy learning debugging agent verifies the modified reinforcement learning algorithm and generates an algorithm verification result. If the algorithm verification result indicates an abnormality in the reward function or hyperparameters, as in step S1121, the policy learning debugging agent sends the algorithm verification abnormality result to the algorithm development and tuning agent. As in step S1122, the algorithm development and tuning agent modifies the modified reinforcement learning algorithm based on the algorithm verification result.

[0159] The method provided in this application introduces a policy learning and debugging agent into the collaborative process after algorithm adaptation, transforming the analysis of algorithm training logs from relying on manual item-by-item inspection to automatic multi-dimensional verification by the agent. The policy learning and debugging agent's performance reward deduction can detect cross-stage reward inversion problems that traditional vertical component review cannot detect, as well as deep-seated defects such as insufficient compensation for dense rewards due to high potential function saturation and directional deviations in dense rewards. At the hyperparameter level, it can identify abnormal signals such as premature convergence of entropy values ​​and oscillations in value loss in the early stages of training. Upon detection of anomalies, the algorithm development and tuning agent is automatically triggered to perform repairs, ensuring that the reward function and hyperparameters are optimized to a reasonable state in the early stages of training, avoiding prolonged ineffective training caused by improper reward design or unreasonable hyperparameter settings.

[0160] In an optional embodiment of the data processing method provided in the above embodiments of this application, after generating the data link verification result, the method further includes: The modified reinforcement learning environment and the modified reinforcement learning algorithm are jointly trained to generate link training logs, which are log data related to the data links generated during the training process. The policy learning and debugging agent is invoked so that it can verify the data link status based on the link training log and generate the link verification result. The policy learning and debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the link verification result indicates that the data link is abnormal, the data link verification agent is invoked so that the data link verification agent can verify the corresponding link of the data link based on the link verification result.

[0161] Understandably, the link training log is log data related to the data link generated during joint training, used to record the data flow status from backend encoding to algorithm training. The data link status refers to the operational status of the data transformation link during training, including performance in aspects such as encoding / decoding alignment, observation range, reward values, and dimensional consistency. The link verification result is the output of the policy learning debugging agent after verifying the data link status based on the link training log, used to indicate whether there are any anomalies in the data link. Data link anomalies are problems that occur in the data transformation link, such as field misalignment, out-of-bounds value range, incomplete sample pool data, or unhealthy training signals.

[0162] The modified reinforcement learning environment and algorithm are jointly trained. During training, a data link training log is generated, recording the actual runtime data from environment encoding to algorithm training. A policy learning debugging agent is invoked. This agent reads the data link training log and analyzes whether the encoding / decoding fields are aligned, whether the observation range continuously exceeds limits, whether the reward signal distribution is reasonable, whether the sample pool data is complete, and whether the training signal is healthy, thereby determining the data link status. After analysis, a data link verification result is generated. If the data link verification result indicates a data link anomaly, such as an excessively high proportion of expired samples in the sample pool or abnormal oscillations in the training signal, the data link verification agent is invoked to perform in-depth verification based on the anomaly indicated by the data link verification result. The data link verification agent performs specific verification on the anomaly to confirm the root cause of the problem. The policy learning debugging agent continuously monitors the data link status during training and can dynamically discover new data link problems that appear after full-link verification has passed. Upon discovering a problem, only the data link verification agent is triggered to perform targeted verification on the anomaly, without re-executing all verification steps, thus improving diagnostic efficiency.

[0163] For easier understanding, please refer to Figure 12 , Figure 12There is shown a schematic diagram of joint testing of reinforcement learning environments and algorithms by a policy learning debugging agent. Joint training is performed on the modified reinforcement learning environment and the modified reinforcement learning algorithm to generate link training logs, wherein the link training logs are log data related to data links generated during training; the policy learning debugging agent is invoked to enable the policy learning debugging agent to check the data link status according to the link training logs and generate link check results, and the policy learning debugging agent is an agent constructed based on a large language model and responsible for policy training diagnosis and analysis; when the link check result indicates that the data link is abnormal, a data link verification agent is invoked to enable the data link verification agent to verify the corresponding link of the data link according to the link check result.

[0164] For ease of understanding, please refer to Figure 13 , Figure 13 there is shown a sequence diagram of a data link abnormality check processing flow provided by an embodiment of the present application. Wherein, steps S1311 to S1318 are the same as steps S411 to S418 in Figure 4 , and details are not described herein again. As shown in step S1320, the policy learning debugging agent checks the data link status and generates a link check result. When the link check result indicates that the data link is abnormal, as shown in step S1321, the policy learning debugging agent sends a link check abnormality result to the data link verification agent. As shown in step S1322, the data link verification agent verifies the corresponding link of the data link according to the link check result.

[0165] The method provided by the embodiment of the present application extends the monitoring range of the policy learning debugging agent to the data link status, so that data link problems can be found not only in post-change verification, but also can be continuously monitored during training. The policy learning debugging agent locates the specific link of abnormality by analyzing the link training logs, and then directionally triggers the data link verification agent for rechecking, instead of performing full verification on all links every time. This process realizes an automatic closed loop from training monitoring to accurate verification, reduces the frequency of full verification, and ensures that the data link is continuously in a healthy state during training.

[0166] In an alternative embodiment of the data processing method provided in the above embodiment of the present application, after generating the change description document, the method further includes: When the reinforcement learning framework interface of the target service is changed, a framework development and maintenance agent is invoked to enable the framework development and maintenance agent to generate framework interface change request information according to the change description document, wherein the framework development and maintenance agent is an agent constructed based on a large language model with the responsibility of framework code maintenance, and the framework interface change request information is used to represent the implementation scheme for the change of the reinforcement learning framework interface; In response to the confirmation operation of the framework interface change request information, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can modify the reinforcement learning framework interface according to the framework interface change request information and generate framework interface change information. The framework interface change information is used to represent the changes in the modified reinforcement learning framework interface. Invoke the environment access and maintenance agent so that the environment access and maintenance agent can modify the modified reinforcement learning environment according to the framework interface change information; Call the algorithm development and tuning agent so that the agent can modify the reinforcement learning algorithm according to the framework interface change information.

[0167] Understandably, the reinforcement learning framework interface is a functional interface in the training framework used for environment access, algorithm callback, sample storage, and multi-process communication. It is used to connect the framework layer with the environment access and maintenance agent and the algorithm development and tuning agent.

[0168] The framework development and maintenance agent is an intelligent agent built on a large language model, responsible for maintaining the framework code. It is responsible for the code quality, performance optimization, and functional evolution of both the framework layer and the algorithm layer. This intelligent agent adheres to six core principles: a human-in-the-Loop mechanism, requiring a change description document to be output and manually confirmed before any code change; zero cross-dependencies, with framework layer code never referencing algorithm layer or feature layer code; an unbreakable opaque buffer, where the framework only operates on three flat 32-bit floating-point arrays: the observation buffer (obs_buf), the action buffer (action_buf), and the extra buffer (extra_buf), without interpreting their semantics; a first-class field, with the version number recorded in the version field of the step record (StepRecord), not hidden in the extra buffer; stable callback interfaces, requiring evaluation of the impact of signature changes on all implementations for the algorithm callback base class; and no performance degradation, with the number of samples per second (SPS) after the change not falling below 90% of the previous level. The change implementation process adopts a two-phase design: Phase 1 first outputs a change description document, including a change overview, impact analysis, operational efficiency impact, data verification plan, and rollback plan, and waits for manual confirmation; Phase 2 executes the code implementation after manual confirmation.

[0169] When the interface of the reinforcement learning framework for the target business changes, the framework development and maintenance agent is invoked. This agent reads the change description document to understand the background of the current environment change, analyzes the scope of impact of the framework interface change, and outputs a framework interface change request. This request includes a change overview, impact analysis, operational efficiency impact assessment, data validation plan, and rollback scheme. After the framework interface change request is output, it awaits manual review. Since the framework layer is at the bottom of a three-layer decoupled architecture, its changes directly affect the stability of environment access and algorithm callbacks. Therefore, a manual confirmation step is introduced before automated execution to prevent automated operations from disrupting the stability of the training framework. Upon receiving the confirmation instruction, the framework development and maintenance agent is invoked again. This agent modifies the corresponding code of the reinforcement learning framework interface according to the confirmed framework interface change request, generates framework interface change information, and records the modified interface definition and change content. After the modification is completed, the environment access and maintenance agent is invoked. This agent synchronously adapts the environment bridging code in the reinforcement learning environment according to the environment interaction interface change content recorded in the framework interface change information. Simultaneously, an algorithm development and tuning agent is invoked. This agent synchronously adapts the callback implementation in the reinforcement learning algorithm based on the callback interface changes recorded in the framework interface change information. After a framework interface change, affected agents are automatically notified, ensuring that the impact of framework layer changes is fully propagated to the environment and algorithm layers, avoiding runtime errors due to interface inconsistencies.

[0170] The framework development and maintenance agent is an intelligent agent built on a large language model, responsible for maintaining the framework code. It is responsible for the code quality, performance optimization, and functional evolution of both the framework layer and the algorithm layer.

[0171] This intelligent agent adheres to six core principles. The first is a human intervention mechanism, requiring that any code change must be documented and manually confirmed before execution. This serves as a safety barrier to prevent automated operations from disrupting the stability of the training framework. The second is the zero-cross-dependency principle, requiring the framework layer code to never reference algorithm or business layer code, ensuring the three-layer decoupled architecture remains intact. The third is the opaque buffer unbreakable principle, requiring the framework layer to operate only on three flat floating-point arrays: the observation buffer, action buffer, and extra buffer. It does not interpret the semantics of these arrays, only handling data movement and storage management; the framework layer only needs the observation dimension, action dimension, and extra dimension to function. The fourth is the version number first-class field principle, requiring the version number to be recorded in the version field of the sample record, not hidden in the extra buffer. The framework layer natively supports a version-based expired sample eviction mechanism. The fifth is the callback interface stability principle, requiring that changes to the signature of the algorithm callback base class must assess the impact on all implementations. The sixth is the performance non-degradation principle, requiring that the sample throughput after a change not be less than 90% of the previous level.

[0172] The change implementation process for this intelligent agent adopts a two-phase design. Phase one is the solution output, where the intelligent agent analyzes the impact scope of the framework interface change based on the change description document, generates a change description document, including a change overview, impact analysis, operational efficiency impact assessment, data verification plan, and rollback plan, and awaits human confirmation after output. Phase two is the code implementation, where, upon receiving the human confirmation instruction, the intelligent agent executes the specific code modifications according to the confirmed solution, and generates framework interface change information upon completion.

[0173] After the framework interface changes are completed, the framework development and maintenance agent, as the initiator of the collaboration, sends synchronization notifications to relevant agents based on the scope of the change's impact. If the change involves environment interaction interfaces, the environment access and maintenance agent is notified to synchronize and adapt the environment bridging code. If the change involves algorithm callback interfaces, the algorithm development and tuning agent is notified to synchronize and adapt the callback implementation.

[0174] The core files maintained by the framework for developing and maintaining intelligent agents encompass the asynchronous frame pushing module, batch inference module, circular sample pool, shared memory inference module, shared memory sample pool, multi-process orchestration module, vectorized environment module, online statistics module, and algorithm callback base class. These files collectively constitute the framework layer in the three-layer decoupled architecture, providing standardized data scheduling and communication services for the upper algorithm layer and the lower business layer.

[0175] For easier understanding, please refer to Figure 14 , Figure 14 The diagram illustrates the processing flow of the framework development and maintenance agent. When the reinforcement learning framework interface for the target business changes, the framework development and maintenance agent is invoked. This agent generates a framework interface change request based on the change description document. The framework development and maintenance agent is an agent built based on a large language model, responsible for maintaining the framework code. The framework interface change request information represents the implementation scheme of the reinforcement learning framework interface change. In response to the confirmation operation of the framework interface change request information, the framework development and maintenance agent is invoked again. This agent modifies the original reinforcement learning framework interface according to the change request information, resulting in a modified reinforcement learning framework interface and generating framework interface change information. This change information represents the content of the changes in the modified reinforcement learning framework interface. The environment access and maintenance agent is invoked to modify the reinforcement learning environment according to the framework interface change information. Finally, the algorithm development and tuning agent is invoked to modify the reinforcement learning algorithm according to the framework interface change information.

[0176] The method provided in this application introduces a framework development and maintenance agent into the collaborative process after environmental changes, transforming framework interface changes from manual modification and verbal notification into a controlled process executed automatically by the agent. Before modification, the framework interface change request information outputs an implementation plan and awaits manual confirmation, achieving a balance between automation efficiency and security stability in framework layer changes. Upon confirmation, the agent automatically executes the modification and generates framework interface change information. The environment access and maintenance agent and the algorithm development and optimization agent adapt synchronously based on this information, automatically propagating the impact of the framework layer change to all dependent parties. This avoids the interface inconsistency problem caused by downstream components not updating synchronously after framework interface changes in the manual mode.

[0177] In an optional embodiment of the data processing method provided in the above embodiments of this application, after generating the framework interface change information, the method further includes: Train the modified reinforcement learning framework interface and generate interface training logs; The policy learning debugging agent is invoked so that it can verify the modified reinforcement learning framework interface based on the interface training log and generate the interface verification result. The policy learning debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the interface verification result indicates that the interface is abnormal, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can modify the modified reinforcement learning framework interface according to the interface verification result.

[0178] Understandably, the interface training log is the runtime record data generated during the training of the modified reinforcement learning framework interface. It records performance-related metrics such as sample throughput, inference latency, sample pool read / write rate, and inter-process communication timeout. The interface verification result is the output of the policy learning debugging agent after verifying the modified reinforcement learning framework interface based on the interface training log. It indicates whether there are performance anomalies in the framework interface. Interface anomalies are performance issues that occur during the training process, such as decreased sample throughput, increased inference latency, sample pool read / write blocking, or inter-process communication timeouts.

[0179] After training begins on the modified reinforcement learning framework interface, an interface training log is generated, recording framework-level performance metrics such as sample throughput, inference latency, and sample pool read / write rates. The policy learning debugging agent is invoked. This agent reads the interface training log, analyzes whether the sample throughput is lower than before the change from the perspective of sample production efficiency, and checks for blockages in sample pool read / write from the perspective of sample usage efficiency, comparing the performance metrics changes before and after the framework interface change. After analysis, an interface verification result is generated. If the interface verification result shows a decrease in sample throughput or an increase in inference latency, the framework development and maintenance agent is invoked. This agent, based on the anomaly type and specific location indicated by the verification result, investigates and modifies the corresponding code of the reinforcement learning framework interface. The policy learning debugging agent transforms framework-level performance monitoring from manual investigation to automatic verification, dynamically detecting performance degradation issues caused by framework interface changes during training. Upon detecting anomalies, the framework development and maintenance agent is automatically triggered to perform repairs, ensuring that framework interface performance issues are corrected early in training and preventing continuous performance degradation from affecting overall training efficiency.

[0180] For easier understanding, please refer to Figure 15 , Figure 15 The diagram illustrates the process of the policy learning debugging agent. The modified reinforcement learning framework interface is trained, generating an interface training log. The policy learning debugging agent is then invoked to verify the modified reinforcement learning framework interface based on the interface training log, generating an interface verification result. This policy learning debugging agent is a large language model-based agent with policy training, diagnostic, and analysis responsibilities. If the interface verification result indicates an interface anomaly, the framework development and maintenance agent is invoked to modify the modified reinforcement learning framework interface based on the verification result.

[0181] The method provided in this application extends the diagnostic scope of the policy learning debugging agent to framework interface performance, enabling dynamic monitoring and automatic detection of performance degradation issues after framework layer changes during training. The policy learning debugging agent analyzes metrics such as sample throughput and inference latency in the interface training logs to detect anomalies in the early stages of performance degradation. Upon detection, the framework development and maintenance agent is automatically triggered to investigate and repair, ensuring that the framework layer performance meets training requirements after interface changes. This avoids the problem in manual mode where framework layer performance degradation may not be detected until after prolonged training.

[0182] In an optional embodiment of the data processing method provided in the above embodiments of this application, after obtaining the task requirement text of the target task, the method further includes: Perform intent analysis on the task requirement text to obtain intent information; When the intent information indicates that the target task is to modify the reinforcement learning environment of the target service, determine the environment access maintenance agent, algorithm development and optimization agent, and data link verification agent used to perform the target task, as well as the calling order of the environment access maintenance agent, algorithm development and optimization agent, and data link verification agent.

[0183] Understandably, intent information is the result of intent analysis of the task requirement text, used to indicate the operation type and scope of the target task, serving as the basis for determining the required intelligent agent to be invoked. The invocation order is the execution sequence of the environment access and maintenance intelligent agent, the algorithm development and optimization intelligent agent, and the data link verification intelligent agent in the collaborative process. The environment access and maintenance intelligent agent performs the change first, the algorithm development and optimization intelligent agent performs the adaptation next, and the data link verification intelligent agent performs the verification last.

[0184] After obtaining the task requirement text, intent analysis is performed to identify the operation types contained in the user's natural language input. If the intent information indicates that the target task is to modify the reinforcement learning environment of the target business, it is determined that the environment access and maintenance agent, the algorithm development and optimization agent, and the data link verification agent need to be invoked, and the invocation order of the three agents is determined. The invocation order is as follows: the environment access and maintenance agent first performs the environment change, the algorithm development and optimization agent then performs algorithm adaptation based on the change results, and the data link verification agent finally performs full-link verification on the change and adaptation results. Intent analysis parses the user's natural language requirement into specific operation types and agent invocation schemes, allowing users to drive multi-agent collaboration processes without understanding the responsibilities and collaboration rules of each agent. Automatically determining the agent set and invocation order based on intent information ensures the integrity and consistency of the collaboration process in each environment change scenario, preventing any agent from being missed due to differences in user description.

[0185] The method provided in this application adds an intent analysis step after obtaining the task requirement text, enabling users to trigger a standardized multi-agent collaboration process simply by inputting their requirements in natural language. The intent analysis automatically identifies the operation type and determines the agents to be invoked and their execution order, ensuring consistency and integrity of the collaboration process in each changing environment scenario. This avoids, from the outset, situations where agents are omitted or invoked in the wrong order due to differences in user descriptions or human error.

[0186] This application also provides a reinforcement learning intelligent agent system; for ease of understanding, please refer to [link to relevant documentation]. Figure 16 , Figure 16This paper presents a reinforcement learning agent system. The entire reinforcement learning development process is broken down into five dedicated agents with single responsibilities, achieving end-to-end automated collaboration through standardized interfaces and linkage protocols. The overall system architecture is divided into four layers from top to bottom: interaction orchestration layer, agent collaboration layer, shared knowledge layer, and project information layer. The reinforcement learning agent system is positioned as a complete RL development chain covering environment access, algorithm development, data validation, and training and debugging.

[0187] The interaction orchestration layer serves as the top-level entry point of the system, responsible for receiving user requests, orchestrating the order of agent calls, and summarizing output results. Users can describe their requests in natural language (such as "accessing a new MOBA game environment" or "diagnosing why the training win rate is not increasing"). The interaction orchestration layer parses the requests into specific agent call sequences and schedules them for execution serially or in parallel according to dependencies.

[0188] The agent collaboration layer comprises five dedicated agents: the environment access and maintenance agent (rl-env-adapter), the algorithm development and optimization agent (rl-algo-developer), the data pipeline verification agent (rl-data-pipeline-checker), the framework development and maintenance agent (rl-framework-maintainer), and the policy learning and debugging agent (rl-policy-learning-debugger). Each agent is built upon a large language model and has independent responsibility boundaries, input / output specifications, and core methodologies. The collaboration relationships among the five agents are shown in Table 3. These relationships include: After an environment change, the environment access and maintenance agent initiates feature adaptation, logical adaptation of the reward shaper, and dimensional synchronization to the algorithm development and optimization agent, and initiates full-pipeline verification to the data pipeline verification agent. After feature and reward modifications, the algorithm development and optimization agent initiates numerical correctness verification to the data pipeline verification agent. The framework development and maintenance agent sends synchronization notifications to the environment access and maintenance agent when the framework interface changes, and to the algorithm development and optimization agent when the callback interface changes. When the policy learning debugging agent discovers problems with hyperparameters or reward functions, it initiates modifications to the algorithm development and tuning agent; when it discovers abnormal data on the environment side, it initiates repairs to the environment access and maintenance agent; when it suspects a silent error in the data link, it initiates full-link verification to the data link verification agent; and when it discovers framework performance issues, it initiates investigations to the framework development and maintenance agent.

[0189] Table 3

[0190] The shared knowledge layer comprises two core components. The first component is the general knowledge base (rl-sys-dev-rule), which is referenced by all five agents and contains nine core methods: a three-layer decoupling architecture, an environment requirements documentation mechanism, a data link correctness methodology, a reward shaping design methodology, a training and diagnostic methodology, standard procedures for environment access, lessons learned, standard operating procedures for environment change coordination, and a methodology for frame push performance optimization. The general knowledge base ensures that all agents follow consistent design principles and methodologies, rather than acting independently.

[0191] The second component is the project information source, namely the environment requirements document. Each reinforcement learning project maintains one environment requirements document as the sole source of project-specific information, containing complete information such as an environment overview, backend information, observation space, action space, reward signal, communication protocol, file structure, training configuration, and known issues. The priority for obtaining the environment requirements document is: direct access to the user takes precedence over automatic search, automatic search takes precedence over automatic generation, and automatic generation takes precedence over interactive acquisition. The environment requirements document mechanism is the foundation for achieving zero hard-coding and cross-environment reuse.

[0192] The project information layer serves as the storage and management layer for environment requirement documents (env-spec), supporting parallel management of multiple projects. Each project has its own independent environment requirement document. All agents retrieve project information through a unified document retrieval interface, ensuring consistency in the information obtained by agents within the same project.

[0193] For easier understanding, please refer to Figure 17 , Figure 17 The diagram illustrates the architecture of a reinforcement learning agent system. The reinforcement learning agent system provided in this application includes an orchestration layer, a framework layer, an algorithm layer, and a business layer. The reinforcement learning agent system adopts a three-layer decoupled architecture, achieving zero cross-dependency between the framework layer, algorithm layer, and business layer.

[0194] The framework layer is responsible for infrastructure functions such as environment scheduling, inference services, sample storage, and multi-process communication. The framework layer never references the code of the algorithm layer or business layer. It only interacts with the upper layer through three opaque buffers: the observation buffer (obs_buf), the action buffer (action_buf), and the extra buffer (extra_buf), all of which are flat 32-bit floating-point arrays. The framework layer does not interpret the semantics of their contents. The core files of the framework layer include: the asynchronous frame push module (actor_worker.py), which provides three callback injection points; the batch inference module (inference_server.py), which provides batch inference callbacks; the circular sample pool module (sample_pool.py), which adopts an expiration and maximum reuse count eviction strategy; the shared memory graphics processor inference module (shm_inference.py), which adopts dual-threading and CUDA Streaming; the shared memory cross-process pool module (shm_sample_pool.py), which maintains a shared version counter; the multi-process orchestration module (mp_actor_group.py), which is responsible for adversary pool routing; the vectorized environment module (vec_env.py), which uses a thread pool executor; the online statistics module (running_stats.py), which uses the Welford mean and standard deviation algorithm; and the algorithm callback base class module (algo_callback.py), which provides interfaces for obtaining the current version, determining segment truncation, and preprocessing segments.

[0195] The algorithm layer is responsible for core algorithmic functions such as policy learning, network structure, and generalization advantage estimation (GAE) calculation. The algorithm layer has zero game dependencies, does not reference business layer code, and injects dimensional information through constructor parameters. The core files of the algorithm layer include: the network structure definition module (networks.py), the near-end policy optimization algorithm implementation module (ppo.py), the near-end policy optimization callback module (ppo_callback.py), and the open neural network exchange format export module (onnx_utils.py). The batch update method of the near-end policy optimization agent receives opaque batch data and is unconcerned with the business semantics of the data source.

[0196] The business layer (Feature) is responsible for game-related functions such as observation serialization, dimension definition, inference callbacks, and reward shaping. The business layer is divided into a feature sublayer and a reward sublayer. The feature sublayer includes the adapter script module (custom_adapter.py), the observation builder (obs_builder.py), the environment bridging module (env_bridge.py), the dimension definition file (dims.py), the game configuration file ({game}_config.py), and the base adapter module (base_adapter.py). The reward sublayer includes the base shaper module (base_shaper.py) and the game shaper module ({game}_shaper.py). The business layer injects business logic into the framework layer and the algorithm layer through three callback interfaces: environment bridging, reward shaping, and algorithm callbacks.

[0197] The core design principles of a three-layer decoupled architecture include: (1) Opaque Buffer Principle: The framework layer only operates on three one-dimensional 32-bit floating-point arrays, does not interpret the semantics of the content, and realizes the complete generalization of the framework. The framework layer only needs three numbers: observation buffer, action buffer, and additional buffer to work.

[0198] (2) Version identifier field principle: Each sample comes with a version identifier field. The framework layer natively supports the old sample elimination mechanism based on version to ensure the freshness of training data.

[0199] (3) Callback Injection Principle: Business / algorithm logic is injected into the framework layer through three callback interfaces: environment bridging, reward shaping, and algorithm callback injection. The framework layer does not directly call the business code.

[0200] (4) Zero cross-dependency principle: The framework directory does not include the algorithm directory or feature directory.

[0201] (5) Dimension parameterization principle: The framework layer only needs to observe three values: the dimension, the action dimension, and the additional dimension. The specific feature construction and interpretation are entirely the responsibility of the business layer.

[0202] This application also provides a data processing apparatus; please refer to [link to relevant documentation]. Figure 19 The data processing device 1900 includes: Acquisition unit 1901 is used to acquire the task requirement text of the target task; The acquisition unit 1901 is also used to acquire an environment requirement document when the target task is used to indicate the modification of the reinforcement learning environment of the target service, wherein the environment requirement document is used to characterize the configuration information of the reinforcement learning environment of the target service. Processing unit 1902 is used to call the environment access and maintenance agent so that the environment access and maintenance agent can generate a change description document based on the task requirement text and the environment requirement document. The environment access and maintenance agent is an agent with environment configuration responsibility built based on a large language model. The change description document is used to describe the changes that occur after the reinforcement learning environment is modified. The processing unit 1902 is also used to call the algorithm development and optimization agent so that the algorithm development and optimization agent generates an adaptation description document based on the change description document. The algorithm development and optimization agent is an agent built based on a large language model with the responsibility of algorithm business configuration. The adaptation description document is used to describe the content generated after the reinforcement learning algorithm is modified. The processing unit 1902 is also used to invoke the data link verification agent so that the data link verification agent generates data link verification results based on the change description document and the adaptation description document. The data link verification agent is an agent with correctness verification responsibility built based on a large language model.

[0203] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to invoke the environment access and maintenance agent, so that the environment access and maintenance agent modifies the reinforcement learning environment according to the task requirement text and the environment requirement document, and generates a change description document.

[0204] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Input the task requirements text and environment requirements document into the environment access and maintenance agent; The environment access and maintenance agent is invoked so that it can modify the reinforcement learning environment according to the task requirement text and environment requirement document, and generate the modified reinforcement learning environment. Based on the differences between the reinforcement learning environment before and after the modification, a change description document is generated.

[0205] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Call the algorithm development and optimization agent so that the agent can modify the reinforcement learning algorithm for the target business based on the change description document and generate an adaptation description document.

[0206] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Input the change description document into the algorithm development and tuning agent; Call the algorithm development and optimization agent so that the algorithm development and optimization agent can modify the reinforcement learning algorithm of the target business according to the change description document and generate the modified reinforcement learning algorithm. The modified reinforcement learning algorithm is adapted to the modified reinforcement learning environment. Based on the differences between the original and modified reinforcement learning algorithms, an adaptation description document is generated.

[0207] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: The data link verification agent is invoked to verify the modified reinforcement learning environment and the modified reinforcement learning algorithm based on the change description document and the adaptation description document, and to generate data link verification results.

[0208] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Input the change description document and the adaptation description document into the data link verification agent; The data link verification agent is invoked to verify at least one link in the data transformation link based on the change description document and the adaptation description document, and to generate a data link verification result. The data transformation link is a data flow path from the environment backend encoding to the algorithm training, and at least one link includes at least one of the following: encoding / decoding field alignment link, observation data value range verification link, reward value verification link, data dimension consistency check link, sample pool data integrity check link, and training signal health check link.

[0209] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Train the modified reinforcement learning environment and generate an environment training log. The policy learning debugging agent is invoked so that it can verify the modified reinforcement learning environment based on the environment training log and generate environment verification results. The policy learning debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the environment verification result indicates that the environment-side data is abnormal, the environment access and maintenance agent is invoked so that the environment access and maintenance agent can modify the modified reinforcement learning environment according to the environment verification result.

[0210] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Train the modified reinforcement learning algorithm and generate an algorithm training log. The policy learning debugging agent is invoked so that it can verify the modified reinforcement learning algorithm based on the algorithm training log and generate the algorithm verification result. The policy learning debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the algorithm verification result indicates that the reward function or hyperparameters are abnormal, the algorithm development and tuning agent is invoked so that the algorithm development and tuning agent can modify the reinforcement learning algorithm according to the algorithm verification result.

[0211] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: The modified reinforcement learning environment and the modified reinforcement learning algorithm are jointly trained to generate link training logs, which are log data related to the data links generated during the training process. The policy learning and debugging agent is invoked so that it can verify the data link status based on the link training log and generate the link verification result. The policy learning and debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the link verification result indicates that the data link is abnormal, the data link verification agent is invoked so that the data link verification agent can verify the corresponding link of the data link based on the link verification result.

[0212] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: When the interface of the reinforcement learning framework for the target business changes, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can generate framework interface change request information based on the change description document. The framework development and maintenance agent is an agent built on a large language model with the responsibility of maintaining the framework code. The framework interface change request information is used to characterize the implementation scheme of the reinforcement learning framework interface change. In response to the confirmation operation of the framework interface change request information, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can modify the reinforcement learning framework interface according to the framework interface change request information and generate framework interface change information. The framework interface change information is used to represent the changes in the modified reinforcement learning framework interface. Invoke the environment access and maintenance agent so that the environment access and maintenance agent can modify the reinforcement learning environment according to the framework interface change information; Call the algorithm development and tuning agent so that the agent can modify the reinforcement learning algorithm according to the framework interface change information.

[0213] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Train the modified reinforcement learning framework interface and generate interface training logs; The policy learning debugging agent is invoked so that it can verify the modified reinforcement learning framework interface based on the interface training log and generate the interface verification result. The policy learning debugging agent is an agent built on a large language model with the responsibility of policy training, diagnosis and analysis. If the interface verification result indicates that the interface is abnormal, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can modify the modified reinforcement learning framework interface according to the interface verification result.

[0214] In an optional embodiment of the data processing apparatus provided in the above embodiments of this application, the processing unit 1902 is further configured to: Perform intent analysis on the task requirement text to obtain intent information; When the intent information indicates that the target task is to modify the reinforcement learning environment of the target service, determine the environment access maintenance agent, algorithm development and optimization agent, and data link verification agent used to perform the target task, as well as the calling order of the environment access maintenance agent, algorithm development and optimization agent, and data link verification agent.

[0215] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0216] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0217] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0218] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0219] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0220] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a server or terminal device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0221] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method, characterized in that, include: Obtain the task requirement text for the target task; When the target task is used to instruct the modification of the reinforcement learning environment of the target service, an environment requirement document is obtained, wherein the environment requirement document is used to characterize the configuration information of the reinforcement learning environment of the target service. The environment access and maintenance agent is invoked so that the environment access and maintenance agent generates a change description document based on the task requirement text and the environment requirement document. The environment access and maintenance agent is an agent with environment configuration responsibility built based on a large language model. The change description document is used to describe the changes that occur after the reinforcement learning environment is modified. An algorithm development and optimization agent is invoked to generate an adaptation description document based on the change description document. The algorithm development and optimization agent is an agent built on a large language model with algorithm business configuration responsibilities. The adaptation description document is used to describe the content generated after modifying the reinforcement learning algorithm. The data link verification agent is invoked so that it generates a data link verification result based on the change description document and the adaptation description document. The data link verification agent is an agent with correctness verification responsibility built based on a large language model.

2. The method as described in claim 1, characterized in that, The invocation of the environment access maintenance agent enables the environment access maintenance agent to generate a change description document based on the task requirement text and the environment requirement document, including: The environment access and maintenance agent is invoked so that it modifies the reinforcement learning environment according to the task requirement text and the environment requirement document, and generates the change description document.

3. The method as described in claim 2, characterized in that, The invocation of the environment access and maintenance agent, enabling the environment access and maintenance agent to modify the reinforcement learning environment based on the task requirement text and the environment requirement document, and generate the change description document, includes: Input the task requirement text and the environment requirement document into the environment access and maintenance agent; The environment access and maintenance agent is invoked so that the environment access and maintenance agent can modify the reinforcement learning environment according to the task requirement text and the environment requirement document, and generate the modified reinforcement learning environment. Based on the differences between the reinforcement learning environment before and after the modification, the change description document is generated.

4. The method as described in claim 1, characterized in that, The algorithm development and tuning agent is invoked to generate an adaptation description document based on the change description document, including: The algorithm development and optimization agent is invoked to modify the reinforcement learning algorithm of the target service according to the change description document, and generate the adaptation description document.

5. The method as described in claim 4, characterized in that, The step of invoking the algorithm development and tuning agent enables the agent to modify the reinforcement learning algorithm for the target service based on the change description document, generating the adaptation description document, including: Input the change description document into the algorithm development and tuning agent; An algorithm development and tuning agent is invoked to modify the reinforcement learning algorithm of the target service according to the change description document, and generate a modified reinforcement learning algorithm, wherein the modified reinforcement learning algorithm is adapted to the modified reinforcement learning environment. The adaptation description document is generated based on the differences between the original and modified reinforcement learning algorithms.

6. The method as described in claim 1, characterized in that, The invocation of the data link verification agent enables the data link verification agent to generate a data link verification result based on the change description document and the adaptation description document, including: The data link verification agent is invoked to verify the modified reinforcement learning environment and the modified reinforcement learning algorithm based on the change description document and the adaptation description document, and to generate data link verification results.

7. The method as described in claim 6, characterized in that, The invocation of the data link verification agent enables the data link verification agent to verify the modified reinforcement learning environment and the modified reinforcement learning algorithm based on the change description document and the adaptation description document, generating data link verification results, including: The change description document and the adaptation description document are input into the data link verification agent; A data link verification agent is invoked to verify at least one link in the data transformation link based on the change description document and the adaptation description document, and to generate the data link verification result. The data transformation link is a data flow path from the environment backend encoding to the algorithm training. The at least one link includes at least one of the following: encoding / decoding field alignment link, observation data value range verification link, reward value verification link, data dimension consistency check link, sample pool data integrity check link, and training signal health check link.

8. The method as described in claim 1, characterized in that, After generating the change description document, the process also includes: Train the modified reinforcement learning environment and generate an environment training log; The policy learning debugging agent is invoked so that the policy learning debugging agent can verify the modified reinforcement learning environment according to the environment training log and generate an environment verification result. The policy learning debugging agent is an agent built based on a large language model and has the responsibility of policy training diagnosis and analysis. If the environment verification result indicates that the environment-side data is abnormal, the environment access and maintenance agent is invoked so that the environment access and maintenance agent can modify the modified reinforcement learning environment according to the environment verification result.

9. The method as described in claim 1, characterized in that, After generating the adaptation description document, the process also includes: Train the modified reinforcement learning algorithm and generate an algorithm training log. The policy learning debugging agent is invoked so that the policy learning debugging agent can verify the modified reinforcement learning algorithm according to the algorithm training log and generate an algorithm verification result. The policy learning debugging agent is an agent built based on a large language model and has the responsibility of policy training diagnosis and analysis. If the algorithm verification result indicates that the reward function or hyperparameters are abnormal, the algorithm development and tuning agent is invoked so that the algorithm development and tuning agent can modify the modified reinforcement learning algorithm according to the algorithm verification result.

10. The method as described in claim 1, characterized in that, After generating the data link verification result, the process also includes: The modified reinforcement learning environment and the modified reinforcement learning algorithm are jointly trained to generate a link training log, wherein the link training log is log data related to the data link generated during the training process; The policy learning and debugging agent is invoked so that the policy learning and debugging agent can verify the data link status according to the link training log and generate a link verification result. The policy learning and debugging agent is an agent built based on a large language model with the responsibility of policy training, diagnosis and analysis. If the link verification result indicates that the data link is abnormal, the data link verification agent is invoked so that the data link verification agent can verify the corresponding link of the data link based on the link verification result.

11. The method as described in claim 1, characterized in that, After generating the change description document, the process also includes: When the reinforcement learning framework interface of the target business changes, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can generate framework interface change request information according to the change description document. The framework development and maintenance agent is an agent built based on a large language model and has the responsibility of maintaining the framework code. The framework interface change request information is used to characterize the implementation scheme of the reinforcement learning framework interface change. In response to the confirmation operation of the framework interface change request information, the framework development and maintenance agent is invoked so that the framework development and maintenance agent modifies the reinforcement learning framework interface according to the framework interface change request information and generates framework interface change information, wherein the framework interface change information is used to characterize the changes in the modified reinforcement learning framework interface. The environment access and maintenance agent is invoked so that the environment access and maintenance agent modifies the reinforcement learning environment according to the framework interface change information; The algorithm development and tuning agent is invoked so that it modifies the reinforcement learning algorithm according to the framework interface change information.

12. The method as described in claim 11, characterized in that, After generating the framework interface change information, it also includes: Train the modified reinforcement learning framework interface and generate interface training logs; The policy learning debugging agent is invoked so that the policy learning debugging agent can verify the modified reinforcement learning framework interface according to the interface training log and generate an interface verification result. The policy learning debugging agent is an agent built based on a large language model with the responsibility of policy training, diagnosis and analysis. If the interface verification result indicates that the interface is abnormal, the framework development and maintenance agent is invoked so that the framework development and maintenance agent can modify the modified reinforcement learning framework interface according to the interface verification result.

13. The method as described in claim 1, characterized in that, After obtaining the task requirement text of the target task, the method further includes: The intent information is obtained by performing intent analysis on the task requirement text. When the intent information indicates that the target task is to modify the reinforcement learning environment of the target service, the environment access maintenance agent, the algorithm development and optimization agent, and the data link verification agent used to execute the target task are determined, as well as the calling order of the environment access maintenance agent, the algorithm development and optimization agent, and the data link verification agent.

14. A data processing apparatus, characterized in that, include: The acquisition unit is used to acquire the task requirement text of the target task. The acquisition unit is further configured to acquire an environment requirement document when the target task indicates that the reinforcement learning environment of the target service should be modified, wherein the environment requirement document is used to characterize the configuration information of the reinforcement learning environment of the target service. The processing unit is used to invoke the environment access and maintenance agent, so that the environment access and maintenance agent generates a change description document based on the task requirement text and the environment requirement document. The environment access and maintenance agent is an agent with environment configuration responsibility built based on a large language model. The change description document is used to describe the changes that occur after modifying the reinforcement learning environment. The processing unit is also used to call the algorithm development and optimization agent, so that the algorithm development and optimization agent generates an adaptation description document according to the change description document, wherein the algorithm development and optimization agent is an agent with algorithm business configuration responsibility built based on a large language model, and the adaptation description document is used to describe the content generated after modifying the reinforcement learning algorithm; The processing unit is further configured to invoke the data link verification agent, so that the data link verification agent generates a data link verification result based on the change description document and the adaptation description document, wherein the data link verification agent is an agent with correctness verification responsibility built based on a large language model.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the data processing method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the data processing method according to any one of claims 1 to 13.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the data processing method according to any one of claims 1 to 13.

Images