A development system for an intelligent model based on a system-on-a-chip (SoC) with in-memory computing

Abstract

This application discloses a development system for an intelligent model based on a system-on-a-chip (SoC) with in-memory computing. When executing a model training task, this system uses various agents to determine different adjustment strategies from different dimensions. These adjustment strategies allow for a comprehensive evaluation of various states during the model training task, ultimately achieving overall optimization of the training process. Furthermore, during the execution of the model training task by multiple agents, information can be exchanged between them. This allows any agent to combine data monitored or determined by other agents to derive a more comprehensive adjustment strategy, ensuring the smooth execution of the model training task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer and artificial intelligence, and particularly relates to a development system of an intelligent model of a system-level chip based on storage-computing integration. BACKGROUND

[0002] At present, artificial intelligence models such as neural network models and Transformers have been widely applied in various fields, bringing great convenience to people's daily life and industrial production activities.

[0003] For the development of artificial intelligence models, at present, various model training strategies (such as resource allocation strategies and training task adjustment strategies) are mainly determined by artificial experience. However, because of the influence of artificial subjective experience, the various strategies determined by artificial experience are not accurate, so that when the model training task is executed according to these strategies, situations such as low model training task execution efficiency and poor model training effect may occur.

[0004] Therefore, how to solve the above problems to improve the execution of the model training task from multiple angles has become a technical problem to be solved. SUMMARY

[0005] The embodiment of the present application provides a development system of an intelligent model of a system-level chip based on storage-computing integration to partially solve the above problems existing in the prior art.

[0006] The present application adopts the following technical solutions:

[0007] The embodiment of the present application provides a development system of an intelligent model of a system-level chip based on storage-computing integration, which comprises: a plurality of node devices, at least one node device in the plurality of node devices is deployed with a configuration module and a multi-agent reinforcement learning module, at least part of the node devices in the plurality of node devices are deployed with a plurality of agents, for each node device in the at least part of the node devices, at least one agent is deployed in the node device, and different agents are used to determine adjustment strategies of different dimensions when executing a model training task for the to-be-trained model.

[0008] The configuration module is configured to receive a task configuration request of a user, determine a model type corresponding to a to-be-trained model according to the task configuration request, determine a model corresponding to the model type from a preset model library as the to-be-trained model, and generate a model training task for the to-be-trained model according to data set information carried in the task configuration request.

[0009] The multi-agent reinforcement learning module is used to determine adjustment strategies for different dimensions when performing the model training task through the multiple agents, so as to adjust and execute the model training task according to the adjustment strategies to obtain the trained model. For each agent, the agent collects key dynamic data of the corresponding dimension of the agent when performing the model training task, and receives associated data transmitted by the associated agent corresponding to the agent, so as to output an adjustment strategy based on the key dynamic data and the associated data. The associated data is the data associated with the corresponding dimension of the agent required by the agent when outputting the adjustment strategy.

[0010] Optionally, the development system further includes: a simulation module;

[0011] The simulation module is used to configure a simulation environment based on the acquired simulation parameters, and to execute a simulation task for the trained model according to the simulation environment, so as to obtain and output the simulation results.

[0012] Optionally, the development system further includes:

[0013] The user permission management module is used to determine the user's operation permissions upon receiving a user's request for the model training task, and to open an operation entry point matching the operation permissions to the user, so that the user can perform the operation specified by the operation entry point.

[0014] Optionally, the multi-agent reinforcement learning module is specifically used to: determine the global reward value obtained after training the model according to the candidate adjustment strategy determined by each agent and the preset global reward function; and for each agent, determine the reward function value in the corresponding dimension of the agent according to the local reward function of the agent, as the local reward value of the agent; and obtain a comprehensive reward value based on the local reward value of each agent and the global reward value, with maximizing the comprehensive reward value as the optimization objective, and control each agent to adjust its own determined candidate adjustment strategy to obtain the adjustment strategy output by each agent.

[0015] Optionally, the plurality of intelligent agents include a task scheduling intelligent agent, a resource scheduling intelligent agent, and a model training intelligent agent, and the development system executes multiple model training tasks in parallel;

[0016] The task scheduling agent is used to receive resource status data monitored by the resource scheduling agent and training execution status information monitored by the model training agent. Based on the resource status data and the training execution status information, it determines the execution priority of each model training task. The resource status data includes the resource utilization rate of the development system when executing each model training task, the resource allocation strategy of the development system for executing each model training task, and the resource idle information of the development system when executing each model training task. The resource idle information is used to indicate the idle status of various resources of the development system when executing each model training task. The training execution status information includes the training progress of each model training task, the resource usage information of each model training task, and the expected accuracy index of each model training task.

[0017] The resource scheduling agent is used to receive the execution priority of each model training task determined by the task scheduling agent, and to receive the training execution status information of the model training agent, and to determine the resource allocation strategy of the development system when executing each model training task based on the execution priority and the training execution status information.

[0018] The model training agent is used to receive the resource allocation strategy determined by the resource scheduling agent when the development system executes each model training task, and to receive the training data used by the development system when executing each model training task. Based on the training data and the resource allocation strategy, the agent determines the training execution status information of the development system when executing each model training task.

[0019] Optionally, the plurality of intelligent agents includes a data management intelligent agent, a task scheduling intelligent agent, and a model training intelligent agent, and the development system executes multiple model training tasks in parallel;

[0020] The data management agent is used to receive the training execution status information of the development system when executing each model training task, which is monitored by the model training agent, and to obtain the data requirement information of the development system when executing each model training task. Based on the training execution status information and the data requirement information, the agent determines the training batch strategy corresponding to each training data when the development system executes each model training task. The data requirement information is used to indicate the characteristics of the training data required when executing each model training task. The training execution status information includes the training progress of each model training task, the resource usage information of each model training task, and the expected accuracy index of each model training task.

[0021] The task scheduling agent is used to receive training execution status information monitored by the model training agent and data usage information sent by the data management agent. Based on the training execution status information and the data usage information, the agent determines the execution priority of the development system when executing each model training task. The data usage information includes the usage status information of each training data when the development system executes each model training task, the data throughput information of the development system when executing each model training task, and the training batch strategy corresponding to each training data when the development system executes each model training task.

[0022] The model training agent is used to receive the training batch strategy corresponding to each training data when the development system executes each model training task, as determined by the data management agent, and to determine the training execution status information of the development system when executing each model training task according to the training batch strategy.

[0023] Optionally, the plurality of intelligent agents includes a development role intelligent agent and a task scheduling intelligent agent, and the development system executes multiple model training tasks in parallel;

[0024] The development role agent is used to receive task status information of the development system when executing each model training task, which is monitored by the task scheduling agent, and to receive operation instructions sent by each user for each model training task. Based on the task status information and the operation instructions, the agent determines the permission control strategy for each model training task. The permission control strategy is used to adjust the operation permissions of each user for each model training task.

[0025] This application provides a model training execution method. The method is applied to a development system for intelligent models based on a system-on-a-chip (SoC) with in-memory computing. The development system includes: multiple node devices, at least some of which are equipped with multiple intelligent agents. For each of the at least some node devices, at least one intelligent agent is deployed. Different intelligent agents are used to determine adjustment strategies of different dimensions when performing model training tasks for the model to be trained.

[0026] Receive user task configuration requests;

[0027] Based on the task configuration request, determine the model type corresponding to the model to be trained;

[0028] The model corresponding to the model type is determined from the preset model library as the model to be trained, and a model training task is generated for the model to be trained based on the dataset information carried in the task configuration request.

[0029] Through the multiple agents, adjustment strategies for different dimensions are determined when executing the model training task. The model training task is adjusted and executed according to the adjustment strategies to obtain the trained model. For each agent, the agent collects key dynamic data of the corresponding dimension when executing the model training task, and receives associated data transmitted by the associated agent. Based on the key dynamic data and the associated data, the agent outputs an adjustment strategy. The associated data is the data associated with the corresponding dimension of the agent required by the agent when outputting the adjustment strategy.

[0030] Optionally, the method further includes:

[0031] Configure the simulation environment based on the obtained simulation parameters;

[0032] Based on the simulation environment, a simulation task is performed on the trained model to obtain and output the simulation results.

[0033] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described model training task execution method.

[0034] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0035] This application provides a development system and model training task execution method for an intelligent model based on a system-on-a-chip (SoC) with in-memory computing. The development system includes multiple node devices. At least one of these node devices is equipped with a configuration module and a multi-agent reinforcement learning module. At least some of the node devices are equipped with multiple agents, and each of these node devices has at least one agent. Different agents determine adjustment strategies for different dimensions when executing a model training task for the model to be trained. The development system receives a user's task configuration request through the configuration module and determines the model type corresponding to the model to be trained based on the request. It then selects the model corresponding to that model type from a preset model library as the model to be trained and generates a model training task for the model to be trained based on the dataset information carried in the task configuration request. Based on the multi-agent reinforcement learning module, the development system uses multiple agents to determine adjustment strategies for different dimensions when executing the model training task. The system adjusts and executes the model training task according to these different adjustment strategies to obtain the trained model.

[0036] As can be seen from the above development system and model training execution method, when the development system executes the model training task, it determines various adjustment strategies from different dimensions through each agent. Through these adjustment strategies, various states during the model training task can be comprehensively evaluated from a holistic perspective, thereby ultimately achieving comprehensive optimization of the model training task. Furthermore, during the execution of the model training task by multiple agents, each agent can transmit information, allowing any agent to combine the data monitored or determined by other agents to derive a more comprehensive adjustment strategy, ensuring the smooth execution of the model training task. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this application, illustrate exemplary embodiments of the present application and, together with their descriptions, serve to explain this specification and do not constitute an undue limitation of the present application. In the drawings:

[0038] Figure 1 A schematic diagram illustrating the deployment of each intelligent agent in each node device of the development system provided in the embodiments of this application;

[0039] Figure 2 This is a schematic diagram illustrating the steps of a model task execution method provided in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram of a model training task execution device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0042] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0043] To address the aforementioned issues, this application provides a development system for an intelligent model based on a system-on-a-chip (SoC) with in-memory computing capabilities. This system deploys a SoC, where a SoC integrates all or most computer hardware components (such as CPU, GPU, memory, I / O interfaces, etc.) onto a single chip. Through highly integrated design, it achieves efficient data processing and low power consumption. Furthermore, the development system further enhances the chip's energy efficiency and computing power by integrating an in-memory computing architecture into the SoC.

[0044] Furthermore, the development system provided in this application embodiment is a distributed architecture, in which multiple node devices are deployed. For any given node device, it can refer to a high-performance computer device, a server, or a dedicated device for executing model training tasks. These node devices can execute multiple training steps in a model training task in parallel, or execute multiple model training tasks simultaneously, or execute each model training task sequentially according to a certain execution priority order.

[0045] For the implementation method of deploying each intelligent agent through each node device in the development system, please refer to Figure 1 The deployment method shown.

[0046] Figure 1 This is a schematic diagram illustrating the deployment method of each intelligent agent in each node device of the development system provided in the embodiments of this application.

[0047] from Figure 1 As can be seen, for a development system containing multiple node devices, at least some of these node devices can deploy multiple intelligent agents. For a node device with deployed intelligent agents, one intelligent agent can be deployed; conversely, multiple intelligent agents can be deployed on a single node device. Therefore, whether to deploy the intelligent agents on one node device or multiple node devices depends on the actual needs and the hardware performance of the node devices.

[0048] Furthermore, multiple agents deployed in the development system can each determine different strategies during the model training task execution process from different dimensions. These different dimensions can be understood as determining adjustment strategies from different perspectives, and can include factors such as resource allocation, task execution priority, use of training data, and data status during the model training task execution process.

[0049] Therefore, in this embodiment, any agent deployed in the development system actually determines the corresponding adjustment strategy in its corresponding dimension. All agents deployed in the entire development system determine various adjustment strategies from a comprehensive perspective, enabling the development system to ultimately adopt the optimal comprehensive strategy to execute the model training task. This significantly improves the execution efficiency of the model training task while fully utilizing system resources.

[0050] In addition, the aforementioned development system also includes a configuration module and a multi-agent reinforcement learning module. The multi-agent reinforcement learning module is mainly used to perform model training tasks through multiple agents. Overall, it uses deep reinforcement learning algorithms such as Deep Q-Network (DQN), policy gradient methods, and hybrid methods to continuously optimize various adjustment strategies.

[0051] The configuration module is mainly used to create the model to be trained and determine the training data used for the model training task based on the user's task configuration request.

[0052] The development system can receive task configuration requests from users through the configuration module. These requests can be generated by users filling in information according to their needs through the interface displayed in the development system.

[0053] Specifically, the user's terminal device (such as a desktop computer or laptop) can be connected to the development system, allowing the connected terminal device to display a configuration page to the user. This configuration page includes various functional controls, enabling users to perform different configuration operations.

[0054] For example, the configuration page can include functional controls for model creation. Users can interact with these controls to build models that need to be trained later, based on their actual needs, using the model library configured in the development system.

[0055] For example, the configuration page can have a function control for configuring the dataset. Users can use this function control to configure the training dataset they want to use, or select the required training dataset from the training datasets pre-saved in the development system to perform subsequent model training tasks.

[0056] For example, after model training is complete, simulation operations can be performed on the trained model to further determine its effectiveness. Therefore, the configuration page mentioned above can also include functional controls for executing simulation tasks. By touching these controls, the user can cause the development system to generate simulation tasks for the trained model and simultaneously create a corresponding simulation environment based on the user-configured simulation environment parameters. This allows the development system to execute the simulation tasks of the trained model within the created simulation environment.

[0057] The configuration module mentioned above can be further divided into its specific functions, and each sub-function can be implemented by a corresponding sub-module.

[0058] Specifically, the aforementioned configuration module may include an algorithm library and template management submodule. The development system can use this submodule to provide users with a pre-defined model library containing various types of basic model templates. These different types of basic model templates differ in their specific functions and architectures. For example, the model library may include basic neural network model templates, deep learning network model templates, Transformer model templates, and convolutional neural network model templates. It can be seen that these basic model templates differ in their specific architectures and actual functions.

[0059] Therefore, in this embodiment, after receiving the user's task configuration request through the aforementioned configuration module, the system can parse the request to determine the model type selected by the user on the configuration page. Then, the development system can use the aforementioned algorithm library and template management submodule to call a preset model library to determine the model corresponding to the determined model type from the library, which will then be used as the model to be trained.

[0060] In the process of determining the model to be trained based on the model library, the development system can first determine the basic model template from the model library according to the above model types, and then initialize the basic model to obtain the model to be trained.

[0061] In practical applications, the task configuration request mentioned above can also carry information such as the number of network layers and parameter precision of the model required by the user. Therefore, the algorithm library and template management submodule can initialize the basic model template determined from the model library based on this information. For example, the number of network layers in the basic model template can be adjusted to the number of network layers required by the user, and the parameter precision of the network parameters in the basic model template can be adjusted to the parameter precision required by the user, thereby obtaining the model to be trained.

[0062] Furthermore, the aforementioned algorithm library and template management submodule can also configure the training dataset required for performing model training tasks, specifically based on the configuration operations performed by the user. In addition to determining the required training dataset, this submodule can also determine the batch of training data to be used when performing model training tasks. For different batches, the amount of training data used in each training step during the model training task may also vary.

[0063] In addition, the aforementioned algorithm library and template management submodule also provide functions such as model deletion and version management. Specifically, when a user performs a model deletion operation based on their actual needs, the development system can use the algorithm library and template management submodule to determine the model corresponding to the user's deletion operation as the model to be deleted, and then perform the deletion operation on the model to be deleted.

[0064] Version management allows the development system to save models at different stages of the training process, with each stage corresponding to a different version. Therefore, the development system can manage different model versions through the algorithm library and template management sub-model, enabling users to selectively save or delete model versions based on their needs.

[0065] Furthermore, users can also input the model name of the model to be trained in the development system, so that the development system can establish a correspondence between the model name and the model to be trained through the algorithm library and template management submodule, so that users can easily query the model to be trained based on the model name.

[0066] In addition, in this embodiment of the application, the development system can automatically import the training dataset through the above-mentioned algorithm library and template management submodule. That is, in the subsequent execution of model training tasks, the development system can automatically import the user-configured training dataset into the node device required to execute the model training task through the template management submodule, so as to further improve the efficiency of model training execution.

[0067] Moreover, in practical applications, the development system can use this template management submodule to perform preprocessing operations on the training dataset. These preprocessing operations include removing redundant data and converting the training data format to meet the input requirements of the model to be trained.

[0068] For the aforementioned training datasets, the development system can also manage them through a template management submodule. This includes importing and deleting training datasets, as well as managing various types of training datasets. This allows users to select the required training datasets based on their actual needs and execute model training tasks through the development system. The development system can also update various training datasets through the template management submodule. For example, by using a preset listener, it can monitor whether the source of the training dataset (such as a third-party dataset, third-party platform, website, etc.) has updated the training dataset. When an update is detected, the updated training dataset is retrieved from its source, and the locally stored training dataset is updated accordingly.

[0069] The configuration module mentioned above also includes an editing submodule, which is mainly used to compile the created model to be trained into executable machine code, and to convert the execution code corresponding to the generated model training task into the corresponding machine code, so as to execute the model training task for the model to be trained through the node device in the development system.

[0070] The aforementioned configuration module also includes a file storage submodule. The development system uses a shared network file system as its file storage solution. This submodule centrally manages the code for all models, the training datasets used to train the models, and intermediate files generated during the model training process. These intermediate files may include transitional models (i.e., the different versions of the models mentioned above) and gradient data generated during the model training process.

[0071] In actual execution, the development system can manage different types of data using different containers. For example, a first container can manage the code for all models, while a second container can manage the training dataset. Therefore, before starting these containers, the development system can use the file synchronization tool in the file storage submodule to synchronize the files needed for model training tasks, either locally or in a preset storage location, to the network file system, and then manage these different files through these containers. In this way, the development system can start these containers and use these files during the model training process.

[0072] When the model training task is completed or terminated according to the user's actual needs, the development system can destroy these containers. At the same time as destroying these containers, the development system synchronizes the various files previously stored in the network file system to the distributed file system or other storage locations through the file storage submodule to ensure the security and traceability of the data.

[0073] The configuration module mentioned above also includes a visualization and monitoring submodule. This visualization and monitoring submodule is used to monitor various data generated during the execution of the model training task, such as the power consumption, network latency, data throughput, resource utilization and other performance indicators generated by the model training task. It also displays the monitored data to the user in the form of charts through visualization, so that the user can clearly understand the actual situation during the entire model training task execution process.

[0074] Meanwhile, the visualization and monitoring submodule can also generate various log data generated during the execution of the model training task, and save these log data through the file storage submodule.

[0075] In addition, in this embodiment of the application, the development system also has an entry point for retrieving log data. Users can perform log retrieval operations on the configuration page or a page specifically for retrieving logs. In response to the log retrieval operation, the development system can query the log data required by the user from the saved log data and return the queried log data to the user for viewing.

[0076] In this embodiment of the application, the development system is also equipped with a simulation module. Through this simulation module, the development system can perform simulation tasks on the trained model, so as to simulate the performance and usage effect of the trained model through the obtained simulation results.

[0077] Specifically, the development system can obtain simulation parameters through the simulation module. These simulation parameters reflect the requirements of the simulation environment when executing simulation tasks. Simulation parameters may include parameters such as memory size, register size, simulation step size (reflecting the time interval between each execution step during the simulation task), and algorithm information of the simulation algorithm used.

[0078] After obtaining the above simulation parameters, the development system can configure the simulation environment through the simulation module, and then execute simulation tasks for the trained model according to the configured simulation environment to obtain and output the simulation results.

[0079] In practical applications, multiple users may share the responsibility for a single model training task. However, different users may have varying levels of access permissions for the same task. For example, some users may be responsible for data configuration, such as creating the model to be trained and configuring the training dataset used to execute the training task; others may be responsible for monitoring the various metrics generated during the model training process and may have the authority to stop or adjust the training task based on the monitored data.

[0080] Based on this, the aforementioned development system also includes a user permission management module. When a user performs an operation on a model training task, the development system can use this user permission management module to determine the user's operation permissions and, based on the determined operation permissions, open operation entry points that match those permissions to the user.

[0081] As mentioned above, when multiple users are jointly responsible for the same model training task, different users have different levels of operation permissions. In the embodiments of this application, the higher the level of operation permission, the more types of operations can be performed; conversely, the lower the level of operation permission, the fewer types of operations can be performed.

[0082] The specific operational permissions corresponding to different users can be determined by the user type. For example, when a user is an administrator, their operational permission level is the highest, and they can perform all operations related to model training tasks. On the other hand, when a user is a tester, their operational permissions only allow them to test and simulate the trained model, and they are not allowed to perform related operations during the model training task execution process.

[0083] In this embodiment of the application, different user types can be pre-entered into the user permission management module of the development system. Therefore, when a user logs into the development system, the development system can determine the user's corresponding type through the user permission management module, and then determine the user's operation permissions based on the user's corresponding type.

[0084] Furthermore, for at least two users with the same or partially overlapping operation permissions, when these users simultaneously execute operations through the same entry point, the development system needs to restrict the operations performed by these users through the user permission management module. For example, the user who executes the operation first can be allowed to continue, while subsequent users executing the same operation can have their corresponding operation instructions intercepted, or the operation instructions temporarily stored in a preset queue, and the subsequent user's operation executed only after the first user has completed their operation.

[0085] Of course, if a single operation entry point allows users to perform multiple different specific operations, then when at least two users simultaneously perform operations through the same entry point, a collaborative operation link can be created for these users. This link allows them to access a collaborative operation page, where they can also perform operations on the entry point simultaneously. However, when at least two users execute the same specific operation through this collaborative operation page, the aforementioned mechanism needs to intercept or temporarily store the operation command for the later execution in a preset queue. Alternatively, when at least two users simultaneously execute the same specific operation, a chat interface displayed on the collaborative operation page should prompt the users that there is an identical specific operation targeting the same object, allowing them to proactively cancel their identical operations through negotiation in the chat interface. However, if at least two users perform different specific operations targeting different objects or different specific operations based on this collaborative operation page, it will not affect the specific operations being performed by each other, and these operations can be executed simultaneously and in parallel.

[0086] It should be noted that after creating the above-mentioned collaborative operation link, the development system actually creates a command masking space for multiple users who perform operations based on the collaborative operation link. That is, the collaborative operation link is equivalent to the entrance to this space. When multiple users access this command masking space through this collaborative operation link, the development system only allows users who have accessed the command masking space to perform operations, while operations performed by users who have accessed the command masking space will be masked or intercepted.

[0087] In addition, when multiple users perform operations on the same model training task, the development system needs to ensure that multiple users are performing operations on the same version of the model. This ensures consistency in the development of the model to be trained.

[0088] When multiple users are collaboratively developing the same model, the operation instructions sent by multiple users to the development system are first synchronized to the node devices in the development system, and then synchronized to the terminal devices used by at least some of the users participating in the collaborative development, thereby achieving synchronized operation effects on the same model on the terminal devices used by each user.

[0089] The above content provides a detailed introduction to the configuration modules included in the development system. The following section will focus on the multi-agent reinforcement learning module deployed in the development system.

[0090] The multi-agent reinforcement learning module mentioned above is mainly used to coordinate various agents to determine adjustment strategies from multiple different dimensions, so as to achieve overall optimization of the model training task from a global perspective.

[0091] For any agent deployed on one or more node devices in the development system, when determining the adjustment strategy for its own corresponding dimension, the agent does not only refer to the data collected by itself, but also refers to the data obtained from other agents, so as to achieve accurate determination of the adjustment strategy.

[0092] Specifically, for each agent, it can collect key dynamic data of its corresponding dimension during model training tasks, as well as receive associated data transmitted by its associated agents. Based on the key dynamic data and the acquired associated data, it can output an adjustment strategy. The associated data mentioned here refers to the data related to the agent's corresponding dimension that the agent needs to obtain from its associated agents when outputting the adjustment strategy.

[0093] The key dynamic data collected by different agents during model training tasks varies. For example, for a resource scheduling agent, the key dynamic data collected during model training tasks includes data reflecting resource usage such as computing resources (e.g., GPU), memory, and network bandwidth. Furthermore, when a development system simultaneously performs multiple model training tasks for different models, the key dynamic data collected by the task scheduling agent during model training tasks may include data reflecting the execution priority of each model training task.

[0094] Different intelligent agents have different associated intelligent agents, and the required data obtained from the associated intelligent agents also differs. The following will explain the different intelligent agents in turn.

[0095] In this embodiment of the application, the intelligent agents deployed in the development system may include: a task scheduling intelligent agent, a resource scheduling intelligent agent, a model training intelligent agent, a data management intelligent agent, and a development role intelligent agent.

[0096] For a task-oriented agent, its associated agents include a resource scheduling agent and a model training agent. In practical applications, the task scheduling agent is mainly used to determine the execution priority of each model training task when the development system is executing multiple different model training tasks simultaneously, based on data collected by itself and data obtained from associated agents. This execution priority determines which model training tasks the development system executes first, and / or which model training tasks are allocated more resources.

[0097] In this embodiment, the task scheduling agent can receive resource status data monitored by the resource scheduling agent and training execution status information monitored by the model training agent. Then, based on the resource status data and training execution status information, the execution priority of each model training task is determined. The resource status data mentioned here includes the resource utilization rate of the development system when executing each model training task, the resource allocation strategy of the development system for each model training task, and resource idle information of the development system when executing each model training task. Resource idle information indicates the idle status of various resources of the development system when executing each model training task, such as how much unused GPU and memory space remain.

[0098] The aforementioned training execution status information may include the training progress of each model training task, the resource usage information of each model training task, and the expected accuracy metric for each model training task. The resource usage information reflects the system's use of resources such as memory and GPU when executing each model training task.

[0099] For the resource scheduling agent, it can receive the execution priorities of each model training task determined by the task scheduling agent, as well as the training execution status information from the model training agent. Based on these priorities and status information, the resource allocation strategy of the development system when executing each model training task is determined. This resource allocation strategy reflects the amount of resources allocated to each model training task and the priority of allocating idle resources to each model training task when available.

[0100] For the model training agent, it can receive the resource allocation strategy determined by the resource scheduling agent when the development system executes each model training task, and receive the training data used by the development system when executing each model training task. Then, based on the training data and the resource allocation strategy, it can determine the training execution status information of the development system when executing each model training task.

[0101] As can be seen from the above, the task scheduling agent, resource scheduling agent, and model training agent actually integrate the data they monitor and the data provided by other related agents. Based on the overall resource usage of each model training task, the resource usage of individual model training tasks, and the required performance indicators, they plan the execution priority of each model training task. This allows for resource allocation tilting based on the fixed resources of the development system and the execution priority, thereby achieving overall optimized resource allocation for each model training task.

[0102] The above are just examples of three types of agents working together to perform model training tasks. As mentioned above, in this application embodiment, the development system can also deploy other agents in the node device. Therefore, the specific forms of agents working together to perform model training tasks are not limited to the above situations.

[0103] For example, the development system also deploys a data management agent that can work collaboratively with the task scheduling agent and the overall model training process.

[0104] Specifically, for the data management agent, the data management agent can receive the training execution status information of the development system when executing each model training task, which is monitored by the model training agent, and obtain the data requirement information of the development system when executing each model training task. Based on the training execution status information and the data requirement information, the data management agent can determine the training batch strategy corresponding to each training data when the development system executes each model training task.

[0105] The data requirement information indicates the characteristics of the training data needed to perform each model training task, such as the quantity and format of the training data. The training batch strategy mentioned above reflects how to plan the batches of training data for each model training task in the current state of the development system's execution of these tasks, and the quantity and size of each batch of training data for any given model training task.

[0106] For the task scheduling agent that collaborates with the data management agent, the task scheduling agent can receive training execution status information monitored by the model training agent, as well as data usage information sent by the data management agent, and determine the execution priority of the development system when executing each model training task based on the training execution status information and data usage information.

[0107] The data usage information includes the usage status information of each training data when the development system executes each model training task (such as the usage status of a batch of training data), the data throughput information of the development system when executing each model training task (such as data throughput), and the training batch strategy corresponding to each training data when the development system executes each model training task.

[0108] For the model training agent that collaborates with the data management agent, the model training agent can receive the training batch strategy corresponding to each training data when the development system executes each model training task, as determined by the data management agent, so as to determine the training execution status information of the development system when executing each model training task according to the training batch strategy.

[0109] For example, the aforementioned development role agent and task scheduling agent can work together to perform model training tasks.

[0110] For the development agent, when the development system executes multiple model training tasks in parallel, it can receive task status information monitored by the task scheduling agent as the development system executes each model training task, as well as operation instructions sent by each user for each model training task. Based on the task status information and the operation instructions, it determines the access control strategy for each model training task. This access control strategy is used to adjust the operation permissions of each user for each model training task. This adjustment of operation permissions includes not only adjusting the user's operation permissions themselves (such as upgrading or downgrading), but also adjusting the execution order of user operation instructions to avoid code conflicts.

[0111] This shows that during the development system's execution of various model training tasks, the user's operation permissions are not fixed. Instead, they can be dynamically adjusted by the aforementioned development role, combined with the current task status of each model training task. The order in which user operation commands are executed can also be adjusted, thereby further improving the overall execution efficiency and effectiveness of each model training task based on the adjusted operation permissions.

[0112] The above only lists some examples of model training tasks performed collaboratively by various agents. In practical applications, all agents can participate in collaborative model training tasks. For some agents, such as resource scheduling agents and task scheduling agents, their associated agents are some of the remaining agents. For the aforementioned developer agent, all agents other than the developer agent itself can be its associated agents. Therefore, the developer agent can receive data from all other agents (whether monitored data or decision data determined by itself) to determine the aforementioned access control strategy.

[0113] Furthermore, in the embodiments of this application, the data output by each agent can be regarded as an adjustment strategy. For example, the execution priority output by the task scheduling agent, the resource allocation strategy output by the resource scheduling agent, the training batch strategy output by the data management agent, the permission control strategy output by the development role agent, and the expected accuracy index output by the model training agent can all be regarded as adjustment strategies.

[0114] Based on this, the development system can use the aforementioned multi-agent reinforcement learning module to determine the global reward value obtained after performing model training tasks according to the candidate adjustment strategies determined by each agent and the preset global reward function. Furthermore, for each agent, the system determines the reward function value in the corresponding dimension based on the agent's local reward function, which serves as the agent's local reward value.

[0115] Subsequently, the development system uses the aforementioned multi-agent reinforcement learning module to obtain a comprehensive reward value based on the local and global reward values ​​of each agent. With the goal of maximizing the comprehensive reward value, it controls each agent to adjust its determined candidate adjustment strategies to obtain the final adjustment strategy output by each agent.

[0116] Before each agent outputs its final adjustment strategy, the determined strategies can be called candidate adjustment strategies. Candidate adjustment strategies can be understood as imperfect strategies that need to be continuously adjusted from a global perspective to obtain the strategy used for training the entire global model.

[0117] Therefore, for each of the aforementioned agents, before determining the final adjustment strategy, their outputs such as resource allocation strategies, execution priorities, expected accuracy metrics, and permission adjustment strategies can all be considered as transitional candidate adjustment strategies. Through the aforementioned global and local reward values, various candidate adjustment strategies are continuously optimized to obtain the optimal adjustment strategy.

[0118] For a multi-agent reinforcement learning module, it can be understood that after executing the model training task according to the candidate adjustment strategies determined by each agent, various state data are obtained. These state data are used to determine the reward values ​​obtained after adopting these candidate adjustment strategies. Feedback information is then sent to each agent based on the received feedback, enabling each agent to adjust or re-determine the candidate adjustment strategies. Through multiple iterations, the final adjustment strategy output by each agent is determined.

[0119] The various state data mentioned here may include at least some of the data transmitted by the aforementioned agents, such as resource availability information, training progress, usage status information of each training data, data throughput information, etc.

[0120] In the embodiments of this application, the local reward functions used by different agents will be listed and described one by one below.

[0121] For a resource scheduling agent, its local reward function can be specifically referred to as the following formula:

[0122] RA = α1·RU-β1·OR-γ1·AR

[0123] In the above formula, RU represents resource utilization, OR represents resource overload rate (reflecting whether over-provisioning or resource contention occurs during the execution of various model training tasks), AR represents resource idle information (mainly reflecting the amount of allocated but unused resources), and α1, β1, and γ1 are the corresponding weight parameters. RA is the local reward value determined by the resource scheduling agent based on the above local reward function.

[0124] Through this local reward function, the resource scheduling agent can determine adjustment strategies that can efficiently use various resources and avoid resource waste during model training tasks, based on the dimension of resource usage.

[0125] For the task scheduling agent, its corresponding local reward function can be specifically referred to as the following formula:

[0126] TA = α²·TP - β²·WT - γ²·STV

[0127] In the above formula, TP is used to represent the task priority satisfaction rate (to reflect the proportion of high-priority model training tasks that are scheduled first), WT is used to represent the average waiting time of the task, STV is used to represent the task start delay volatility, α2, β2 and γ2 are the corresponding weight parameters, and TA is the local reward value determined by the task scheduling agent based on the above local reward function.

[0128] Through this local reward function, the task scheduling agent can reduce the waiting time of each model training task and rationally prioritize each model training task to improve the overall efficiency of the development system when executing each model training task.

[0129] For a data management agent, its local reward function can be specifically referred to as the following formula:

[0130] DA = α3·DT - β3·LD - γ3·E

[0131] In the above formula, DT is used to represent data throughput information, LD is used to represent the data transmission latency of calling various data when performing model training tasks, E is used to represent the error rate (such as packet loss rate, number of preprocessing errors, etc.), α3, β3 and γ3 are the corresponding weight parameters, and DA is used to represent the local reward value determined by the data management agent based on the above local reward function.

[0132] Through this local reward function, the development system, based on this data management agent, can achieve efficient and secure data transmission and data processing during the model training process.

[0133] For model training agents, the corresponding local reward function can be referred to as the following formula:

[0134] MA = α4·Acc - β4·TL - γ4·C

[0135] In the above formula, Acc represents the expected accuracy index, TL represents the training time corresponding to the model training task, C represents the resource usage information, α4, β4 and γ4 are the corresponding weight parameters, and MA represents the local reward value determined by the data training agent based on the above local reward function.

[0136] By using this local reward function, the development system can improve the training efficiency and accuracy of the model to be trained by training agents based on this model, and reduce the waste of resources when performing model training tasks.

[0137] For developing a role-based intelligent agent, its corresponding local reward function can be referenced in the following formula:

[0138] DevA = α5·VS - β5·CD - γ5·LT

[0139] In the above formula, VS represents the version synchronization success rate, CD represents the code conflict frequency after multiple users perform operations on the same model training task, LT represents the development latency (used to represent the waiting time for compiling or running the model), α5, β5 and γ5 are the corresponding weight parameters, and DevA represents the local reward value determined by the development agent based on the above local reward function.

[0140] Through this local reward function, the development system, based on the development role intelligent agent, can ensure version consistency when multiple users perform operations on the same model training task, reducing the occurrence of conflicts and further improving the development and training efficiency of the model to be trained.

[0141] The above are examples of various local reward functions. In practical applications, the local reward functions used by each agent are not limited to these. As mentioned above, each agent will obtain the required data from other associated agents. Therefore, the local reward function of each agent can actually include the data items corresponding to the data obtained from the associated agents.

[0142] For example, in the local reward function corresponding to the aforementioned development role agent, LT is a key dynamic data sent by the task scheduling agent (i.e., the associated agent corresponding to the development role agent).

[0143] Furthermore, in the various local reward functions listed above as examples, the specific values ​​corresponding to some data items can be determined based on key dynamic data sent by the associated agents. For example, the development delay LT in the aforementioned development agent can be determined by the task status information (such as the start time of the monitored model training task and the start time of the model to be trained) transmitted from the associated agent: the task scheduling agent. Other examples will not be elaborated here.

[0144] The above are examples of the local reward functions for each agent. However, during the model training process of the development system, it is also necessary to provide feedback from a global perspective to guide each agent to provide the optimal adjustment strategy. The global reward function used will be illustrated below; please refer to the following formula for details:

[0145] global = λ1·TCR - λ2·AR - λ3·AL - λ4·ER

[0146] In the above formula, TCR represents the task completion rate (reflecting the proportion of all model training tasks completed within the predetermined time), AR represents the average resource idle rate (reflecting the proportion of resources occupied but not used during the execution of each model training task to the total resources occupied from an average perspective), AL represents the average response latency (reflecting the speed of the development system's response when executing each model training task), ER represents the system error rate (reflecting the number of errors that occur in the development system when executing each model training task), λ1, λ2, λ3, and λ4 are the corresponding weight parameters, and global represents the global reward value determined by the global reward function.

[0147] As can be seen from the above formula, the global reward function comprehensively evaluates the optimization degree of the candidate adjustment strategies output by each agent from four aspects: resource utilization, latency of executing model training tasks, completion status of the development system for executing all model training tasks within the specified time, and error occurrence during the execution of all model training tasks. In other words, it aims to achieve overall optimal performance by ensuring reasonable resource allocation and utilization, low latency during task execution, maximizing task completion within the specified time, and minimizing errors during task execution.

[0148] As mentioned above, the global reward function described above is an illustrative example. In practical applications, other forms of global reward functions can also be used. For example, based on the global reward function described above, other indicator data items that can measure the performance of the model training task from a global perspective can be further introduced.

[0149] After determining the global reward value and each local reward value through the global reward function and the local reward function corresponding to each agent, the comprehensive reward value can be further determined through the global reward value and the comprehensive reward value can be maximized as the optimization objective. Each agent is then controlled to adjust its determined candidate adjustment strategy, thereby obtaining the final adjustment strategy.

[0150] The above process is actually the process of determining the optimal adjustment strategy from both global and local perspectives. By continuously adjusting the candidate adjustment strategies, the global reward value and the local reward values ​​can be optimized overall. Once the optimization is achieved, the final adjustment strategy will be determined accordingly.

[0151] During the initial model training process, each agent first determines its initial adjustment strategy (which can be randomly determined or a pre-defined strategy). The training is then executed according to these initial strategies. During training, each agent continuously monitors and transmits its data to its associated agents, generating candidate adjustment strategies. The training continues according to these candidate strategies, with agents again monitoring and transmitting their data to their associated agents, thus creating an iterative process that ultimately yields the optimal adjustment strategies.

[0152] As can be seen from the above method, when the development system executes the model training task, it determines various adjustment strategies from different dimensions through each agent. Through these adjustment strategies, various states during the model training task can be comprehensively evaluated from a holistic perspective, thereby ultimately achieving comprehensive optimization of the model training task. Furthermore, during the execution of the model training task through multiple agents, each agent can transmit information, allowing any agent to combine the data monitored or determined by other agents to derive a more comprehensive adjustment strategy, ensuring the smooth execution of the model training task.

[0153] In addition, in practical applications, each model training task is executed by each node device. The number of model training tasks or the partial task steps of a model training task that each node device is responsible for may be different, which may result in different loads on each node device when executing each model training task.

[0154] In practical applications, the aforementioned agents need to be deployed on various node devices or some node devices. This may cause the load on one or more node devices to increase further during the execution of various model training tasks, thereby affecting the smooth execution of model training tasks.

[0155] Therefore, in this embodiment, when the load level of a node device deploying an agent exceeds a preset level, it can be determined whether some tasks or some task steps of the currently executed tasks of that node device can be allocated to other node devices for execution. If it is determined that some tasks or some task steps of the currently executed tasks cannot be transferred from that node device to other node devices, the code of the agent deployed on that node device can be transmitted to a target node device with a load level lower than the preset level. The target node device then deploys the corresponding agent using the received agent code and determines the corresponding adjustment strategy by running the agent.

[0156] In the embodiments of this application, both the multi-agent reinforcement learning module and each agent use reinforcement learning algorithms to implement the policy determination and feedback mechanism. That is, each agent determines its adjustment strategy through reinforcement learning algorithms, and the multi-agent reinforcement learning model evaluates the optimization degree of the adjustment strategy determined by each agent in an overall manner through the global reward value and the comprehensive reward value, thereby realizing positive feedback to each agent, so that each agent continuously adjusts its determined adjustment strategy based on the feedback from the multi-agent reinforcement learning module.

[0157] In this embodiment, the development system can also periodically and proactively perform analysis operations on the generated log data to determine the status of the model training task during execution, and thereby adjust the weight parameters in the global reward function and at least some of the local reward functions corresponding to the agents. That is, by monitoring the execution of the model training task through log data, and thereby providing feedback on the adjustment of weight parameters in the global reward function and some of the local reward functions, the aim is to obtain the optimal adjustment strategy overall.

[0158] The above describes a development system provided by an embodiment of this application. Additionally, this application also provides a method for executing model training tasks based on the aforementioned development system.Figure 2 As shown.

[0159] Figure 2 A schematic diagram illustrating the steps of a model task execution method provided in this application embodiment specifically includes the following steps:

[0160] S201: Receive the user's task configuration request.

[0161] S202: Determine the model type corresponding to the model to be trained based on the task configuration request.

[0162] S203: Determine the model corresponding to the model type from the preset model library as the model to be trained, and generate a model training task for the model to be trained based on the dataset information carried in the task configuration request.

[0163] S204: Through the multiple agents, determine adjustment strategies for different dimensions when executing the model training task, so as to adjust and execute the model training task according to the adjustment strategies to obtain the trained model. For each agent, the agent collects key dynamic data of the corresponding dimension of the agent when executing the model training task, and receives associated data transmitted by the associated agent corresponding to the agent, so as to output an adjustment strategy based on the key dynamic data and the associated data. The associated data is the data associated with the corresponding dimension of the agent required by the agent when outputting the adjustment strategy.

[0164] The model training task execution method provided in this application embodiment is executed by the aforementioned development system. This development system may have the same functional modules as the aforementioned development system, or it may have different or a different number of functional modules to execute the various steps involved in the entire model training task execution method. Apart from this, the development system executing the model training task is basically the same as the aforementioned development system in terms of function and overall architecture, and it also uses reinforcement learning algorithms, relying on the collaborative data transmission between various agents to determine various adjustment strategies for optimizing the model training task. The detailed process of the entire model training task execution has been described in detail in the introduction of the development system above, and will not be repeated here.

[0165] The above are one or more embodiments of this application providing a development system for an intelligent model based on a system-on-a-chip (SoC) with in-memory computing capabilities, and a method for executing model training tasks. Based on the same idea, this application also provides a corresponding device for executing model training tasks, such as... Figure 3 As shown.

[0166] Figure 3This is a schematic diagram of a model training task execution device provided in an embodiment of this application. The device is applied to a development system for intelligent models based on a system-on-a-chip (SoC) with in-memory computing capabilities. The development system includes: multiple node devices, at least some of which are equipped with multiple intelligent agents. For each of the node devices, at least one intelligent agent is deployed. Different intelligent agents are used to determine adjustment strategies in different dimensions when executing a model training task for the model to be trained, specifically including:

[0167] The receiving module 301 is used to receive the user's task configuration request;

[0168] The determining module 302 is used to determine the model type corresponding to the model to be trained based on the task configuration request.

[0169] The generation module 303 is used to determine the model corresponding to the model type from the preset model library as the model to be trained, and generate a model training task for the model to be trained according to the dataset information carried in the task configuration request.

[0170] The training module 304 is used to determine, through the multiple agents, adjustment strategies for different dimensions when performing the model training task, so as to adjust and execute the model training task according to the adjustment strategies to obtain the trained model. For each agent, the agent collects key dynamic data for its corresponding dimension when performing the model training task, and receives associated data transmitted by its associated agents. Based on the key dynamic data and the associated data, the agent outputs an adjustment strategy. The associated data is the data associated with the corresponding dimension of the agent required by the agent when outputting the adjustment strategy.

[0171] Optionally, the device further includes:

[0172] The simulation module 305 is used to configure the simulation environment according to the acquired simulation parameters; and to execute the simulation task for the trained model according to the simulation environment, so as to obtain and output the simulation results.

[0173] This application also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 2 This provides a method for performing model training tasks.

[0174] Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0175] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0176] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0177] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0178] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0179] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0180] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0181] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0182] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0183] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0184] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0185] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0186] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0187] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0189] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0190] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A development system of an intelligent model based on a system-on-a-chip of compute-in-memory, characterized in that, The development system comprises: a plurality of node devices, at least one node device of the plurality of node devices being provided with a configuration module and a multi-agent reinforcement learning module, at least part of the node devices being provided with a plurality of agents, for each node device of the at least part of the node devices, at least one agent is provided in the node device, and different agents are used to determine adjustment strategies of different dimensions when performing a model training task for a to-be-trained model; The configuration module is configured to receive a task configuration request of a user, determine a model type corresponding to a to-be-trained model according to the task configuration request, determine a model corresponding to the model type from a preset model library as the to-be-trained model, and generate a model training task for the to-be-trained model according to data set information carried in the task configuration request. The multi-agent reinforcement learning module is configured to determine adjustment strategies for different dimensions when performing the model training task by the plurality of agents, to adjust and perform the model training task according to the adjustment strategies, and obtain a trained model, wherein for each agent, the agent collects key dynamic data of a corresponding dimension of the agent when performing the model training task, and receives associated data transmitted by an associated agent corresponding to the agent, to output an adjustment strategy according to the key dynamic data and the associated data, and the associated data is data associated with the corresponding dimension of the agent required by the agent when outputting the adjustment strategy.

2. The development system of claim 1, wherein, The development system further comprises a simulation module; The simulation module is configured to configure a simulation environment according to obtained simulation parameters, to perform a simulation task for the trained model according to the simulation environment, and obtain and output a simulation result.

3. The development system of claim 1, wherein, The development system further comprises: A user permission management module is configured to, upon receiving a model training task sent by a user, determine an operation permission of the user, open an operation portal matching the operation permission to the user according to the operation permission, and enable the user to perform an operation specified by the operation portal through the operation portal.

4. The development system of claim 1, wherein, The multi-agent reinforcement learning module is specifically configured to determine a global reward value obtained after performing the model training according to a candidate adjustment strategy determined by each agent and a preset global reward function, determine, for each agent, a reward function value in a corresponding dimension of the agent as a local reward value of the agent according to a local reward function corresponding to the agent, and obtain a comprehensive reward value according to the local reward value of each agent and the global reward value, to maximize the comprehensive reward value as an optimization target, control each agent to adjust the candidate adjustment strategy determined by the agent, and obtain an adjustment strategy output by each agent.

5. The development system of claim 1, wherein, The plurality of agents comprises a task scheduling agent, a resource scheduling agent, and a model training agent, and the development system performs a plurality of model training tasks in parallel. The task scheduling agent is configured to receive resource state data monitored by the resource scheduling agent and receive training execution state information monitored by the model training agent, determine execution priorities of the model training tasks according to the resource state data and the training execution state information, wherein the resource state data comprises resource utilization of the development system when executing the model training tasks, resource allocation strategies of the development system for executing each model training task, and resource idle information of the development system when executing the model training tasks, the resource idle information is used to indicate idle conditions of various resources of the development system when executing the model training tasks, and the training execution state information comprises training progress of the model training tasks, resource usage information of the model training tasks, and expected accuracy indicators of the model training tasks; The resource scheduling agent is configured to receive execution priorities of the model training tasks determined by the task scheduling agent and receive training execution state information of the model training agent, and determine resource allocation strategies of the development system when executing the model training tasks according to the execution priorities and the training execution state information. The model training agent is configured to receive resource allocation strategies of the development system when executing the model training tasks determined by the resource scheduling agent and receive training data used by the development system when executing the model training tasks, and determine training execution state information of the development system when executing the model training tasks according to the training data and the resource allocation strategies.

6. The development system of claim 1 or 5, wherein, The plurality of agents comprises a data management agent, a task scheduling agent, and a model training agent, and the development system executes a plurality of model training tasks in parallel; The data management agent is configured to receive training execution state information of the development system when executing the model training tasks monitored by the model training agent and obtain data requirement information of the development system when executing the model training tasks, and determine training batch strategies of the training data corresponding to the development system when executing the model training tasks according to the training execution state information and the data requirement information, wherein the data requirement information is used to indicate characteristics of the training data required when executing the model training tasks, and the training execution state information comprises training progress of the model training tasks, resource usage information of the model training tasks, and expected accuracy indicators of the model training tasks. The task scheduling agent is configured to receive the training execution state information monitored by the model training agent, and receive data usage information sent by the data management agent, determine an execution priority of the development system when executing each model training task according to the training execution state information and the data usage information, and the data usage information includes usage state information of each training data when the development system executes each model training task, data throughput information of the development system when executing each model training task, and a training batch strategy corresponding to each training data when the development system executes each model training task. The model training agent is configured to receive the training batch strategy corresponding to each training data when the development system executes each model training task determined by the data management agent, and determine training execution state information of the development system when executing each model training task according to the training batch strategy.

7. The development system of claim 3, wherein, The plurality of agents includes a development role agent and a task scheduling agent, and the development system executes a plurality of model training tasks in parallel. The development role agent is configured to receive task state information of the development system when executing each model training task monitored by the task scheduling agent, and receive operation instructions for each model training task sent by each user, determine a permission control strategy for each model training task according to the task state information and the operation instructions, and the permission control strategy is used to adjust the operation permission of each user for each model training task.

8. A model training task execution method, comprising: The method is applied to a development system of an intelligent model of a system-on-chip based on a storage-computing integrated system, and the development system includes a plurality of node devices, at least part of the plurality of node devices are deployed with a plurality of agents, for each node device in the at least part of the plurality of node devices, at least one agent is deployed in the node device, and different agents are used to determine adjustment strategies of different dimensions when executing a model training task for a to-be-trained model. A task configuration request of a user is received. According to the task configuration request, a model type corresponding to a to-be-trained model is determined. A model corresponding to the model type is determined from a preset model library as a to-be-trained model, and a model training task for the to-be-trained model is generated according to data set information carried in the task configuration request. Through the plurality of agents, adjustment strategies for different dimensions when executing the model training task are determined, so that the model training task is adjusted and executed according to the adjustment strategies, and a trained model is obtained, wherein for each agent, the agent collects key dynamic data of a corresponding dimension of the agent when executing the model training task, and receives associated data transmitted by an associated agent corresponding to the agent, so as to output an adjustment strategy according to the key dynamic data and the associated data, and the associated data is data associated with the corresponding dimension of the agent required by the agent when the adjustment strategy is output.

9. The method of claim 8, wherein, The method further includes: According to the obtained simulation parameters, a simulation environment is configured. According to the simulation environment, a simulation task is performed on the trained model, a simulation result is obtained, and output.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 8-9.

Images