Intelligent agent development platform, method, medium and equipment

Abstract

The invention provides an agent development platform and method, a medium and equipment, and relates to the technical field of intelligent algorithm development. The platform comprises a hybrid cloud heterogeneous computing resource management module used for monitoring a plurality of integrated computing resources and scheduling and distributing the plurality of computing resources; the algorithm model automatic development module is used for providing a target development environment and a target algorithm template for a user and generating an initial intelligent algorithm model in response to an editing instruction of the user for the target algorithm template in the target development environment; the intelligent game distributed training module is used for performing distributed training on the initial intelligent algorithm model based on the target data set to obtain a target intelligent algorithm model; and the intelligent algorithm model evaluation module is used for evaluating the target intelligent algorithm model based on the evaluation index and verifying the effectiveness of the target intelligent algorithm model. Therefore, a user is comprehensively assisted in developing a new algorithm model, and the research and development threshold and time cost of intelligent task planning are reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent algorithm development technology, and in particular to an intelligent agent development platform, method, medium and equipment. Background Art

[0002] A typical algorithm model development platform generally includes three modules: data preparation module, model building module and model training module. It can realize simple data processing, model parameter adjustment and model training functions. It is suitable for project management and model development in simple scenarios. However, when faced with projects and tasks in complex scenarios, it is difficult to achieve effective development of algorithm models. Summary of the invention

[0003] The present application provides an intelligent agent development platform, method, medium and equipment, which can fully assist users in developing intelligent decision-making algorithm models for task planning, reduce the R&D threshold and time cost of intelligent task planning, and promote the rapid upgrading and evolution of task planning algorithms.

[0004] A first aspect of an embodiment of the present application provides an intelligent agent development platform, the intelligent agent development platform comprising: A hybrid cloud heterogeneous computing resource management module, which is used to monitor the integrated multiple computing resources and schedule and allocate the multiple computing resources in response to user resource requests; The algorithm model automatic development module is used to provide a target development environment and a target algorithm template for the user, and generate an initial intelligent algorithm model in response to the user's editing instruction of the target algorithm template in the target development environment; An intelligent game distributed training module is used to perform distributed training on the initial intelligent algorithm model based on a target data set to obtain a target intelligent algorithm model; The intelligent algorithm model evaluation module is used to respond to the user's indicator selection instructions, determine the evaluation indicators, and evaluate the target intelligent algorithm model based on the evaluation indicators to verify the effectiveness of the target intelligent algorithm model.

[0005] Optionally, the hybrid cloud heterogeneous computing resource management module includes: An integrated resource submodule is used to perform unified cloudification and system adaptation optimization on the various computing resources to obtain integrated various computing resources; The containerized deployment submodule is used to package the intelligent agent development platform and the simulation environment it depends on into a lightweight container, so that the user can deploy and run the intelligent agent development platform in the target operating system based on the lightweight container.

[0006] Optionally, the algorithm model automation development module includes: An integrated development environment submodule, which is used to integrate a variety of artificial intelligence computing framework engines and algorithm development tools, and in response to the user's task creation instruction, determine a target framework engine from a variety of artificial intelligence computing framework engines, and install the dependency package corresponding to the target framework engine in the target development environment, and provide the algorithm development tool to the user; The auxiliary development tool sub-module is used to integrate algorithm templates developed by various intelligent agents, and generate the initial intelligent algorithm model in response to the user's editing instructions for the algorithm template through the algorithm development tool.

[0007] Optionally, the intelligent algorithm model evaluation module includes: A first evaluation submodule is used to determine a first evaluation indicator in response to a first indicator selection instruction of a user, and compare and evaluate multiple versions of the target intelligent algorithm model based on the first evaluation indicator to obtain an optimal target intelligent algorithm model; the multiple versions of the target intelligent algorithm model are obtained after the user adjusts the initial intelligent algorithm model and performs distributed training; The second evaluation submodule is used to determine the second evaluation indicator in response to the user's second indicator selection instruction, and compare and evaluate the optimal target intelligent algorithm model with the integrated multiple baseline algorithm models based on the second evaluation indicator to verify the effectiveness of the optimal target intelligent algorithm model.

[0008] Optionally, the first evaluation submodule includes: A horizontal evaluation unit, used for comparing and evaluating different versions of the target intelligent algorithm model under the same training task, and determining the optimal version of the target intelligent algorithm model according to the evaluation result; The longitudinal evaluation unit is used to compare and evaluate multiple target intelligent algorithm models under the optimal version under the same training task, and determine the optimal target intelligent algorithm model according to the evaluation results; the multiple target intelligent algorithm models under the optimal version are obtained by distributed training of the initial intelligent algorithm model based on different training parameters.

[0009] Optionally, the intelligent agent development platform further includes: The algorithm model deployment module is used to store the network parameter file of the target intelligent algorithm model, and in response to the user's deployment instructions, deploy the target intelligent algorithm model offline or online to the target operating system based on the network parameter file.

[0010] Based on the same inventive concept, a second aspect of the embodiment of the present application provides an agent development method, the method comprising: Monitor the integrated multiple computing resources and schedule and allocate the multiple computing resources in response to user resource requests; Providing a target development environment and a target algorithm template for a user, and generating an initial intelligent algorithm model in response to an editing instruction of the target algorithm template by the user in the target development environment; Performing distributed training on the initial intelligent algorithm model based on the target data set to obtain a target intelligent algorithm model; In response to the user's indicator selection instruction, an evaluation indicator is determined, and the target intelligent algorithm model is evaluated based on the evaluation indicator to verify the effectiveness of the target intelligent algorithm model.

[0011] Based on the same inventive concept, the third aspect of an embodiment of the present application provides a storage medium, in which machine executable instructions are stored. When the machine executable instructions are executed by a processor, the intelligent agent development method proposed in the second aspect of the present application is implemented.

[0012] Based on the same inventive concept, the fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the intelligent agent development method proposed in the second aspect of the present application is implemented.

[0013] Compared with the prior art, this application has the following advantages: An intelligent agent development platform provided by the embodiment of the present application includes: a hybrid cloud heterogeneous computing resource management module, which is used to monitor the integrated multiple computing resources, and in response to the user's resource request, schedule and allocate multiple computing resources; an algorithm model automation development module, which is used to provide the user with a target development environment and a target algorithm template, and in response to the user's editing instructions for the target algorithm template in the target development environment, generate an initial intelligent algorithm model; an intelligent game distributed training module, which is used to perform distributed training on the initial intelligent algorithm model based on the target data set to obtain the target intelligent algorithm model; an intelligent algorithm model evaluation module, which is used to respond to the user's indicator selection instructions, determine the evaluation index, and evaluate the target intelligent algorithm model based on the evaluation index to verify the validity of the target intelligent algorithm model. Thus, by adopting technologies such as precise resource scheduling based on cloud services, flexible integration of computing frameworks based on container virtualization, and full-process service-oriented application systems, users are fully assisted in developing intelligent decision-making algorithm models for task planning, reducing the research and development threshold and time cost of intelligent task planning, and promoting the rapid upgrade and evolution of task planning algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0015] Figure 1 This is a schematic diagram of the functional modules of an intelligent agent development platform in one embodiment of the present application; Figure 2 This is a functional module diagram of an intelligent agent development platform in another embodiment of the present application; Figure 3 is a flow chart of an agent development method in one embodiment of the present application; Figure 4 It is a structural schematic diagram of an electronic device in one embodiment of the present application.

[0016] Figure numerals: 101, hybrid cloud heterogeneous computing resource management module; 102, algorithm model automation development module; 103, intelligent game distributed training module; 104, intelligent algorithm model evaluation module. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] A typical algorithm model development platform generally includes three modules: data preparation module, model building module and model training module. It can realize simple data processing, model parameter adjustment and model training functions, and is suitable for project management and model development in simple scenarios. A typical algorithm model development platform has the following defects: Typical algorithm model development platforms can only process homogeneous computing resources (domestic or non-domestic computer resources), and cannot unify heterogeneous resources for resource scheduling and management, and cannot adapt to the special requirements for hardware resources in the project; Typical algorithm model development platforms basically develop algorithm models in the form of uploading and editing codes, which has high R&D threshold and low development efficiency; Typical algorithm model development platforms can only rely on the parameters of the algorithm itself to evaluate algorithm performance. They lack the means of comparative evaluation and are unable to effectively verify the comprehensive performance and intelligence level of the developed algorithm.

[0019] These defects make it difficult for typical algorithm model development platforms to effectively develop algorithm models when facing projects and tasks in complex scenarios. In view of this, the present application provides an intelligent agent development platform that uses technologies such as precise resource scheduling based on cloud services, flexible integration of computing frameworks based on container virtualization, and full-process service-oriented application systems to uniformly schedule and manage heterogeneous resources, and integrates automated auxiliary development tools for development and design personnel. It can realize the development of algorithm models through interface interaction, and also integrates many baseline algorithms for evaluating the developed algorithm models, assisting in the implementation of the optimization comparison function, so as to obtain the target intelligent algorithm model that meets the actual needs of users.

[0020] Please refer to Figure 1 , Figure 1 Schematic diagram of the functional modules of an intelligent agent development platform proposed in one embodiment of the present application. Figure 1 As shown in the figure, the intelligent agent development platform mainly includes: The hybrid cloud heterogeneous computing resource management module 101 is used to monitor the integrated multiple computing resources and schedule and allocate the multiple computing resources in response to the user's resource request.

[0021] In this embodiment, when constructing the hybrid cloud heterogeneous computing resource management module 101, by using technologies such as containers and k8s (Kubernetes, an open source container orchestration and management system), the underlying independent and scattered multiple computing resources are unified into a cloud, and a "hybrid cloud" with sufficient computing power is formed to provide users with it for users to choose on demand. For example, it includes domestic computing resources such as Feiteng and V-GOOD, non-domestic heterogeneous computing power resources, and some general computing resources and other computing power hardware resources.

[0022] The hybrid cloud heterogeneous computing resource management module 101 can monitor the integrated multiple computing resources in real time, and provide users with information such as computing power list, computing power attributes and computing power occupancy, so that users can understand the current resource usage. When developing a new algorithm model, the user can select the required GPU, memory, CPU and other hardware resources according to the scale of training and deployment of the model, and generate a resource request. The hybrid cloud heterogeneous computing resource management module 101 will respond to the user's resource request and uniformly schedule, manage and allocate the GPU, memory, CPU and other hardware resources selected by the user. In this way, sufficient domestic or non-domestic heterogeneous computing power resources are automatically provided for intelligent computing on demand for different applications, realizing the automation of heterogeneous computing power resource management and scheduling and maximizing resource utilization.

[0023] The algorithm model automatic development module 102 is used to provide the user with a target development environment and a target algorithm template, and generate an initial intelligent algorithm model in response to the user's editing instructions for the target algorithm template in the target development environment.

[0024] In this embodiment, the algorithm model automation development module 102 integrates intelligent decision-making algorithm model development and design auxiliary tools and a model operator library for optimization problems. It aims to assist users in completing the automated development and research of intelligent decision-making models and traditional optimization algorithms through good interaction with users throughout the entire process.

[0025] Specifically, on the one hand, the algorithm model automation development module 102 integrates the commonly used artificial intelligence computing framework engine and intelligent algorithm development tools such as VS Code (Visual Studio Code, a cross-platform source code editor) and Canvas (canvas, used for dynamic graphics drawing) with containerized technology, providing a basic development environment for intelligent agent development, supporting applications such as autonomous learning and evolution of decision-making intelligent agents, parallel deduction and evaluation, and prediction of the effects of multi-branch solutions. At the same time, the module has built-in multiple deep reinforcement learning frameworks, such as TensorFlow (a symbolic mathematical system based on data flow programming, an open source software library that uses data flow graphics for numerical calculations), PyTorch (an open source Python machine learning library), etc., which are used for programming implementation of various machine learning algorithms.

[0026] Furthermore, the framework is integrated in a plug-in manner, and users can choose the most suitable framework to build deep reinforcement learning tasks, that is, to program the initial intelligent algorithm model in the target development environment. The distributed reinforcement learning library provided by the deep reinforcement learning framework supports the implementation of various reinforcement learning algorithms, including deep Q network (DQN), proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), etc. The algorithm model automation development module 102 can support complex multi-agent scenarios, in which each agent can independently learn and make decisions through a multi-layer reinforcement learning framework, and cooperate or confront each other.

[0027] On the other hand, the algorithm model automation development module 102 integrates a variety of intelligent agent development algorithm templates, including full-process algorithm development templates for intelligent agents from simulation environment construction, decision process model construction, intelligent strategy model construction to intelligent model training and evaluation. These templates provide predefined algorithm structures and parameter configurations, and users can use the template tool to quickly implement the design, training, verification and release of planning and decision-making deep reinforcement learning intelligent agents.

[0028] For users, they can select the appropriate target development environment and target algorithm template according to the current task scenario, and modify and adjust the target algorithm template to conveniently generate the initial intelligent algorithm model. Thus, it helps users quickly build the initial intelligent algorithm model for the current task scenario and improves the efficiency of intelligent algorithm model development.

[0029] The intelligent game distributed training module 103 is used to perform distributed training on the initial intelligent algorithm model based on the target data set to obtain the target intelligent algorithm model.

[0030] In this embodiment, the intelligent game distributed training module 103 flexibly integrates the ultra-real-time distributed computing framework. The platform is designed from the underlying architecture, free from the constraints of the operating system and hardware platform, and uses the Python language to build a set of basic models and basic simulation service specifications, thereby providing a flexible architecture with a cross-platform and reusable modeling and simulation concept system, basic models and underlying basic services.

[0031] Specifically, the intelligent game distributed training module 103 adopts a combination of a distributed computing framework and a reinforcement learning library to provide users with an efficient training environment and supports different training modes, including synchronous learning training, asynchronous reinforcement learning training, and game confrontation training. Among them, the intelligent game distributed training module 103 realizes synchronous learning training between multiple nodes by training multiple agents on different computing nodes at the same time, synchronizing the gradient information to the master node at the end of each training cycle, and then updating the global parameters. At the same time, the intelligent game distributed training module 103 implements asynchronous reinforcement learning training through the Ray framework (an open source distributed computing framework for building and expanding distributed applications). Ray provides a flexible scheduling and parallel execution mechanism, allowing agents to perform actions and collect experience in an asynchronous manner in their own independent environments. In addition, the intelligent game distributed training module 103 also realizes the function of game confrontation training by integrating the reinforcement learning library. Multiple agents compete with each other in a confrontation environment and improve their decision-making ability by constantly adjusting strategies. For example, self-confrontation and evolution of agents are realized through various reinforcement learning algorithms such as policy gradients and Q learning.

[0032] The intelligent game distributed training module 103 combines the distributed computing framework and the reinforcement learning library technology to realize the diversified functions of synchronous learning training, asynchronous reinforcement learning training and game confrontation training, and can realize multi-node ultra-real-time parallel training, and supports two training methods: single machine multi-card (referring to the use of multiple GPUs on one machine for parallel computing) and multiple machines multi-card (referring to the use of multiple GPUs on multiple machines for distributed computing), providing support for the rapid evolution of intelligent agents. In addition, the creation, startup, operation and stop of intelligent agent training tasks in different modes can be managed and controlled throughout the life cycle, effectively supporting ultra-real-time parallel training of intelligent agents in different tasks and modes, so as to quickly train the target intelligent algorithm model and shorten the development cycle of the intelligent algorithm model.

[0033] The intelligent algorithm model evaluation module 104 is used to respond to the user's indicator selection instruction, determine the evaluation indicator, and evaluate the target intelligent algorithm model based on the evaluation indicator to verify the effectiveness of the target intelligent algorithm model.

[0034] In this embodiment, the intelligent algorithm model evaluation module 104 is used to compare and evaluate the capability levels of the intelligent agents in the intelligent agent development and training platform, including the evaluation of newly added intelligent algorithms, the evaluation of modified intelligent algorithms, the evaluation of querying intelligent algorithms, the evaluation of deleted intelligent algorithms and the startup evaluation, so as to verify the comprehensive performance of the model, the model application efficiency and the model intelligence level.

[0035] Specifically, the intelligent algorithm model evaluation module 104 integrates a large number of general intelligent models, professional intelligent models and traditional optimization algorithms to form a model algorithm library such as a neural network model library, reinforcement learning, and machine learning algorithm library, which is used for performance or efficiency comparison evaluation of newly developed intelligent algorithm models, and provides a basis for the use of algorithm applications. At the same time, users can directly customize and expand the integrated algorithm models DQN (Deep Q Network, an algorithm that combines neural networks and Q-learning), PPO (Proximal Policy Optimization, proximal policy optimization), A3C (Asynchronous Advantage Actor-Critic, a reinforcement learning algorithm), etc., so that users can seamlessly integrate customized algorithms with actual applications and quickly deploy intelligent agents. In addition, by designing a unified deep learning API interface, users can customize neural network structures and deep learning reinforcement learning algorithms, as well as define network layers, activation functions, loss functions, etc., and apply them to different training tasks.

[0036] Furthermore, the intelligent algorithm model evaluation module 104 uses a configuration-driven approach and a RESTful API (an application programming interface based on Representational State Transfer) interface protocol to define and manage evaluation indicators, extensions, and customized indicators. When applying, users can choose from existing evaluation indicators, or define evaluation indicators through configuration files YAML (a highly readable data serialization language that uses indentation to represent data hierarchical relationships) and JSON (JavaScript Object Notation, a lightweight data exchange format), including the calculation method, data source, and display format of the indicators, to evaluate the comprehensive performance, model application efficiency, and model intelligence level of the developed target intelligent algorithm model. If the evaluation result is poor, the user can be prompted to make modifications and retrain and optimize, and evaluate again, and so on and so forth, until a target intelligent algorithm model that can be effectively applied to the current task scenario is developed.

[0037] This application firstly clouds a variety of heterogeneous computing resources to automatically provide sufficient domestic or non-domestic heterogeneous computing resources for intelligent computing on demand, and realize the automation of heterogeneous computing resource management and scheduling and maximize resource utilization. Secondly, it provides users with a variety of intelligent engine frameworks and intelligent agent development algorithm templates, so that users can quickly build intelligent agent models based on templates, thereby reducing the R&D threshold and time cost of intelligent task planning. In addition, it integrates many baseline algorithms for evaluating the developed algorithm models, assisting in the implementation of the optimization comparison function, so as to obtain the target intelligent algorithm model that meets the actual needs of users, thereby assisting users in the rapid development of intelligent algorithm models throughout the entire process.

[0038] Please refer to Figure 2 , Figure 2 FIG. 1 is a functional module diagram of an intelligent agent development platform proposed in another embodiment of the present application. Figure 2 As shown in the figure, the composition of the intelligent agent development platform is as follows: Optionally, the above-mentioned intelligent agent development platform also includes: a platform comprehensive management module, which is used to comprehensively manage the resources, resource groups, users, roles, project groups, permissions, images, logs, data sets, models and tasks of the intelligent agent development platform, and supports data set annotation, data set preprocessing, etc. to meet the development and application needs of different types of models and provide users with friendly human-computer interaction management services.

[0039] In this embodiment, the platform integrated management module provides users with intuitive and efficient task management, project management, effect monitoring, resource monitoring, and user management functions through a visual operation and management interface based on the B / S (browser / server) architecture.

[0040] Specifically, the platform integrated management module uses the front-end framework Vue.js (progressive JavaScript framework) to develop the user interface. Users can access functional modules such as task management and project management in the browser, and easily create, edit and manage tasks and projects. At the same time, the module uses the Spring Cloud (distributed microservice architecture) back-end framework to provide data processing and business logic support. This framework not only supports high-concurrency request processing, but also seamlessly integrates with the front-end framework to ensure fast data delivery and real-time updates. Task management and project management functions implement complex business logic processing through these back-end frameworks, such as task creation, status tracking, and project version management.

[0041] In addition, the platform's comprehensive management module integrates monitoring tools such as Grafana (an open source data visualization tool) and Prometheus (an open source system monitoring and alarm toolkit) to monitor simulation effects and resource usage, and provide rich visual charts. Users can monitor various indicators in the simulation process in real time, such as resource utilization, system performance, etc. Prometheus is responsible for data collection and storage to ensure the accuracy and timeliness of monitoring data.

[0042] In addition, the user management function of the platform's integrated management module implements user security authentication and permission management through OAuth2 (an open standard for authorizing third-party applications to access user resources without sharing user credentials). Users can assign roles and set permissions in the management interface to ensure system security and data confidentiality.

[0043] When applying, users can use their user account to log in to the development interface on the intelligent agent development platform login interface, view the usage of resources such as GPU, CPU, and memory, and then create a new intelligent agent development task and enter the task development interface to apply for the resources required for this intelligent agent development and create a data set. Based on the data set, they can develop models or label, train, set, and test data. They can also share, publish, and clone data to prepare data for the construction and training of intelligent algorithm models.

[0044] Optionally, the hybrid cloud heterogeneous computing resource management module 101 includes: The integrated resource submodule is used to unify the cloudification and system adaptation optimization of various computing resources to obtain integrated various computing resources.

[0045] In this implementation, the integrated resource submodule uses technologies such as containers and k8s (Kubernetes, an open source container orchestration and management system) to unify the underlying independent and scattered computing resources into a unified cloud, and form a "hybrid cloud" with sufficient computing power to provide it to users. In addition, it further ensures that all components can run normally on these systems through in-depth adaptation of domestic operating systems (such as UOS-Universal Operating System, universal operating system, Kylin, etc.). The adaptation work includes the optimization of the system kernel, file system, compiler, and library dependencies, so that the intelligent agent development platform can run efficiently and stably in a domestic environment. In addition, the integrated resource submodule has also been specially optimized for domestic CPU architectures (such as Loongson, Feiteng, etc.) to ensure compatibility and performance in different hardware environments.

[0046] The containerized deployment submodule is used to package the intelligent agent development platform and its dependent simulation environment into a lightweight container, so that users can deploy and run the intelligent agent development platform in the target operating system based on the lightweight container.

[0047] In this embodiment, the containerized deployment submodule supports packaging the intelligent agent development platform and its dependent simulation environment into lightweight containers by integrating Docker containerization (the process of integrating applications into containers and running them). Through Docker, users can quickly deploy and run the intelligent agent development platform in different operating systems and hardware environments to achieve complete cluster management capabilities and heterogeneous computing resource scheduling capabilities, and can automatically provide sufficient domestic or non-domestic heterogeneous computing resources for intelligent computing on demand for different applications, realize the automation of heterogeneous computing resource management and scheduling, and maximize resource utilization.

[0048] Optionally, the algorithm model automatic development module 102 includes: The integrated development environment sub-module is used to integrate multiple artificial intelligence computing framework engines and algorithm development tools, and respond to the user's task creation instructions, determine the target framework engine from multiple artificial intelligence computing framework engines, and install the dependency packages corresponding to the target framework engine in the target development environment to provide users with algorithm development tools.

[0049] In this implementation, the integrated development environment submodule packages many different artificial intelligence computing frameworks (Tensorflow, Pytorch, PaddlePaddle, etc.) and their dependent packages and operating environments into an image file based on the efficient integration technology of container virtualization, and then integrates them uniformly and flexibly into the intelligent agent development platform in the form of an image.

[0050] When applying, after creating a task, the user can determine the target framework engine from a variety of artificial intelligence computing framework engines according to the current task scenario, such as selecting Tensorflow or Pytorch artificial intelligence computing framework development environment, and enter the corresponding target development environment after entering the task interface, and install the corresponding dependency packages in the target development environment. Then enter the VScode development tool interface of the artificial intelligence computing framework engine, edit the intelligent agent algorithm in this interface, and perform editing, testing, and training of the intelligent agent algorithm.

[0051] The auxiliary development tool sub-module is used to integrate algorithm templates developed by various intelligent agents, and generate an initial intelligent algorithm model in response to the user's editing instructions for the algorithm template through the algorithm development tool.

[0052] In this implementation, in order to quickly and conveniently develop intelligent agents, the auxiliary development tool submodule integrates a variety of intelligent agent development algorithm templates. The templates provide predefined algorithm structures and parameter configurations, and users can quickly build intelligent agent models based on the templates. For example, for rule-based intelligent agents, the templates contain the definition and application of expert rules, and the rule engine converts domain knowledge into the behavior decision logic of the intelligent agent; and for deep reinforcement learning intelligent agents, the templates have built-in algorithms such as deep Q network (DQN) and proximal policy optimization (PPO), which support users to train and optimize models based on the templates.

[0053] In addition, the auxiliary development tool submodule also integrates the monitoring and automated management tools of the training process, supports real-time adjustment of parameters and algorithm iteration, and ensures that the intelligent agent continuously improves its performance during the training process. When applied, the user quickly builds the intelligent agent model based on the template, and after obtaining the initial intelligent algorithm model, the code can be tested, trained and optimized in the VScode integrated development environment. That is, the initial intelligent algorithm model is distributedly trained through the above-mentioned intelligent game distributed training module 103 to obtain the target intelligent algorithm model.

[0054] Optionally, after obtaining the target intelligent algorithm model, the model needs to be evaluated by the intelligent algorithm model evaluation module 104 to verify the validity of the target intelligent algorithm model. The intelligent algorithm model evaluation module 104 includes: The first evaluation submodule is used to determine a first evaluation indicator in response to a first indicator selection instruction from a user, and to compare and evaluate multiple versions of target intelligent algorithm models based on the first evaluation indicator to obtain an optimal target intelligent algorithm model.

[0055] In this embodiment, the first evaluation submodule is used to perform version evaluation within the algorithm model. Multiple versions of the target intelligent algorithm model are obtained after the user adjusts the initial intelligent algorithm model and performs distributed training. It can be understood that different versions of the target intelligent algorithm model have a certain degree of difference in model structure. The first evaluation index is used to compare and evaluate multiple versions of the target intelligent algorithm model to obtain the optimal target intelligent algorithm model.

[0056] The second evaluation submodule is used to respond to the user's second indicator selection instruction, determine the second evaluation indicator, and compare and evaluate the optimal target intelligent algorithm model with the integrated multiple baseline algorithm models based on the second evaluation indicator to verify the effectiveness of the optimal target intelligent algorithm model.

[0057] In this embodiment, the second evaluation submodule is used to perform application evaluation between algorithms. The second evaluation index is used to compare and evaluate the optimal target intelligent algorithm model with the integrated multiple baseline algorithm models to verify the effectiveness of the optimal target intelligent algorithm model. Thus, by performing internal version evaluation of the algorithm model and application evaluation between different algorithm models, the comprehensive performance, application efficiency and intelligence level evaluation of the benchmark intelligent algorithm model can be achieved, and the optimization comparison function can be assisted to obtain the target intelligent algorithm model that meets the actual needs of users.

[0058] Furthermore, the first evaluation submodule and the second evaluation submodule provide a RESTful API interface protocol to achieve the expansion and management of evaluation indicators through interfaces. The interface supports the dynamic addition and update of the first or second evaluation indicators to ensure that the system can adapt to the ever-changing user needs. Thus, for different training tasks and application scenarios, users can customize the weights and calculation methods of evaluation indicators. At the same time, users can quickly add new evaluation indicators according to their needs, such as training performance indicators or task success rates. For example, in reinforcement learning tasks, users customize the evaluation indicators of the reward function, while in deep learning tasks, they focus on the accuracy and loss value of the model. Thereby achieving flexible adjustment of evaluation indicators and improving the evaluation effect and efficiency of algorithm models.

[0059] Furthermore, the process of performing version evaluation within the model mainly includes horizontal evaluation and vertical evaluation. Specifically, the first evaluation submodule includes: The horizontal evaluation unit is used to compare and evaluate different versions of the target intelligent algorithm model under the same training task, and determine the optimal version of the target intelligent algorithm model based on the evaluation results.

[0060] In this implementation, because the task scenarios may be similar, during a development process, multiple versions of the algorithm model saved previously and the algorithm model adjusted by the user this time may all be applicable to this type of task scenario. Therefore, it is necessary to perform internal version evaluation on these different versions of the algorithm model, and ultimately select the version of the algorithm model that performs best in the current task, that is, the optimal version of the target intelligent algorithm model.

[0061] A longitudinal evaluation unit is used to compare and evaluate multiple target intelligent algorithm models under the optimal version under the same training task, and determine the optimal target intelligent algorithm model according to the evaluation results; In this implementation, multiple target intelligent algorithm models under the optimal version are obtained by distributed training of the initial intelligent algorithm model based on different training parameters. Because for the same version of the initial intelligent algorithm model, if different training parameters are used for training, target intelligent algorithm models with different performance will be obtained, so it is necessary to determine the best performance from the multiple target intelligent algorithm models under the optimal version, and determine it as the optimal target intelligent algorithm model, which is used for comparison with the integrated multiple baseline algorithms to provide a basis for the application of the algorithm.

[0062] Optionally, the intelligent agent development platform also includes: an algorithm model deployment module, which is used to store the network parameter file of the target intelligent algorithm model, and in response to the user's deployment instructions, deploy the target intelligent algorithm model offline or online to the target operating system based on the network parameter file.

[0063] In this implementation, the algorithm model deployment module provides users with online algorithm deployment application services through port mapping technology, and at the same time packages the algorithm into an independent image through container technology to provide users with offline algorithm deployment application services.

[0064] Specifically, before deployment, the ability to export agent network parameter files is also provided. By combining the distributed file system (Ceph) with containerization technology, the agent network parameter file can be exported after training is completed. The distributed file system Ceph technology can provide reliable storage support for the large number of model parameters generated during the agent training process. After the training is completed, the computing resource scheduling management tool will export the agent network parameter file TensorFlow or PyTorch model weight file generated by the training to the specified storage location through the containerized storage volume (Volume). At the same time, in order to facilitate subsequent deployment, the algorithm model deployment module provides an automated export script. Through the script, users can complete the export of model parameter files with one click, and the exported model files can be directly used for deployment in the production environment.

[0065] When applying, users can select the optimal target intelligent algorithm model obtained through evaluation to configure the environment, including selecting the target deployment environment, operating system, hardware device and container platform, and configuring environmental parameters such as CPU, memory and storage, to ensure that the optimal target intelligent algorithm model can run correctly in the target environment. After confirming all configurations, the user clicks the deployment trigger button or option to start the offline deployment operation of the model. The system will start deploying the model application according to the configuration information provided by the user, and provide the function of real-time monitoring of the deployment status. The algorithm model deployment module also supports online deployment, by deploying the optimal target intelligent algorithm model as a Web Service (a program that can be accessed through the network) and providing an online test UI (test interface) and monitoring functions, so that users can easily test and monitor model applications.

[0066] Please refer to Figure 3 Based on the same inventive concept, the second aspect of the embodiment of the present application provides an agent development method, the method comprising: S301: monitor the integrated multiple computing resources, and schedule and allocate the multiple computing resources in response to the user's resource request; S302: providing a target development environment and a target algorithm template for the user, and generating an initial intelligent algorithm model in response to the user's editing instruction for the target algorithm template in the target development environment; S303: Perform distributed training on the initial intelligent algorithm model based on the target data set to obtain a target intelligent algorithm model; S304: In response to the user's indicator selection instruction, determine the evaluation indicator, and evaluate the target intelligent algorithm model based on the evaluation indicator to verify the effectiveness of the target intelligent algorithm model.

[0067] Optionally, the integrated multiple computing resources are monitored, and in response to the user's resource request, the multiple computing resources are scheduled and allocated, including: Unify various computing resources into a unified cloud and optimize system adaptation to obtain integrated computing resources; and, The intelligent agent development platform and its dependent simulation environment are packaged into a lightweight container so that users can deploy and run the intelligent agent development platform in the target operating system based on the lightweight container.

[0068] Optionally, providing a target development environment and a target algorithm template for a user, and generating an initial intelligent algorithm model in response to an editing instruction of the target algorithm template by the user in the target development environment, including: Integrate multiple artificial intelligence computing framework engines and algorithm development tools, and respond to the user's task creation instructions, determine the target framework engine from multiple artificial intelligence computing framework engines, and install the corresponding dependency package of the target framework engine in the target development environment to provide the user with algorithm development tools; Integrate algorithm templates developed by multiple intelligent agents, and generate an initial intelligent algorithm model in response to the user's editing instructions for the algorithm template through the algorithm development tool.

[0069] Optionally, in response to an indicator selection instruction from a user, determining an evaluation indicator, and evaluating the target intelligent algorithm model based on the evaluation indicator to verify the validity of the target intelligent algorithm model, including: In response to the user's first indicator selection instruction, a first evaluation indicator is determined, and multiple versions of target intelligent algorithm models are compared and evaluated based on the first evaluation indicator to obtain an optimal target intelligent algorithm model; the multiple versions of the target intelligent algorithm model are obtained after the user adjusts the initial intelligent algorithm model and performs distributed training; In response to the user's second indicator selection instruction, a second evaluation indicator is determined, and based on the second evaluation indicator, the optimal target intelligent algorithm model is compared and evaluated with the integrated multiple baseline algorithm models to verify the effectiveness of the optimal target intelligent algorithm model.

[0070] Optionally, based on the first evaluation index, a comparison and evaluation is performed on multiple versions of the target intelligent algorithm model to obtain the optimal target intelligent algorithm model, including: Under the same training task, different versions of the target intelligent algorithm model are compared and evaluated, and the optimal version of the target intelligent algorithm model is determined based on the evaluation results; Under the same training task, multiple target intelligent algorithm models under the optimal version are compared and evaluated, and the optimal target intelligent algorithm model is determined based on the evaluation results; the multiple target intelligent algorithm models under the optimal version are obtained by distributed training of the initial intelligent algorithm model based on different training parameters.

[0071] Optionally, the above method further includes: The network parameter file of the target intelligent algorithm model is stored, and in response to the user's deployment instructions, the target intelligent algorithm model is deployed offline or online to the target operating system based on the network parameter file.

[0072] As for the method embodiment, since it is basically similar to the platform embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the platform embodiment.

[0073] On the third aspect, based on the same inventive concept, an embodiment of the present application provides a storage medium, in which machine executable instructions are stored. When the machine executable instructions are executed by a processor, the intelligent agent development method proposed in the first aspect of the present application is implemented.

[0074] It should be noted that the specific implementation of the storage medium of the embodiment of the present application refers to the specific implementation of the intelligent agent development method proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0075] In the fourth aspect, based on the same inventive concept, refer to Figure 4 An embodiment of the present application provides an electronic device 400, including a processor 401 and a memory 402; the memory 402 stores machine executable instructions that can be executed by the processor 401, and the processor 401 is used to execute the machine executable instructions to implement the intelligent agent development method proposed in the first aspect of the present application.

[0076] It should be noted that the specific implementation of the electronic device 400 in the embodiment of the present application refers to the specific implementation of the intelligent agent development method proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0077] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0078] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0080] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0082] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0083] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0084] The above is a detailed introduction to an intelligent body development platform, method, medium and equipment provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An intelligent agent development platform, characterized in that: The agent development platform comprises: A hybrid cloud heterogeneous computing resource management module, which is used to monitor the integrated multiple computing resources and schedule and allocate the multiple computing resources in response to user resource requests; The algorithm model automatic development module is used to provide a target development environment and a target algorithm template for the user, and generate an initial intelligent algorithm model in response to the user's editing instruction of the target algorithm template in the target development environment; An intelligent game distributed training module is used to perform distributed training on the initial intelligent algorithm model based on a target data set to obtain a target intelligent algorithm model; The intelligent algorithm model evaluation module is used to respond to the user's indicator selection instructions, determine the evaluation indicators, and evaluate the target intelligent algorithm model based on the evaluation indicators to verify the effectiveness of the target intelligent algorithm model.

2. The intelligent agent development platform according to claim 1, characterized in that: The hybrid cloud heterogeneous computing resource management module includes: An integrated resource submodule is used to perform unified cloudification and system adaptation optimization on the various computing resources to obtain integrated various computing resources; The containerized deployment submodule is used to package the intelligent agent development platform and the simulation environment it depends on into a lightweight container, so that the user can deploy and run the intelligent agent development platform in the target operating system based on the lightweight container.

3. The intelligent agent development platform according to claim 1, characterized in that: The algorithm model automation development module includes: An integrated development environment submodule, which is used to integrate a variety of artificial intelligence computing framework engines and algorithm development tools, and in response to the user's task creation instruction, determine a target framework engine from a variety of artificial intelligence computing framework engines, and install the dependency package corresponding to the target framework engine in the target development environment, and provide the algorithm development tool to the user; The auxiliary development tool sub-module is used to integrate algorithm templates developed by various intelligent agents, and generate the initial intelligent algorithm model in response to the user's editing instructions for the algorithm template through the algorithm development tool.

4. The intelligent agent development platform according to claim 1, characterized in that: The intelligent algorithm model evaluation module includes: A first evaluation submodule is used to determine a first evaluation indicator in response to a first indicator selection instruction of a user, and compare and evaluate multiple versions of the target intelligent algorithm model based on the first evaluation indicator to obtain an optimal target intelligent algorithm model; the multiple versions of the target intelligent algorithm model are obtained after the user adjusts the initial intelligent algorithm model and performs distributed training; The second evaluation submodule is used to determine the second evaluation indicator in response to the user's second indicator selection instruction, and compare and evaluate the optimal target intelligent algorithm model with the integrated multiple baseline algorithm models based on the second evaluation indicator to verify the effectiveness of the optimal target intelligent algorithm model.

5. The intelligent agent development platform according to claim 4, characterized in that: The first evaluation submodule comprises: A horizontal evaluation unit, used for comparing and evaluating different versions of the target intelligent algorithm model under the same training task, and determining the optimal version of the target intelligent algorithm model according to the evaluation result; The longitudinal evaluation unit is used to compare and evaluate multiple target intelligent algorithm models under the optimal version under the same training task, and determine the optimal target intelligent algorithm model according to the evaluation results; the multiple target intelligent algorithm models under the optimal version are obtained by distributed training of the initial intelligent algorithm model based on different training parameters.

6. The intelligent agent development platform according to claim 1, characterized in that: The intelligent agent development platform also includes: The algorithm model deployment module is used to store the network parameter file of the target intelligent algorithm model, and in response to the user's deployment instructions, deploy the target intelligent algorithm model offline or online to the target operating system based on the network parameter file.

7. A method for developing an intelligent agent, characterized in that: The method comprises: Monitor the integrated multiple computing resources and schedule and allocate the multiple computing resources in response to user resource requests; Providing a target development environment and a target algorithm template for a user, and generating an initial intelligent algorithm model in response to an editing instruction of the target algorithm template by the user in the target development environment; Performing distributed training on the initial intelligent algorithm model based on the target data set to obtain a target intelligent algorithm model; In response to the user's indicator selection instruction, an evaluation indicator is determined, and the target intelligent algorithm model is evaluated based on the evaluation indicator to verify the effectiveness of the target intelligent algorithm model.

8. The agent development method according to claim 7, characterized in that: In response to the user's indicator selection instruction, determining the evaluation indicator, and evaluating the target intelligent algorithm model based on the evaluation indicator to verify the validity of the target intelligent algorithm model, including: In response to a first indicator selection instruction of a user, a first evaluation indicator is determined, and based on the first evaluation indicator, multiple versions of the target intelligent algorithm model are compared and evaluated to obtain an optimal target intelligent algorithm model; the multiple versions of the target intelligent algorithm model are obtained after the user adjusts the initial intelligent algorithm model and performs distributed training; In response to the user's second indicator selection instruction, a second evaluation indicator is determined, and based on the second evaluation indicator, the optimal target intelligent algorithm model is compared and evaluated with the integrated multiple baseline algorithm models to verify the effectiveness of the optimal target intelligent algorithm model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the intelligent agent development method as described in any one of claims 7 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the intelligent agent development method as described in any one of claims 7 to 8.

Images