A localized one-stop AI development platform and method

Abstract

The application belongs to the technical field of AI development, and particularly relates to a localized one-stop AI development platform and method, which comprises at least one first control server, multiple computing power servers and a data center level Ethernet switch. The Ethernet switch and the multiple computing power servers are located at a computing interconnection layer. The at least one first control server is located at a data access layer and is a core management layer of a business network. The application adopts a hierarchical design network architecture to solve the technical problem of how to build an AI development platform with high performance, high reliability and resource isolation capability.

Description

Technical Field

[0001] This application belongs to the field of AI development technology, specifically involving a localized one-stop AI development platform and method. Background Technology

[0002] In traditional artificial intelligence (AI) development environments, especially in scenarios involving large-scale distributed training, computing, storage, and data access networks often employ a converged or simply isolated architecture. Parameter exchange between computing nodes (such as gradient aggregation and weight synchronization) typically relies on the standard TCP / IP protocol stack, resulting in high communication latency and protocol overhead, making it difficult to fully utilize the performance of heterogeneous computing hardware. Simultaneously, critical traffic such as training dataset distribution and activation value transmission shares the same physical channel with storage access traffic, easily leading to I / O contention and network congestion, resulting in decreased training efficiency. Furthermore, the data access link lacks a dynamic load balancing mechanism, making it difficult to guarantee deterministic latency and bandwidth during multi-task concurrency. Therefore, how to build a high-performance, highly reliable AI development platform with resource isolation capabilities has become a pressing technical problem that needs to be solved. Summary of the Invention

[0003] In view of this, this application provides a localized one-stop AI development platform and method, which solves the technical problem of how to build a high-performance, highly reliable AI development platform with resource isolation capabilities by adopting a layered network architecture.

[0004] The first aspect of this application provides a localized one-stop AI development platform, which includes at least one primary control server, multiple computing servers, and a data center-level Ethernet switch. The Ethernet switch and multiple computing servers are located at the computing interconnect layer. The at least one primary control server is located at the data access layer and serves as the core management layer of the business network.

[0005] The nodes of multiple computing servers are all connected to an Ethernet switch via a 200GE network. Each node of the computing server is equipped with storage media, and the distributed storage pool integrates multiple discrete storage media into a unified parallel file system through a distributed storage software stack. The multiple computing servers have model training, fine-tuning, and inference service capabilities.

[0006] The primary control server is connected to the distributed storage pool via a dedicated 25GE network for storage-intensive operations and provides petabyte-level namespace and I / O bandwidth. It also connects to external systems located in the data center core via a 100GE network to receive all raw datasets, preprocessed feature-engineered data, model training tasks, inference requests, and user operations.

[0007] In one specific embodiment of this application, at least one first control server is provided, and the two first control servers are redundantly configured and equipped with a fault detection and automatic switching mechanism.

[0008] In one specific embodiment of this application, the number of multiple computing servers is 5 or 6.

[0009] In one specific embodiment of this application, the first control server adopts a modular design, mainly including a user management module, a database management module, a file management module, an AI development module, a device management module, and a log management module.

[0010] In one specific embodiment of this application, the localized one-stop AI development platform further includes an out-of-band management system. The out-of-band management system is connected to at least one first control server and is used to control the power-off restart operation and access management of the at least one first control server.

[0011] In one specific embodiment of this application, the out-of-band management system includes a main control server and at least one secondary control server, each connected to the main control server. The number of secondary control servers corresponds one-to-one with the number of primary control servers. Each secondary control server is connected to one primary control server.

[0012] The second aspect of this application provides a localized one-stop AI development method, which includes the following steps.

[0013] Step 1: The first control server receives the data, image, computing resources, and start command specified by the user.

[0014] Step 2: The first control server creates a training task based on the data, image, computing resources, and startup command specified by the user.

[0015] Step 3: After the task stops, the first control server evaluates the model's performance and uses the inference mirror to build an inference service based on the model output generated by the training task, so as to debug the model according to the evaluation results and the inference service.

[0016] Step 4: Repeat steps 2 and 3 until the model performance is stable and meets the standards. Then, the first control server requests computing resources to deploy the task output as an inference service.

[0017] The beneficial effects of the technical solution in this application are as follows: It adopts a layered design approach to construct a high-performance, highly reliable heterogeneous computing infrastructure. At the computing interconnect layer, all computing server nodes are connected to a data center-level Ethernet switch via a 200GE (200 Gigabit Ethernet) high-speed network interface, forming a low-latency, high-bandwidth computing network. This interconnect network enables remote direct memory access and carries critical communication traffic during distributed model training, including training dataset distribution, forward propagation activation value transfer, backpropagation gradient aggregation, and model weight synchronization. This effectively supports various distributed training strategies such as data parallelism, model parallelism, and hybrid parallelism.

[0018] At the data access layer, a 100GE high-speed uplink is deployed between the business network and the data center core area, forming the sole business data channel between the platform and external systems. All raw datasets, preprocessed feature engineering data, model training tasks, and inference requests to be processed enter the business network from the data center core area via this link. This connection uses a dynamic routing protocol to achieve multi-path load balancing, ensuring deterministic latency and bandwidth guarantees for data transmission.

[0019] In terms of the storage subsystem, a primary control server is connected to the distributed storage pool via an independent 25GE network to achieve physical isolation between computing and storage traffic and avoid I / O contention. Attached Figure Description

[0020] Figure 1 The diagram shown is a structural schematic of a localized one-stop AI development platform provided in an embodiment of this application.

[0021] Figure 2 The image shows a schematic diagram of the training parameter configuration interface for the first control server.

[0022] Figure 3 The image shown is a schematic diagram of the interface for the model training task list.

[0023] Figure 4 The image shows a schematic diagram of the interface for copying the model output address after training.

[0024] Figure 5 The image shown is a schematic diagram of the interface for configuring the inference service. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] At least one embodiment of this application provides a localized, one-stop AI development platform, see reference. Figure 1 This localized, one-stop AI development platform includes at least one primary control server 10, multiple computing servers 20, and a data center-grade Ethernet switch 30. The Ethernet switch 30 and the multiple computing servers 20 are located at the computing interconnect layer. The primary control server 10 is located at the data access layer and serves as the core management layer of the business network.

[0027] The nodes of multiple computing servers 20 are all connected to Ethernet switch 30 via a 200GE (200 Gigabit Ethernet) network. Each node of the computing server 20 is configured with storage media, and the distributed storage pool integrates multiple discrete storage media into a unified parallel file system through a distributed storage software stack. The multiple computing servers 20 have model training, fine-tuning, and inference service functions.

[0028] The first control server 10 is connected to the distributed storage pool via a separate 25GE network for storage-intensive operations and provides petabyte-level namespace and I / O bandwidth to the outside world. The first control server 10 is connected to the external system located in the core area of ​​the data center via a 100GE network to receive all raw datasets to be processed, preprocessed feature engineering data, model training tasks and inference requests, and user operations from the external system.

[0029] It should be noted that the data center-grade Ethernet switch 30 can also be called an AI switch. The storage medium can be a high-capacity NVMe SSD. The 25GE network can also be called a 25GE storage access network. The I / O bandwidth can be high aggregated I / O bandwidth. A PB-level namespace refers to a namespace with a capacity reaching the petabyte (PB) level. The first control server 10 can connect to the second control server 1 of an external system via a 100GE network. The area where multiple computing servers 20 are located can be called the AI ​​training area.

[0030] Storage-intensive operations include, but are not limited to, loading training datasets for business networks, persisting model checkpoints, layered transport of container images, and distribution of user-defined algorithm libraries.

[0031] It should be noted that the distributed storage pool integrates discrete storage resources into a unified parallel file system through a distributed storage software stack, providing petabyte-level namespaces and high aggregated I / O bandwidth. The first control server 10 is used to control the service continuity of multiple computing server nodes 20. Different first control servers 10 can be connected via a GE network.

[0032] The network architecture of the localized one-stop AI development system provided in this application adopts a layered design concept, constructing a high-performance, high-reliability heterogeneous computing infrastructure. At the computing interconnection layer, all 20 computing server nodes are connected to a data center-level Ethernet switch 30 through a 200GE (200 Gigabit Ethernet) high-speed network interface, forming a low-latency, high-bandwidth computing network. This interconnection network enables remote direct memory access and carries key communication traffic such as training dataset distribution, forward propagation activation value transmission, backpropagation gradient aggregation, and model weight synchronization during distributed model training, effectively supporting various distributed training strategies such as data parallelism, model parallelism, and hybrid parallelism.

[0033] At the data access layer, a 100GE high-speed uplink is deployed between the business network and the data center core area, forming the sole business data channel between the platform and external systems. All raw datasets, preprocessed feature engineering data, model training tasks, and inference requests to be processed enter the business network from the data center core area via this link. This connection uses a dynamic routing protocol to achieve multi-path load balancing, ensuring deterministic latency and bandwidth guarantees for data transmission.

[0034] In terms of the storage subsystem, a first control server 10 is connected to the distributed storage pool via an independent 25GE network to achieve physical isolation between computing and storage traffic and avoid I / O contention.

[0035] Furthermore, due to the layered design of the network architecture of the localized one-stop AI development platform, users do not need to directly operate the computing server 20; all operations are performed on the first control server 10 of the business network. The layered network architecture achieves physical-level isolation and differentiated service delivery for the three types of information flows: computing, storage, and management. This ensures that training tasks achieve predictable throughput and microsecond-level latency stability, while providing deterministic response capabilities for online inference services.

[0036] In at least one embodiment of this application, at least one first control server 10 is provided in pairs, and the two first control servers 10 are redundantly configured and equipped with a fault detection and automatic switching mechanism. In this way, by setting the control plane to adopt a dual-machine hot standby architecture, the core management layer of the business network is formed by two redundantly configured control servers. This enables the AI ​​development cluster (i.e., the high-availability cluster) to ensure the service continuity of the nodes of the localized one-stop AI development system controlling multiple computing power servers 20 through the fault detection and automatic switching mechanism. The single-point failure switching time (RTO) can be controlled within seconds, thereby ensuring the 24 / 7 uninterrupted operation of key management functions such as platform orchestration scheduling, task management, and user authentication.

[0037] In at least one embodiment of this application, the number of multiple computing servers 20 is 5 or 6.

[0038] It should be noted that the model of computing server 20 can be a 910B computing server.

[0039] In at least one embodiment of this application, the first control server 10 adopts a modular design, mainly including a user management module, a database management module, a file management module, an AI development module, a device management module, and a log management module.

[0040] The user management module manages user account registration / login, defines user roles (such as administrators and developers), and implements role-based access control (RBAC) to ensure that different users can only access the data and tools they are authorized to, thus ensuring system security.

[0041] The database management module is responsible for the storage and maintenance of structured data, metadata, and vector data. It supports data versioning to ensure data security and consistency.

[0042] The file management module is responsible for the storage, organization, and collaboration of unstructured data. It stores raw datasets, source code, model weight files, etc., and supports efficient uploading / downloading of large files.

[0043] The AI ​​development module is responsible for receiving user training instructions to perform model fine-tuning and inference tasks. It provides a model training pipeline, hyperparameter tuning tools, and model inference service interfaces, lowering the barrier to entry for deploying AI algorithms.

[0044] The device management module is responsible for the scheduling, monitoring, and maintenance of underlying hardware resources. It unifies the management of computing resources such as NPUs and CPUs, monitors hardware load and health status in real time, and ensures efficient utilization of computing power, especially in multi-user environments to achieve resource isolation and fair allocation.

[0045] The log management module is responsible for recording, auditing, and troubleshooting the system's operational status. It collects user operation logs, system operation logs, and program error information; provides log search and visual analysis to help operations and maintenance personnel quickly locate problems, while also meeting compliance audit requirements and tracking model iteration records.

[0046] In at least one embodiment of this application, the localized one-stop AI development platform further includes an out-of-band management system. The out-of-band management system is connected to at least one first control server 10 and is used to control the power-off restart operation and permission management of the at least one first control server 10.

[0047] In at least one embodiment of this application, the out-of-band management system includes a main control server 2 and at least one secondary control server 3, each connected to the main control server 2. The number of secondary control servers 3 corresponds one-to-one with the number of primary control servers 10. Each secondary control server 3 is connected to one primary control server 10.

[0048] It should be noted that the main control server 2 and the secondary control server 3 can be connected via a 10GE network.

[0049] At least one embodiment of this application also provides a localized one-stop AI development method, which is implemented by a localized one-stop AI development platform as described in the above embodiments. The localized one-stop AI development platform is deployed locally and used by users for AI development. The localized one-stop AI development method includes the following steps.

[0050] Step 1: The first control server receives the data, image, computing resources, and start command specified by the user.

[0051] Specifically, data preparation involves uploading user data (business data, custom images, and open-source model files) to the first control server and then registering it in the file management service; alternatively, files can be uploaded directly through the file management service.

[0052] Step 2: The first control server creates a training task based on the data, image, computing resources, and startup command specified by the user.

[0053] Specifically, the process involves creating a task. The user specifies data, image, computing resources, and the start command to create a training task. For fine-tuning tasks, an existing model must be specified. The device management module will allocate resources based on available availability and initiate model training / fine-tuning.

[0054] For example, refer to Figure 2 Users specify data, image, computing resources, and startup commands on the training parameter configuration interface of the primary control server. New tasks are added to the training task list after clicking "OK" in the task configuration interface. It should be noted that the startup command specifies the training objective for the fine-tuning task; parameters such as the learning rate allow the model to converge and stabilize quickly. Fine-tuning adapts the model from a general semantic space to a specific task domain or application scenario. It optimizes parameters on high-quality labeled datasets using a supervised learning paradigm, thereby significantly improving the model's performance on the target task, aligning with human preferences, and enhancing output controllability. The parameters for the fine-tuning task mainly configure the fine-tuning training epochs, the requested resources, and environmental variables during training.

[0055] Step 3: After the task stops, the first control server evaluates the model's performance and builds an inference service based on the model's output generated by the training task, so as to debug the model according to the evaluation results and the inference service.

[0056] Step 4: Repeat steps 2 and 3 until the model performance is stable and meets the standards. Then, the first control server requests computing resources from multiple computing servers to deploy the task output as an inference service.

[0057] For example, refer to Figure 3 The training task list displays currently completed and ongoing tasks. Clicking "Download Log" will download the log file as a compressed attachment without redirecting to another page. Users can... Figure 4 Address of the product in the copy model. Figure 5 The Chinese text refers to the image information, environment variables, and resource request status of the configuration inference service.

[0058] It should be noted that users can configure the inference service address on the first control server for the data center to access.

[0059] The one-stop AI development cluster supports diverse model development paradigms. It allows users to upload custom model assets conforming to formats such as ONNX and TorchScript via standardized interfaces on the first control server. It also provides lightweight fine-tuning capabilities based on open-source pre-trained model repositories (such as LoRA and QLoRA, which offer lightweight fine-tuning options). By integrating efficient parameter fine-tuning techniques like LoRA, QLoRA, and Adapter, users can update only a small number of adapter parameters on the first control server 10 while keeping the backbone network weights fixed. This enables efficient domain adaptation and task transfer in resource-constrained scenarios, meeting the refined training needs of complex cognitive tasks such as natural language understanding, computer vision, and multimodal generation.

[0060] It should be noted that the combination of the technical features in the embodiments of this application is not limited to the combination methods described in the embodiments of this application or the combination methods described in specific embodiments. All technical features described in this application can be freely combined or combined in any way, unless they contradict each other.

[0061] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the term "comprising" only indicates that it includes the explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0062] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0063] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A localized, one-stop AI development platform, characterized in that, It includes at least one primary control server, multiple computing servers, and data center-grade Ethernet switches. Ethernet switches and multiple computing servers are located at the computing interconnection layer. At least one primary control server is located at the data access layer and serves as the core management layer of the service network. The nodes of multiple computing servers are connected to Ethernet switches via a 200GE network. Each node of the computing server is equipped with storage media. The distributed storage pool integrates multiple discrete storage media into a unified parallel file system through a distributed storage software stack. Multiple computing servers have model training, fine-tuning and inference service functions. The first control server is connected to the distributed storage pool via a separate 25GE network for storage-intensive operations and provides petabyte-level namespace and I / O bandwidth to the outside world. The first control server is connected to the external system located in the core area of ​​the data center via a 100GE network to receive all raw datasets to be processed, preprocessed feature engineering data, model training tasks and inference requests, and user operations from the external system.

2. The localized one-stop AI development platform according to claim 1, characterized in that, At least one primary control server is provided, and two primary control servers are redundantly configured and equipped with fault detection and automatic switching mechanisms.

3. The localized one-stop AI development platform according to claim 1, characterized in that, The number of computing servers is 5 or 6.

4. The localized one-stop AI development platform according to claim 1, characterized in that, The first control server adopts a modular design, mainly including a user management module, a database management module, a file management module, an AI development module, a device management module, and a log management module.

5. A localized one-stop AI development platform according to any one of claims 1 to 4, characterized in that, It also includes an out-of-band management system, which is connected to at least one first control server and is used to control the power outage restart operation and access management of at least one first control server.

6. The localized one-stop AI development platform according to claim 5, characterized in that, The out-of-band management system includes a main control server and at least one secondary control server connected to the main control server. The number of secondary control servers corresponds one-to-one with the number of primary control servers. Each secondary control server is connected to one primary control server.

7. A localized, one-stop AI development method, characterized in that, include: Step 1: The first control server receives the user-specified data, image, computing resources, and startup command; Step 2: The first control server creates a training task based on the data, image, computing resources, and start command specified by the user. Step 3: After the task stops, the first control server evaluates the model performance and uses the inference mirror to build an inference service based on the model output generated by the training task, so as to debug the model according to the evaluation results and the inference service. Step 4: Repeat steps 2 and 3 until the model performance is stable and meets the standards. Then, the first control server requests computing resources to deploy the task output as an inference service.

Images