A storage expansion method, device and equipment of an AI training platform and a medium

Abstract

The application relates to the technical field of cluster capacity expansion, in particular to a storage capacity expansion method and device of an AI training platform, equipment and a medium. The method comprises the following steps: mounting backup storage to each physical node of a cluster; executing a modified deployment file script, wherein the modified deployment file script is used for scanning and inquiring each physical node deployed microservice and returning storage information of the backup storage; selecting a microservice needing to use the backup storage in the AI training platform as a target microservice; and executing a dynamic mounting script based on istorage, wherein the dynamic mounting script is used for finding the process of each target microservice through an ssh connection mode, and performing a hot mounting operation on the process based on the storage information of the backup storage. The scheme can perform storage capacity expansion without stopping the microservice framework process, and improves the stability of software running.

Description

Technical Field

[0001] This invention relates to the field of cluster expansion technology, and in particular to a storage expansion method, apparatus, device and medium for an AI training platform. Background Technology

[0002] With the widespread adoption of AI training platforms and the increasing number of enterprise-level applications, users' business volume is gradually increasing, leading to a greater demand for platform storage. The single storage in existing AI platforms is no longer sufficient to meet user needs. Therefore, rapidly, efficiently, and stably expanding user storage in AI training platforms is an urgent problem to be solved and a concern for many customers.

[0003] Currently, the AI ​​training platform supports adding multiple storage units, allowing users to expand their storage capacity through the multi-storage management page. To address insufficient storage space, users install a storage system outside the AI ​​cluster as backup storage to be added to the AI ​​platform. The administrator logs into the physical nodes of the AI ​​cluster and, depending on the backup storage type (e.g., Network File System (NFS), High-Performance Parallel File System (BeegFS), Distributed File System (Lustre), etc.), executes mount commands or installs storage clients to mount the backup storage to various physical nodes in the cluster. Then, the deployment files of the microservices related to the backup storage are modified. After modification, the services related to the newly added storage are restarted to make the new storage path effective. Finally, the system administrator logs into the platform's system settings module and adds the backup storage through the multi-storage management page, filling in the storage name, selecting the storage type (e.g., NFS, NFS4, BeegFSR), storage path, and mount point.

[0004] However, the current AI platform expansion process is cumbersome and prone to problems such as errors in configuration file modification or omissions in the configuration modification of certain services. Moreover, restarting various microservices related to backup storage can lead to service interruptions, temporary platform anomalies that prevent login, and failure of running training tasks. If users need to expand again after using the platform for a period of time, they still need to repeat the above steps, which will cause the platform service to restart multiple times, greatly reducing the stability of the software. Summary of the Invention

[0005] In view of this, it is necessary to provide a storage expansion method, device, equipment and medium for an AI training platform to address the above technical problems.

[0006] According to a first aspect of the present invention, a method for expanding the storage capacity of an AI training platform is provided, the method comprising:

[0007] Mount the backup storage onto each physical node of the cluster;

[0008] Execute the script to modify the deployment file, wherein the script is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage;

[0009] Select the microservice in the AI ​​training platform that requires backup storage as the target microservice;

[0010] The dynamic mounting script is executed based on istorage. The dynamic mounting script is used to find the processes of each target microservice through SSH connection and perform hot mounting operation on the processes based on the storage information of the backup storage.

[0011] In some embodiments, mounting the backup storage onto each physical node of the cluster includes:

[0012] Log in to the physical node of the cluster;

[0013] Depending on the type of backup storage, execute the mount command or install the storage client on each physical node.

[0014] In some embodiments, the backup storage type includes a network shared file system, a high-performance parallel file system, and a distributed file system.

[0015] In some embodiments, the modified deployment file script is used to perform the following steps:

[0016] Scan the file contents based on the directory where the microservice deployment files are located;

[0017] Find the deployment files of all microservices based on the key value defined in the main storage;

[0018] The sed command is used to append the stored configuration content to the main storage configuration.

[0019] Output the storage name, storage path, and mounted physical node of the backup storage.

[0020] In some embodiments, the microservices in the AI ​​training platform that require backup storage are selected as target microservices, including:

[0021] Receive the microservice name input by the user;

[0022] The AI ​​training platform filters all microservices based on the microservice name used as input, and the filtered microservices are used as target microservices.

[0023] In some embodiments, the dynamic mounting script application performs the following steps:

[0024] Use the kubectl command to query the IP addresses of each management node and compute node in the cluster;

[0025] Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port;

[0026] Use the `docker ps|grep` command to query the container ID corresponding to the target microservice;

[0027] The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID.

[0028] Based on the backup storage information, the process ID (PID) is mounted using the mount command.

[0029] In some embodiments, the method further includes: displaying the storage information returned by the modified deployment file script on a console page.

[0030] According to a second aspect of the present invention, a storage expansion device for an AI training platform is provided, the device comprising:

[0031] The mount module is configured to mount backup storage to each physical node in the cluster.

[0032] The first execution module is configured to execute a script to modify deployment files, wherein the script is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage.

[0033] Select the module and configure it to select the microservices in the AI ​​training platform that need to use backup storage as the target microservices;

[0034] The second execution module is configured to execute a dynamic mounting script based on iStorage. The dynamic mounting script is used to locate the processes of each target microservice via SSH connection and perform a hot mounting operation on the processes based on the storage information in the backup storage.

[0035] According to a third aspect of the present invention, a computer device is also provided, the computer device comprising:

[0036] At least one processor; and

[0037] The memory stores computer programs that can run on the processor, and the processor executes the aforementioned storage expansion method of the AI ​​training platform when executing the program.

[0038] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, performs the aforementioned storage expansion method for the AI ​​training platform.

[0039] The aforementioned storage expansion method for an AI training platform utilizes a self-developed script for modifying deployment files to dynamically scan microservice configuration files, simplifying the user's configuration modification process and avoiding the omission of modifications to certain service configuration files or errors in configuration file modification. It also employs a self-developed dynamic mounting script to achieve hot mounting of storage, resolving issues such as service interruptions caused by microservice restarts, temporary platform anomalies preventing login, and failures of running training tasks. This method enables storage expansion without stopping the microservice framework process, improving software stability and simplifying the tedious process of manually modifying deployment files, thus reducing the error rate.

[0040] In addition, the present invention also provides a storage expansion device for an AI training platform, a computer device, and a computer-readable storage medium, which can achieve the above-mentioned technical effects, and will not be described in detail here. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a storage expansion method for an AI training platform according to an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of a storage expansion scheme for an AI training platform provided in another embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the structure of a storage expansion device for an AI training platform provided in another embodiment of the present invention;

[0045] Figure 4 This is an internal structural diagram of a computer device according to another embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0047] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0048] In one embodiment, please refer to Figure 1 As shown, the present invention provides a storage expansion method 100 for an AI training platform. Specifically, the method includes the following steps:

[0049] Step 101: Mount the backup storage to each physical node of the cluster;

[0050] Step 102: Execute the script to modify the deployment file, wherein the script to modify the deployment file is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage.

[0051] Step 103: Select the microservice in the AI ​​training platform that needs to use backup storage as the target microservice;

[0052] Step 104: Execute the dynamic mounting script based on `istorage`. This script locates the processes of each target microservice via SSH connection and performs a hot mount operation on the processes based on the storage information of the backup storage. `istorage` is the name of the core storage module microservice of the AI ​​platform; in this embodiment, `istorage` is used as the entry point service for issuing storage additions.

[0053] The aforementioned storage expansion method for an AI training platform utilizes a self-developed script for modifying deployment files to dynamically scan microservice configuration files, simplifying the user's configuration modification process and avoiding the omission of modifications to certain service configuration files or errors in configuration file modification. It also employs a self-developed dynamic mounting script to achieve hot mounting of storage, resolving issues such as service interruptions caused by microservice restarts, temporary platform anomalies preventing login, and failures of running training tasks. This method enables storage expansion without stopping the microservice framework process, improving software stability and simplifying the tedious process of manually modifying deployment files, thus reducing the error rate.

[0054] In some embodiments, step 101, which involves mounting the backup storage onto each physical node of the cluster, includes:

[0055] Log in to the physical node of the cluster;

[0056] Depending on the type of backup storage, execute the mount command or install the storage client on each physical node.

[0057] In some embodiments, the backup storage type includes a network shared file system, a high-performance parallel file system, and a distributed file system.

[0058] In some embodiments, the modified deployment file script is used to perform the following steps:

[0059] Scan the file contents based on the directory where the microservice deployment files are located;

[0060] Find the deployment files of all microservices based on the key value defined in the main storage;

[0061] The sed command is used to append the stored configuration content to the main storage configuration.

[0062] Output the storage name, storage path, and mounted physical node of the backup storage.

[0063] In some embodiments, step 103 above, selecting the microservice in the AI ​​training platform that needs to use backup storage as the target microservice, includes:

[0064] Receive the microservice name input by the user;

[0065] The AI ​​training platform filters all microservices based on the microservice name used as input, and the filtered microservices are used as target microservices.

[0066] In some embodiments, the dynamic mounting script application performs the following steps:

[0067] Use the kubectl command to query the IP addresses of each management node and compute node in the cluster;

[0068] Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port;

[0069] Use the `docker ps|grep` command to query the container ID corresponding to the target microservice;

[0070] The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID.

[0071] Based on the backup storage information, the process ID (PID) is mounted using the mount command.

[0072] In some embodiments, the method further includes: displaying the storage information returned by the modified deployment file script on a console page.

[0073] In yet another embodiment, for ease of understanding the present invention, please refer to... Figure 2 As shown, the following example uses a cluster with three nodes: node1, node2, and node3. Let's assume that podname1-podname3 represent the microservices to be hot-mounted on the AI ​​platform. It should be noted that the number of cluster nodes and the number of microservices are for illustrative purposes only and should not be construed as limiting the method of this invention. The storage expansion method for the AI ​​training platform in this scenario will be described in detail below, specifically including the following steps:

[0074] Step 1: The backend uses a script to modify deployment files, dynamically scans the deployment configuration files of each microservice, modifies them with one click, and outputs the backup storage information, including the backup storage name and path, to the console, intuitively displaying the backup storage information added by the user on the page.

[0075] Step 2: Define a dynamic mounting script. The main function of this script is to enable hot mounting of each microservice in the cluster.

[0076] A cluster can contain multiple nodes, including management nodes and compute nodes. Each node runs microservices that require backup storage. The goal of hot mounting is to hot mount backup storage to each microservice while it is running.

[0077] We will use a three-node cluster as an example. The nodes in the cluster are: node1, node2, and node3. Node1 is the management node, and node2 and node3 are compute nodes. Both the management node and the compute nodes host the microservices that need to be hot-mounted.

[0078] Below, we will explain the implementation process of dynamically mounting scripts.

[0079] (1) First, you need to use the kubectl command:

[0080] kubectl get nodes

[0081] The IP addresses of each management node and compute node in the cluster can be retrieved.

[0082] (2) Then, using the obtained IP address and the configured fixed SSH port, connect to each node via SSH and execute the following command:

[0083] ssh -o IP -p port "docker ps|grep -E 'services to be hot-mounted'"

[0084] The container ID of the microservice that needs to be hot-mounted was found (Note: the microservice name is the service name, which is known and determined by the platform, but the container ID is uncertain and needs to be queried according to the actual deployment environment).

[0085] (3) Then execute the following command:

[0086] docker inspect --format "{{.State.Pid}}" container ID

[0087] Query the process ID (PID) of the container.

[0088] (4) Finally, execute the mount command on the container's PID:

[0089] PID "mount -t nfs -o rw 'backup storage path' 'mount point'"

[0090] The backup storage is mounted to each microservice in a read-write manner to achieve hot mounting.

[0091] Step 3: Administrators can add backup storage with one click through the page. The operation is simple and quick. The page is an entry point. By calling the interface for adding storage, the application calls the script defined in Step 2 above and persists the storage information to the database for business management.

[0092] The storage expansion method of the AI ​​training platform in this embodiment has at least the following beneficial technical effects: First, since there are many microservices involved, the script dynamically scans the service configuration files, simplifying the user's configuration modification operation and avoiding the problem of missing some service configuration file modifications or incorrect configuration file modifications; Second, by executing the dynamic mounting script through an SSH connection, the process of each service is found, and the mount operation is directly performed on the process ID of the service to mount the backup storage into the service, which solves the problems of service interruption caused by microservice restarts of the AI ​​platform, temporary platform anomalies that prevent login, and failure of running training tasks; Third, it can be executed repeatedly, and when performing storage expansion multiple times, it is not necessary to restart the service each time, which improves the stability of software operation.

[0093] In some embodiments, please refer to Figure 3 As shown, the present invention also provides a storage expansion device 200 for an AI training platform, the device comprising:

[0094] Mounting module 201 is configured to mount backup storage to each physical node of the cluster.

[0095] The first execution module 202 is configured to execute a script to modify deployment files, wherein the script to modify deployment files is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage.

[0096] Select module 203 and configure it to select the microservices in the AI ​​training platform that need to use backup storage as the target microservices;

[0097] The second execution module 204 is configured to execute a dynamic mounting script based on iStorage. The dynamic mounting script is used to find the processes of each target microservice via SSH connection and perform a hot mounting operation on the processes based on the storage information of the backup storage.

[0098] The aforementioned storage expansion device for an AI training platform utilizes a self-developed script for modifying deployment files to dynamically scan microservice configuration files. This simplifies the user's configuration modification process, avoiding the omission of modifications to certain service configuration files or errors in configuration file modification. Furthermore, it employs a self-developed dynamic mounting script to achieve hot mounting of storage, resolving issues such as service interruptions caused by microservice restarts, temporary platform anomalies preventing login, and failures of running training tasks. This allows for storage expansion without stopping the microservice framework process, improving software stability and simplifying the tedious process of manually modifying deployment files, thus reducing the error rate.

[0099] In some embodiments, the mounting module 201 is further configured to:

[0100] Log in to the physical node of the cluster;

[0101] Depending on the type of backup storage, execute the mount command or install the storage client on each physical node.

[0102] In some embodiments, the backup storage type includes a network shared file system, a high-performance parallel file system, and a distributed file system.

[0103] In some embodiments, the modified deployment file script is used to perform the following steps:

[0104] Scan the file contents based on the directory where the microservice deployment files are located;

[0105] Find the deployment files of all microservices based on the key value defined in the main storage;

[0106] The sed command is used to append the stored configuration content to the main storage configuration.

[0107] Output the storage name, storage path, and mounted physical node of the backup storage.

[0108] In some embodiments, the selection module 203 is further configured to:

[0109] Receive the microservice name input by the user;

[0110] The AI ​​training platform filters all microservices based on the microservice name used as input, and the filtered microservices are used as target microservices.

[0111] In some embodiments, the dynamic mounting script application performs the following steps:

[0112] Use the kubectl command to query the IP addresses of each management node and compute node in the cluster;

[0113] Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port;

[0114] Use the `docker ps|grep` command to query the container ID corresponding to the target microservice;

[0115] The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID.

[0116] Based on the backup storage information, the process ID (PID) is mounted using the mount command.

[0117] In some embodiments, the method further includes configuring a module for performing some steps: displaying the storage information returned by the modified deployment file script on a console page.

[0118] It should be noted that the specific limitations regarding the storage expansion device of the AI ​​training platform can be found in the limitations on the storage expansion method of the AI ​​training platform mentioned above, and will not be repeated here. Each module in the aforementioned storage expansion device of the AI ​​training platform can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0119] According to another aspect of the present invention, a computer device is provided, which may be a server, and its internal structure diagram is shown below. Figure 4 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the storage expansion method of the AI ​​training platform described above. Specifically, the method includes the following steps:

[0120] Mount the backup storage onto each physical node of the cluster;

[0121] Execute the script to modify the deployment file, wherein the script is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage;

[0122] Select the microservice in the AI ​​training platform that requires backup storage as the target microservice;

[0123] The dynamic mounting script is executed based on istorage. The dynamic mounting script is used to find the processes of each target microservice through SSH connection and perform hot mounting operation on the processes based on the storage information of the backup storage.

[0124] In some embodiments, mounting the backup storage onto each physical node of the cluster includes:

[0125] Log in to the physical node of the cluster;

[0126] Depending on the type of backup storage, execute the mount command or install the storage client on each physical node.

[0127] In some embodiments, the backup storage type includes a network shared file system, a high-performance parallel file system, and a distributed file system.

[0128] In some embodiments, the modified deployment file script is used to perform the following steps:

[0129] Scan the file contents based on the directory where the microservice deployment files are located;

[0130] Find the deployment files of all microservices based on the key value defined in the main storage;

[0131] The sed command is used to append the stored configuration content to the main storage configuration.

[0132] Output the storage name, storage path, and mounted physical node of the backup storage.

[0133] In some embodiments, the microservices in the AI ​​training platform that require backup storage are selected as target microservices, including:

[0134] Receive the microservice name input by the user;

[0135] The AI ​​training platform filters all microservices based on the microservice name used as input, and the filtered microservices are used as target microservices.

[0136] In some embodiments, the dynamic mounting script application performs the following steps:

[0137] Use the kubectl command to query the IP addresses of each management node and compute node in the cluster;

[0138] Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port;

[0139] Use the `docker ps|grep` command to query the container ID corresponding to the target microservice;

[0140] The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID.

[0141] Based on the backup storage information, the process ID (PID) is mounted using the mount command.

[0142] In some embodiments, the method further includes: displaying the storage information returned by the modified deployment file script on a console page.

[0143] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the storage expansion method of the AI ​​training platform described above, specifically including the following steps:

[0144] Mount the backup storage onto each physical node of the cluster;

[0145] Execute the script to modify the deployment file, wherein the script is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage;

[0146] Select the microservice in the AI ​​training platform that requires backup storage as the target microservice;

[0147] The dynamic mounting script is executed based on istorage. The dynamic mounting script is used to find the processes of each target microservice through SSH connection and perform hot mounting operation on the processes based on the storage information of the backup storage.

[0148] In some embodiments, mounting the backup storage onto each physical node of the cluster includes:

[0149] Log in to the physical node of the cluster;

[0150] Depending on the type of backup storage, execute the mount command or install the storage client on each physical node.

[0151] In some embodiments, the backup storage type includes a network shared file system, a high-performance parallel file system, and a distributed file system.

[0152] In some embodiments, the modified deployment file script is used to perform the following steps:

[0153] Scan the file contents based on the directory where the microservice deployment files are located;

[0154] Find the deployment files of all microservices based on the key value defined in the main storage;

[0155] The sed command is used to append the stored configuration content to the main storage configuration.

[0156] Output the storage name, storage path, and mounted physical node of the backup storage.

[0157] In some embodiments, the microservices in the AI ​​training platform that require backup storage are selected as target microservices, including:

[0158] Receive the microservice name input by the user;

[0159] The AI ​​training platform filters all microservices based on the microservice name used as input, and the filtered microservices are used as target microservices.

[0160] In some embodiments, the dynamic mounting script application performs the following steps:

[0161] Use the kubectl command to query the IP addresses of each management node and compute node in the cluster;

[0162] Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port;

[0163] Use the `docker ps|grep` command to query the container ID corresponding to the target microservice;

[0164] The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID.

[0165] Based on the backup storage information, the process ID (PID) is mounted using the mount command.

[0166] In some embodiments, the method further includes: displaying the storage information returned by the modified deployment file script on a console page.

[0167] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for expanding the storage capacity of an AI training platform, characterized in that, The method includes: Mount the backup storage onto each physical node of the cluster; Execute the script to modify the deployment file, wherein the script is used to scan and query the microservices deployed on each physical node and return the storage information of the backup storage; the script is used to perform the following steps: Scan the file contents based on the directory where the microservice deployment files are located; Find the deployment files of all microservices based on the key value defined in the main storage; The sed command is used to append the stored configuration content to the main storage configuration. Output the storage name, storage path, and mounted physical node of the backup storage; Select the microservice in the AI ​​training platform that requires backup storage as the target microservice; The dynamic mounting script is executed based on `istorage`, where `istorage` is the name of the core storage module microservice of the AI ​​platform. `istorage` serves as the entry point for issuing storage additions. The dynamic mounting script uses SSH connections to locate the processes of each target microservice and performs hot mounting operations on the processes based on the backup storage information. The dynamic mounting script executes the following steps: Use the kubectl command to query the IP addresses of each management node and compute node in the cluster; Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port; Use the `docker ps |grep` command to query the container ID corresponding to the target microservice; The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID. Based on the backup storage information, the process ID (PID) is mounted using the mount command.

2. The storage expansion method for the AI ​​training platform according to claim 1, characterized in that, Mount the backup storage to each physical node of the cluster, including: Log in to the physical node of the cluster; Depending on the type of backup storage, execute the mount command or install the storage client on each physical node.

3. The storage expansion method for the AI ​​training platform according to claim 2, characterized in that, The types of backup storage include network shared file systems, high-performance parallel file systems, and distributed file systems.

4. The storage expansion method for the AI ​​training platform according to claim 1, characterized in that, Select the microservices in the AI ​​training platform that require backup storage as the target microservices, including: Receive the microservice name input by the user; The AI ​​training platform filters all microservices based on the microservice name used as input, and the filtered microservices are used as target microservices.

5. The storage expansion method for the AI ​​training platform according to claim 1, characterized in that, The method also includes displaying the storage information returned by the modified deployment file script on the console page.

6. A storage expansion device for an AI training platform, characterized in that, The device includes: The mount module is configured to mount backup storage to each physical node in the cluster. The first execution module is configured to execute a script to modify deployment files. This script scans and queries the microservices deployed on each physical node and returns the storage information for backup storage. The script also performs the following steps: Scan the file contents based on the directory where the microservice deployment files are located; Find the deployment files of all microservices based on the key value defined in the main storage; The sed command is used to append the stored configuration content to the main storage configuration. Output the storage name, storage path, and mounted physical node of the backup storage; Select the module and configure it to select the microservices in the AI ​​training platform that need to use backup storage as the target microservices; The second execution module is configured to execute a dynamic mounting script based on `istorage`, where `istorage` is the name of the core storage module microservice of the AI ​​platform. `istorage` serves as the entry point for issuing storage additions. The dynamic mounting script is used to locate the processes of each target microservice via SSH connection and perform hot mounting operations on the processes based on the backup storage information. The dynamic mounting script executes the following steps: Use the kubectl command to query the IP addresses of each management node and compute node in the cluster; Connect to each node via SSH using the retrieved IP address and the pre-configured fixed SSH port; Use the `docker ps |grep` command to query the container ID corresponding to the target microservice; The process ID (PID) of the running container can be retrieved using the `docker inspect` command based on the container ID. Based on the backup storage information, the process ID (PID) is mounted using the mount command.

7. A computer device, characterized in that, include: At least one processor; as well as A memory storing a computer program executable in the processor, wherein the processor executes the program and performs the storage expansion method of the AI ​​training platform according to any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it performs the storage expansion method of the AI ​​training platform according to any one of claims 1-5.

Images