Method and apparatus for loading file, and electronic device and medium

Abstract

The embodiments of the present disclosure relate to a method and apparatus for loading a file, and an electronic device and a medium. The method comprises: determining one or more source devices where machine learning model files are stored, and one or more destination devices. The method further comprises: on the basis of a path policy table, determining one or more file loading paths that load the machine learning model files to the one or more destination devices, wherein the path policy table is constructed on the basis of topological relationships between a plurality of devices in a distributed file system, and the plurality of devices comprise a plurality of accelerators. In addition, the method further comprises: on the basis of the one or more file loading paths, loading the machine learning model files from the one or more source devices to the one or more destination devices. According to the embodiments of the present disclosure, a path policy table is constructed on the basis of topological relationships between devices, and a preferred loading path for machine learning model files is then determined on the basis of the path policy table, thereby reducing the time for loading the machine learning model files.

Description

Method, device, electronic device and medium for loading files

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese invention patent application with application number 202311229017.X, entitled “Method, device, electronic device and medium for loading files” and filing date September 21, 2023, and the entire application is incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to the field of storage, and more particularly, to a method, apparatus, electronic device, and medium for loading files. Background Art

[0004] Large language models are deep learning models trained using large amounts of text data. They can generate natural language text or understand the meaning of text. They can handle a variety of natural language tasks, such as text classification, question-answering, and conversation. The introduction of large language models has brought a wider range of application scenarios and higher performance to machine learning, but also increased computing and storage costs. Given the massive scale of model training parameters, loading parameter files or models from storage systems to accelerators can take several seconds or even longer.

[0005] Currently, many service providers are offering their own models and services to different tenants. This model eliminates the need for tenants to build their own infrastructure; multiple tenants share the same infrastructure, allowing them to access the latest models without investing in expensive hardware and software. However, given the massive scale of parameters required for machine learning model training, it can take even longer for service providers to use the same infrastructure to serve different tenants.

[0006] Summary of the Invention

[0007] Embodiments of the present disclosure provide a method, apparatus, electronic device, and medium for loading a file.

[0008] According to the first aspect of the disclosure, a method for loading a file is provided. The method includes determining one or more source devices and one or more destination devices where a machine learning model file is stored. The method also includes determining one or more file loading paths for loading the machine learning model file to the one or more destination devices according to a path policy table, wherein the path policy table is constructed according to the topological relationship between multiple devices in a distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and host memory. In addition, the method also includes loading the machine learning model file from the one or more source devices to the one or more destination devices according to the one or more file loading paths.

[0009] In a second aspect of the disclosure, a device for loading a file is provided. The device includes a location determination module configured to determine one or more source devices and one or more destination devices where a machine learning model file is stored. The device also includes a path determination module configured to determine one or more file loading paths for loading the machine learning model file into one or more destination devices according to a path policy table, wherein the path policy table is constructed according to a topological relationship between multiple devices in a distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and a host. In addition, the device also includes a file loading module configured to load the machine learning model file from one or more source devices to one or more destination devices according to one or more file loading paths.

[0010] In a third aspect of the present disclosure, an electronic device is provided, comprising a processor and a memory coupled to the processor, wherein the memory has instructions stored therein, and when the instructions are executed by the processor, the electronic device executes the method according to the first aspect.

[0011] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method according to the first aspect.

[0012] This summary is intended to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0014] FIG1 illustrates a schematic diagram of an example environment in which some embodiments of the present disclosure may be implemented;

[0015] FIG2 shows a flowchart of a method for loading a file according to some embodiments of the present disclosure;

[0016] FIG3 shows a schematic diagram of a distributed system architecture according to some embodiments of the present disclosure;

[0017] FIG4 is a schematic diagram showing a topological relationship between devices according to some embodiments of the present disclosure;

[0018] FIG5 shows a schematic diagram of point-to-point direct memory access (P2PDMA) according to some embodiments of the present disclosure;

[0019] FIG6 shows a schematic diagram of file transmission through multi-threading in some embodiments of the present disclosure;

[0020] 7A, 7B and 7C are schematic diagrams showing asynchronous pre-caching of files according to some embodiments of the present disclosure;

[0021] FIG8 shows a block diagram of an apparatus for loading a file according to some embodiments of the present disclosure; and

[0022] FIG9 shows a block diagram of an electronic device according to some embodiments of the present disclosure.

[0023] Throughout the drawings, the same or similar reference numbers denote the same or similar elements. DETAILED DESCRIPTION

[0024] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0025] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0026] For example, upon receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0029] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0030] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. can refer to different or the same objects, unless explicitly stated otherwise. Other explicit and implicit definitions may also be included below.

[0031] Given the massive size of the parameters required for machine learning model training, loading machine learning model files from the file system takes a significant amount of time. This can be even more time-consuming if a service provider wishes to use a single infrastructure to serve different tenants. Furthermore, because the memory size of accelerators (such as graphics processing units (GPUs)) is limited, all tenants' models cannot fit into the accelerator's memory. It's necessary to constantly swap out a tenant's model file or save the model parameters after each training session before loading another tenant's model. The latency associated with loading files and the time it takes to switch between tenants impact the efficiency of model inference and training, resulting in significant model file loading times.

[0032] To this end, during the file loading process, the embodiments of the present disclosure obtain basic information about the file to be loaded, including the current storage location and target storage location of the file, obtain the transmission parameters of each device in the system, such as the accelerator, host memory, and file system, and construct a topology map with the devices as nodes based on the obtained transmission parameters. Then, based on the topology map, the loading paths between the devices are obtained, and a policy table for the file loading path is generated. The file loading path is then determined and the file is loaded to the target storage device. According to the embodiments of the present disclosure, the file loading path of the loaded file can be determined, effectively reducing the delay of file loading.

[0033] FIG1 illustrates a schematic diagram of an example environment 100 in which some embodiments of the present disclosure may be implemented. As shown in FIG1 , example environment 100 includes at least a distributed file system 112, a network 104, and tenants 102. Distributed file system 112 can provide cloud services for multiple tenants, including N tenants 102-1, 102-2, 102-N. In some embodiments, network 104 can be the Internet, a local area network, a wide area network, and / or a wireless network, and host 106 can be a virtual host, a dedicated host, or a managed host.

[0034] 1 , the distributed file system 112 includes a host 106, an accelerator set 108, and a file system 110. In some embodiments, the distributed file system 112 may include a local file system and a remote file system. The accelerator set 108 may include M accelerators, such as accelerators 108-1, accelerator 108-2, to accelerator 108-M. In some embodiments, the accelerators in the accelerator set 108 are homogeneous, i.e., accelerators of the same type. In some embodiments, the accelerators in the accelerator set 108 are heterogeneous, i.e., the accelerator set 108 includes multiple accelerators of different types or from different manufacturers. The file system 110 stores a large number of model files, including model files 110-1, model files 110-2, to model files 110-N. In the process of providing services to multiple tenants, the model files in the file system 110 are loaded into the accelerator set 108 for operation.

[0035] Continuing with Figure 1 , tenant 102-1 accesses distributed file system 112 via network 104. Model file 110-1 corresponding to tenant 102-1 in file system 110 is loaded into accelerator set 108 for computation. Tenant 102-2 also accesses distributed system 112 via network 104. Model file 110-2 corresponding to tenant 102-1 in file system 110 is loaded into accelerator set 108 for computation after model file 110-1 corresponding to tenant 102-1 is switched out of accelerator set 108.

[0036] It should be understood that the architecture and functions in the example environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. The embodiments of the present disclosure may also be applied to other environments with different structures and / or functions.

[0037] The following describes in detail the process according to an embodiment of the present disclosure in conjunction with Figures 2 to 9. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection of the present disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or may omit the actions shown, and the scope of the present disclosure is not limited in this respect.

[0038] FIG2 illustrates a flow chart of a method 200 for loading a file according to some embodiments of the present disclosure. At block 202, one or more source devices and one or more destination devices where a machine learning model file is stored are determined. For example, referring to FIG1 , the model file 110-1 is stored in the host 106 or in the accelerator 108-1, and its destination device may be the accelerator 108-2. In some embodiments, the model file 110-1 is stored in the host 106, and its destination device is the accelerator 108-1 or the accelerator 108-2.

[0039] In box 204, one or more file loading paths for loading the machine learning model file to one or more destination devices are determined based on the path strategy table, wherein the path strategy table is constructed based on the topological relationship between multiple devices in the distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and host memory. In some embodiments, a topology graph can be constructed based on at least one of the transmission protocol, network bandwidth, and transmission rate between devices. For example, a topology graph can be generated with a remote file system, multiple accelerators, host memory, and a local file system as nodes. The remote file system is connected to the local file system via the Transmission Control / Internet Protocol (TCP / IP), the host memory is connected to the accelerator via the high-speed Serial Computer Expansion Bus standard (PCIe), and the accelerators are connected to each other via specific hardware technology. In some embodiments, if the accelerator is a GPU, it can be connected via GPU storage Direct technology.

[0040] TCP / IP is a layered protocol that defines the standards for how electronic devices connect to the internet and how data is transferred between them. PCI-Express is a computer internal bus transmission protocol primarily used for high-speed data transfer. GPU Storage Direct is a technology that connects storage directly to the GPU, bypassing the central processing unit (CPU) for faster data transfer and processing.

[0041] In some embodiments, hosts can be connected via Non-Volatile Memory Architecture over Remote Direct Memory (NVMe-oF RDMA) to increase transfer rates. NVMe over Remote Direct Memory (NVMe-oF RDMA) is an NVMe protocol implementation based on RDMA technology that allows for efficient, low-latency data transfer between storage devices and remote servers.

[0042] In some embodiments, the path strategy table includes the current storage device, target storage device, and file load path from the storage device to the target storage device for each machine learning model file. The load path may be one or more. For example, if model file 110-1 is stored in host 106 and accelerator 108-1, and the target storage device of model file 110-1 is accelerator 108-2, then there will be a load path from accelerator 108-1 to accelerator 108-2. In some embodiments, the path strategy table is updated in real time based on changes in the current storage location of the machine learning model file. For example, if model file 110-1 is previously stored on host 106 and accelerator 108-1, and its target storage device is accelerator 108-2, there will be a load path from host 106 to accelerator 108-2 and a load path from accelerator 108-1 to accelerator 108-2. When model file 110-1 stored on accelerator 108-1 is released, only the path from host 106 to accelerator 108-2 remains. The path policy table is updated in real time based on this change. By updating the path policy table in real time, the timeliness and accuracy of network data are guaranteed, further improving file transfer efficiency.

[0043] In block 206, the machine learning model file is loaded from one or more source devices to one or more destination devices according to one or more file loading paths. In some embodiments, if the bandwidth between the accelerator 108-1 and the accelerator 108-2 is greater than the bandwidth between the host 106 and the accelerator 108-2, the path between the accelerator 108-1 and the accelerator 108-2 is determined to be the loading path, and the model file 110-1 is loaded from the accelerator 108-1 to the accelerator 108-2. In some embodiments, if the transmission rate between the accelerator 108-2 and the accelerator 108-M is less than the transmission rate between the host 106 and the accelerator 108-M, the path between the host 106 and the accelerator 108-M is determined to be the loading path, and the model file 110-1 is loaded from the host 106 to the accelerator 108-M. According to the solution of the present disclosure, the time for loading and switching machine learning model files can be effectively reduced.

[0044] FIG3 shows a schematic diagram of a distributed system architecture 300 of some embodiments of the present disclosure. As shown in FIG3 , the distributed system architecture 300 includes a host 318, a host 322, a cloud file system 320, a container set (POD) 302, a POD 304, a POD 306, and a POD 308, and the host 318 is configured with accelerators 324, 326, and 328, and the host 322 is configured with accelerators 330, 332, and 334. In some embodiments, the container cluster platform communicates with the host through a container storage interface. Each POD may include one or more model containers, such as the model container 310, the model container 312, the model container 314, and the model container 316 shown in FIG3 .

[0045] The container cluster platform is used to manage containerized applications on multiple hosts in a cloud platform. In the container cluster platform, a POD is the smallest basic unit of deployment and management in the cluster. It is a collection of one or more containers. Each container is isolated from each other and each container has its own file system. In some embodiments, the model file 110-1 is loaded from the cloud file system 320 to the host 318 or the host 318 has its own local system (not shown). The host 318 loads the model file 110-1 into the model container 310 through the container storage interface.

[0046] The container cluster platform's scheduler can schedule processes for each POD, each of which corresponds to a tenant's needs. For example, after the request process for POD 302 completes, the request process for POD 304 can proceed under the scheduling of the container cluster platform. In some embodiments, after selecting host 318 for POD 302 or POD 304, the cluster platform's scheduler loads the model files in cloud file system 320 to the target storage device according to the path policy table. In some embodiments, scheduling information can be obtained from the container cluster platform's scheduler, and the order in which the model files are switched is determined based on the scheduling information.

[0047] In some embodiments, a model file corresponding to one tenant has been loaded from host memory into an accelerator set for training. When another tenant with a higher priority initiates a service request, the container cluster platform schedules the model file corresponding to one tenant and the model parameters saved after training to be swapped out of the accelerator set, and then the model file corresponding to the other tenant is loaded into the accelerator set. In some embodiments, the model file includes the model parameters saved after each training session.

[0048] Figure 4 shows a schematic diagram of a possible topological relationship 400 between devices in some embodiments of the present disclosure. In some embodiments, a model file is currently stored in host memory 401, an accelerator 402, a local file system 403, and a remote file system 405, and the destination storage device of the model file is an accelerator 404. The nodes of the topology diagram can be determined based on the current storage device and the destination device of the model file, and the topology diagram can be generated by taking into account the transmission protocol, network bandwidth, or transmission rate of the connection between the devices.

[0049] In some embodiments, host memory 401 is connected to accelerator 402 and accelerator 404 via PCI-e. In some embodiments, accelerator 402 and accelerator 404 may be isomorphic and directly connected. In some embodiments, local file system 403 and remote file system 405 may be connected via TCP / IP. In some embodiments, local file system 403 and remote file system 405 may be connected via the NVMe-oF RDMA protocol. In some embodiments, a path policy table may be generated based on a topology diagram between devices.

[0050] Table 1: Path strategy table

[0051] For example, if the network bandwidth of the model file 110-1 in the transmission path from accelerator 402 to accelerator 404 is greater than the bandwidth in the transmission path from host memory 401 to accelerator 404, and the network bandwidth in the transmission path from accelerator 402 to accelerator 404 is greater than the bandwidth between the local file system 403 and accelerator 404, and the network bandwidth in the transmission path from accelerator 402 to accelerator 404 is greater than the bandwidth in the transmission path from remote file system 405 to accelerator 404, then the path between accelerator 402 and accelerator 404 can be determined as the loading path of the model file 110-1.

[0052] In some embodiments, model file 110-2 is currently stored in host memory 401 and remote file system 405, and its destination storage device is accelerator 404. Host memory 401 is connected to accelerator 402 and accelerator 404 via PCI-E, and accelerators are connected to each other via GPU Storage Direct. A topology map can be generated based on the above connection method and a path strategy table can be constructed based on the generated topology map. Furthermore, if the transmission rate of model file 110-2 between host memory 401 and accelerator 404 is better than the transmission rate between remote file system 405 and accelerator 404, the path between host memory 401 and accelerator 404 is determined to be the loading path of model file 110-2.

[0053] In some embodiments, the accelerator 402 has switched out the model file 110-1. At this time, the current storage location of the model file 110-1 is updated in real time in the policy table to the local file system 403, the host memory 401, and the remote file system 405. If the transmission rate of the model file 110-1 from the host memory 401 to the accelerator 404 is better than the transmission rate of loading from the local file system 403 to the accelerator 404 and is also better than the transmission rate of the remote file system 405 to the accelerator 404, then the loading path of the model file 110-1 is updated in real time to the loading path between the host memory 401 and the accelerator 404.

[0054] In some embodiments, the accelerator 402 has already switched the model file 110-1 out, and the current storage location of the model file 110-1 is updated in real time in the policy table to the local file system 403, the host memory 401, and the remote file system 405. If the network bandwidth of the model file 110-1 from the host memory 401 to the accelerator 404 is better than the transfer rate from the local file system 403 to the accelerator 404 and is also better than the transfer rate from the remote file system 405 to the accelerator 404, the loading path of the model file 110-1 is updated in real time to the loading path from the host memory 401 to the accelerator 404. In some embodiments, the model file includes model parameters saved after each training.

[0055] FIG5 shows a schematic diagram 500 of P2P DMA according to the present disclosure. In some embodiments, direct communication can be achieved between device 504 and device 506 without passing through host memory 502. That is, device 504 can directly load a model file to device 506, or device 506 can directly load a model file to device 504. In some embodiments, device 504 and device 506 are homogeneous or heterogeneous GPUs that can be configured with direct-attached storage technology in hardware. In this case, there is no need to load the model file from device 506 to host memory 502 and then from host memory 502 to device 506. Instead, direct communication can be achieved between device 504 and device 506.

[0056] In some embodiments, device 504 and device 506 can be configured with a local file system and a remote file system using the same communication protocol in software, such as an NVMe-oF RDMA protocol interface or a similar protocol interface that increases transmission rate and reduces latency. This allows the local file system and the remote file system to communicate directly without passing through host memory, thereby improving communication efficiency. In some embodiments, the model file includes model parameters saved after each training session.

[0057] FIG6 shows a schematic diagram of a file transfer 600 via multiple threads according to some embodiments of the present disclosure. After determining the file loading path, the model file 612 can be split into multiple parts in the accelerator 610, namely, a model file 612-1, a model file 612-2, and a model file 612-3. These parts are then transferred to the accelerator 620 in parallel via multiple threads 630-1, 630-2, and 630-3, to load the model file 612 from the accelerator 610 to the accelerator 620. As shown in FIG6 , thread 630-1 transfers the model file 612-1, thread 630-2 transfers the model file 612-2, and thread 630-3 transfers the model file 612-3. In the accelerator 620, the model files 612-1, 612-2, and 612-3 are merged into the model file 612. In some embodiments, the accelerator 610 and the accelerator 620 may be heterogeneous. In some embodiments, accelerator 610 and accelerator 620 may also be isomorphic. In some embodiments, the model file includes model parameters saved after each training.

[0058] Figures 7A, 7B, and 7C illustrate schematic diagrams of asynchronous pre-caching of files 700 according to some embodiments of the present disclosure. Referring to Figure 7A , a model file needs to be loaded from current storage device 702 into accelerators 704, 706, 708, and 710. While accelerators 708 and 710 are still in operation, the model file can be loaded into accelerators 702 and 704 first, as shown in Figure 7B . Then, while waiting for accelerators 708 and 710 to become idle, the machine model file is loaded into the third accelerator 708 and the fourth accelerator 710, as shown in Figure 7C .

[0059] In some embodiments, a model file loading queue can be obtained from the scheduling information, where the queue indicates that the model file 110-2 corresponding to the tenant 102-2 and the model file 110-1 corresponding to the tenant 102-1 are about to be loaded into the accelerator set 108, and that the model file 110-1 is loaded into the accelerator set 108 before the model file 110-2. When the model file 110-1 of the tenant 102-1 has been loaded into the accelerator set 108 and has not yet been released by the accelerator set 108, the model file 110-2 of the tenant 102-2 can be loaded into the host 106 in advance, and then loaded into the accelerator set 108 after the model file of the tenant 102-1 is released from the accelerator set 108. According to the embodiment of asynchronous pre-loading of files disclosed herein, the time for multi-tenant switching can be further reduced.

[0060] In some embodiments, the model file 110-2 of tenant 102-2 can be pre-loaded into accelerator 108-1 or accelerator 108-2 that has released model file 110-1, without having to wait for accelerator 108-M to also release model file 110-1. The model file of tenant 102-2 can then be loaded into accelerator 108-M when accelerator 110-M is idle. The asynchronous pre-loaded file embodiment disclosed herein can reduce the time required for multi-tenant switching. In some embodiments, the model file includes model parameters saved after each training session.

[0061] Figure 8 shows a block diagram of an apparatus 800 for loading files in accordance with some embodiments of the present disclosure. As shown in Figure 8, the apparatus 800 includes a location determination module 802 configured to determine one or more source devices and one or more destination devices where the machine learning model file is stored. The apparatus 800 also includes a path determination module 804 configured to determine one or more file loading paths for loading the machine learning model file into one or more destination devices according to a path policy table, wherein the path policy table is constructed according to a topological relationship between multiple devices in a distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and a host. The apparatus 800 also includes a file loading module 806 configured to load the machine learning model file from one or more source devices into one or more destination devices according to the one or more file loading paths.

[0062] Figure 9 shows a block diagram of an electronic device 900 of some embodiments of the present disclosure, and device 900 can be the device or apparatus described in the embodiments of the present disclosure. As shown in Figure 9, device 900 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 902, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 904 or loaded from a storage unit 916 into a random access memory (RAM) 906. CPU / GPU 902, ROM 904 and RAM 906 are connected to each other via a bus 908. Input / output (I / O) interface 910 is also connected to bus 908. Although not shown in Figure 9, device 900 may also include a coprocessor.

[0063] Various components in the device 900 are connected to the I / O interface 910, including an input unit 912, such as a keyboard, a mouse, etc.; an output unit 914, such as various types of displays, speakers, etc.; a storage unit 916, such as a magnetic disk, an optical disk, etc.; and a communication unit 918, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 918 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0064] The various methods or processes described above may be performed by the CPU / GPU 902. For example, in some embodiments, the methods may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 916. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 900 via the ROM 904 and / or the communication unit 918. When the computer program is loaded into the RAM 906 and executed by the CPU / GPU 902, one or more steps or actions in the methods or processes described above may be performed.

[0065] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0066] Computer-readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device.Computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device or any suitable combination thereof.More specific examples (non-exhaustive list) of computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove having instructions stored thereon, and any suitable combination thereof.Computer-readable storage medium used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0067] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0068] The computer program instructions for performing the disclosed operation can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or the object code written in any combination of one or more programming languages, programming languages ​​include object-oriented programming languages, and conventional procedural programming languages.Computer-readable program instructions can be performed completely on a user's computer, partially on a user's computer, performed as an independent software package, partly on a user's computer and partly on a remote computer, or performed completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network-including local area network (LAN) or wide area network (WAN), or can be connected to an external computer (such as utilizing an internet service provider to connect by the internet). In certain embodiments, by utilizing the state information of computer-readable program instructions to carry out personalized customization electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLA), this electronic circuit can perform computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0069] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0070] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0071] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented by a special hardware-based system that performs the specified function or action, or can be implemented by a combination of special hardware and computer instructions.

[0072] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, practical applications, or technical improvements to existing technologies, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

[0073] Some example implementations of the present disclosure are listed below.

[0074] Example 1. A method for loading a file, comprising:

[0075] determining one or more source devices and one or more destination devices where the machine learning model file is stored;

[0076] Determining, according to a path policy table, one or more file loading paths for loading the machine learning model file to the one or more destination devices, wherein the path policy table is constructed according to a topological relationship between a plurality of devices in a distributed file system, and the plurality of devices include a plurality of accelerators, a plurality of file systems, and a host memory; and

[0077] The machine learning model file is loaded from the one or more source devices to the one or more destination devices according to the one or more file loading paths.

[0078] Example 2. The method of Example 1, further comprising:

[0079] generating a topology map between the plurality of devices in the distributed file based on transmission parameters between the devices, the transmission parameters including at least one of a transmission protocol, a network bandwidth, and a transmission rate; and

[0080] For each of the multiple machine learning model files:

[0081] Determine the current storage device for each machine learning model file;

[0082] Determine the target storage device for each machine learning model file; and

[0083] Based on the topology map, a file loading path is generated to load each machine learning model file from the current storage device to the target storage device.

[0084] Example 3. The method of any one of Examples 1-2, wherein the path policy table includes at least the following:

[0085] File identifier of the machine learning model file;

[0086] The current storage device of the machine learning model file;

[0087] The target storage device for the machine learning model files; and

[0088] The file loading path of the machine learning model file.

[0089] Example 4. The method of any one of Examples 1-3, wherein generating a file load path for loading each machine learning model file from the current storage device to the target storage device comprises:

[0090] Based on at least one of the transmission protocol, the network bandwidth, and the transmission rate, determine a file loading path for each machine learning model file.

[0091] Example 5. The method of any one of Examples 1-4, wherein determining the file load path of each machine learning model file comprises:

[0092] Based on the topology graph, determining multiple candidate file loading paths for loading the target machine learning model file;

[0093] In response to the first communication protocol of the first candidate file loading path among the multiple candidate file loading paths being better than the second communication protocol of the second candidate file loading path, the first candidate file loading path is determined as the file loading path of the target machine learning model file.

[0094] Example 6. The method of any one of Examples 1-5, further comprising:

[0095] In response to a change in a current storage device of one or more machine learning model files among the multiple machine learning model files, the path strategy table is updated.

[0096] Example 7. The method of any one of Examples 1-6, further comprising:

[0097] Obtain scheduling information for a target machine learning model file, where the scheduling information indicates that the target machine learning model file is to be scheduled to a first accelerator set;

[0098] In response to a first subset of accelerators in the first accelerator set being in a working state and a second subset of accelerators being in an idle state, loading the target machine learning model file into the second subset of accelerators; and

[0099] In response to the first accelerator subset transitioning to an idle state, the target machine learning model file is loaded from the second accelerator subset to the first accelerator subset.

[0100] Example 8. The method of any one of Examples 1-7, wherein a first accelerator and a second accelerator of the plurality of accelerators are different types of accelerators.

[0101] Example 9. The method of any of Examples 1-8, wherein loading the machine learning model file from the one or more source devices to the one or more destination devices comprises:

[0102] The machine learning model file is loaded directly from the first file system to the first accelerator without loading through the host memory.

[0103] Example 10. The method of any of Examples 1-9, wherein loading the machine learning model file from the one or more source devices to the one or more destination devices comprises:

[0104] Splitting the machine learning model file into multiple parts;

[0105] The portions of the machine learning model file are loaded in parallel to the one or more destination devices using multiple threads.

[0106] Example 11. An apparatus for determining attribution of behavior, comprising:

[0107] a location determination module configured to determine one or more source devices and one or more destination devices where the machine learning model file is stored;

[0108] a path determination module configured to determine one or more file loading paths for loading the machine learning model file into the one or more destination devices according to a path policy table, wherein the path policy table is constructed according to a topological relationship between multiple devices in a distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and a host; and

[0109] The file loading module is configured to load the machine learning model file from the one or more source devices to the one or more destination devices according to the one or more file loading paths.

[0110] Example 12. The apparatus of Example 11, further comprising:

[0111] a topology map generating module configured to generate a topology map between the plurality of devices in the distributed file based on transmission parameters between the devices, the transmission parameters including at least one of a transmission protocol, a network bandwidth, and a transmission rate; and

[0112] The path generation module is configured to:

[0113] Determine the current storage device for each machine learning model file;

[0114] Determine the target storage device for each machine learning model file; and

[0115] Based on the topology map, a file loading path is generated to load each machine learning model file from the current storage device to the target storage device.

[0116] Example 13. The apparatus of any of Examples 11-12, wherein the path policy table comprises at least the following:

[0117] File identifier of the machine learning model file;

[0118] The current storage device of the machine learning model file;

[0119] The target storage device for the machine learning model files; and

[0120] The file loading path of the machine learning model file.

[0121] Example 14. The apparatus of any of Examples 11-13, wherein the path generation module comprises:

[0122] The second path determination module is configured to determine the file loading path of each machine learning model file based on at least one of the transmission protocol, the network bandwidth and the transmission rate.

[0123] Example 15. The apparatus of any of Examples 11-14, wherein the path determination module comprises:

[0124] a candidate loading path determining module, configured to determine, based on the topology graph, a plurality of candidate file loading paths for loading a target machine learning model file; and

[0125] The first candidate file path determination module is configured to determine the first candidate file loading path as the file loading path of the target machine learning model file in response to the first communication protocol of the first candidate file loading path among the multiple candidate file loading paths being better than the second communication protocol of the second candidate file loading path.

[0126] Example 16. The apparatus of any of Examples 11-15, further comprising:

[0127] The path strategy table update module is configured to update the path strategy table in response to a change in the current storage device of one or more machine learning model files among the multiple machine learning model files.

[0128] Example 17. The apparatus of any of Examples 11-16, further comprising:

[0129] a scheduling information acquisition module, configured to acquire scheduling information for a target machine learning model file, wherein the scheduling information indicates that the target machine learning model file is to be scheduled to a first accelerator set;

[0130] a second file loading module configured to, in response to a first accelerator subset in the first accelerator set being in a working state and a second accelerator subset being in an idle state, load the target machine learning model file into the second accelerator subset; and

[0131] The third file loading module is configured to load the target machine learning model file from the second accelerator subset to the first accelerator subset in response to the first accelerator subset being converted to an idle state.

[0132] Example 18. The apparatus of Examples 11-17, wherein a first accelerator and a second accelerator of the plurality of accelerators are different types of accelerators.

[0133] Example 19. The apparatus of any of Examples 11-18, wherein the file loading module comprises:

[0134] A fourth file loading module is configured to load the machine learning model file from the first file system directly to the first accelerator without loading it through the host memory.

[0135] Example 20. The apparatus of any of Examples 11-19, wherein the file loading module comprises:

[0136] A file splitting module is configured to split the machine learning model file into multiple parts;

[0137] The file parallel loading module is configured to use multiple threads to load the multiple parts of the machine learning model file in parallel to the one or more destination devices.

[0138] Example 21. An electronic device comprising:

[0139] processor; and

[0140] A memory coupled to the processor, the memory having instructions stored therein, wherein when the instructions are executed by the processor, the electronic device performs actions, the actions comprising:

[0141] determining one or more source devices and one or more destination devices where the machine learning model file is stored;

[0142] Determining, according to a path policy table, one or more file loading paths for loading the machine learning model file to the one or more destination devices, wherein the path policy table is constructed according to a topological relationship between a plurality of devices in a distributed file system, and the plurality of devices include a plurality of accelerators, a plurality of file systems, and a host memory; and

[0143] The machine learning model file is loaded from the one or more source devices to the one or more destination devices according to the one or more file loading paths.

[0144] Example 22. The electronic device of Example 21, wherein the actions further comprise:

[0145] generating a topology map between the plurality of devices in the distributed file based on transmission parameters between the devices, the transmission parameters including at least one of a transmission protocol, a network bandwidth, and a transmission rate; and

[0146] For each of the multiple machine learning model files:

[0147] Determine the current storage device for each machine learning model file;

[0148] Determine the target storage device for each machine learning model file; and

[0149] Based on the topology map, a file loading path is generated to load each machine learning model file from the current storage device to the target storage device.

[0150] Example 23. The electronic device of any of Examples 21-22, wherein the path strategy table includes at least the following:

[0151] File identifier of the machine learning model file;

[0152] The current storage device of the machine learning model file;

[0153] The target storage device for the machine learning model files; and

[0154] The file loading path of the machine learning model file.

[0155] Example 24. The electronic device of any one of Examples 21-23, wherein generating a file load path for loading each machine learning model file from the current storage device to the target storage device comprises:

[0156] Based on at least one of the transmission protocol, the network bandwidth, and the transmission rate, determine a file loading path for each machine learning model file.

[0157] Example 25. The electronic device of any of Examples 21-24, wherein determining a file load path for each machine learning model file comprises:

[0158] Based on the topology graph, determining multiple candidate file loading paths for loading the target machine learning model file;

[0159] In response to the first communication protocol of the first candidate file loading path among the multiple candidate file loading paths being better than the second communication protocol of the second candidate file loading path, the first candidate file loading path is determined as the file loading path of the target machine learning model file.

[0160] Example 26. The electronic device of any of Examples 21-25, wherein the actions further comprise:

[0161] In response to a change in a current storage device of one or more machine learning model files among the multiple machine learning model files, the path strategy table is updated.

[0162] Example 27. The electronic device of any of Examples 21-26, wherein the actions further comprise:

[0163] Obtain scheduling information for a target machine learning model file, where the scheduling information indicates that the target machine learning model file is to be scheduled to a first accelerator set;

[0164] In response to a first subset of accelerators in the first accelerator set being in a working state and a second subset of accelerators being in an idle state, loading the target machine learning model file into the second subset of accelerators; and

[0165] In response to the first accelerator subset transitioning to an idle state, the target machine learning model file is loaded from the second accelerator subset to the first accelerator subset.

[0166] Example 28. The electronic device of any of Examples 21-27, wherein a first accelerator and a second accelerator of the plurality of accelerators are different types of accelerators.

[0167] Example 29. The electronic device of any of Examples 21-28, wherein loading the machine learning model file from the one or more source devices to the one or more destination devices comprises:

[0168] The machine learning model file is loaded directly from the first file system to the first accelerator without loading through the host memory.

[0169] Example 30. The electronic device of any of Examples 21-29, wherein loading the machine learning model file from the one or more source devices to the one or more destination devices comprises:

[0170] Splitting the machine learning model file into multiple parts;

[0171] The portions of the machine learning model file are loaded in parallel to the one or more destination devices using multiple threads.

[0172] Example 31. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of Examples 1 to 10.

[0173] Example 32. A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method of any one of Examples 1 to 10.

[0174] Although the present disclosure has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for loading a file, comprising: determining one or more source devices and one or more destination devices where the machine learning model file is stored; Determining one or more file loading paths for loading the machine learning model file to the one or more destination devices according to a path policy table, wherein the path policy table is constructed according to a topological relationship between multiple devices in a distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and a host memory; and According to the one or more file loading paths, the machine learning model file is loaded from the one or more source devices to the one or more destination devices.

2. The method according to claim 1, further comprising: Generate a topology map between the plurality of devices in the distributed file based on transmission parameters between the devices, the transmission parameters including at least one of a transmission protocol, a network bandwidth, and a transmission rate; as well as For each of the multiple machine learning model files: Determine the current storage device for each machine learning model file; Determine the target storage device for each machine learning model file; as well as Based on the topology map, a file loading path is generated to load each machine learning model file from the current storage device to the target storage device.

3. The method according to claim 2, wherein the path strategy table comprises at least the following items: The file identifier of the machine learning model file; The current storage device of the machine learning model file; The target storage device for the machine learning model files; and The file load path for the machine learning model file.

4. The method according to claim 2, wherein generating a file loading path for loading each machine learning model file from the current storage device to the target storage device comprises: Based on at least one of the transmission protocol, the network bandwidth, and the transmission rate, determine the file loading path of each machine learning model file.

5. The method according to claim 4, wherein determining the file loading path of each machine learning model file comprises: Based on the topology graph, determine multiple candidate file loading paths for loading the target machine learning model file; In response to a first communication protocol of a first candidate file loading path among the multiple candidate file loading paths being superior to a second communication protocol of a second candidate file loading path, the first candidate file loading path is determined as the file loading path of the target machine learning model file.

6. The method according to claim 5, further comprising: In response to a change in a current storage device of one or more machine learning model files among the multiple machine learning model files, the path strategy table is updated.

7. The method according to claim 1, further comprising: Obtain scheduling information for a target machine learning model file, wherein the scheduling information indicates that the target machine learning model file is to be scheduled to a first accelerator set; In response to a first accelerator subset in the first accelerator set being in a working state and a second accelerator subset being in an idle state, loading the target machine learning model file into the second accelerator subset; as well as In response to the first accelerator subset being converted to an idle state, the target machine learning model file is loaded from the second accelerator subset to the first accelerator subset. The method according to claim 1 , wherein a first accelerator and a second accelerator of the plurality of accelerators are accelerators of different types.

9. The method of claim 1, wherein loading the machine learning model file from the one or more source devices to the one or more destination devices comprises: The machine learning model file is loaded directly from the first file system to the first accelerator without loading through the host memory.

10. The method of claim 1, wherein loading the machine learning model file from the one or more source devices to the one or more destination devices comprises: Splitting the machine learning model file into multiple parts; The multiple portions of the machine learning model file are loaded in parallel to the one or more destination devices using multiple threads.

11. A device for loading a file, comprising: a location determination module configured to determine one or more source devices and one or more destination devices where the machine learning model file is stored; a path determination module configured to determine one or more file loading paths for loading the machine learning model file to the one or more destination devices according to a path policy table, wherein the path policy table is constructed according to a topological relationship between multiple devices in a distributed file system, and the multiple devices include multiple accelerators, multiple file systems, and a host; as well as The file loading module is configured to load the machine learning model file from the one or more source devices to the one or more destination devices according to the one or more file loading paths.

12. An electronic device comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, wherein when the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 10.

Images