Training task execution method and device, equipment and storage medium

Abstract

This application discloses a training task execution method, apparatus, device, and storage medium, belonging to the field of artificial intelligence technology. Optionally, this application relates to cloud storage technology, using cloud storage to store training data to address the needs of multi-terminal training tasks. In this application's embodiments, the mounting step of the network storage system is moved to before container creation. The time spent on the mounting step is not included in the time statistics of the task execution process, thus improving task execution efficiency. When creating the container, the mounted directory is mapped to the directory where the container is located. This directory mapping process is a localized operation, avoiding mounting failures that may occur due to network communication. The directory mapping process is almost time-efficient, resulting in high task execution efficiency and improving the success rate of task execution. Decoupling the network storage system mounting step from the task execution process avoids task execution failures caused by network storage system mounting failures, further improving the success rate of task execution.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training task execution method, apparatus, device, and storage medium. Background Technology

[0002] Container technology refers to the technique of effectively dividing the resources of a single operating system into isolated groups to balance conflicting resource usage demands among these groups. The application of container technology is becoming increasingly widespread, using containers to execute tasks and improve task processing efficiency. In the field of artificial intelligence, containers can be used to execute training tasks. Because training tasks require large amounts of training data, this data is typically stored in a network storage system. When a container executes a training task, the network storage system is mounted onto the container so that the container can retrieve the necessary training data from it. This network storage system can be implemented using cloud storage technology, that is, storing the training data in a distributed storage system.

[0003] Currently, training task execution typically involves creating a container corresponding to the task when a training task is needed. During the container creation process, a network storage system is mounted to the directory where the container is located. The container can then retrieve training data from the mounted directory to execute the training task.

[0004] In the above method, the mounting process of the network storage system is coupled into the critical path of container creation and execution. The mounting step is performed during container creation, which introduces a time delay, resulting in relatively low task execution efficiency. Furthermore, the mounting step carries the risk of failure, directly leading to task execution failure. This is especially true since the mounting step relies on network communication; poor network communication can severely impact the success rate of task execution. Summary of the Invention

[0005] This application provides a training task execution method, apparatus, device, and storage medium, which can improve task execution efficiency and success rate. The technical solution is as follows:

[0006] On the one hand, a training task execution method is provided, the method comprising:

[0007] Obtain the address of the network storage system, wherein the training data required to perform the training task is stored in the subdirectory under the root directory of the network storage system.

[0008] Mount the root directory in the address to the local directory of the current device;

[0009] In response to the creation of a container corresponding to any training task, a link is created between the directory where the container is located and the subdirectory of the root directory in the local directory;

[0010] During the execution of the training task in the container, the training data required to execute the training task is read from the subdirectory based on the link.

[0011] In some embodiments, the network storage system is a distributed storage system.

[0012] In some embodiments, the network storage system is a distributed portable operating system interface (POSIX) system.

[0013] In some embodiments, the network storage system is a blockchain system.

[0014] In some embodiments, the root directory that is already mounted in the local directory can be reused.

[0015] In some embodiments, the container corresponding to the training task is a first container; the method further includes:

[0016] In response to the creation of a second container corresponding to another training task, a link is created between the directory where the second container is located and a subdirectory of the root directory in the local directory.

[0017] On one hand, a training task execution device is provided, the device comprising:

[0018] The acquisition module is used to acquire the address of the network storage system, wherein the training data required to perform the training task is stored in the subdirectory under the root directory of the network storage system.

[0019] The mounting module is used to mount the root directory in the address to the local directory of the current device;

[0020] A creation module is used to create a link between the directory where the container is located and the subdirectories of the root directory in the local directory in response to the creation of a container corresponding to any training task.

[0021] An execution module is used to read training data required for executing the training task from the subdirectory based on the link during the execution of the training task in the container.

[0022] In some embodiments, the acquisition module is configured to perform any of the following:

[0023] In response to a current device reboot, the step of obtaining the address of the network storage system is executed;

[0024] In response to the first creation of a container corresponding to a training task in the current device, the step of obtaining the address of the network storage system is performed.

[0025] In some embodiments, the creation module is used for:

[0026] In response to the fact that the root directory includes at least two subdirectories, a target subdirectory is determined from the at least two subdirectories, the target subdirectory being the subdirectory where the training data required by the container to perform the training task is located;

[0027] Create a link between the directory containing the container and the target subdirectory.

[0028] In some embodiments, the creation module is used for:

[0029] In response to creating a container corresponding to any training task, query whether the root directory of the network storage system is contained in the local directory of the current device;

[0030] In response to the local directory containing the root directory of the network storage system, the step of creating a link between the directory where the container is located and the subdirectories of the root directory in the local directory is performed.

[0031] In some embodiments, the acquisition module and the mounting module are configured to, in response to the local directory not containing the root directory of the network storage system, perform the steps of acquiring the address of the network storage system and mounting the root directory in the address to the local directory of the current device.

[0032] In some embodiments, the apparatus further includes:

[0033] The deletion module is used to delete the link between the directory where the container is located and the subdirectories of the root directory in the local directory in response to the destruction command of any container.

[0034] In some embodiments, the deletion module is further configured to retry deleting the link in response to a failure to delete the link between the directory where the container is located and a subdirectory of the root directory in the local directory.

[0035] In some embodiments, the network storage system is a distributed storage system.

[0036] In some embodiments, the network storage system is a distributed portable operating system interface (POSIX) system.

[0037] In some embodiments, the network storage system is a blockchain system.

[0038] In some embodiments, the root directory that is already mounted in the local directory can be reused.

[0039] In some embodiments, the container corresponding to the training task is a first container; the creation module is further configured to create a link between the directory where the second container is located and the subdirectory of the root directory in the local directory in response to creating a second container corresponding to another training task.

[0040] On one hand, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement various optional implementations of the training task execution method described above.

[0041] On the one hand, a computer-readable storage medium is provided, wherein at least one computer program is stored therein, and the at least one computer program is loaded and executed by a processor to implement various optional implementations of the above-described training task execution method.

[0042] In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising one or more lines of program code stored in a computer-readable storage medium. One or more processors of an electronic device are capable of reading the one or more lines of program code from the computer-readable storage medium, and the one or more processors execute the one or more lines of program code, enabling the electronic device to perform a training task execution method according to any of the above possible embodiments.

[0043] This application provides a method for pre-mounting a network storage system. By advancing the mounting step of the network storage system to before container creation, instead of performing the mounting step during container creation, it improves task execution efficiency. Firstly, by prioritizing the mounting step, the time spent on it is not included in the task execution time statistics. Secondly, by mapping the mounted directory to the container's directory during container creation, this directory mapping process is a local operation, requiring no network communication and avoiding mounting failures that might occur due to network communication. Furthermore, the directory mapping process is minimal, resulting in high task execution efficiency and a higher success rate. Finally, by decoupling the network storage system mounting step from the task execution flow, if the network storage system mounting fails, it can be remounted without directly affecting the task execution flow, further improving the task execution success rate. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the implementation environment of a training task execution method provided in an embodiment of this application;

[0046] Figure 2 This is a flowchart of a training task execution method provided in an embodiment of this application;

[0047] Figure 3 This is a flowchart of a training task execution method provided in an embodiment of this application;

[0048] Figure 4 This is a flowchart of a training task execution method provided in an embodiment of this application;

[0049] Figure 5 This is a flowchart of a training task execution method provided in an embodiment of this application;

[0050] Figure 6 This is a flowchart of a training task execution method provided in an embodiment of this application;

[0051] Figure 7 This is a flowchart of a training task execution method provided in an embodiment of this application;

[0052] Figure 8 This is a schematic diagram of the structure of a training task execution device provided in an embodiment of this application;

[0053] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0054] Figure 10 This is a structural block diagram of a terminal provided in an embodiment of this application;

[0055] Figure 11 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0057] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. Both the first image and the second image can be images, and in some cases, they can be separate and distinct images.

[0058] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple data packets means two or more data packets.

[0059] It should be understood that the terminology used in the descriptions of the various examples in this document is for the purpose of describing the specific examples only and is not intended to be limiting.

[0060] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0061] It should also be understood that determining B based on A does not mean determining B solely based on A; it is also possible to determine B based on A and / or other information.

[0062] The following is an explanation of the terms used in this application.

[0063] Network storage systems are systems used to provide network storage services. Network storage is a method of data storage. It is typically implemented using a specialized dedicated data storage server. This server includes storage devices (such as disk arrays, CD / DVD drives, tape drives, or removable storage media) and embedded system software, thereby providing cross-platform file sharing capabilities.

[0064] This network storage system can be represented as a cloud drive. Cloud storage, also known as online hard drive, network disk, network space, or cloud drive, is a website that provides file hosting and file upload / download services. The file hosting service offered by cloud drives is mostly similar to a file transfer protocol (FTP) network service, with added basic upload / download functionality to facilitate user file access. Compared to storing files on a local hard drive, which results in poor file mobility and sharing, the advantage of cloud storage is that files are stored on the service provider's servers, allowing anyone to access them anytime, anywhere via the network. With fast broadband speeds, the access time is similar to using a local hard drive, enabling quick file access.

[0065] Containers provide lightweight virtualization to isolate a group of processes and control their resource usage. They work by offering different system views to different processes and using the Linux namespace mechanism to isolate processes. Processes within the same namespace can see, access, and communicate with each other. A namespace is a collection of resource localizations, composed of namespaces from multiple subsystems. Objects in each subsystem are localized from a globally unique instance to multiple instances, which do not interfere with each other, and one instance cannot access elements within another instance. Linux is a free and open-source UNIX-like operating system. UNIX (Uniplexed Information and Computing Service), also known as UnICS, is a multi-user, multi-process computer operating system. This container can execute tasks and is therefore also called a task container.

[0066] In everyday life, a task refers to the various purposeful activities people engage in during their daily routines, work, and leisure activities. It typically refers to tasks assigned by superiors or responsibilities undertaken. In computer systems, a task is a technical term for the basic unit of work; that is, the work, process, or procedure that the computer system needs to execute or complete.

[0067] Training Task: In the field of artificial intelligence, training tasks are performed to enable computers to make human-like responses, thereby replacing humans in handling tasks. For example, an image classification model can be trained using sample images, allowing it to classify input images and determine their type. Similarly, a text processing model can be trained using sample text, enabling the image classification model to process text using natural language processing techniques, producing summaries or translating text into other languages. Of course, the above examples are only for image classification and text processing scenarios; this training task can be applied to any scenario, such as speech recognition, image processing, video processing, and text processing. This application does not limit the scope of the examples.

[0068] The following is a brief introduction to artificial intelligence.

[0069] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0070] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0071] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0072] The implementation environment of this application is described below.

[0073] Figure 1This is a schematic diagram of an implementation environment for a training task execution method provided in an embodiment of this application. The implementation environment includes a terminal 101 and a network storage system 102. The terminal 101 is connected to the network storage system 102 via a wireless network or a wired network.

[0074] Terminal 101 is at least one of a desktop computer, smartphone, game console, tablet computer, e-book reader, and laptop computer. Terminal 101 has an application installed and running that supports the execution of training tasks. When executing training tasks, Terminal 101 creates containers and uses these containers to execute the training tasks.

[0075] The network storage system 102 includes at least one of a server, multiple servers, a cloud computing platform, and a virtualization center. The network storage system 102 provides data services for training tasks. Training data can be stored in the network storage system 102. When executing a training task, the terminal 101 needs to retrieve the training data from the network storage system 102 to perform the training task based on the training data. In this embodiment, the network storage system 102 is mounted on the terminal 101, and the location where the network storage system 102 is mounted can be called a mount point. When executing a training task, the terminal 101 accesses the mount point to access the training data stored in the network storage system 102.

[0076] Optionally, the network storage system 102 includes at least one server 1021 and at least one database 1022. The database 1022 is used to store training data. In this embodiment, the database 1022 stores training data and provides data services to at least one server 1021.

[0077] Optionally, the network storage system 102 may include at least one server 1021, which locally stores the training data and does not require data services from the database 1022.

[0078] A server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. The following sections provide detailed explanations regarding whether the network storage system 102 is a distributed system, whether this distributed system can be a blockchain system, and whether this network storage system can utilize cloud technology; these details will not be elaborated upon further here.

[0079] Those skilled in the art will know that the number of terminals 101 and servers 1021 may be more or less. For example, there may be only one terminal 101 or server 1021, or there may be dozens or hundreds of terminals 101 or servers 1021, or even more. The embodiments of this application do not limit the number or type of terminals or servers.

[0080] The aforementioned network storage system can be implemented using cloud technology. Specifically, this network storage system involves cloud storage technology within cloud technology. The following section will explain cloud technology and cloud storage.

[0081] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computing, storage, processing, and sharing. Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0082] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network to work together through application software or application interfaces to provide data storage and business access functions to the outside world.

[0083] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0084] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0085] The aforementioned network storage system 102 can be a distributed system. This distributed system uses a server cluster to distribute data storage, which can reduce the storage burden on individual servers and improve the accuracy of the stored data.

[0086] Optionally, the distributed system can be a blockchain system, and the training data required for the training task can be stored on the blockchain of the blockchain system. For example, corresponding blocks can be generated based on the training data, and these blocks, which include the training data, can then be stored on the blockchain in the form of blocks. In some embodiments, the data obtained after the training task is executed can also be stored on the blockchain of the blockchain system. The following describes the distributed system and the blockchain system.

[0087] The network storage system involved in the embodiments of the present invention can be a distributed system formed by connecting clients and multiple nodes (any form of computing device in the network, such as servers and user terminals) through network communication.

[0088] Taking a distributed system as an example, see blockchain system. Figure 2 , Figure 2This is an optional structural diagram of the distributed system 200 provided in this embodiment of the invention applied to a blockchain system. It consists of multiple nodes 201 (any form of computing device in the network, such as a server or user terminal) and clients 202. The nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In the distributed system, any machine, such as a server or terminal, can join and become a node. A node includes a hardware layer, a middleware layer, an operating system layer, and an application layer.

[0089] See Figure 2 The functions of each node in the blockchain system shown include:

[0090] 1) Routing: A basic function of nodes used to support communication between nodes.

[0091] In addition to routing capabilities, nodes can also have the following functions:

[0092] 2) Applications are deployed in the blockchain to implement specific business needs. They record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system. When other nodes successfully verify the source and integrity of the record data, they add the record data to a temporary block.

[0093] For example, the business logic implemented by the application includes:

[0094] 2.1) A wallet is used to provide the function of conducting electronic currency transactions, including initiating transactions (i.e., sending the transaction record of the current transaction to other nodes in the blockchain system; after other nodes successfully verify the transaction, they store the transaction record data in the temporary block of the blockchain as a response to acknowledge the validity of the transaction; of course, the wallet also supports querying the remaining electronic currency in the electronic currency address.

[0095] 2.2) Shared ledger, used to provide functions such as storage, query and modification of ledger data. It sends the record data of the operation on the ledger data to other nodes in the blockchain system. After the other nodes verify the validity, as a response to acknowledge the validity of the ledger data, they store the record data in a temporary block. They can also send confirmation to the node that initiated the operation.

[0096] 2.3) Smart contracts are computerized protocols that can execute the terms of a contract. They are implemented through code deployed on a shared ledger that executes when certain conditions are met. Based on actual business needs, the code is used to complete automated transactions, such as querying the logistics status of goods purchased by a buyer and transferring the buyer's electronic money to the merchant's address after the buyer signs for the goods. Of course, smart contracts are not limited to executing contracts for transactions; they can also execute contracts for processing received information.

[0097] 3) A blockchain consists of a series of blocks that are sequentially generated. Once a new block is added to the blockchain, it will not be removed. The blocks contain the data submitted by the nodes in the blockchain system.

[0098] Figure 3 This is a schematic diagram of a training task execution system architecture provided in an embodiment of this application. In this embodiment, a cloud storage system is used as an example for illustration. See also... Figure 3 The training task execution system architecture may include a network storage layer 301, a physical device layer 302, and a task container layer 303. The network storage layer 301 can be implemented as a network storage system, which can be a cloud drive. The physical device layer 302 can be implemented as a physical device with computing capabilities, i.e., an electronic device. Here, a single physical device is referred to as a node, and multiple physical devices are referred to as multiple nodes. This electronic device can be a server or a terminal. The task container layer 303 can be implemented as a container. Since in this embodiment, the container is used to execute training tasks, it can also be called a task container. The task container layer 303 can employ Kubernetes system orchestration and management. Kubernetes is an open-source container orchestration engine from Google. The task container layer 303 uses Docker technology to achieve the delivery of containerized resources and the start and stop of tasks.

[0099] In this embodiment, the cloud storage layer 301 can store training data. If the cloud storage layer 301 is mounted to a physical device (node) in the physical device layer 302, the physical device can create a task container and execute training tasks through the task container.

[0100] This physical device can create one or more task containers to execute tasks. If it includes a single task, a task container can be created to execute the task; if it includes a single task, a group of task containers can also be created to execute the task. If it includes multiple tasks, multiple task containers can be created to process the multiple tasks in parallel. Each task container is used to process one task, and a task can be executed by one or a group of task containers. Here, the task containers included in the task container are represented by pods (a C-compatible data structure). A pod is a data structure that can encapsulate a task container or a group of task containers. A physical device (node) can create one pod or multiple pods; this embodiment of the application does not limit this.

[0101] In this embodiment of the application, the electronic device can mount the cloud drive to the local physical device. When the task container is created, the subdirectories of the mount point can be mapped to the container through directory mapping. This directory mapping process can be called the binding process, which is to establish a link between the subdirectories of the mount point and the directory where the container is located, or to establish a mapping relationship between the two.

[0102] It should be noted that, Figure 3 The number of physical devices, cloud drives, and task containers shown is merely an illustrative example. The number of physical devices, cloud drives, and task containers can be one or more, and this application embodiment does not limit this.

[0103] Figure 4 This is a flowchart illustrating a training task execution method provided in an embodiment of this application. The method is applied to an electronic device, which may be a terminal or a server. See also... Figure 2 The method includes the following steps.

[0104] 401. The electronic device obtains the address of the network storage system, in which the training data required to perform the training task is stored in the subdirectory under the root directory of the network storage system.

[0105] This address is used to uniquely identify the network storage system, and the network storage system can be accessed through this address.

[0106] The root directory refers to the top-level directory of a logical drive, relative to subdirectories. For example, an electronic device may have a local hard drive with drives C and D. Double-clicking drive C takes you to its root directory, and double-clicking drive D takes you to its root directory. The same principle applies to other drives. The root directory is created when the file system is established, and its purpose is to store directory entries for subdirectories (also called folders) or files. If you think of a directory as a tree, the root is the most fundamental part of the tree, and that root is the root directory.

[0107] A subdirectory is a directory within a parent directory. The parent directory is relative to the subdirectory. For example, the root directory can contain one or more subdirectories; here, the root directory is the parent directory of these subdirectories. Let's take a subdirectory of the root directory as an example. We'll call this the first subdirectory. This first subdirectory can also contain subdirectories, which we'll call the second subdirectory. In this case, the second subdirectory is a subdirectory of the first subdirectory, and the first subdirectory is the parent directory of the second subdirectory.

[0108] Training data refers to the data required for training tasks. For example, the training data may include sample data, initial model data, or model configuration data. The specific content of the training data can be set by relevant technical personnel according to their needs, and this application embodiment does not limit it.

[0109] In this embodiment, the network storage system stores training data in its file system, which is located in a subdirectory of the file system's root directory. The electronic device can mount the network storage system to its local directory, and then access the mounted directory to retrieve the training data during subsequent training tasks. Therefore, the electronic device can first obtain the address of the network storage system to be mounted, and then access its file system through that address to mount its root directory locally.

[0110] 402. The electronic device mounts the root directory at that address to the local directory of the current device.

[0111] Mounting refers to the process by which the operating system makes computer files and directories on a network storage device (such as a hard drive, CD-ROM, or shared resource) accessible to users through the computer's file system. In other words, the network storage device is mounted to the computer, allowing users to access the computer files or directories on the network storage device through the computer's file system.

[0112] The file system, also known as the information management module or file management module, is primarily responsible for managing software resources. All software resources are stored as files on storage media, and information is transferred within the computer in units of files. Therefore, a file is defined as a collection of related information elements. All files in the computer form a file system, which, although sharing the same name as a management module in the operating system, is usually distinguishable due to their different contexts and applications.

[0113] Cloud storage pre-mounting: During AI training, the training data used by the business is typically large, and the local hard drive space of the device (the device executing the training task) is usually insufficient. Therefore, it's common practice to mount a network storage system (i.e., a cloud drive) onto the device. This network storage system is used to store the business's training data. Therefore, mounting the cloud drive before the business starts training—that is, pre-mounting the disk operation to the task startup phase—is called cloud storage pre-mounting.

[0114] A local directory is a directory on the current device, which is essentially a path on the current device. Searching along this path does not require an internet connection.

[0115] In step 402, the electronic device performs a mounting step, also known as a pre-mounting process, before creating a container to perform the training task. During mounting, the root directory of the network storage system's address is mounted to the local directory of the current device. This way, when it is necessary to access the network storage system later, the data or directories on the network storage system can be accessed by accessing the mounted location.

[0116] In steps 401 and 402 above, the electronic device has already performed the mounting step of the network storage system. At this time, there may not be any training tasks to be executed yet. When a training task needs to be executed later, the electronic device can execute steps 403 and 404 below to retrieve training data from the network storage system by accessing the mounted location to support the execution of the training task.

[0117] 403. In response to creating a container for any training task, the electronic device creates a link between the directory containing the container and a subdirectory of the root directory in the local directory.

[0118] When an electronic device needs to perform a training task, it creates a container for that task and executes the training task within the container. This container needs to acquire training data and then performs training based on that data.

[0119] The root directory of the network storage system is already mounted in the local directory of this electronic device. Since the root directory is already mounted to this electronic device, its subdirectories are naturally also mounted. Training data can be stored in the subdirectories under this root directory, so if the container needs to access the training data, it can simply access the subdirectories under the mounted root directory.

[0120] Considering the operating characteristics of this container, it typically extracts data from the directory where the container is located during runtime, while the training data is located in the local directory of the electronic device. Therefore, it is necessary to map the two directories so that the container can extract the required data from the local directory through the mapping.

[0121] 404. During the execution of the training task in the container, the electronic device reads the training data required to perform the training task from the subdirectory based on the link.

[0122] Once this link is established, that is, the directory mapping is set up, the electronic device can read the training data from the local directory based on this link when the container is running. In this way, the extraction of the training data does not require connecting to the network to mount the network storage system to the directory where the container is located. The container only needs to perform local read and write operations.

[0123] This application provides a method for pre-mounting a network storage system. By advancing the mounting step of the network storage system to before container creation, instead of performing the mounting step during container creation, it improves task execution efficiency. Firstly, by prioritizing the mounting step, the time spent on it is not included in the task execution time statistics. Secondly, by mapping the mounted directory to the container's directory during container creation, this directory mapping process is a local operation, requiring no network communication and avoiding mounting failures that might occur due to network communication. Furthermore, the directory mapping process is minimal, resulting in high task execution efficiency and a higher success rate. Finally, by decoupling the network storage system mounting step from the task execution flow, if the network storage system mounting fails, it can be remounted without directly affecting the task execution flow, further improving the task execution success rate.

[0124] Figure 5 This is a flowchart of a training task execution method provided in an embodiment of this application. See also... Figure 5 The method includes the following steps.

[0125] 501. In response to a current device restart or the first creation of a container corresponding to a training task in the current device, the electronic device obtains the address of a network storage system, in which the subdirectories under the root directory of the network storage system store the training data required to perform the training task.

[0126] Because the training data is quite large, it is stored through a network storage system. Then, when an electronic device needs to use the training data to perform a training task, the network storage system can be mounted on the electronic device. In this way, the container on the electronic device can obtain the training data from the network storage system through the mounted location.

[0127] In some embodiments, the training data is stored in a subdirectory of the root directory of the network storage system. In some embodiments, the root directory may include at least one subdirectory. The at least one subdirectory may be one subdirectory or multiple subdirectories.

[0128] In some embodiments, different training tasks may require different training data, and the different training data may be stored in different subdirectories. When the container executes a training task subsequently, it can retrieve the training data from the subdirectory containing the training data required for that specific training task.

[0129] Step 501 involves obtaining the address of the network storage system. In step 501, the electronic device obtains the address of the network storage system when it restarts or when the container corresponding to the training task is created for the first time. The following explanation addresses these two scenarios separately.

[0130] In the first scenario, when the device restarts, it can fulfill some business requirements, such as performing training tasks. At this time, the electronic device obtains the address of the network storage system and performs the subsequent mounting steps. Placing the mounting steps of the network storage system before creating the container to execute the training task ensures that the latency caused by the mounting process and failures caused by poor network conditions will not affect the subsequent training task execution process, which can greatly improve the task execution efficiency and success rate.

[0131] In the second scenario, when the device creates a container for the first time to perform a training task, it can perform the mounting step of the network storage system. In this way, the mounting step is completed when the current training task is executed. When there are other training tasks in the future, the mounted network storage system will already be available, and the mounted network storage system can be directly applied, which improves the efficiency and success rate of subsequent task execution.

[0132] In some embodiments, the electronic device may have already undergone a mounting step previously. Therefore, the electronic device can also determine whether a network storage system is mounted in the local directory of the current device in response to a device restart or the first creation of a container corresponding to a training task in the current device. If a network storage system is mounted in the local directory of the current device, the electronic device can skip steps 501 and 502 and directly execute the subsequent step 503. If no network storage system is mounted in the local directory of the current device, the electronic device can execute steps 501 and 502.

[0133] 502. The electronic device mounts the root directory at that address to the local directory of the current device.

[0134] Once an electronic device obtains the address of a network storage system, it can access the file system on that network storage system through that address, and naturally, it can also access the file directory and the files within that directory stored on the network storage system.

[0135] The mounting process can be a mount process. The electronic device itself has its own file system. In this process, the network storage system is mounted to an existing directory in the electronic device's own file system. In this way, the network storage system is connected to the electronic device, and accessing the mounted directory will access the network storage system.

[0136] In some embodiments, the network storage system may provide a target interface for providing data operation services. The address of the network storage system obtained by the electronic device is also the identification information of this interface. Through the identification information of the target interface, the electronic device can access the target interface and access the file system in the network storage system through the target interface.

[0137] Specifically, the electronic device can access the target interface to obtain the root directory of the network storage system. The root directory may include one or more subdirectories. The electronic device then establishes a link between the root directory and the local directory. When we access the local directory, we can access the root directory of the network storage system through the link.

[0138] In some embodiments, the network storage system can be a distributed storage system, that is, the network storage system can consist of multiple computer devices. For these multiple computer devices, the distributed storage system can maintain an address list of the computer devices. The distributed storage system can use this address list to map the directories of these multiple computer devices to the root directory provided by the target interface. Through this mapping, the distributed storage system can determine which directories of which computer devices in the address list need to retrieve data. The address list of the distributed storage system can be user-invisible data; that is, the distributed storage system maintains an address list, and the electronic device accessing the target interface accesses a general directory obtained by the distributed storage system after aggregation or directory mapping, without needing the electronic device to retrieve relevant data from individual computer devices based on the address list.

[0139] In one specific possible embodiment, the network storage system is a distributed portable operating system interface (POSIX) system. Accordingly, the target interface can be a POSIX file operation interface. For example, the network storage system could be a storage system such as Ceph or GlusterDS. For this type of storage system, developers need to prepare for the deployment and setup of distributed storage in advance, which will not be detailed here.

[0140] 503. In response to creating a first container corresponding to the first training task, the electronic device creates a link between the directory where the first container is located and the subdirectory of the root directory in the local directory.

[0141] Electronic devices may receive training tasks and use training data to perform these tasks. When performing a training task, the electronic device can create a container and execute the task based on that container. The training dataset required for the training task is stored in the aforementioned network storage system, and the container needs to retrieve the necessary training data from this system during the execution of the training task.

[0142] Through steps 501 and 502 above, the electronic device has mounted the network storage system to the local directory. Therefore, when creating a container, directory mapping can be performed between the container's directory and the local directory to guide the container to access the local directory to obtain the required training data.

[0143] In some embodiments, different training data can be stored in different subdirectories, with the root directory including one or more subdirectories. If the root directory contains only one subdirectory, the electronic device can create a link between the directory containing the first container and that subdirectory within the local directory.

[0144] If the root directory includes at least two subdirectories, the electronic device can determine which subdirectory contains the training data required by the first container, and then access that subdirectory to retrieve the training data. Specifically, in response to the root directory including at least two subdirectories, the electronic device can determine a target subdirectory from these at least two subdirectories. This target subdirectory is the subdirectory containing the training data required by the first container to perform the training task. The electronic device can then create a link between the directory containing the first container and the target subdirectory.

[0145] When creating a link, this electronic device can first obtain the directory containing the first container and the subdirectories of the root directory, and then create a link between them. This directory mapping process is a local directory read / write operation, which does not require an internet connection and can achieve a success rate of up to 100%.

[0146] In some embodiments, when an electronic device needs to create a container to perform a training task, it can first check whether the root directory of the network storage system is already mounted in its local directory. If so, it performs the directory mapping process shown in step 503. Specifically, in response to creating a first container corresponding to a first training task, the electronic device can query whether the local directory of the current device contains the root directory of the network storage system. In response to the local directory containing the root directory of the network storage system, the electronic device can perform the step of linking the directory where the first container is created with the subdirectories of the root directory in the local directory.

[0147] Based on the above embodiments of querying whether there is a mount point and querying subdirectories, a specific possible embodiment is provided. The electronic device can respond to the fact that the local directory contains the root directory of the network storage system, and the root directory includes at least two subdirectories. From the at least two subdirectories, the target subdirectory is determined. The target subdirectory is the subdirectory where the data required by the container to perform the training task is located. Then the electronic device creates a link between the directory where the container is located and the target subdirectory.

[0148] In the process of checking whether the root directory of the network storage system is already mounted in the local directory, there is another possibility: the local directory may not contain the root directory of the network storage system. In this case, the electronic device needs to re-mount the system, i.e., execute steps 501 and 502. Specifically, in response to the fact that the local directory does not contain the root directory of the network storage system, the electronic device can execute the steps of obtaining the address of the network storage system and mounting the root directory at that address to the local directory of the current device.

[0149] This query or detection step ensures that if problems occur in the preceding mounting steps or at the mounting point, appropriate measures can be taken to remount the data to guarantee the acquisition of training data and the success rate of task execution.

[0150] It should be noted that step 503 is the process of creating a link between the directory where the container is located and the subdirectory of the root directory in the local directory in response to the creation of a container corresponding to any training task. Here, only the first container corresponding to the first training task is used as an example for illustration. It can also be the second container corresponding to the second training task. This application embodiment does not limit this.

[0151] 504. During the execution of the first training task in the first container, the electronic device reads the training data required to execute the first training task from the subdirectory based on the link.

[0152] The electronic device creates links between the container's directory and its subdirectories. During the execution of the first training task in the first container, when it reaches the training data acquisition step, it can use these links to navigate from the container's directory to a subdirectory. This subdirectory is connected to a subdirectory in the network storage system, allowing the container to retrieve the corresponding training data from the network storage system. Once the first container has acquired the training data, it can continue executing the first training task. The training process itself will not be elaborated upon here.

[0153] 505. In response to the destruction command of the first container, the electronic device deletes the link between the directory where the first container is located and the subdirectory of the root directory in the local directory.

[0154] After the first container completes the first training task, it can be destroyed. When the electronic device receives the destruction command for the first container, it can delete the link created in step 503 above, but not delete the root directory where the network storage system is mounted. In this way, the network storage system is still mounted in the local directory of the electronic device, and the mount point can continue to be used for subsequent training tasks without having to perform the mounting step again.

[0155] It should be noted that, in this embodiment, the network storage system is mounted to the local directory of the electronic device, rather than being directly mounted to the container's directory when the container is created. Therefore, when the container is destroyed, the mount point is not destroyed, eliminating the need to perform a mounting step for each training task. This significantly reduces the number of mounting operations and improves task execution efficiency.

[0156] The mounting step-before-mounting method provided in this application embodiment enables the reuse of the root directory already mounted in the local directory. When each container uses the end root directory to destroy the container, only the link created during the directory mapping process is destroyed. This link can naturally be created for other containers without repeated mounting. Furthermore, this mounting step-before-mounting method also supports the parallelism of multiple training tasks; that is, it can create links between subdirectories under the root directory and the directories of multiple containers, thereby executing multiple training tasks simultaneously.

[0157] In some embodiments, regarding the link deletion step described above, if the container is destroyed and the link is also deleted, the container can be considered successfully destroyed. If the link is not successfully deleted, the electronic device can try to delete it again. Specifically, in response to the failure to delete the link between the directory where the container is located and the subdirectory of the root directory in the local directory, the electronic device retryes deleting the link.

[0158] In other words, the destruction process mainly involves deleting the links to the bind mapped directories, without performing the umount operation on the electronic device mount point. Once the link is successfully deleted, the container is destroyed. If the link deletion fails, a retry operation is performed. Since deleting links is a local file system operation, it almost never fails. Adding a retry ensures 100% success.

[0159] 506. In response to creating a second container corresponding to the second training task, the electronic device queries whether the root directory of the network storage system is contained in the local directory of the current device.

[0160] In step 506, the electronic device needs to perform another training task: a second training task. The training data required for this second training task may be the same as or different from the training data required for the first training task. For this second training task, the electronic device can create a second container and perform the second training task based on the second container.

[0161] The electronic device can first query whether the root directory of the network storage system is already mounted in the local directory. If so, steps 507 and 508 can be executed. If not, the electronic device still needs to perform the mounting step. Considering that no umount operation is performed when the container is destroyed in this embodiment, the query result may be yes. The root directory of the already mounted network storage system can be reused.

[0162] 507. In response to the local directory containing the root directory of the network storage system, the electronic device creates a link between the directory containing the second container and a subdirectory of the root directory in the local directory.

[0163] 508. During the execution of the second training task in the second container, the electronic device reads the training data required to execute the second training task from the subdirectory based on the link.

[0164] 509. In response to the destruction command of the second container, the electronic device deletes the link between the directory where the second container is located and the subdirectory of the root directory in the local directory.

[0165] Steps 507 to 509 are the same as steps 503 to 505 above, and will not be elaborated on here.

[0166] Regarding the pre-mounting method provided above, the mounting steps, directory mapping, and container destruction process are explained in detail below. Taking a network storage system as an example, the method provided in this application embodiment can be used in scenarios where business operations require GPU (Graphics Processing Unit) computing power for AI training. The training data stored in the network file system needs to be pre-mounted. The operation of mounting the network disk in the task startup process is moved forward, that is, the operation of mounting the network disk is placed before the task starts. The task startup path no longer includes the network disk mounting operation, thus reducing the length of the task startup path. Moreover, the mounting process is decoupled from the task startup process, thereby avoiding the task failure to start due to mounting failure.

[0167] The cloud drive pre-mounting operation, also known as pre-disk mounting, occurs during the device startup phase (i.e., device restart) or when the first task container on the device starts. In these two phases, the cloud drive root directory is mounted, making the cloud drive directory visible on the device (corresponding to steps 401 and 402, or steps 501 and 502). The training data used by the business is stored in some subdirectories of the mounted cloud drive root directory. When the business task container starts, directory mapping maps the subdirectories containing the training data to the task container, allowing the task container to access the training data through file system read and write operations. Pre-mounting the cloud drive significantly improves the delivery efficiency of task containers, increases the success rate of AI training processes, and reduces training task latency.

[0168] like Figure 6 As shown, when the physical device restarts (601) or the task container is created (602), the process proceeds to step 603, which checks if the physical device already has a mounted root directory for the cloud storage. If such a mount point already exists, step 605, the directory binding mapping operation, is executed, corresponding to step 503 above, mapping the subdirectories of the training data into the container and completing the creation of the task container. If no mount point exists yet, step 604, the cloud storage root directory mounting operation, is executed first, mounting the root directory to the physical device. This process communicates with the cloud storage to complete the mount process, corresponding to steps 501 and 502 above.

[0169] Combination Figure 3 and Figure 6The mounting process from cloud storage to physical devices is called the `mount` process. This process occurs during device restart or the initial creation of a task container on the device. This `mount` involves network communication. Once the mount is successful, subsequent container training uses the same mount point to continuously reuse training data. This is primarily because the mounted directory is the root directory of the storage system, and the training data are all subdirectories of the root directory, allowing for continuous reuse of existing root directory mount points. The process from the physical device to the produced task container is the `bind` directory mapping stage. This operation occurs during the task container creation process. Because it's a local directory operation, the local file system guarantees 100% success, and the operation is instantaneous, taking less than milliseconds. Even during the task container startup process, it doesn't affect the startup time or success rate. The key point of the `bind` directory is the mapping of the training data subdirectories, which doesn't require network communication with the storage system and can be completed solely within the local file system.

[0170] like Figure 7 As shown, during the container destruction process, the task container destruction 701 mainly involves deleting the link of the bind mapped directory 702, without performing the umount operation on the physical device mount point. Then, it can be determined whether the deletion was successful 703. If the link is successfully deleted, the container is destroyed 704. If the link deletion is unsuccessful, a retry operation is performed. Since deleting the link is a local file system operation, it almost never fails. Adding a retry ensures 100% success.

[0171] As can be seen from the above description of the embodiments, the method provided in this application embodiment may include the following beneficial effects:

[0172] First, mounting the cloud drive before the task container starts avoids task startup failures caused by cloud drive mounting failures, improving the success rate of task startup and execution. Second, the time spent on cloud drive mounting is not included in the task container startup process time statistics, reducing the time spent in the task startup phase. Third, mounting training data through directory mapping eliminates network communication during the mounting operation, ensuring near 100% availability. Fourth, pre-mounting the root directory and mapping subdirectories within the cluster, especially in scenarios with multiple task containers on a single device, not only reduces cloud drive mounting for containers but also lowers the load on the storage cluster, benefiting its stability. Fifth, during the container destruction process after task completion, only the directory mapping is reclaimed; network communication is not required to unmount the cloud drive, resulting in fast container destruction and a high success rate.

[0173] This application provides a method for pre-mounting a network storage system. By advancing the mounting step of the network storage system to before container creation, instead of performing the mounting step during container creation, it improves task execution efficiency. Firstly, by prioritizing the mounting step, the time spent on it is not included in the task execution time statistics. Secondly, by mapping the mounted directory to the container's directory during container creation, this directory mapping process is a local operation, requiring no network communication and avoiding mounting failures that might occur due to network communication. Furthermore, the directory mapping process is minimal, resulting in high task execution efficiency and a higher success rate. Finally, by decoupling the network storage system mounting step from the task execution flow, if the network storage system mounting fails, it can be remounted without directly affecting the task execution flow, further improving the task execution success rate.

[0174] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0175] Figure 8 This is a schematic diagram of the structure of a training task execution device provided in an embodiment of this application. See also... Figure 8 The device includes:

[0176] The acquisition module 801 is used to acquire the address of the network storage system, which stores the training data required to perform the training task in the subdirectory under the root directory of the network storage system.

[0177] Mount module 802 is used to mount the root directory at this address to the local directory of the current device;

[0178] Create module 803, which is used to create a link between the directory where the container is located and the subdirectory of the root directory in the local directory in response to the creation of a container corresponding to any training task.

[0179] Execution module 804 is used to read the training data required to perform the training task from the subdirectory based on the link during the execution of the training task in the container.

[0180] In some embodiments, the acquisition module 801 is configured to perform any of the following:

[0181] In response to a current device reboot, execute the step of obtaining the address of the network storage system;

[0182] In response to the first creation of a container corresponding to a training task in the current device, the step of obtaining the address of the network storage system is performed.

[0183] In some embodiments, the creation module 803 is used for:

[0184] In response to the fact that the root directory includes at least two subdirectories, a target subdirectory is determined from the at least two subdirectories. The target subdirectory is the subdirectory where the training data required by the container to perform the training task is located.

[0185] Create a link between the directory containing the container and the target subdirectory.

[0186] In some embodiments, the creation module 803 is used for:

[0187] In response to the creation of a container for any training task, check whether the local directory of the current device contains the root directory of the network storage system;

[0188] In response to the local directory containing the root directory of the network storage system, the step of creating a link between the directory where the container is located and the subdirectories of the root directory in the local directory is performed.

[0189] In some embodiments, the acquisition module 801 and the mounting module 802 are configured to, in response to the fact that the local directory does not contain the root directory of the network storage system, perform the steps of acquiring the address of the network storage system and mounting the root directory at that address to the local directory of the current device.

[0190] In some embodiments, the device further includes:

[0191] The delete module is used to delete the links between the directory where the container is located and the subdirectories of the root directory in the local directory in response to the destruction command of any container.

[0192] In some embodiments, the deletion module is further configured to retry deleting the link in response to a failure to delete the link between the directory where the container is located and a subdirectory of the root directory in the local directory.

[0193] In some embodiments, the network storage system is a distributed storage system.

[0194] In some embodiments, the network storage system is a distributed portable operating system interface (POSIX) system.

[0195] In some embodiments, the network storage system is a blockchain system.

[0196] In some embodiments, the root directory that is already mounted in the local directory can be reused.

[0197] In some embodiments, the container corresponding to the training task is a first container; the creation module 803 is further configured to create a link between the directory where the second container is located and the subdirectory of the root directory in the local directory in response to creating a second container corresponding to another training task.

[0198] The apparatus provided in this application improves task execution efficiency by moving the mounting step of the network storage system before container creation, rather than performing the mounting step during container creation. Firstly, by moving the mounting step forward, the time spent on the mounting step is not included in the task execution time statistics. Secondly, by mapping the mounted directory to the container's directory during container creation, this directory mapping process is a local operation, requiring no network communication and avoiding mounting failures that might occur due to network communication. Furthermore, the directory mapping process is minimal, resulting in high task execution efficiency and a higher success rate. Finally, by decoupling the network storage system mounting step from the task execution process, if the network storage system mounting fails, it can be remounted without directly affecting the task execution process, further improving the task execution success rate.

[0199] It should be noted that the training task execution device provided in the above embodiments is only illustrated by the division of the above functional modules when executing training tasks. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the training task execution device can be divided into different functional modules to complete all or part of the functions described above. In addition, the training task execution device and the training task execution method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0200] Figure 9 This is a schematic diagram of the structure of an electronic device 900 provided in an embodiment of this application. The electronic device 900 can vary considerably due to differences in configuration or performance. It can include one or more Central Processing Units (CPUs) 901 and one or more memories 902. The memory 902 stores at least one computer program, which is loaded and executed by the processor 901 to implement the training task execution methods provided in the various method embodiments described above. The electronic device can also include other components for implementing device functions. For example, the electronic device can also have wired or wireless network interfaces and input / output interfaces for input and output. Further details are not elaborated upon in this embodiment.

[0201] The electronic device in the above method embodiments can be implemented as a terminal. For example, Figure 10 This is a structural block diagram of a terminal provided in an embodiment of this application. The terminal 1000 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, laptop computer, or desktop computer. The terminal 1000 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0202] Typically, terminal 1000 includes a processor 1001 and a memory 1002.

[0203] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a GPU, which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0204] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the training task execution method provided in the method embodiments of this application.

[0205] In some embodiments, the terminal 1000 may also optionally include a peripheral device interface 1003 and at least one peripheral device. The processor 1001, memory 1002, and peripheral device interface 1003 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, a positioning assembly 1008, and a power supply 1009.

[0206] Peripheral device interface 1003 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002 and peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1001, memory 1002 and peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0207] The radio frequency (RF) circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1004 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0208] Display screen 1005 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1005 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1001 for processing. In this case, display screen 1005 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1005, disposed on the front panel of terminal 1000; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 1000 or in a folded design; in still other embodiments, display screen 1005 may be a flexible display screen, disposed on a curved or folded surface of terminal 1000. Furthermore, display screen 1005 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0209] The camera assembly 1006 is used to acquire images or videos. Optionally, the camera assembly 1006 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0210] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1001 for processing, or input to the radio frequency circuit 1004 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1000. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0211] The positioning component 1008 is used to determine the current geographical location of the terminal 1000 in order to enable navigation or LBS (Location Based Service). The positioning component 1008 can be a positioning component based on the US GPS (Global Positioning System), China's BeiDou system, or Russia's Galileo system.

[0212] Power supply 1009 is used to power the various components in terminal 1000. Power supply 1009 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1009 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0213] In some embodiments, the terminal 1000 further includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: an accelerometer 1011, a gyroscope 1012, a pressure sensor 1013, a fingerprint sensor 1014, an optical sensor 1015, and a proximity sensor 1016.

[0214] Accelerometer 1011 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 1000. For example, accelerometer 1011 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1001 can control display screen 1005 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1011. Accelerometer 1011 can also be used for games or for acquiring user motion data.

[0215] The gyroscope sensor 1012 can detect the orientation and rotation angle of the terminal 1000. The gyroscope sensor 1012, in conjunction with the accelerometer sensor 1011, can collect 3D motion data from the user on the terminal 1000. Based on the data collected by the gyroscope sensor 1012, the processor 1001 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0216] The pressure sensor 1013 can be disposed on the side bezel of the terminal 1000 and / or on the lower layer of the display screen 1005. When the pressure sensor 1013 is disposed on the side bezel of the terminal 1000, it can detect the user's grip signal on the terminal 1000, and the processor 1001 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1013. When the pressure sensor 1013 is disposed on the lower layer of the display screen 1005, the processor 1001 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0217] The fingerprint sensor 1014 is used to collect a user's fingerprint. The processor 1001 identifies the user based on the fingerprint collected by the fingerprint sensor 1014, or vice versa. When the user's identity is identified as trusted, the processor 1001 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1014 can be located on the front, back, or side of the terminal 1000. When the terminal 1000 has physical buttons or a manufacturer's logo, the fingerprint sensor 1014 can be integrated with the physical buttons or manufacturer's logo.

[0218] An optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 based on the ambient light intensity collected by the optical sensor 1015. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is decreased. In another embodiment, the processor 1001 can also dynamically adjust the shooting parameters of the camera assembly 1006 based on the ambient light intensity collected by the optical sensor 1015.

[0219] The proximity sensor 1016, also known as a distance sensor, is typically mounted on the front panel of the terminal 1000. The proximity sensor 1016 is used to detect the distance between the user and the front of the terminal 1000. In one embodiment, when the proximity sensor 1016 detects that the distance between the user and the front of the terminal 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from a screen-on state to a screen-off state; when the proximity sensor 1016 detects that the distance between the user and the front of the terminal 1000 is gradually increasing, the processor 1001 controls the display screen 1005 to switch from a screen-off state to a screen-on state.

[0220] Those skilled in the art will understand that Figure 10 The structure shown does not constitute a limitation on terminal 1000 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0221] The electronic device in the above method embodiments can be implemented as a server. For example, Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It can include one or more Central Processing Units (CPUs) 1101 and one or more memories 1102. The memories 1102 store at least one computer program, which is loaded and executed by the processor 1101 to implement the training task execution methods provided in the various method embodiments described above. Of course, the server can also have wired or wireless network interfaces and input / output interfaces for input / output. The server can also include other components for implementing device functions, which will not be elaborated here.

[0222] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including at least one computer program, which is executable by a processor to perform the training task execution method in the above embodiments. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0223] In an exemplary embodiment, a computer program product or computer program is also provided, comprising one or more lines of program code stored in a computer-readable storage medium. One or more processors of an electronic device are capable of reading the one or more lines of program code from the computer-readable storage medium, and the one or more processors execute the one or more lines of program code, enabling the electronic device to perform the training task execution method described above.

[0224] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0225] It should be understood that determining B based on A does not mean determining B solely based on A; it also means determining B based on A and / or other information.

[0226] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0227] The above description is only an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for executing a training task, characterized in that, The method includes: Obtain the address of the network storage system, wherein the training data required to perform the training task is stored in the subdirectory under the root directory of the network storage system. Mount the root directory in the address to the local directory of the current device; In response to the creation of a container corresponding to any training task, a link is created between the directory where the container is located and the subdirectory of the root directory in the local directory; During the execution of the training task in the container, the training data required to execute the training task is read from the subdirectory based on the link.

2. The method according to claim 1, characterized in that, The acquisition of the network storage system address includes any of the following: In response to a current device reboot, the step of obtaining the address of the network storage system is executed; In response to the first creation of a container corresponding to a training task in the current device, the step of obtaining the address of the network storage system is performed.

3. The method according to claim 1, characterized in that, The step of responding to the creation of a container corresponding to any training task, and creating a link between the directory where the container is located and a subdirectory of the root directory in the local directory, includes: In response to the fact that the root directory includes at least two subdirectories, a target subdirectory is determined from the at least two subdirectories, the target subdirectory being the subdirectory where the training data required by the container to perform the training task is located; Create a link between the directory containing the container and the target subdirectory.

4. The method according to claim 1, characterized in that, The step of responding to the creation of a container corresponding to any training task, and creating a link between the directory where the container is located and a subdirectory of the root directory in the local directory, includes: In response to creating a container corresponding to any training task, query whether the root directory of the network storage system is contained in the local directory of the current device; In response to the local directory containing the root directory of the network storage system, the step of creating a link between the directory where the container is located and the subdirectories of the root directory in the local directory is performed.

5. The method according to claim 4, characterized in that, The method further includes; If the local directory does not contain the root directory of the network storage system, the steps of obtaining the address of the network storage system and mounting the root directory in the address to the local directory of the current device are executed.

6. The method according to claim 1, characterized in that, The method further includes: In response to a destroy command for any container, delete the link between the directory where the container is located and the subdirectories of the root directory in the local directory.

7. The method according to claim 6, characterized in that, The method further includes: In response to the failure to delete the link between the directory where the container is located and the subdirectory of the root directory in the local directory, the link deletion is retried.

8. A training task execution device, characterized in that, The device includes: The acquisition module is used to acquire the address of the network storage system, wherein the training data required to perform the training task is stored in the subdirectory under the root directory of the network storage system. The mounting module is used to mount the root directory in the address to the local directory of the current device; A creation module is used to create a link between the directory where the container is located and the subdirectories of the root directory in the local directory in response to the creation of a container corresponding to any training task. An execution module is used to read training data required for executing the training task from the subdirectory based on the link during the execution of the training task in the container.

9. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the training task execution method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the training task execution method as described in any one of claims 1 to 7.

Images