File security acquisition method and device, medium and electronic equipment

Abstract

The embodiment of the present disclosure provides a file security acquisition method, device, medium and electronic equipment, and relates to the technical field of data processing. The method comprises the following steps: obtaining a model training request about a preset task initiated by a requestor for a to-be-trained model, the model training request carrying a training configuration parameter, verifying the model training request, confirming a container group and a resource access key corresponding to the training configuration parameter after the verification is passed, and allocating computing resource and data resource required by the container group; and accessing training corpus in the data resource according to the resource access key in the container group, so as to train the to-be-trained model according to the computing resource and the training corpus. The file security acquisition method adopted by the present disclosure controls the access authority of data through the resource access key, so that the model training is carried out under the condition of ensuring data security, the risk of data and privacy leakage is reduced, and the effect of model training is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of data processing, and more particularly, to a text security acquisition method, a text security acquisition device, a computer readable storage medium and an electronic device. BACKGROUND

[0002] This section is intended to provide background information to facilitate a better understanding of embodiments of the present disclosure. Information in this section is not admitted to be prior art.

[0003] Federated machine learning (Federated machine learning / Federated Learning) is a new machine learning paradigm. In the process of machine learning, participants can jointly model with the help of other party data, and each party does not need to share data resources, that is, joint training is performed without data leaving the local.

[0004] Federated machine learning requires each training node (which can represent an individual or an institution) to encrypt the trained model and transmit it to the center server for aggregation, which increases the additional computing cost. When the model is large, it is difficult to meet the network bandwidth and network stability required by data transmission, and there are problems such as difficulty in unifying the training environment and training method of each training node. SUMMARY

[0005] To this end, the present disclosure proposes a text security acquisition method to allocate data resources and computing power resources of each cluster through containerization technology, and to ensure data security by controlling data access rights, so as to provide a safe and controllable data reading, model training and model sharing environment without changing the data ownership.

[0006] In this context, embodiments of the present disclosure aim to provide a text security acquisition method, a text security acquisition device, a computer readable storage medium and an electronic device.

[0007] In a first aspect of the embodiments of the present disclosure, a file secure acquisition method is provided, including: obtaining a model training request initiated by a requestor for a to-be-trained model with respect to a preset task, the model training request carrying a training configuration parameter; verifying the model training request, and confirming a container group and a resource access key corresponding to the training configuration parameter after verification; allocating computing resource and data resource required by the container group, the data resource containing training corpus; accessing the training corpus in the data resource according to the resource access key in the container group, to train the to-be-trained model according to the computing resource and the training corpus for the preset task; and if the to-be-trained model meets a training completion requirement, taking the to-be-trained model as a target model, and feeding back a file of the target model to the requestor.

[0008] In a second aspect of the embodiments of the present disclosure, a file secure acquisition apparatus is provided, including: a request acquisition module, configured to obtain a model training request initiated by a requestor for a to-be-trained model with respect to a preset task, the model training request carrying a training configuration parameter; a container group and key confirmation module, configured to verify the model training request, and confirm a container group and a resource access key corresponding to the training configuration parameter after verification; a resource allocation module, configured to allocate computing resource and data resource required by the container group, the data resource containing training corpus; a training module, configured to access the training corpus in the data resource according to the resource access key in the container group, to train the to-be-trained model according to the computing resource and the training corpus for the preset task; and a file feedback module, configured to, if the to-be-trained model meets a training completion requirement, take the to-be-trained model as a target model, and feed back a file of the target model to the requestor.

[0009] In a third aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the file secure acquisition method as described above.

[0010] In a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the file secure acquisition method as described above.

[0011] According to the technical scheme of the embodiment of the present disclosure, on the one hand, the scheduling of the model training task is implemented by the container group to implement the model training, so as to standardize the training environment and training method of the model, and improve the effect of the model. On the other hand, the data resources and computing resources required for model training are allocated to the container group, and the access permission of the data resources is controlled through the resource access key, so as to ensure the data security of each cluster, and provide a safe and controllable data reading, model training and model sharing environment without changing the data ownership. On the other hand, since the central server of the traditional federated training is abandoned, the model does not need to be encrypted and transmitted, and the computing cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0013] Figure 1 A schematic diagram of federated learning in the related art is schematically shown;

[0014] Figure 2 A schematic diagram of an exemplary implementation environment according to some embodiments of the present disclosure is schematically shown;

[0015] Figure 3 A flowchart of a file secure acquisition method according to some embodiments of the present disclosure is schematically shown;

[0016] Figure 4 A schematic diagram of a file secure acquisition method according to some embodiments of the present disclosure is schematically shown;

[0017] Figure 5 A schematic diagram of cluster access according to some embodiments of the present disclosure is schematically shown;

[0018] Figure 6 A schematic diagram of cluster resource decoupling according to some embodiments of the present disclosure is schematically shown;

[0019] Figure 7 A schematic diagram of data resource virtualization according to some embodiments of the present disclosure is schematically shown;

[0020] Figure 8 A schematic diagram of scheduling of GPU according to some embodiments of the present disclosure is schematically shown;

[0021] Figure 9 A schematic diagram of distributed model training according to some embodiments of the present disclosure is schematically shown;

[0022] Figure 10 A schematic diagram of non-distributed model training is shown according to some embodiments of the present disclosure;

[0023] Figure 11 A schematic block diagram of a file security acquisition apparatus is shown according to some embodiments of the present disclosure;

[0024] Figure 12 A schematic diagram of a storage medium is shown according to example embodiments of the present disclosure;

[0025] Figure 13 A block diagram of an electronic device is shown according to example embodiments of the present disclosure.

[0026] In the drawings, the same or similar notations are used to represent the same or similar parts. DETAILED DESCRIPTION

[0027] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and in no way limit the scope of the present disclosure. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0028] Those skilled in the art know that the embodiments of the present disclosure can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure can be embodied as a complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0029] The data involved in the present disclosure can be data authorized by users or fully authorized by all parties, and the collection, dissemination, use, etc. of the data comply with the requirements of relevant national laws and regulations.

[0030] The flowcharts shown in the drawings are only exemplary illustrations, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further divided, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0031] According to embodiments of the present disclosure, a text security acquisition method, a text security acquisition apparatus, a computer readable storage medium and an electronic device are proposed.

[0032] In this document, it is understood that the terms involved, such as:

[0033] Pre-trained large model: The introduction of the Transformer structure in 2017 made the deep learning model parameters break through 100 million. Starting from the initial LeNet, AlexNet, and Deep residual network (ResNet) model, the model parameters are getting larger and larger. The introduction of the Bidirectional Encoder Representations from Transformers (BERT) model made the parameter size exceed 300 million for the first time. The General Pre-trained Transformer-3 (GPT-3) model exceeds 10 billion. The Pengcheng discoloration realizes the scale of 100 billion dense, and the appearance of the Switch Transformer also breaks through the scale of 100 billion. Such a large model needs to be pre-trained on a large amount of data in advance. This process is called "pre-training", and the resulting model is a pre-trained large model.

[0034] Model fine-tuning: Based on the pre-trained large model, the pre-trained large model is retrained for different types of downstream tasks to ensure that the pre-trained large model is effective for different types of downstream tasks, i.e., model fine-tuning.

[0035] Computer cluster: A computer system that is connected by a group of loosely integrated computer software or hardware to work closely together to complete computing work. In a sense, they can be regarded as a computer.

[0036] Kubernetes: A open source platform for automating container operations, including deployment, scheduling, and node cluster expansion, etc. Kubernetes cluster represents a group of nodes (nodes), which can be physical servers or virtual machines, and the kubernetes platform is installed on them.

[0037] Pod: The smallest Application Program Interface (API) object in the kubernetes project, composed of multiple containers and related configuration information. Pod is a layer of encapsulation of kubernetes on containers, composed of one or more containers running on the same host.

[0038] Container: Used to define the environment parameters of model training.

[0039] In addition, the number of any elements in the drawings is used for example and not limitation, and any naming is only used for distinction and does not have any limiting meaning.

[0040] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments. SUMMARY

[0042] Federated learning is a machine learning framework that effectively helps multiple training nodes jointly train machine learning or deep learning models while meeting data privacy protection requirements.

[0043] refer to Figure 1 , Figure 1 This is a schematic diagram of federated learning in related technologies, such as Figure 1 As shown, a central server communicates with multiple training nodes, each storing training data. The central server manages the models output by each training node in federated learning, while the training nodes train their own models based on their stored training data. Federated learning includes the following steps: Step 1. Training nodes download the global model from the central server; Step 2. Each training node trains the global model using its own stored training data, outputting intermediate models; Step 3. Each training node encrypts its trained intermediate models and uploads the encrypted intermediate models to the central server; Step 4. The central server decrypts the received encrypted intermediate models, aggregates the decrypted intermediate models to obtain the updated global model, and determines whether the updated global model meets the model training termination condition. If it does, the updated global model is used as the target model; otherwise, steps 1 to 4 are continued.

[0044] Figure 1 In federated learning, the training process involves a large amount of data transmission, which makes it dependent on the network environment. If the network security is poor or the network transmission speed is slow, it will affect the training process of each federated learning participant, and thus affect the training effect of the model. In particular, pre-trained large models usually have very large model parameters. Using federated learning for training will lead to extremely high requirements for network bandwidth and stability. Moreover, the excessively large model parameters will require more computational cost when encrypting the model, increasing the computing load.

[0045] Based on the above, the basic idea of the present disclosure is that a requestor acquires a model training request initiated by a to-be-trained model for a preset task, the model training request carrying a training configuration parameter, the model training request is verified, the container group corresponding to the training configuration parameter and the resource access key are confirmed after the verification is passed, and the required computing resource and data resource of the container group are allocated, the data resource containing training corpus, the training corpus in the data resource is accessed in the container group according to the resource access key, and the to-be-trained model is trained for the preset task according to the computing resource and the training corpus. If the to-be-trained model meets the training completion requirement, the to-be-trained model is used as a target model, and the file of the target model is fed back to the requestor. Further, the data resource and the computing resource can be virtualized through containerization technology, and the data access permission is controlled through the resource access key, so that the model training is performed in the case of ensuring data security, the risk of data and privacy leakage is reduced, and the effect of model training is improved.

[0046] After introducing the basic principle of the present disclosure, various non-limiting embodiments of the present disclosure will be specifically introduced below.

[0047] OVERVIEW OF APPLICATION SCENARIOS

[0048] Firstly, reference is made to Figure 2 , Figure 2 An implementation environment schematic diagram of a text security acquisition method to which the embodiments of the present disclosure can be applied is shown.

[0049] As shown in Figure 2 , the implementation environment can include a terminal 201 and a plurality of clusters 202. The plurality of clusters 202 are respectively connected to a resource co-signal platform for managing the resources of each cluster 202, so as to access the data resources and computing resources of each cluster 202 to the resource co-signal platform. The terminal 201 allocates the data resources and computing resources of the cluster 202 through the resource co-signal platform to realize model training. The terminal 201 and the resource co-signal platform are directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0050] The terminal 201 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. Optionally, the terminal 201 is a terminal used by a user, the user configures a model training task based on the terminal, and then generates a model training request according to the model training task, so as to send the model training request to the resource co-signal platform, and call the data resources and computing resources provided by the cluster 202 to run the AI model training task through the resource co-signal platform.

[0051] The cluster 202 is used to provide data resources and computing resources. Optionally, the cluster 202 is a server cluster or a distributed system composed of multiple physical servers, and can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0052] The resource sharing platform can be a Kubernetes platform, which is pre-deployed with multiple container groups. The resource sharing platform runs in a terminal or a server. The terminal can be the terminal 201 in Figure 2 , and can also be another terminal. The server can be a server contained in the cluster 202 in Figure 2 , and can also be another server. The present application does not limit this.

[0053] For example, in this implementation environment, the terminal 201 submits a model training request about a preset task to the resource sharing platform. The resource sharing platform needs to create a container group running the model training task according to the model training request, allocate the computing resources and data resources required by the container group, and allocate a resource access key to the terminal 201, so that the data resources are accessed in the container group according to the resource access key to perform model training about the preset task according to the computing resources and data resources.

[0054] It should be understood that, Figure 2 The implementation environment shown is only one example in which embodiments of the present disclosure can be implemented. The scope of application of the embodiments of the present disclosure is not limited by any aspect of this implementation environment.

[0055] EXEMPLARY METHOD

[0056] The file security acquisition method according to the exemplary embodiments of the present disclosure will be described below in conjunction with the implementation environment of Figure 2 , with reference to Figure 3 . It should be noted that the above implementation environment is only shown for the purpose of facilitating understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any implementation environment as applicable.

[0057] The present disclosure first provides a file security acquisition method. The execution subject of the method can be a terminal device or a server. The present disclosure does not specially limit this, and the method is described in the present example embodiment as being executed by a server.

[0058] Referring to Figure 3As shown, in an exemplary embodiment, the file security acquisition method can include the following steps S310 to S350, which are described in detail as follows:

[0059] Step S310, obtaining a model training request initiated by a requestor for a preset task for a to-be-trained model, the model training request carrying a training configuration parameter.

[0060] It should be noted that the training configuration parameter is used to limit the related training rules in the training process of the to-be-trained model, such as the required training corpus, the required number of computing resource, the training environment, the training method, etc. The content of the training configuration parameter is not limited in the present application. The preset task refers to the training target of the to-be-trained model, such as indicating the training of the image recognition capability of the to-be-trained model, indicating the training of the text recognition capability of the to-be-trained model, or indicating the training of the audio recognition capability of the to-be-trained model.

[0061] For example, the terminal device can display a model training configuration page to the requestor, obtain the training configuration parameter input by the requestor for the preset task through receiving the parameter input operation performed by the requestor on the model training configuration page, and then generate a model training request according to the training configuration parameter and send the model training request to the server corresponding to the resource sharing platform.

[0062] For example, the training configuration parameter includes computing power configuration, such as the required amount of computing resource for training, the container group image required by the training environment, whether it needs to be scheduled to a specified GPU type, etc. The training configuration parameter also includes data configuration, such as data volume path, specific mounting directory, etc.

[0063] Step S320, verifying the model training request, and confirming the container group and resource access key corresponding to the training configuration parameter after verification.

[0064] It should be noted that the container group (Pod) is pre-configured with various environment parameters required in the model training process, and the operating system and application environment required for model training are restored by activating the Pod. The resource access key is used to control the access permission of the data resource, i.e. only with the resource access key can the data resource be accessed, and then the model training is realized.

[0065] The server receives the model training request sent by the requester, and verifies the model training request. Illustratively, the model training request also carries user information of the requester, and the server performs authority identification on the user information of the requester to identify whether the requester has access authority of the data resource. For example, a list of accessible users is set for each data resource, the data resource accessed by the model training request is confirmed according to the training configuration parameter carried by the model training request, the list of accessible users of the data resource is obtained, and then it is judged whether the user information carried by the training request exists in the list of accessible users. If it exists in the list of accessible users, it is judged that the verification result of the model training request is verified, and if it does not exist in the list of accessible users, it is judged that the verification result of the model training request is verification failure.

[0066] After verification, the container group corresponding to the training configuration parameter and the resource access key are confirmed, the model training environment is deployed through the container group, and the data resource is accessed through the resource access key.

[0067] Step S330, allocate the computing resource and data resource required by the container group, and the data resource contains training corpus.

[0068] It should be noted that the computing resource includes but is not limited to central processing unit (CPU) resource, graphics processing unit (GPU) resource, etc., which is used to provide computing power support for model training. The data resource is a storage resource containing training corpus, which has the ability of data storage and reading. In addition to containing training corpus, the data resource can also contain model parameters of the model to be trained, training scripts and other data.

[0069] After confirming the container group corresponding to the training configuration parameter, the corresponding computing resource and data resource are allocated to the container group in the resource sharing platform, and the resource access key is issued to the container group. The training corpus in the data resource is accessed through the resource access key, and the computing power support for model training is provided through the computing resource, so as to realize the model training task by combining the computing resource and the data resource.

[0070] Step S340, access the training corpus in the data resource in the container group according to the resource access key, so as to train the model to be trained according to the preset task of the computing resource and the training corpus.

[0071] Illustratively, the container group is started, and the resource access key is mounted in the container group. The training task in the container accesses the data resource through the key, and the data resource includes training corpus data volume and model parameter data volume of the model to be trained, wherein the training corpus data volume is used to save all training corpus of each cluster.

[0072] In some embodiments, in order to ensure the security of the training corpus, the data resource containing the training corpus and the computing resource belong to the same cluster, that is, the training corpus in cluster A can only be mounted to the computing resource of cluster A, so as to ensure that the access of the training corpus is carried out under the condition of being able to be constrained and monitored.

[0073] The to-be-trained model is iteratively trained according to the computing resource and the training corpus to update the model parameters of the to-be-trained model.

[0074] Step S350, if the to-be-trained model meets the training completion requirement, the to-be-trained model is taken as a target model, and the file of the target model is fed back to the requester.

[0075] The training completion requirement can be that the number of iterations of the to-be-trained model reaches a preset number, or that the loss function of the to-be-trained model converges, and the present application does not limit the training completion requirement.

[0076] If it is detected that the current to-be-trained model meets the training completion requirement, the current to-be-trained model is taken as a target model, and then the target model is packaged into a file and fed back to the requester.

[0077] Exemplarily, please refer to Figure 4 , Figure 4 The schematic diagram of the file security acquisition method provided by an exemplary embodiment of the present application is shown in Figure 4 The resource co-sensing platform includes data resource pools and computing resource pools corresponding to each cluster, the requester sends a model training request to the resource co-sensing platform, the model training request carries training configuration parameters, the training configuration parameters are used to indicate the computing resource and the data resource required for model training. After the resource co-sensing platform verifies the model training request, the Pods matching the training configuration parameters are confirmed, and appropriate computing resources and data resources are selected in the data resource pool and the computing resource pool corresponding to the resource co-sensing platform, and the corresponding resource access keys are generated, the computing resources and the data resources and the resource access keys are allocated to the Pods, the training corpus in the data resource, the model parameters of the to-be-trained model and other data are accessed in the Pods according to the resource access keys, the to-be-trained model is trained according to the computing resource and the training corpus for a preset task, and the target model obtained by the training is fed back to the requester. Wherein, after verifying the model training request, the model training task monitoring link can also be fed back to the requester, so that the requester can view the training progress and other information of the to-be-trained model through the model training task monitoring link.

[0078] In the file security acquisition method provided by the example embodiment, on the one hand, the scheduling of the model training task is implemented by the container group to implement the model training, so as to unify the training environment and training method of the model through the container group specification, and improve the effect of the model. On the other hand, the data resources and computing resources required for model training are allocated to the container group, and the access permission of the data resources is controlled through the resource access key, so as to ensure the data security of each cluster, and provide a safe and controllable data reading, model training and model sharing environment without changing the data ownership. On the other hand, since the central server of the traditional federated training is abandoned, the model does not need to be encrypted and transmitted, and the computing cost is reduced.

[0079] Next, the above steps of the file security acquisition method of the example embodiment will be described in more detail.

[0080] In one embodiment of the present disclosure, before the model training request initiated by the acquisition request party for the to-be-trained model, it further includes: a cluster access request initiated by the resource provider; decoupling the computing resources and data resources of the resource provider according to the cluster access request to obtain a plurality of data units and computing units.

[0081] Please refer to Figure 5 , Figure 5 The schematic diagram of cluster access provided by an example embodiment is shown in Figure 5 The resource provider is cluster D, and the access agent interface of the Kubernetes platform receives the cluster access request of cluster D. The cluster access request carries the cluster information of cluster D, such as the physical machines contained in cluster D, cluster configuration and other key information. According to the cluster access request received by the access agent interface, cluster D is registered and accessed to decouple the computing resources and data resources of cluster D to obtain a plurality of data units and computing units, and then the access of computing resources and data resources is realized.

[0082] Among them, the resource sharing environment constructed by the Kubernetes platform contains the computing resources and data resources of each accessed cluster, such as the clusters A, B, C and D accessed by the Kubernetes platform. The resource sharing environment constructed by the Kubernetes platform respectively includes the computing resources and data resources of cluster A, the computing resources and data resources of cluster B, the computing resources and data resources of cluster C and the computing resources and data resources of cluster D, to form a shared cluster accessible by the outside, and after receiving the model training request of the request party, the physical machine in the shared cluster accessed by the Kubernetes which matches the model training request is taken as a production cluster, and the model training is performed based on the data resources and computing resources of the production cluster.

[0083] Further, the cluster accessed by the Kubernetes platform has a virtual node registered respectively, and the external access to each cluster is realized by accessing the virtual node. The virtual node can be provided by the Virtual-Kubelet, which is disguised as a Kubelet upwards, thereby simulating a scheduling node object to interface with the native resource object of Kubernetes; and provides an API downwards, which can interface with the Provider provided by other resource management platforms, and the Provider can manage other clusters accessed by Kubernetes through the Provider. The visual kubelet is used to provide resource orchestration, container scheduling and other capabilities for each cluster, and the mixed scheduling and resource isolation of virtual resources and containers under multiple clusters are realized, without data exchange, and the security of data resource access is ensured directly through the resource access key.

[0084] Each cluster is accessed into the resource co-sensing platform to virtualize the computing power resources and data resources of each cluster, and then a resource co-sensing environment containing multiple cluster resources is constructed, in which resource sharing can be realized, the training efficiency of the model is improved, and since the data ownership of each cluster is not changed, data isolation between each cluster is realized, and the security of data is ensured.

[0085] Please refer to Figure 6 , Figure 6 The schematic diagram for decoupling the cluster resources is shown in Figure 6 , multiple clusters are accessed as scheduling nodes into Kubernetes to virtualize the data resources and computing power resources of each cluster, and split into multiple data units and computing power units to provide data storage and computing power support. According to the model training request of the requester, a model training task is constructed to obtain the computing power unit and the data unit required by the model training task to obtain a model computing unit, and then the data resources and computing power resources in the model computing unit are scheduled through the virtual node registered by each cluster to execute the model training task. It can be understood that, Figure 6 The number of computing power units and data units in the model computing unit in

[0086] In an embodiment of the present disclosure, the specific steps of virtualizing the data resources and splitting them into multiple data units include: establishing a PVC resource in the cluster according to the model training request; mounting the PVC resource to the container group and confirming the PV resource corresponding to the PVC resource; binding the PV resource and the PVC resource to allocate the data unit corresponding to the PV resource to the container group, and allocating the computing power unit to the container group according to the data unit.

[0087] In Kubernetes, in order to shield the details of the underlying storage implementation, make it convenient for users to use and convenient for administrators to manage, two resource objects, Persistent Volume (PV) and Persistent Volume Claim (PVC), are introduced to implement the storage management system.

[0088] The PV resource is an abstraction of the storage resource, and the storage is defined as a resource that can be used by a container. The PV is created and configured by an administrator, and it is directly related to the specific implementation of the storage provider, for example: Gluster File System (GlusterFS), Internet Small Computer System Interface (iSCSI) storage technology, Reliable Block Device (RBD), Network File System (NFS) storage technology, etc. It is managed through a plug-in mechanism for application access and use.

[0089] The PVC resource is a user's application for a storage resource, and the PVC consumes the resources of the PV. The PVC can apply for the size of the storage space (Size) and the access mode (for example, ReadWriteOnce, ReadOnlyMany or ReadWriteMany).

[0090] Please refer to Figure 7 , Figure 7 For the schematic diagram of data resource virtualization, as shown in Figure 7 , the underlying storage can be Ceph, which is a distributed storage system. Multiple physical machines can be combined into a cluster, and the disk resources in these physical machines can be integrated together to form a large resource pool. Then, Kubernetes can uniformly abstract the storage resources in the resource pool as PVs. It can be understood that the underlying storage can be of various types, including NFS, iSCSI, etc. The present application does not limit this.

[0091] Further, according to the requirements of the requester for the data resource, the PVC resource is created, which is a user's request for storage. A PVC is usually defined in a Pod in a Namespace. When defining, the relevant information of the PVC will be specified, such as the size of the space, the read-write attribute, etc. However, the PVC is not a real storage space. The PVC and the PV of the Pod must be connected, so that the Pod can call the actual storage space, that is, the PV resource needs to be bound to the PVC resource, so as to allocate the data unit corresponding to the PV resource to the container group.

[0092] By virtualizing the data resources of each cluster, when scheduling the data resources in the model training task, the corresponding data unit can be directly scheduled according to the resource access key, thereby reducing the complexity of model training between multiple clusters under the premise of ensuring data security.

[0093] Exemplarily, the specific steps of virtualizing the computing power resources and splitting the computing power resources include: loading and registering the computing power resources (such as GPU, CPU, etc.) on Kubernetes, and then virtualizing the computing power resources, such as cutting the GPU into multiple virtual graphics processing units (vGPU) to obtain multiple computing power units.

[0094] After the data unit is allocated to the Pod, the corresponding computing power unit is allocated to the Pod according to the model training request, so as to provide scheduling of the computing power resources for the model training task in the Pod.

[0095] For example, taking scheduling of the GPU as an example, please refer to Figure 8 , Figure 8 is a schematic diagram of scheduling of the GPU in an exemplary embodiment, as shown in Figure 8 The environment parameters such as application framework, graphics card numerical calculation library, etc. in the physical machine in the cluster are configured in the container image, and the link of the graphics card driver installed on the physical machine is mapped to the container image. Among them, the graphics card device plugin is deployed on the Kubernetes in the form of Deamonset, and Deamonset is deployed on each physical machine that meets the requirements, that is, all GPU hosts accessing the cluster are installed with the graphics card device plugin. Then, after the Pod is activated through the container image, the corresponding GPU can be called on the operating system through the link of the graphics card driver.

[0096] In an embodiment of the present disclosure, the computing power unit is allocated to the container group according to the data unit, comprising: confirming the candidate computing power unit according to the resource provider to which the data unit belongs; selecting the target computing power unit meeting the training configuration parameter from the candidate computing power unit; and allocating the target computing power unit to the container group.

[0097] For example, according to the model training request of the requestor, the resource provider of the required data unit is the cluster D, then all the computing power units belonging to the cluster D in the resource pool are taken as the candidate computing power units, and then the target computing power unit matching the training configuration parameter of the model training request is selected from the candidate computing power units, so as to allocate the target computing power unit to the container group.

[0098] By allocating the data units and computing units belonging to the same resource provider to the container group, i.e., any party data volume is not mounted to the node of the other cluster, the physical isolation of data access can be ensured, and the security of data access is further improved.

[0099] In one embodiment of the present disclosure, the model training request further includes identity information of the requester; the model training request is verified, and after verification, the container group corresponding to the training configuration parameter and the resource access key are confirmed, including: verifying the identity information and the training configuration parameter; generating the resource access key according to the identity information and the training configuration parameter after verification; and assigning the resource access key to the container group corresponding to the training configuration parameter.

[0100] Exemplarily, an authentication center can be provided in the resource sharing platform, and a user and permission mapping relationship table and a resource and permission mapping relationship table are stored in the authentication center. The user and permission mapping relationship table is used to store the mapping relationship between the identity information and the permission level of each registered user, and the resource and permission mapping relationship table is used to store the mapping relationship between each data unit or computing unit accessed to the cluster and the permission level. According to the identity information of the requester contained in the model training request, the user and permission mapping relationship table is queried to obtain the permission level corresponding to the requester, then the resource and permission mapping relationship table is queried according to the data volume requested in the training configuration parameter to obtain the permission level corresponding to the training configuration parameter, and then it is detected whether the permission level corresponding to the requester matches the permission level corresponding to the training configuration parameter. If it is detected that the permission level corresponding to the requester is lower than the permission level corresponding to the training configuration parameter, it is judged that the verification is not passed, and the requester is fed back prompt information that the verification is not passed. If it is detected that the permission level corresponding to the requester is higher than or equal to the permission level corresponding to the training configuration parameter, it is judged that the verification is passed, and the resource access key is generated according to the identity information and the training configuration parameter carried in the model training request. It can be understood that if the requester has not been registered, the corresponding prompt information is fed back to the requester to prompt the requester to register.

[0101] Exemplarily, each cluster can be respectively provided with an authentication center, and each authentication center can correspondingly store a corresponding user and permission mapping relationship table and a resource and permission mapping relationship table. After receiving the model training request, the resource sharing platform can query the configuration information of each cluster according to the requested data resource in the training configuration parameter of the model training request, to confirm the cluster to which the data resource belongs, and then forward the model training request to the cluster, so that the cluster verifies the model training request and receives the verification result fed back by the cluster. Wherein, the authentication center of the cluster can generate a resource access key according to the identity information and the training configuration parameter carried in the model training request after obtaining the authentication result of passing the verification; or the resource sharing platform can generate a resource access key according to the identity information and the training configuration parameter carried in the model training request after receiving the verification result fed back by the cluster, which is not limited in the present application.

[0102] The security of data access is ensured by verifying whether the resource requested by the requestor matches the permission level of the requestor.

[0103] Optionally, after the identity information and the training configuration parameter are verified, it is also necessary to judge whether the resources that can be provided by the current cluster meet the requested resources corresponding to the training configuration parameter. For example, if the computing resource requested by the requestor according to the training configuration parameter is greater than the computing resource that can be provided by the current cluster, the requestor is fed back the corresponding resource application failure information.

[0104] Optionally, the resource access key is bound to the requestor, that is, only the container activated by the requestor can use the resource access key to access the resource, so as to further improve the security of data access.

[0105] Optionally, the resource access key is limited by a use period, so that the resource access key can only be used for data access within the use period. The use period can be determined according to the permission level of the requestor or the permission level of the data volume accessed by the requestor. For example, the higher the permission level of the requestor, the longer the use period of the resource access key, and the lower the permission level of the requestor, the shorter the use period of the resource access key, so as to facilitate the management of the access permission of data and improve the security of data access.

[0106] In an embodiment of the present disclosure, the resource access key is generated according to the identity information and the training configuration parameter after the verification, comprising: confirming the data resource allocated by the container group according to the training configuration parameter, and generating the resource access key of the container group according to the identity information and the data resource allocated by the container group.

[0107] For example, the corresponding resource access key is generated according to the identity of the requestor and the data volume identifier of the requested data resource.

[0108] Exemplarily, after the resource access key is mounted into the Pod, the Pod needs to detect the resource access key when accessing the data resource, to determine whether the resource access key is legal, and if so, access is allowed, and if not, access is denied. Wherein, the key set corresponding to the data resource can be stored in the resource sharing platform or the cluster corresponding to the data resource, the key set includes the resource access key that can access the data resource and the available period of the resource access key, when accessing the data resource, it is detected whether the resource access key in the Pod exists in the key set, if so, the Pod is allowed to access the data resource, and if not, the Pod is not allowed to access the data resource.

[0109] In the embodiment, the resource access key is generated according to the identity information of the requester and the requested data resource, and when detecting the resource access key mounted by the Pod, the resource access key mounted by the Pod is decrypted, and it is detected whether the decrypted information contains the data volume identifier of the data resource accessed by the Pod, if so, the Pod is allowed to access the data resource, and if not, the Pod is not allowed to access the data resource, so as to improve the association between the resource access key and the data resource, and further improve the accuracy of resource access key verification.

[0110] In an embodiment of the present disclosure, the number of container groups corresponding to the training configuration parameters is at least two; the training corpus in the data resource is accessed in the container group according to the resource access key, to train the to-be-trained model according to the computing resource and the training corpus, including: accessing the training corpus in the data resource according to the resource access key corresponding to each container group; training the to-be-trained model in each container group according to the training corpus, and in the training process, each container group updates the parameters of the to-be-trained model of itself according to the parameters of the to-be-trained model of other container groups.

[0111] Exemplarily, the model training request sent by the requester is used to indicate distributed model training, for example, the training configuration parameter of the model training request indicates that the data resources of multiple clusters need to be applied, and then multiple Pods are respectively started on the physical machines that meet the resource requirements in the multiple clusters, wherein each Pod is mounted with a resource access key, and distributed model training is performed through these Pods.

[0112] For example, please refer to Figure 9 , Figure 9 An exemplary embodiment provides a schematic diagram of distributed model training, as shown in Figure 9As shown, the resource sharing platform starts multiple Pods on the host computer that meets the resource requirement according to the application resource, and respectively mounts the resource access key. For example, the container group for model training includes Pod1, Pod2, and Pod3, and a communication channel is established between each container group. Pod1 accesses the training corpus and the model parameters of the to-be-trained model in the data resource according to the resource access key, trains the to-be-trained model, Pod2 accesses the training corpus and the model parameters of the to-be-trained model in the data resource according to the resource access key, trains the to-be-trained model, and Pod3 accesses the training corpus and the model parameters of the to-be-trained model in the data resource according to the resource access key, trains the to-be-trained model. During the model training process, Pod1 obtains the parameters of the to-be-trained model of Pod2 and Pod3 according to the communication channel between Pod1, Pod2, and Pod3, and then Pod1 updates the parameters of the to-be-trained model of Pod1 according to the obtained parameters, so as to train the to-be-trained model in a distributed manner. This can meet the training requirements of a model with large parameters and ensure the security of data.

[0113] It can be understood that the model training request sent by the requester can also indicate single-machine training of the to-be-trained model, for example, Figure 10 As shown, the number of Pods corresponding to the training configuration parameters is one, the resource access key of the Pod is generated, and the resource access key is mounted to the Pod. After the Pod is activated, the training corpus and the model parameters of the to-be-trained model in the data resource are accessed according to the resource access key, so as to perform the model training task on the model parameters of the to-be-trained model according to the training corpus.

[0114] In an embodiment of the present disclosure, the training corpus in the data resource is accessed in the container group according to the resource access key, and the to-be-trained model is trained according to the computing resource and the training corpus, including: building a model training environment according to the container group, and accessing the training corpus and the model parameters in the data resource according to the resource access key; and training the to-be-trained model in the model training environment according to the computing resource and the training corpus.

[0115] The model training environment required by model training is set in the Pod, such as a software environment, an algorithm application framework, a graphics card numerical calculation library, and the like. When it is necessary to perform a model training task, the corresponding container image is confirmed according to the model training request sent by the requester, the corresponding Pod is activated according to the container image, the model training environment required by model training is pulled by the Pod, and the training corpus and the model parameters in the data resource are accessed in the model training environment according to the resource access key, so as to train the to-be-trained model.

[0116] In the traditional federated learning, due to the difficulty in unifying the model training environment of each training node, the model training effect is poor. Therefore, the environment parameters of the model training are pre-configured to obtain a container image. When the model training environment corresponding to the container image is needed in subsequent model training, the corresponding Pod can be activated according to the required container image to quickly build the expected model training environment, thereby improving the model training effect.

[0117] EXEMPLARY DEVICE

[0118] After introducing the method of the example embodiment of the present disclosure, next, with reference to Figure 11 The file security acquisition device of the example embodiment of the present disclosure is described.

[0119] In Figure 11 The file security acquisition device 1100 can include a request acquisition module 1110, a container group and key confirmation module 1120, a resource allocation module 1130, a training module 1140, and a file feedback module 1150.

[0120] The request acquisition module 1110 is configured to acquire a model training request about a preset task initiated by a requestor for a to-be-trained model, the model training request carrying a training configuration parameter;

[0121] The container group and key confirmation module 1120 is configured to verify the model training request, and confirm a container group and a resource access key corresponding to the training configuration parameter after the verification is passed;

[0122] The resource allocation module 1130 is configured to allocate a computing resource and a data resource required by the container group, the data resource containing training corpus;

[0123] The training module 1140 is configured to access the training corpus in the data resource according to the resource access key in the container group, and train the to-be-trained model according to the computing resource and the training corpus to complete the preset task;

[0124] The file feedback module 1150 is configured to, if the to-be-trained model meets the training completion requirement, take the to-be-trained model as a target model, and feed back a file of the target model to the requestor.

[0125] In an embodiment of the present disclosure, the file security acquisition device 1100 can further include a cluster access request acquisition unit and a resource decoupling unit. The cluster access request acquisition unit is configured to acquire a cluster access request initiated by a resource provider. The resource decoupling unit is configured to decouple the computing resource and the data resource of the resource provider according to the cluster access request to obtain a plurality of data units and computing units.

[0126] In an embodiment of the present disclosure, the resource allocation module 1130 comprises a PVC resource establishing unit, a PV resource confirming unit, and a data resource allocation unit: the PVC resource establishing unit is configured to establish a PVC resource in the cluster according to the model training request; the PV resource confirming unit is configured to mount the PVC resource to the container group and confirm a PV resource corresponding to the PVC resource; and the data and computing resource allocation unit is configured to bind the PV resource and the PVC resource, to allocate a data unit corresponding to the PV resource to the container group, and to allocate a computing unit to the container group according to the data unit.

[0127] In an embodiment of the present disclosure, the data and computing resource allocation unit comprises a candidate computing unit confirming unit, a target computing unit confirming unit, and a computing allocation unit: the candidate computing unit confirming unit is configured to confirm a candidate computing unit according to a resource provider to which a data unit belongs; the target computing unit confirming unit is configured to select a target computing unit satisfying the training configuration parameter from the candidate computing unit; and the computing allocation unit is configured to allocate the target computing unit to the container group.

[0128] In an embodiment of the present disclosure, the model training request further comprises identity information of a requestor; and the container group and key confirming module 1120 comprises a verifying unit, a key generating unit, and a key allocating unit: the verifying unit is configured to verify the identity information and the training configuration parameter; the key generating unit is configured to generate a resource access key according to the identity information and the training configuration parameter after verification; and the key allocating unit is configured to allocate the resource access key to the container group corresponding to the training configuration parameter.

[0129] In an embodiment of the present disclosure, the key generating unit comprises a data resource confirming unit and a key generating sub-unit: the data resource confirming unit is configured to confirm a data resource allocated to the container group according to the training configuration parameter; and the key generating sub-unit is configured to generate a resource access key of the container group according to the identity information and the data resource allocated to the container group, respectively.

[0130] In an embodiment of the present disclosure, the number of the container groups corresponding to the training configuration parameter is at least two; and the training module 1140 comprises a training corpus accessing unit and a model training unit: the training corpus accessing unit is configured to access training corpus in the data resource and model parameters of a to-be-trained model according to a resource access key of each container group, respectively; and the model training unit is configured to train the to-be-trained model in each container group according to the training corpus, and to update the parameters of the to-be-trained model of each container group according to the parameters of the to-be-trained model of other container groups during the training process.

[0131] In an embodiment of the present disclosure, the training module 1140 comprises a training environment construction and resource access unit and a training unit: the training environment construction and resource access unit is configured to construct a model training environment according to the container group, and access training corpus and model parameters of the to-be-trained model in a data resource according to a resource access key; the training unit is configured to train the to-be-trained model according to the computing resource and the training corpus in the model training environment.

[0132] Since the various functional modules of the file security acquisition device of the example embodiments of the present disclosure correspond to the steps of the example embodiments of the file security acquisition method described above, for details not disclosed in the device embodiments of the present disclosure, please refer to the above-mentioned embodiments of the file security acquisition method of the present disclosure, which will not be described here.

[0133] It should be noted that, although several modules or units of the file security acquisition device are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units.

[0134] In a third aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the file security acquisition method of the first aspect described above.

[0135] EXEMPLARY MEDIUM

[0136] After introducing the device of the example embodiments of the present disclosure, next, with reference to the Figure 12 The storage medium of the example embodiments of the present disclosure is described.

[0137] In some embodiments, various aspects of the present disclosure can also be implemented as a medium having program code stored thereon, which is executed by a processor of a device to implement the steps of the file security acquisition method according to various example embodiments of the present disclosure described in the above “example method” section of the present specification.

[0138] For example, the processor of the device can implement the file security acquisition method according to various example embodiments of the present disclosure when executing the program code. Figure 3The execution steps of the file security acquisition method described in the embodiment include: step S310, obtaining a model training request about a preset task initiated by a requestor for a to-be-trained model, the model training request carrying a training configuration parameter; step S320, verifying the model training request, and confirming a container group and a resource access key corresponding to the training configuration parameter after verification; step S330, allocating computing resource and data resource required by the container group, the data resource containing training corpus; step S340, accessing the training corpus in the data resource according to the resource access key in the container group, so as to train the to-be-trained model according to the computing resource and the training corpus; and step S350, if the to-be-trained model meets the training completion requirement, taking the to-be-trained model as a target model, and feeding back a file of the target model to the requestor.

[0139] Reference Figure 12 As shown in the embodiment of the present disclosure, a program product 1200 for implementing the above-mentioned file security acquisition method is described, which can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0140] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0141] The computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, in which readable program code is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable signal medium can also be any readable medium other than the readable storage medium.

[0142] The program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0143] EXEMPLARY COMPUTING DEVICE

[0144] After introducing the file security acquisition method, the file security acquisition device, and the storage medium of the exemplary embodiments of the present disclosure, next, the file security acquisition method of the exemplary embodiments of the present disclosure will be described with reference to Figure 13 An electronic device of the exemplary embodiments of the present disclosure will be described.

[0145] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be specifically implemented as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0146] In some possible embodiments, the electronic device according to the present disclosure can at least include at least one processing unit and at least one storage unit. The storage unit stores program code which, when executed by the processing unit, causes the processing unit to perform the steps of the file security acquisition method according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification. For example, the processing unit can perform the execution steps of the file security acquisition method as shown in Figure 3 In some possible embodiments, the electronic device according to the present disclosure can at least include at least one processing unit and at least one storage unit. The storage unit stores program code which, when executed by the processing unit, causes the processing unit to perform the steps of the file security acquisition method according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification. For example, the processing unit can perform the execution steps of the file security acquisition method as shown in Figure 3 In some possible embodiments, the electronic device according to the present disclosure can at least include at least one processing unit and at least one storage unit. The storage unit stores program code which, when executed by the processing unit, causes the processing unit to perform the steps of the file security acquisition method according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification. For example, the processing unit can perform the execution steps of the file security acquisition method as shown in

[0147] The following reference Figure 13 To describe an electronic device 1300 according to an example embodiment of the present disclosure. Figure 13 The electronic device 1300 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0148] like Figure 13 As shown, the electronic device 1300 is manifested in the form of a general-purpose computing device. The components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, a bus 1330 connecting different system components (including storage unit 1320 and processing unit 1310), and a display unit 1340.

[0149] The storage unit stores program code that can be executed by the processing unit 1310, causing the processing unit 1310 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0150] Storage unit 1320 may include readable media in the form of volatile storage units, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0151] Storage unit 1320 may also include a program / utility 1324 having a set (at least one) program module 1325, such program module 1325 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0152] Bus 1330 may include a data bus, an address bus, and a control bus.

[0153] The electronic device 1300 can also communicate with one or more external devices 1370 such as a keyboard, a pointing device, a Bluetooth device, etc. through an input / output (I / O) interface 1350. Further, the electronic device 1300 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet, through a network adapter 1360. As depicted, the network adapter 1360 is in communication with the other modules of the electronic device 1300 through the bus 1330. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the electronic device 1300. Such as, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0154] It should be noted that although several units / modules or sub-units / modules of the file security acquisition apparatus are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into units / modules embodied by several units / modules.

[0155] Moreover, while operations of the methods of the present disclosure are described in a particular order in the drawings, this is not required or implied, and the desired results can be achieved without performing all of the operations shown, or performing the operations in the particular order shown. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or split into multiple steps.

[0156] While the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it is to be understood that the present disclosure is not limited to the specific embodiments disclosed and that the division of aspects is not meant to imply that features from one aspect cannot be combined with features from another aspect to advantage, but is merely for convenience of description. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for securely acquiring files, characterized in that, Applied to a cluster containing a group of containers; the method includes: The request policy obtains a model training request initiated by the model to be trained for a preset task, the model training request carrying training configuration parameters; The model training request is verified, and the container group and resource access key corresponding to the training configuration parameters are confirmed after the verification is successful. According to the model training request, a PVC resource is established in the cluster; the PVC resource is mounted to the container group, and the PV resource corresponding to the PVC resource is confirmed; the PV resource is bound to the PVC resource to allocate the data unit corresponding to the PV resource to the container group, and computing power units are allocated to the container group according to the data unit. The data unit is obtained by virtualizing and splitting the data resource, and the computing power unit is obtained by virtualizing and splitting the computing power resource. The data resource contains training corpus. In the container group, the training corpus in the data resource is accessed according to the resource access key, so as to train the model to be trained on the preset task according to the computing power resources and the training corpus; If the model to be trained meets the training completion requirements, then the model to be trained is used as the target model, and the file of the target model is returned to the requesting party.

2. The method according to claim 1, characterized in that, Before obtaining the model training request initiated by the model to be trained, the method further includes: Obtain cluster access requests initiated by resource providers; Based on the cluster access request, the computing power resources and data resources of the resource provider are decoupled to obtain multiple data units and computing power units.

3. The method according to claim 1, characterized in that, The step of allocating computing power units to the container group according to the data unit includes: Candidate computing power units are identified based on the resource provider to which the data unit belongs; Select a target computing power unit that meets the training configuration parameters from the candidate computing power units; The target computing unit is allocated to the container group.

4. The method according to claim 1, characterized in that, The model training request also includes the identity information of the requester; the verification of the model training request, and the confirmation of the container group and resource access key corresponding to the training configuration parameters after successful verification, includes: The identity information and the training configuration parameters are verified. After successful verification, a resource access key is generated based on the identity information and the training configuration parameters. The resource access key is assigned to the container group corresponding to the training configuration parameters.

5. The method according to claim 4, characterized in that, The step of generating a resource access key based on the identity information and the training configuration parameters after successful verification includes: The data resources allocated to the container group are confirmed based on the training configuration parameters. Based on the identity information and the data resources allocated to the container group, a resource access key for the container group is generated respectively.

6. The method according to claim 1, characterized in that, The number of container groups corresponding to the training configuration parameters is at least two; the step of accessing the training corpus in the data resources within the container group according to the resource access key, and training the model to be trained according to the computing power resources and the training corpus, includes: Access the training corpus and model parameters of the model to be trained in the data resources according to the resource access key corresponding to each container group; The training model is trained in each container group according to the training corpus, and during the training process, each container group updates the parameters of its own training model according to the parameters of the training models of other container groups.

7. The method according to claim 1, characterized in that, The step of accessing the training corpus in the data resources within the container group according to the resource access key, and training the model to be trained based on the computing resources and the training corpus, includes: The model training environment is built according to the container group, and the training corpus and model parameters of the model to be trained are accessed in the data resources according to the resource access key. The model to be trained is trained in the model training environment based on the computing resources and the training corpus.

8. A file secure acquisition device, characterized in that, include: The request acquisition module is used to acquire model training requests initiated by the request policy for a preset task from the model to be trained, wherein the model training requests carry training configuration parameters. The container group and key verification module is used to verify the model training request and, after successful verification, confirm the container group and resource access key corresponding to the training configuration parameters. A PVC resource creation unit is used to create PVC resources in the cluster according to the model training request. The PV resource confirmation unit is used to attach the PVC resource to the container group and confirm the PV resource corresponding to the PVC resource. A data and computing power resource allocation unit is used to bind the PV resource and the PVC resource to allocate the data unit corresponding to the PV resource to the container group, and allocate computing power units to the container group according to the data unit. The data unit is obtained by virtualizing and splitting the data resource, and the computing power unit is obtained by virtualizing and splitting the computing power resource. The data resource contains training corpus. The training module is used to access the training corpus in the data resources in the container group according to the resource access key, so as to train the model to be trained on the preset task according to the computing power resources and the training corpus. The file feedback module is used to, if the model to be trained meets the training completion requirements, use the model to be trained as the target model and feed back the file of the target model to the requester.

9. The document secure acquisition device according to claim 8, characterized in that, The file secure acquisition device further includes a cluster access request acquisition unit and a resource decoupling unit: The cluster access request acquisition unit is used to acquire cluster access requests initiated by the resource provider; The resource decoupling unit is used to decouple the computing power resources and data resources of the resource provider according to the cluster access request, so as to obtain multiple data units and computing power units.

10. The document secure acquisition device according to claim 8, characterized in that, The data and computing resource allocation unit includes a candidate computing unit confirmation unit, a target computing unit confirmation unit, and a computing power allocation unit: The candidate computing power unit confirmation unit is used to confirm the candidate computing power unit according to the resource provider to which the data unit belongs; The target computing power unit confirmation unit is used to select a target computing power unit that meets the training configuration parameters from the candidate computing power units. The computing power allocation unit is used to allocate the target computing power unit to the container group.

11. The document secure acquisition device according to claim 8, characterized in that, The model training request also includes the requester's identity information; the container group and key verification module includes a verification unit, a key generation unit, and a key distribution unit. The verification unit is used to verify the identity information and the training configuration parameters; The key generation unit is used to generate a resource access key based on the identity information and the training configuration parameters after the verification is successful. The key allocation unit is used to allocate the resource access key to the container group corresponding to the training configuration parameters.

12. The document secure acquisition device according to claim 11, characterized in that, The key generation unit includes a data resource verification unit and a key generation subunit: The data resource confirmation unit is used to confirm the data resources allocated to the container group according to the training configuration parameters; The key generation subunit is used to generate resource access keys for the container group based on the identity information and the data resources allocated to the container group.

13. The document secure acquisition device according to claim 8, characterized in that, The number of container groups corresponding to the training configuration parameters is at least two; the training module includes a training corpus access unit and a model training unit: The training corpus access unit is used to access the training corpus and model parameters of the model to be trained in the data resource according to the resource access key corresponding to each container group. The model training unit is used to train the model to be trained in each of the container groups according to the training corpus, and during the training process, each container group updates the parameters of its own model to be trained according to the parameters of the models to be trained in other container groups.

14. The document secure acquisition device according to claim 8, characterized in that, The training module includes a training environment construction and resource access unit and a training unit: The training environment construction and resource access unit is used to build a model training environment according to the container group, and access the training corpus and model parameters of the model to be trained in the data resources according to the resource access key. The training unit is used to train the model to be trained in the model training environment based on the computing resources and the training corpus.

15. An electronic device, characterized in that, include: processor; as well as A memory storing computer-readable instructions that, when executed by the processor, implement the secure file acquisition method as described in any one of claims 1-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the file secure acquisition method as described in any one of claims 1-7.

Images