Model increment training method and device based on differential disk, equipment and product

Abstract

The invention discloses a model increment training method, device, equipment and product based on differential disks, and the method comprises the steps: creating a plurality of differential disks in a network memory, and then mounting the disks for use through an intranet NAS server; therefore, different users can directly access the mounted disk for training without downloading a model file, so that the model transmission time can be saved, and the training efficiency is improved; meanwhile, the differential disk only stores the data changed after the basic disk image is created, so that the actual disk space occupation of the model file after incremental training can be reduced by utilizing the characteristic that the differential disk only records the data change relative to the basic disk; therefore, a brand new training mode is provided for training of the incremental model, and the method is very suitable for large-scale application and popularization in the field of model training.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a model incremental training method, device, equipment and product based on differential disks. Background Art

[0002] With the development of information technology, people have an increasing demand for incremental training of large models. When training large vertical models in specific fields, they often add additional industry data sets to perform incremental training based on the trained and solidified basic general models. When the incremental training is completed, a full model file is also output, which will take up double the storage space. As the number of parameters continues to expand, the model file often reaches hundreds of GB or even TB levels.

[0003] Among them, in actual applications, in experiments to explore the impact of different data sets on model performance and accuracy, it is often necessary to perform incremental training on the same basic model with different data sets. At the same time, when the team has large model training directions for different industries, different team members are required to train different industry models based on the same basic model; however, when performing incremental model training, the existing technology needs to copy a basic model separately for different data sets for separate training. Since the model file is as large as hundreds of GB or even 1TB, a lot of time will be wasted on the transmission of the model file, thereby greatly reducing the efficiency of model training; at the same time, the existing incremental training will output the full model file, which will take up a large storage space. Therefore, how to provide an efficient and space-saving model incremental training method has become a problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a model incremental training method, device, equipment and product based on differential disk, so as to solve the problems of low training efficiency and large space occupation in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a model incremental training method based on differential disk is provided, comprising:

[0007] Creating a virtual disk in a network storage and storing a base model in the virtual disk;

[0008] Creating at least one difference disk for the virtual disk, and inheriting the basic model in the virtual disk to each difference disk, wherein each difference disk is located in the network storage and has different disk paths;

[0009] receiving a first disk mount request sent by at least one training machine, and determining a difference disk corresponding to each training machine from the at least one difference disk based on the first disk mount request of each training machine in the at least one training machine;

[0010] Establish a communication channel between each training machine and the differential disk corresponding to each training machine, so that each training machine can mount the corresponding differential disk after the establishment, so that each training machine can perform incremental training on the basic model in the mounted differential disk to obtain the basic model after each incremental training.

[0011] Based on the above disclosed content, the present invention creates a virtual disk in the network storage, and creates at least one differential disk for the virtual disk, so that parallel incremental training of the basic model can be performed based on each differential disk; wherein, during training, each training machine sends a first disk mount request to the NAS server, and the NAS server determines the differential disk corresponding to each training machine based on each first disk mount request; then, a communication channel is established between each training machine and the corresponding differential disk, so as to realize the mounting of the corresponding differential disk by each training machine; finally, each training machine can use its own training data set to complete the incremental training of the basic model in the mounted differential disk.

[0012] Through the above design, the present invention creates multiple difference disks in the network storage, and then mounts these disks for use through the intranet NAS server; in this way, different users can directly access the mounted disks for training without downloading model files, based on this, the model transmission time can be saved, thereby improving the training efficiency; at the same time, the difference disk only stores the data that has changed since the basic disk image was created. Therefore, by utilizing the characteristic that the difference disk only records the data changes relative to the basic disk, the actual disk space occupied by the model file after incremental training can be reduced; thus, the present invention provides a new training method for the training of incremental models, which is very suitable for large-scale application and promotion in the field of model training.

[0013] In a possible design, after any training machine obtains the corresponding incrementally trained basic model, the method further includes:

[0014] Receive a differential disk creation request sent by any training machine, wherein the differential disk creation request is generated when any training machine performs a version update on the model in the currently mounted differential disk, and a differential disk creation request is generated once for each version update, and when performing the first version update, the model in the currently mounted differential disk is the basic model after incremental training corresponding to any training machine;

[0015] Based on the difference disk creation request, a new difference disk is created for the target disk in the network storage, wherein the target disk is the difference disk currently mounted on any one of the training machines, and the new difference disk inherits the model stored in the target disk, the model stored in the target disk is the model of the previous version, and when the first version update is performed, the model in the target disk is the basic model after incremental training corresponding to any one of the training machines;

[0016] A communication channel is established between the new difference disk and any one of the training machines. After the communication channel is established, any one of the training machines mounts the new difference disk, so that any one of the training machines performs version updates on the model stored in the new difference disk, and stores the updated model in the new difference disk, so that after any one of the training machines completes all version updates of the basic model after the corresponding incremental training, a new difference disk corresponding to each version is obtained.

[0017] In one possible design, the method further includes:

[0018] receiving a disk unloading request sent by any training machine, and determining a first difference disk of any training machine based on the disk unloading request, wherein the first difference disk is one of the new difference disks corresponding to each version;

[0019] Disconnecting the communication channel between any one of the training machines and the first difference disk to complete the uninstallation of the first difference disk by any one of the training machines;

[0020] After disconnecting the communication channel between any one of the training machines and the first difference disk, receiving a second disk mount request sent by any one of the training machines;

[0021] Based on the second disk mount request, determining a version switching disk from new difference disks corresponding to each version;

[0022] A communication channel is established between any one of the training machines and the version switching disk to complete the mounting of the version switching disk by any one of the training machines, so that after the mounting, the version switching of the basic model after the incremental training is completed.

[0023] In a possible design, after storing the updated model in the new difference disk, the method further includes:

[0024] Change the file system permissions of the new differencing disk to read-only.

[0025] In one possible design, a virtual disk is created on a network storage, including:

[0026] Receiving a virtual disk creation request sent by a NAS management device, wherein the virtual disk creation request is generated by the NAS management device after responding to a system command prompt window opening instruction and running a virtual disk creation command after opening a system operating environment window;

[0027] The virtual disk is created in the network storage according to the virtual disk creation request.

[0028] In a possible design, the virtual disk creation command includes a save path and a maximum storage capacity of the virtual disk, wherein the virtual disk is created according to the save path and the maximum storage capacity.

[0029] In one possible design, when each training machine performs incremental training on a base model in a mounted difference disk, it mounts a data storage disk from the network storage and reads a training data set from the data storage disk so as to perform incremental training of the corresponding base model using the read training data set, wherein the data storage disks mounted on each training machine are different.

[0030] In a second aspect, a model incremental training device based on a difference disk is provided, comprising:

[0031] A virtual disk creation unit, used for creating a virtual disk in a network storage and storing a basic model in the virtual disk;

[0032] a difference disk creation unit, configured to create at least one difference disk for the virtual disk, and inherit the basic model in the virtual disk to each difference disk, wherein each difference disk is located in the network storage, and the disk paths are different from each other;

[0033] a disk mounting unit, configured to receive a first disk mounting request sent by at least one training machine, and determine a difference disk corresponding to each training machine from the at least one difference disk based on the first disk mounting request of each training machine in the at least one training machine;

[0034] The disk mounting unit is used to establish a communication channel between each training machine and the difference disk corresponding to each training machine, so as to enable each training machine to mount the corresponding difference disk after the establishment, so that each training machine can perform incremental training on the basic model in the mounted difference disk to obtain the basic model after each incremental training.

[0035] In the third aspect, another model incremental training device based on differential disk is provided. Taking the device as an electronic device as an example, it includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the model incremental training method based on differential disk as in the first aspect or any possible design of the first aspect.

[0036] In a fourth aspect, a storage medium is provided, on which instructions are stored. When the instructions are executed on a computer, the differential disk-based model incremental training method as described in the first aspect or any possible design of the first aspect is executed.

[0037] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to execute the differential disk-based model incremental training method as described in the first aspect or any possible design of the first aspect.

[0038] Beneficial effects:

[0039] (1) The present invention creates multiple difference disks in a network storage, and then mounts these disks for use through an intranet NAS server; thus, different users can directly access the mounted disks for training without downloading model files, thereby saving model transmission time and improving training efficiency; at the same time, the difference disk only stores data that has changed since the creation of the basic disk image, so the actual disk space occupied by the model file after incremental training can be reduced by utilizing the characteristic that the difference disk only records data changes relative to the basic disk; thus, the present invention provides a new training method for incremental model training, which is very suitable for large-scale application and promotion in the field of model training.

[0040] (2) Each time a model version is updated, the present invention will create a new difference disk based on the difference disk storing the previous model, and when the current version is updated, the new difference disk corresponding to the current version will be mounted to perform incremental training; in this way, the above method can realize a version chain structure based on the model; based on this, the model version of the key training node can be saved through the chain difference disk, so that the model version can be freely switched by mounting different difference disk files; therefore, the flexibility and convenience of model version control are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of the steps of a model incremental training method based on difference disks provided in an embodiment of the present invention;

[0042] Figure 2A schematic diagram of file reading during incremental model training provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a version chain structure provided in an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the structure of a model incremental training device based on difference disks provided in an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0047] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0048] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0049] Example:

[0050] See also Figure 1As shown, the incremental model training method based on differential disk provided in this embodiment uses the differential disk technology of virtual disk to save the large model file after incremental training. In this way, the characteristic that the differential disk only records the data changes relative to the basic disk can be used to reduce the actual disk space occupied by the model file after incremental training; at the same time, these differential disks are mounted on the intranet NAS server for use, so that different users can directly access the mounted disk for training without downloading the model file. Based on this, the model transmission time can be saved, thereby improving the training efficiency; in addition, the method realizes the preservation of the model version of the key training node through the chain differential disk, so the free switching of the model version can be realized by mounting different differential disk files; thus, the method not only improves the efficiency of incremental model training and the flexibility of version control, but also reduces the disk space occupied, so it is very suitable for large-scale application and promotion; among them, for example, the method can be but not limited to running on the training machine side, NAS management device and NAS server side, and optionally, the embodiment is preferably run on the NAS server side; it can be understood that the aforementioned execution subject does not constitute a limitation on the embodiment of the present application, and accordingly, the running steps of the method can be but not limited to the following steps S1 to S4.

[0051] S1. Create a virtual disk in a network storage, and store a basic model in the virtual disk; in a specific implementation, the example may include but is not limited to first receiving a virtual disk creation request sent by a NAS management device; then, according to the virtual disk creation request, the virtual disk is created in the network storage; wherein, the example virtual disk creation request is generated by the NAS management device in response to a system command prompt window opening instruction, and after the system operating environment window is opened, the virtual disk creation command is run, and the virtual disk creation command includes a virtual disk save path and a maximum storage capacity; thus, it is equivalent to creating the virtual disk according to the save path and the maximum storage capacity.

[0052] Specifically, this embodiment takes the Windows system installed in the NAS management device as an example to illustrate the generation process of the aforementioned virtual disk creation request:

[0053] First, the NAS management device opens the Windows command prompt window in response to a system command prompt window opening instruction (the instruction may be generated by a user operation, such as opening a Windows command prompt window as an administrator); then, the diskpart command is obtained (which may be, but is not limited to, obtained during human-computer interaction with the user) and the diskpart command is run to enter the diskpart operation environment window; then, in response to the human-computer interaction operation, a virtual disk creation command is obtained; finally, the virtual disk creation command is run to generate a virtual disk creation request, thereby completing the creation of the virtual disk based on the NAS server; based on this, the virtual disk creation request includes information in the virtual disk creation command, such as the save path and the maximum storage capacity.

[0054] In this embodiment, the virtual disk creation command may be, but is not limited to, create vdisk file = "D:\data\base.vhdx" maximum = 100; wherein file is the save path of the created virtual disk file; the maximum parameter specifies the maximum size of the virtual disk (i.e., the maximum storage capacity, in GB), which is of course not the actual size used, and the actual size used will expand as the data grows; of course, the aforementioned save path is the save path in the network storage.

[0055] In this way, after the virtual disk is created, the trained basic model can be copied to the virtual disk; the basic model is the core algorithm and architecture in the field of artificial intelligence and machine learning, which is widely used in various tasks; it usually includes but is not limited to: neural network model, decision tree and support vector machine, etc.

[0056] Therefore, after a virtual disk is created based on the aforementioned step S1, a differential disk can be established using the virtual disk as a base disk, so that incremental training of the model can be performed based on the differential disk later; wherein, the differential disk creation process can be but is not limited to as shown in the following step S2.

[0057] S2. Create at least one difference disk for the virtual disk, and inherit the basic model in the virtual disk to each difference disk, wherein each difference disk is located in the network storage, and the disk paths are different; in specific implementation, the creation of the difference disk is the same as the virtual disk, and both are achieved by the NAS management device sending a corresponding request; specifically, the NAS management device enters the diskpart operating environment window when responding to the human-computer interaction operation; then, obtains the difference disk creation command to generate a corresponding request based on the difference disk creation command, which is then transmitted to the NAS server, thereby completing the creation of the difference disk.

[0058] Optionally, the differential disk creation command is divided into two parts, which may be but are not limited to:

[0059] The first part: elect vdisk file = "D:\data\base.vhdx", where the file path refers to the full path of the virtual disk that stores the base model file; after running this command, the second part of the command can be obtained, namely, create vdisk type = differencing parent = "path\base disk.vhdx" file = "path\difference disk.vhdx"; where the path here refers to the location where the difference disk is stored in the network storage, for example: createvdisk type = differencing parent = "D:\data\base.vhdx" file = "D:\data\difference disk (model A).vhdx".

[0060] Based on this, when creating a differential disk, you need to first determine the path of the base disk (i.e., the virtual disk in the aforementioned step S1); then, create a differential disk based on the base disk; then, the differential disk can inherit the model file (i.e., the basic model) from the base disk, thereby facilitating subsequent incremental training.

[0061] In this embodiment, the above is explained by taking the creation and management of differential disks under Windows as an example. Correspondingly, there are corresponding commands in the Linux system to provide the same operations. In the actual application process, you can refer to the command rules of the corresponding operating system, and this is not limited to the above examples.

[0062] After the difference disk is created, the difference disk can be mounted to achieve parallel incremental training of the basic model; wherein the mounting training process is shown in the following steps S3 and S4.

[0063] S3. Receive a first disk mount request sent by at least one training machine, and based on the first disk mount request of each training machine in the at least one training machine, determine the difference disk corresponding to each training machine from the at least one difference disk; in specific implementation, during the actual training process, each training machine can send a corresponding first disk mount request to the NAS server, wherein any first disk mount request records an address of a difference disk, therefore, the NAS server can determine the difference disk to be mounted by each training machine according to the address of the difference disk in the first disk mount request.

[0064] After determining the difference disks to be mounted on each training machine, a disk communication channel can be established to complete the mounting of the difference disks by the training machine, thereby realizing incremental training of the model; wherein, the mounting and training process is shown in the following step S4.

[0065] S4. Establish a communication channel between each training machine and the differential disk corresponding to each training machine, so that each training machine can mount the corresponding differential disk after the establishment, so that each training machine can perform incremental training on the basic model in the mounted differential disk to obtain the basic model after each incremental training; in this embodiment, for example, any training machine can first establish a communication connection between the device and the NAS server; then, create a directory locally as a mount point, and then determine the address of the differential disk to be mounted in the NAS management interface, and generate a corresponding first disk mount request and send it to the NAS server; then, after receiving the response information sent by the NAS server (the NAS server will return a response information after suggesting the communication channel between any training machine and the corresponding differential disk), mount the NAS volume (i.e., the differential disk) to the local mount point; in this way, after the NAS server establishes the communication channel between the training machine and the differential disk, the training machine can realize the mounting of the corresponding differential disk, thereby realizing different incremental training of the basic model in parallel on each training machine.

[0066] Optional, see Figure 2 As shown in the figure, each difference disk inherits the basic model from the virtual disk, and then each training machine reads the basic model from its mounted difference disk for incremental training. After the training is completed, it is written to the corresponding difference disk, thereby obtaining different incrementally trained basic models (i.e. Figure 2 Model A, Model B, Model C, etc.)

[0067] At the same time, in this embodiment, for example, when each training machine performs incremental training on the basic model in the mounted difference disk, it will first mount the data storage disk from the network storage, and then read the training data set from the data storage disk. Finally, the read training data set can be used to perform incremental training on the corresponding basic model; see Figure 2 As shown, Figure 2 The training set data storage NAS in the example represents the data storage disk, where each training machine mounts a different data storage disk, that is, the stored training data sets are different. In other words, different training machines use different industry data sets (referring to training data sets) to implement different incremental training of the basic model, thereby obtaining different incrementally trained models.

[0068] Through the above design, by creating different difference disks in NAS, different users can directly access the mounted disk for training without downloading model files. Based on this, the model transmission time can be saved, thereby improving the training efficiency. At the same time, the difference disk only stores the data that has changed since the base disk image was created. Therefore, by utilizing the feature that the difference disk only records the data changes relative to the base disk, the actual disk space occupied by the model file after incremental training can be reduced.

[0069] In a possible design, the second aspect of this embodiment is optimized on the basis of the first aspect of the embodiment, and a chain difference disk is provided to realize the preservation and switching of different versions of models, and the process can be but not limited to the following steps S5 to S7; of course, this embodiment is explained by taking any training machine as an example, that is, after any training machine obtains the corresponding basic model after incremental training, it executes the following steps S5 to S7 to complete the preservation of different versions of models.

[0070] S5. Receive a differential disk creation request sent by any training machine, wherein the differential disk creation request is generated when any of the training machines performs a version update on the model in the currently mounted differential disk, and a differential disk creation request is generated for each version update, and when the first version update is performed, the model in the currently mounted differential disk is the basic model after incremental training corresponding to any of the training machines; in this embodiment, the process of the differential disk creation request is the same as the aforementioned step S2, the only difference being that the basic disks are set differently, and the principle will not be repeated here.

[0071] After receiving a difference disk creation request sent by any training machine, a new difference disk can be created, and the process is shown in the following step S6.

[0072] S6. Based on the difference disk creation request, a new difference disk is created for the target disk in the network storage, wherein the target disk is the difference disk currently mounted by any one of the training machines, and the new difference disk inherits the model stored in the target disk, the model stored in the target disk is the model of the previous version, and when the first version update is performed, the model in the target disk is the basic model after incremental training corresponding to any one of the training machines; in this embodiment, it is equivalent to creating a sub-difference disk for the difference disk in step S2 as the disk for this version update, that is, any one of the training machines completes the version update of this model by mounting the sub-difference disk, and the process is shown in the following step S7.

[0073] S7. Establish a communication channel between the new difference disk and any one of the training machines, and after the establishment, enable any one of the training machines to mount the new difference disk, so that any one of the training machines can perform version updates on the model stored in the new difference disk, and store the updated model in the new difference disk, so that after any one of the training machines completes all version updates of the basic model after the corresponding incremental training, a new difference disk corresponding to each version is obtained.

[0074] In this embodiment, the principle of the aforementioned steps S5 to S7 is: after completing the training of the specified incremental data based on the differential disk, if the current model needs to be updated, refer to step S2 to create a new differential disk with the current differential disk as the base disk path, and the training of subsequent versions is performed on the newly created differential disk; that is, for each version update, a corresponding new differential disk is established based on the differential disk established by the previous version, and the model version is saved based on this principle until the update is completed, and a new differential disk corresponding to each version is obtained.

[0075] See also Figure 3 As shown, Figure 3 The version chain structure of model A is given, namely Figure 3 The base.vhdx in the figure is the differential disk of the base model A after incremental training, that is, the initial differential disk. When model A needs to be updated, a new differential disk is created on base.vhdx as model A_V1.vhdx. Among them, model A_V1.vhdx inherits the base model A after the initial incremental training. Then, the training machine mounts the model A_V1.vhdx and performs a version update (that is, further incremental training) on ​​this basis to obtain model A_V1 and store it in the model A_V1.vhdx disk. At this time, a version update is completed.

[0076] Next, if the version needs to be updated again, a new difference disk is created based on the model A_V1.vhdx to obtain the model A_V2.vhdx difference disk (another new difference disk); then, the A_V2.vhdx difference disk is mounted for further incremental training and storage; at this point, the second version update is completed; in this way, the version update is continuously performed according to this principle, and new difference disks are created. After the version update is completed, a new difference disk corresponding to each version can be obtained; further, in this embodiment, after the version-updated model is stored in the new difference disk (that is, after each version update is completed), the file system permissions of the corresponding new difference disk need to be changed to read-only permissions, so as to prevent the model of this version from being updated again on the current difference disk.

[0077] Therefore, after the version chain structure is constructed in the aforementioned steps S5 to S7, the model version can be flexibly switched based on the structure, and the process is shown in the following steps S8 to S12.

[0078] S8. Receive a disk unmount request sent by any of the training machines, and determine the first difference disk of any of the training machines based on the disk unmount request, wherein the first difference disk is one of the new difference disks corresponding to each version; in this embodiment, the disk unmount request is generated by running the disk unmount command in the diskpart operating environment window, that is, input detach vdisk X to unmount the mounted virtual disk, wherein X is the mounted disk letter; in this way, after receiving the disk unmount request, the NAS server can determine the aforementioned first difference disk according to the disk letter; then, the first difference disk can be unmounted, as shown in the following step S9.

[0079] S9. Disconnect the communication channel between any one of the training machines and the first difference disk to complete the uninstallation of the first difference disk by any one of the training machines.

[0080] After the mounted difference disk is uninstalled, the difference disk can be switched, and the process is shown in the following steps S10 to S12.

[0081] S10. After disconnecting the communication channel between any one of the training machines and the first difference disk, receiving a second disk mount request sent by any one of the training machines; in this embodiment, after disconnecting the communication channel between any one of the training machines and the first difference disk, the mounted disk can be unmounted; then, a new difference disk can be mounted, and the process is shown in the following steps S11 and S12.

[0082] S11. Based on the second disk mount request, the version switching disk is determined from the new differential disks corresponding to each version; in this embodiment, the disk mount command is still run in the diskpart operating environment window to generate the second disk mount request, that is, the NAS management device runs select vdisk file="path\another version of the model differential disk file.vhdx" in the diskpart operating environment window, and runs attach vdisk to generate the corresponding second disk mount request, thereby realizing the mounting of different differential disks; wherein, the second disk mount request includes the drive letter or name of the mounted disk, and the NAS server can mount the disk based on this; wherein, the mounting process still establishes a communication channel, and the process is shown in the following step S12.

[0083] S12. Establish a communication channel between any one of the training machines and the version switching disk to complete the mounting of the version switching disk by any one of the training machines, so as to complete the version switching of the basic model after incremental training after mounting.

[0084] Therefore, through the above-mentioned design, this embodiment realizes the preservation of the model version of the key training nodes through the chain difference disk. In this way, the model version can be freely switched by mounting different difference disk files; therefore, the flexibility and convenience of model version control are improved.

[0085] like Figure 4 As shown, the third aspect of this embodiment provides a hardware device for implementing the model incremental training method based on differential disk described in the first aspect and the second aspect of the embodiment, taking the device as a NAS server as an example, including:

[0086] The virtual disk creation unit is used to create a virtual disk in the network storage and store the basic model in the virtual disk.

[0087] The difference disk creation unit is used to create at least one difference disk for the virtual disk and inherit the basic model in the virtual disk to each difference disk, wherein each difference disk is located in the network storage and has different disk paths.

[0088] The disk mounting unit is used to receive a first disk mounting request sent by at least one training machine, and based on the first disk mounting request of each training machine in the at least one training machine, determine the difference disk corresponding to each training machine from the at least one difference disk.

[0089] The disk mounting unit is used to establish a communication channel between each training machine and the difference disk corresponding to each training machine, so as to enable each training machine to mount the corresponding difference disk after the establishment, so that each training machine can perform incremental training on the basic model in the mounted difference disk to obtain the basic model after each incremental training.

[0090] The working process, working details and technical effects of the device provided in this embodiment can be found in the first and second aspects of the embodiments, and will not be described in detail here.

[0091] like Figure 5 As shown, the fourth aspect of this embodiment provides another model incremental training device based on differential disk, taking the device as an electronic device as an example, comprising: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program, and execute the model incremental training method based on differential disk as described in the first and second aspects of the embodiments.

[0092] For example, the memory may include, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory, first input first output (FIFO) and / or first in last out (FILO), etc. Specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). At the same time, the processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0093] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may not be limited to a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (General Packet Radio Service, GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0094] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first and second aspects of the embodiments, and will not be described in detail here.

[0095] The fifth aspect of this embodiment provides a storage medium storing instructions for the incremental model training method based on differential disk as described in the first and second aspects of the embodiments, that is, the storage medium stores instructions, and when the instructions are run on a computer, the incremental model training method based on differential disk as described in the first and second aspects of the embodiments is executed.

[0096] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0097] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first and second aspects of the embodiments, and will not be described in detail here.

[0098] The sixth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the differential disk-based model incremental training method as described in the first and second aspects of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0099] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A model incremental training method based on differential disk, characterized in that: include: Creating a virtual disk in a network storage and storing a base model in the virtual disk; Creating at least one difference disk for the virtual disk, and inheriting the basic model in the virtual disk to each difference disk, wherein each difference disk is located in the network storage and has different disk paths; receiving a first disk mount request sent by at least one training machine, and determining a difference disk corresponding to each training machine from the at least one difference disk based on the first disk mount request of each training machine in the at least one training machine; Establish a communication channel between each training machine and the differential disk corresponding to each training machine, so that each training machine can mount the corresponding differential disk after the establishment, so that each training machine can perform incremental training on the basic model in the mounted differential disk to obtain the basic model after each incremental training.

2. The method according to claim 1, characterized in that After any training machine obtains the corresponding incrementally trained basic model, the method further includes: Receive a differential disk creation request sent by any training machine, wherein the differential disk creation request is generated when any training machine performs a version update on the model in the currently mounted differential disk, and a differential disk creation request is generated once for each version update, and when performing the first version update, the model in the currently mounted differential disk is the basic model after incremental training corresponding to any training machine; Based on the difference disk creation request, a new difference disk is created for the target disk in the network storage, wherein the target disk is the difference disk currently mounted on any one of the training machines, and the new difference disk inherits the model stored in the target disk, the model stored in the target disk is the model of the previous version, and when the first version update is performed, the model in the target disk is the basic model after incremental training corresponding to any one of the training machines; A communication channel is established between the new difference disk and any one of the training machines. After the communication channel is established, any one of the training machines mounts the new difference disk, so that any one of the training machines performs version updates on the model stored in the new difference disk, and stores the updated model in the new difference disk, so that after any one of the training machines completes all version updates of the basic model after the corresponding incremental training, a new difference disk corresponding to each version is obtained.

3. The method according to claim 2, characterized in that The method further comprises: receiving a disk unloading request sent by any training machine, and determining a first difference disk of any training machine based on the disk unloading request, wherein the first difference disk is one of the new difference disks corresponding to each version; Disconnecting the communication channel between any one of the training machines and the first difference disk to complete the uninstallation of the first difference disk by any one of the training machines; After disconnecting the communication channel between any one of the training machines and the first difference disk, receiving a second disk mount request sent by any one of the training machines; Based on the second disk mount request, determining a version switching disk from new difference disks corresponding to each version; A communication channel is established between any one of the training machines and the version switching disk to complete the mounting of the version switching disk by any one of the training machines, so that after the mounting, the version switching of the basic model after the incremental training is completed.

4. The method according to claim 2, characterized in that: After storing the updated version of the model in the new difference disk, the method further includes: Change the file system permissions of the new differencing disk to read-only.

5. The method according to claim 1, characterized in that Create a virtual disk on the network storage, including: Receiving a virtual disk creation request sent by a NAS management device, wherein the virtual disk creation request is generated by the NAS management device after responding to a system command prompt window opening instruction and running a virtual disk creation command after opening a system operating environment window; The virtual disk is created in the network storage according to the virtual disk creation request.

6. The method according to claim 5, characterized in that The virtual disk creation command includes a storage path and a maximum storage capacity of the virtual disk, wherein the virtual disk is created according to the storage path and the maximum storage capacity.

7. The method according to claim 1, characterized in that When each training machine performs incremental training on the basic model in the mounted difference disk, the data storage disk is mounted from the network storage, and the training data set is read from the data storage disk so as to perform incremental training of the corresponding basic model using the read training data set, wherein the data storage disks mounted on each training machine are different.

8. A model incremental training device based on differential disk, characterized in that: include: A virtual disk creation unit, used for creating a virtual disk in a network storage and storing a basic model in the virtual disk; a difference disk creation unit, configured to create at least one difference disk for the virtual disk, and inherit the basic model in the virtual disk to each difference disk, wherein each difference disk is located in the network storage, and the disk paths are different from each other; a disk mounting unit, configured to receive a first disk mounting request sent by at least one training machine, and determine a difference disk corresponding to each training machine from the at least one difference disk based on the first disk mounting request of each training machine in the at least one training machine; The disk mounting unit is used to establish a communication channel between each training machine and the difference disk corresponding to each training machine, so as to enable each training machine to mount the corresponding difference disk after the establishment, so that each training machine can perform incremental training on the basic model in the mounted difference disk to obtain the basic model after each incremental training.

9. An electronic device, characterized in that: include: A memory, a processor and a transceiver that are sequentially communicatively connected, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program, and execute the differential disk-based model incremental training method as described in any one of claims 1 to 7.

10. A computer program product comprising instructions, characterized in that When the instructions are executed on a computer, the computer is caused to execute the incremental model training method based on differential disk as described in any one of claims 1 to 7.

Images