Data processing method and device for model training and electronic equipment

Abstract

The invention provides a data processing method and device for model training and electronic equipment, and relates to the technical field of computers. In the application, the data processing modes are respectively set for the plurality of temporary data generated by the target model in the forward propagation process in advance, so that whether the corresponding temporary data are unloaded from the target video memory of the target model in the forward propagation process or not is determined; and reloading the corresponding temporary data into the target video memory in the back propagation process. Therefore, in the model training process of the target model, once the data processing mode of the first temporary data generated by the target model in the forward propagation process is determined, data processing can be performed on the first temporary data based on the data processing mode of the first temporary data. By adopting the mode, the target video memory can be effectively managed and saved under the condition that the model training speed is not influenced by unloading and reloading part of the temporary data in the plurality of pieces of temporary data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device and electronic device for model training. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, the demand for computing resources for model training is also increasing. Among them, the limitation of video memory (for example, the memory of the graphics processing unit (GPU)) has become an important factor affecting the development of artificial intelligence technology. For example, for models with large magnitudes during training, the video memory capacity required for model training is much larger than the video memory capacity provided by a single GPU.

[0003] Therefore, how to effectively manage and save video memory has become a key issue in model training. Currently, for temporary data (such as activation values) that occupy video memory during model training, gradient check-pointing can be used to reduce video memory usage by recalculating rather than storing certain intermediate activation values.

[0004] However, the above-mentioned video memory saving method sacrifices computational efficiency in exchange for reducing video memory usage, which increases computational complexity during model training and causes the model training time to become longer. Therefore, how to effectively manage and save video memory without affecting the model training speed is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, device and electronic device for model training, which are used to effectively manage and save video memory without affecting the model training speed.

[0006] In a first aspect, an embodiment of the present application provides a data processing method for model training, the method comprising:

[0007] In response to a model training operation for a target model, a data processing mode of first temporary data generated by the target model during a forward propagation process is determined; the data processing mode is used to indicate: whether to unload the first temporary data from a target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during a backward propagation process;

[0008] The first temporary data is processed based on the data processing method of the first temporary data.

[0009] In an optional embodiment, before responding to the model training operation for the target model, the method further includes:

[0010] Record multiple temporary data generated by the target model during the forward propagation process, and their corresponding generation time and usage time; and,

[0011] Performing time consumption tests on multiple temporary data respectively, and obtaining unloading time consumption and reloading time consumption corresponding to the multiple temporary data respectively; wherein each unloading time consumption represents the time required to unload the corresponding temporary data from the target video memory, and each reloading time consumption represents the time required to reload the corresponding temporary data into the target video memory;

[0012] Based on the generation time, use time, unloading time and reloading time respectively corresponding to the plurality of temporary data, the data processing methods respectively corresponding to the plurality of temporary data are determined.

[0013] In an optional embodiment, the process of recording multiple temporary data generated by the target model during the forward propagation process, respectively corresponding to the generation time and the use time, further includes:

[0014] Cache multiple temporary data through the target video memory, and read the cumulative usage of the target video memory in real time;

[0015] When it is determined that the accumulated occupancy reaches a preset occupancy threshold, at least one temporary data not stored in the plurality of temporary data is cached by the standby video memory.

[0016] In an optional embodiment, based on the generation time, usage time, unloading time and reloading time respectively corresponding to the multiple temporary data, determining the data processing modes respectively corresponding to the multiple temporary data includes:

[0017] Based on the generation time and the use time respectively corresponding to the multiple temporary data, the data cache duration respectively corresponding to the multiple temporary data is obtained, and based on the unloading time and the reloading time respectively corresponding to the multiple temporary data, the data processing duration respectively corresponding to the multiple temporary data is obtained;

[0018] Based on the data cache duration, data processing duration and video memory occupancy respectively corresponding to the multiple temporary data, a data processing strategy for the multiple temporary data is determined; the data processing strategy includes: a data processing method respectively corresponding to the multiple temporary data.

[0019] In an optional embodiment, determining a data processing method for first temporary data generated by a target model during forward propagation includes:

[0020] Obtaining a data identifier of the first temporary data; the data identifier is a unique identifier generated according to the data type of the first temporary data;

[0021] The data processing method associated with the data identifier is used as the data processing method of the first temporary data.

[0022] In an optional embodiment, performing data processing on the first temporary data based on the data processing method of the first temporary data includes:

[0023] If the data processing mode of the first temporary data is the first processing mode, then in the forward propagation process, according to the generation time of the first temporary data, the first temporary data is unloaded from the target video memory and stored in the preset memory, and in the reverse propagation process, according to the use time of the first data and the reload time determined by the reload time, the first temporary data is reloaded from the preset memory to the target video memory;

[0024] If the data processing method of the first temporary data is the second processing method, the first temporary data is cached in the target video memory, and the first temporary data is called at the usage time of the first temporary data.

[0025] In a second aspect, an embodiment of the present application further provides a data processing device for model training, the device comprising:

[0026] A mode determination module, for responding to a model training operation for a target model and determining a data processing mode for first temporary data generated by the target model during a forward propagation process; the data processing mode is used to indicate: whether to unload the first temporary data from a target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during a backward propagation process;

[0027] The data processing module is used to process the first temporary data based on the data processing method of the first temporary data.

[0028] In an optional embodiment, before responding to the model training operation for the target model, the mode determination module is further used to:

[0029] Record multiple temporary data generated by the target model during the forward propagation process, and their corresponding generation time and usage time; and,

[0030] Performing time consumption tests on multiple temporary data respectively, and obtaining unloading time consumption and reloading time consumption corresponding to the multiple temporary data respectively; wherein each unloading time consumption represents the time required to unload the corresponding temporary data from the target video memory, and each reloading time consumption represents the time required to reload the corresponding temporary data into the target video memory;

[0031] Based on the generation time, use time, unloading time and reloading time respectively corresponding to the plurality of temporary data, the data processing methods respectively corresponding to the plurality of temporary data are determined.

[0032] In an optional embodiment, in the process of recording the generation time and usage time of multiple temporary data generated by the target model during the forward propagation process, the device also includes a data cache module, which is specifically used to:

[0033] Cache multiple temporary data through the target video memory, and read the cumulative usage of the target video memory in real time;

[0034] When it is determined that the accumulated occupancy reaches a preset occupancy threshold, at least one temporary data not stored in the plurality of temporary data is cached by the standby video memory.

[0035] In an optional embodiment, when determining the data processing modes respectively corresponding to the plurality of temporary data based on the generation time, usage time, unloading time and reloading time respectively corresponding to the plurality of temporary data, the mode determination module is specifically used to:

[0036] Based on the generation time and the use time respectively corresponding to the multiple temporary data, the data cache duration respectively corresponding to the multiple temporary data is obtained, and based on the unloading time and the reloading time respectively corresponding to the multiple temporary data, the data processing duration respectively corresponding to the multiple temporary data is obtained;

[0037] Based on the data cache duration, data processing duration and video memory occupancy respectively corresponding to the multiple temporary data, a data processing strategy for the multiple temporary data is determined; the data processing strategy includes: a data processing method respectively corresponding to the multiple temporary data.

[0038] In an optional embodiment, when determining the data processing method of the first temporary data generated by the target model during the forward propagation process, the method determination module is specifically used to:

[0039] Obtaining a data identifier of the first temporary data; the data identifier is a unique identifier generated according to the data type of the first temporary data;

[0040] The data processing method associated with the data identifier is used as the data processing method of the first temporary data.

[0041] In an optional embodiment, when the first temporary data is processed based on the data processing method of the first temporary data, the data processing module is specifically used to:

[0042] If the data processing mode of the first temporary data is the first processing mode, then in the forward propagation process, according to the generation time of the first temporary data, the first temporary data is unloaded from the target video memory and stored in the preset memory, and in the reverse propagation process, according to the use time of the first data and the reload time determined by the reload time, the first temporary data is reloaded from the preset memory to the target video memory;

[0043] If the data processing method of the first temporary data is the second processing method, the first temporary data is cached in the target video memory, and the first temporary data is called at the usage time of the first temporary data.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0045] Processor; and

[0046] Memory for storing programs,

[0047] The program includes instructions, which, when executed by a processor, cause the processor to execute the data processing method for model training as described in the first aspect.

[0048] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the data processing method for model training as described in the first aspect.

[0049] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the data processing method for model training as described in the first aspect.

[0050] The beneficial effects of this application are as follows:

[0051] In the data processing method for model training provided in the embodiment of the present application, in response to the model training operation for the target model, and determining the data processing mode of the first temporary data generated by the target model during the forward propagation process; the first temporary data is processed based on the data processing mode of the first temporary data. Wherein, the data processing mode is used to indicate: whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the reverse propagation process. In this way, during the model training of the target model, according to the data processing modes respectively set for the multiple temporary data generated during the forward propagation process, it is determined whether to unload the aforementioned multiple temporary data from the target video memory of the target model during the forward propagation process. And, when it is determined that part of the aforementioned multiple temporary data needs to be unloaded from the target video memory of the target model, the aforementioned part of the temporary data can be unloaded from the target video memory, so as to achieve effective management and saving of the target video memory without affecting the model training speed.

[0052] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described here are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0054] Figure 1 A schematic diagram of an optional system architecture applicable to the embodiments of the present application;

[0055] Figure 2 A schematic diagram of an implementation flow of a data processing method for model training provided in an embodiment of the present application;

[0056] Figure 3 A logical schematic diagram of a data processing method for temporary data provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of an implementation flow of a method for determining a data processing method provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of unloading time and reloading time provided in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of a specific application scenario of unloading and reloading temporary data provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of a data processing device for model training provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0063] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0064] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0065] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0066] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0067] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0068] (1) Video memory: Also known as graphics card memory, or frame buffer, it is used to store rendering data that has been processed or is about to be extracted by the graphics card chip (GPU). Like the computer's memory, video memory is a component used to store graphics information to be processed.

[0069] (2) Forward propagation (FP): refers to the process of information transfer from the input layer to the output layer in a neural network. Specifically, in the forward propagation process, the output of the previous layer is used as the input of the next layer, and the output of the next layer is calculated until the operation reaches the output layer.

[0070] (3) Back propagation (BP): also known as backward propagation, is essentially a method of calculating the gradient of parameters in each layer of the neural network. Specifically, the chain rule is used to find the partial derivatives of the loss function with respect to the weights and biases of each neuron layer by layer, forming the gradient of the loss function with respect to the weights and bias vectors, which serves as the basis for modifying the weights.

[0071] (4) Intermediate activation: It is the intermediate result that needs to be retained during the forward propagation process in order for the backward propagation to complete the calculation. For the sake of convenience, the intermediate activation value is referred to as the activation value in the following text.

[0072] Based on the above-mentioned nouns and related terminology explanations, the design concept of the embodiments of the present application is briefly introduced below:

[0073] Model training, especially large-scale deep learning model training, has an increasing demand for computing resources. GPU memory capacity has become an important factor affecting model training.

[0074] For example, as the scale of models continues to expand, reaching giant models with billions or even trillions of parameters (such as the third-generation generative pre-trained transformer 3 (GPT-3) model and Megatron model, etc.), the required video memory capacity far exceeds the video memory capacity that a single GPU can provide.

[0075] In this case, effectively managing and saving video memory becomes a key issue in the model training process. Model training not only involves the storage of hyperparameters, but also includes temporary data (such as activation values), gradient calculations, and optimizer states during forward and backward propagation, which will occupy the model's video memory.

[0076] Currently, for temporary data that occupies video memory during model training, gradient checkpoints can be used to reduce video memory usage by recalculating rather than storing certain intermediate activation values.

[0077] However, adopting the above-mentioned video memory saving method, by sacrificing computing efficiency in exchange for reducing the use of video memory, will increase the computational complexity during the model training process, resulting in a longer model training time.

[0078] In view of this, in order to solve or improve the above problems, the embodiment of the present application proposes a data processing method for model training, which specifically includes: responding to the model training operation for the target model, and determining the data processing method of the first temporary data generated by the target model during the forward propagation process; performing data processing on the first temporary data based on the data processing method of the first temporary data. The data processing method is used to indicate: whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the reverse propagation process.

[0079] Based on the above method, during the model training of the target model, according to the data processing methods respectively set for the multiple temporary data (such as the first temporary data) generated during the forward propagation process, it is determined whether to unload the aforementioned multiple temporary data from the target video memory of the target model during the forward propagation process. In addition, when it is determined that part of the aforementioned multiple temporary data needs to be unloaded from the target video memory of the target model, the aforementioned part of the temporary data can be unloaded from the target video memory, thereby achieving effective management and saving of the target video memory without affecting the model training speed.

[0080] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0081] See also Figure 1 As shown, it is a schematic diagram of a system architecture applicable to an embodiment of the present application, and the system architecture may include: a terminal device (101a, 101b) and a server 102. The terminal device (101a, 101b) and the server 102 may exchange information through a communication network, wherein the communication mode adopted by the communication network may include: a wireless communication mode and a wired communication mode. Exemplarily, the terminal device (101a, 101b) may access the network through cellular mobile communication technology and communicate with the server 102. Wherein, the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th generation mobile networks, 5G) technology or the next generation mobile communication technology. Optionally, the terminal device (101a, 101b) may access the network through a short-range wireless communication mode and communicate with the server 102. Wherein, the short-range wireless communication mode, for example, includes wireless fidelity (wireless fidelity, Wi-Fi) technology.

[0082] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above system architecture. For example, the above system architecture may include more terminal devices, or fewer terminal devices, or other network devices. Figure 1 As shown, only the terminal devices (101a, 101b) and the server 102 are described as examples, and the above communication devices and their respective functions are briefly introduced below.

[0083] The terminal device (101a, 101b) is a device that can provide voice and / or data connectivity to users, and can be a device that supports wired and / or wireless connection.

[0084] Exemplarily, the terminal devices (101a, 101b) may include, but are not limited to: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in unmanned driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0085] In addition, a related client may be installed on the terminal device (101a, 101b), and the client may be software, for example, an application (APP), a browser, a short video software, etc., or a web page, a mini-program, etc.; it should be noted that the terminal device (101a, 101b) in the embodiment of the present application may enable the above-mentioned client related to the data processing for model training to send the training sample set and / or test sample set of the target model, as well as the model training operation for the target model, to the server 102, so as to perform subsequent method steps such as model training for the target model.

[0086] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0087] It should be noted that if Figure 1 As shown, in the embodiment of the present application, a target model is also deployed on the server 102. After receiving the training sample set and / or test sample set of the target model, the server 102 can perform model training and / or model testing on the target model, thereby obtaining a final model that can complete the corresponding task or business. In the embodiment of the present application, the specific type of the target model and the business completed by the target model are not limited.

[0088] It is worth mentioning that in the embodiment of the present application, the server 102 can respond to the model training operation for the target model and determine the data processing method of the first temporary data generated by the target model during the forward propagation process; then, the first temporary data is processed based on the data processing method of the first temporary data. The data processing method can be used to indicate whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the reverse propagation process.

[0089] The following describes the data processing method for model training provided by the exemplary embodiment of the present application in combination with the above-mentioned system architecture and with reference to the accompanying drawings. It should be noted that the above-mentioned system architecture is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.

[0090] See also Figure 2 As shown, it is a schematic diagram of the implementation process of a data processing method for model training provided in an embodiment of the present application. The execution subject takes a server as an example. The specific implementation process of the method is as follows:

[0091] S201: Responding to a model training operation for a target model, and determining a data processing method for first temporary data generated by the target model during a forward propagation process.

[0092] The above-mentioned model training operation is also the triggering operation of model training. When the server receives the model training operation for the target model, it can start model training for the target model according to the sample training set. The above-mentioned first temporary data can be any one of the multiple temporary data generated by the target model during the forward propagation process. In addition, the above-mentioned first temporary data can be the activation value calculated by the activation function of each network layer during the forward propagation process; of course, it can also be other temporary data that will be used in the subsequent back propagation process, and the embodiment of the present application is not limited to this.

[0093] Furthermore, the data processing method of the first temporary data can be used to indicate whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the backward propagation process. For example, the target video memory can be the memory of the GPU corresponding to the target model.

[0094] For example, see Figure 3As shown in (a) in FIG. 1 , the data processing method of the first temporary data Tem.Data.1 may be to unload the first temporary data Tem.Data.1 from the memory of the GPU corresponding to the target model Tar.Model during the forward propagation process, and to overload (i.e., reload) the first temporary data Tem.Data.1 into the memory of the GPU corresponding to the target model during the backward propagation process. Optionally, see Figure 3 As shown in (b), the data processing method of the first temporary data Tem.Data.1 can be to cache the first temporary data Tem.Data.1 in the memory of the GPU corresponding to the target model.

[0095] In order to ensure that the temporary data can save the memory usage of the target model's target memory during the training process, before executing step S201, the server can set data processing methods for multiple temporary data generated during the forward propagation process, see Figure 4 As shown, the specific process is as follows:

[0096] S401: Record multiple temporary data generated by the target model during the forward propagation process, and the corresponding generation time and usage time.

[0097] Among them, the above-mentioned multiple temporary data are all temporary parameters generated by the target model during a forward propagation process. Each generation time can represent the time when the corresponding temporary data is generated during the forward propagation process, and each usage time can represent the time when the corresponding temporary data is used or needed during the backward propagation process.

[0098] Exemplarily, when executing step S401, during a historical model training process for the target model, the generation time corresponding to each temporary data generated during the forward propagation process can be recorded during the model training of the target model, and the usage time corresponding to each temporary data generated during the backward propagation process can be recorded.

[0099] Because the target model generates multiple temporary data during the forward propagation process, the target video memory cache of the target model will be used. In order to ensure that the aforementioned multiple temporary data can be cached, when it is determined that the remaining video memory of the target video memory is obviously insufficient for caching at least one temporary data generated by the subsequent target model during the forward propagation process, the aforementioned at least one temporary data can be cached through the standby video memory. In an optional implementation, the server can cache multiple temporary data through the target video memory and read the cumulative occupancy of the target video memory in real time, so that when it is determined that the aforementioned cumulative occupancy reaches a preset occupancy threshold, at least one temporary data not stored in the multiple temporary data is cached through the standby video memory.

[0100] Assuming that the preset occupancy threshold is 95%, the server reads the cumulative occupancy of the target video memory in real time. Once it is determined that the cumulative occupancy reaches the preset occupancy threshold of 95%, the temporary data generated by the target model in the subsequent forward propagation process and the temporary data that has been generated by the target model in the forward propagation process but has not yet been stored can be cached through the standby video memory. On the contrary, if the aforementioned cumulative occupancy has not reached the preset occupancy threshold of 95%, the temporary data generated by the target model in the subsequent forward propagation process and the temporary data that has been generated by the target model in the forward propagation process but has not yet been stored can continue to be cached through the target video memory. In other words, the standby video memory is used to cache the temporary data generated in the forward propagation process when it is determined that the target video memory cannot continue to cache the temporary data, so as to achieve the caching of all temporary data generated in the forward propagation process.

[0101] It should be noted that the number of spare video memories is not specifically limited in the embodiments of the present application. For example, the number of spare video memories may be determined based on the total amount of video memory occupied by the above-mentioned multiple data, the video memory capacity of the target video memory, and the video memory capacity of the spare video memory. It should also be noted that the video memory capacities of different spare video memories may be the same or different.

[0102] It should be understood that when it is determined that the aforementioned cumulative occupancy reaches a preset occupancy threshold, at least one temporary data not stored in multiple temporary data is cached by the spare video memory, which is equivalent to splitting the target model into two or more sub-models, wherein the video memory of one sub-model is the aforementioned target video memory and the video memories of other sub-models are spare video memories.

[0103] For example, when it is determined that the target video memory of the target model of the 32-layer network structure cannot cache all temporary data generated by the target model during the forward propagation process, the target model of the aforementioned 32-layer network structure can be split into two sub-models of the 16-layer network structure, for example, the first sub-model and the second sub-model. The temporary data generated by the first sub-model can be cached in the target video memory of the target model, and the temporary data generated by the second sub-model can be cached in the corresponding backup video memory, thereby realizing the caching of all temporary data generated by the target model of the aforementioned 32-layer network structure.

[0104] In another optional implementation, the server may also determine the cumulative occupancy of the target video memory after caching the temporary data to be cached (i.e., the second cumulative occupancy) when the current cumulative occupancy of the target video memory (i.e., the first cumulative occupancy) is less than the above-mentioned preset occupancy threshold, based on the video memory occupancy of the temporary data to be cached. If the second cumulative occupancy is greater than the above-mentioned preset occupancy threshold, the server may cache the above-mentioned temporary data to be cached in the standby video memory. Conversely, if the second cumulative occupancy is less than or equal to the above-mentioned preset occupancy threshold, the server may cache the above-mentioned temporary data to be cached in the target video memory.

[0105] Taking the above first cumulative occupancy of 89.5% and the above preset occupancy threshold of 90% as an example, if the video memory occupancy of the temporary data Tem.Data.2 to be cached is 0.6%, the server can determine that the second cumulative occupancy is 91.1% based on the first cumulative occupancy of 89.5% and the video memory occupancy of Tem.Data.2 of 0.6%, at which point Tem.Data.2 can be cached in the standby video memory. If the video memory occupancy of the temporary data Tem.Data.3 to be cached is 0.37%, the server can determine that the second cumulative occupancy is 89.87% based on the first cumulative occupancy of 89.5% and the video memory occupancy of Tem.Data.3 of 0.37%, at which point Tem.Data.3 can be cached in the target video memory.

[0106] S402: Perform time consumption tests on multiple temporary data respectively to obtain unloading time consumption and reloading time consumption corresponding to the multiple temporary data respectively.

[0107] Each unloading time may represent the time required to unload the corresponding temporary data from the target video memory, and each reloading time may represent the time required to reload the corresponding temporary data into the target video memory.

[0108] See also Figure 5 As shown, the unloading time of any one of the above-mentioned multiple temporary data (e.g., the second temporary data) can be the time required to unload the second temporary data from the target video memory and store it in a preset memory (e.g., the memory of the central processing unit (CPU)), which can be represented as t1. The reloading time of the second temporary data can be the time required to reload the second temporary data from the preset memory to the target video memory, which can be represented as t2.

[0109] It should also be noted that step S402 may occur before step S401 or after step S401. Of course, step S402 and step S401 may also occur at the same time, which is not specifically limited in the present embodiment of the application. Figure 4 Step S401 in occurs before step S402.

[0110] S403: Determine data processing methods corresponding to the plurality of temporary data respectively based on the generation time, use time, unloading time and reloading time respectively corresponding to the plurality of temporary data.

[0111] Specifically, when executing step S403, the server can determine a globally optimal data processing strategy based on the generation time, usage time, unloading time and reloading time corresponding to multiple temporary data, so as to unload and reload certain temporary data at the appropriate time to achieve the lowest video memory occupancy.

[0112] In an optional implementation, the server can obtain the data cache duration corresponding to multiple temporary data based on the generation time and usage time corresponding to the multiple temporary data, and obtain the data processing duration corresponding to the multiple temporary data based on the unloading time and reloading time corresponding to the multiple temporary data.

[0113] Optionally, the calculation formula for the above data cache duration can be specifically expressed as follows:

[0114] ΔT=T2-T1

[0115] Wherein, ΔT may represent the data cache duration of temporary data, T2 may represent the usage time of temporary data, and T1 may represent the generation time of temporary data.

[0116] The above data processing time can be specifically expressed as follows:

[0117] t c =t1+t2

[0118] Among them, t c It can represent the data processing time of temporary data, t1 can represent the time consumed for unloading temporary data, and t2 can represent the time consumed for reloading temporary data.

[0119] Exemplarily, if the data cache duration of a temporary data is greater than the data processing duration of the temporary data, the temporary data can be unloaded from the target video memory; conversely, if the data cache duration of a temporary data is less than or equal to the data processing duration of the temporary data, the temporary data does not need to be unloaded from the target video memory.

[0120] Furthermore, the server may determine the data processing strategies for the above-mentioned multiple temporary data based on the data cache duration, data processing duration and video memory occupancy corresponding to the above-mentioned multiple temporary data. The aforementioned data processing strategies may include: the data processing methods corresponding to the above-mentioned multiple temporary data. In this way, combined with the video memory occupancy of the temporary data, the amount of unloaded temporary data can be greatly increased or reduced to meet the corresponding data processing requirements.

[0121] Optionally, the server may also determine data processing strategies for the multiple temporary data based on the data cache durations, data processing durations and video memory occupancy corresponding to the multiple temporary data, and the remaining video memory of the target video memory.

[0122] Taking three temporary data (for example, Tem.Data.a~Tem.Data.c) as an example, the data processing method corresponding to the temporary data Tem.Data.a may be that the temporary data Tem.Data.a needs to be unloaded and reloaded, the data processing method corresponding to the temporary data Tem.Data.b may be that the temporary data Tem.Data.b does not need to be unloaded and reloaded, and the data processing method corresponding to the temporary data Tem.Data.c may be that the temporary data Tem.Data.c needs to be unloaded and reloaded.

[0123] In order to increase the speed of determining the data processing method of the first temporary data, the data identifier of the first temporary data can be associated with the corresponding data processing method in advance. In this way, when determining the data processing method of the first temporary data, the server can obtain the data identifier of the first temporary data, and thus determine the data processing method of the first temporary data according to the data identifier, for example, using the data processing method associated with the data identifier as the data processing method of the first temporary data.

[0124] Optionally, the data identifier of the first temporary data may be a unique identifier generated according to the data type of the first temporary data. Exemplarily, the data type may characterize the data shape and value type of the first temporary data. The data shape may be determined according to the input of the target model and itself (e.g., an activation function), and the value type may be any one of integer (int), floating point number (float), complex number (complex), and long integer (long int).

[0125] It should be noted that the data type of the first temporary data may also be recorded during the historical model training process of the target model. Moreover, by adopting the above method, the accuracy of the data processing method for obtaining the first temporary data according to the data identifier can be improved, thereby improving the problem that the data identifiers corresponding to different temporary data may be the same but the data processing methods are different, resulting in inaccurate data processing methods for the first temporary data.

[0126] S202: Perform data processing on the first temporary data based on the data processing method of the first temporary data.

[0127] Since the data processing mode is used to indicate whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and to reload the first temporary data into the target video memory during the backward propagation process, the data processing mode of the first temporary data can be divided into two data processing modes, namely, the first processing mode and the second processing mode. Optionally, the first processing mode can be used to indicate that the first temporary data is unloaded from the target video memory of the target model during the forward propagation process, and the first temporary data is reloaded into the target video memory during the backward propagation process, and the second processing mode can be used to indicate that the first temporary data does not need to be unloaded from the target video memory of the target model during the forward propagation process, and the first temporary data does not need to be reloaded into the target video memory during the backward propagation process.

[0128] Therefore, when executing step S202, if the data processing method of the first temporary data is the above-mentioned first processing method, the server can unload the first temporary data from the target video memory and store it in the preset memory according to the generation time of the first temporary data during the forward propagation process, and can reload the first temporary data from the preset memory to the target video memory according to the reload time determined by the usage time of the first data and the reloading time during the reverse propagation process; if the data processing method of the first temporary data is the above-mentioned second processing method, the server can cache the first temporary data in the target video memory, and call the first temporary data at the usage time of the first temporary data.

[0129] Take the three temporary data generated by the target model during the forward propagation (FP) process as an example. The three temporary data are: Act.Fun.Value.1, Act.Fun.Value.2 and Act.Fun.Value.3. Assume that the target video memory is the GPU memory and the preset memory is the CPU memory. Figure 6As shown, the server can determine the data processing strategies of the above three temporary data, that is, the data processing methods corresponding to the above three temporary data are: unloading (offload) temporary data Act.Fun.Value.1 during forward propagation and reloading (reload) temporary data Act.Fun.Value.2 during forward propagation and reloading temporary data Act.Fun.Value.3 during reverse propagation. Figure 6 As shown, t 1,1 Indicates the time taken to uninstall the temporary parameter Act.Fun.Value.1, t 2,1 Indicates the time taken to reload the temporary parameter Act.Fun.Value.1, t 1,2 Indicates the time taken to uninstall the temporary parameter Act.Fun.Value.2, t 2,2 Indicates the time taken to reload the temporary parameter Act.Fun.Value.2.

[0130] In summary, in the data processing method for model training provided in the embodiment of the present application, the data processing method is set in advance for the multiple temporary data generated by the target model during the forward propagation process, so as to determine whether to unload the corresponding temporary data from the target video memory of the target model during the forward propagation process, and reload the corresponding temporary data into the target video memory during the reverse propagation process. Therefore, in the process of model training of the target model, once the data processing method of the first temporary data generated by the target model during the forward propagation process is determined, the first temporary data can be processed based on the data processing method of the first temporary data. In this way, by unloading and reloading part of the temporary data in the multiple temporary data, it is possible to effectively manage and save the target video memory without affecting the speed of model training. In addition, the technical drawbacks of reducing the use of video memory by sacrificing computing efficiency in the related art are avoided. Therefore, the data processing method for model training provided in the embodiment of the present application can reduce the video memory occupancy of model training without loss of training speed.

[0131] Further, based on the same technical concept, the embodiment of the present application provides a data processing device for model training, and the data processing device for model training is used to implement the above method flow of the embodiment of the present application. Figure 7 As shown, the data processing device 700 for model training includes: a mode determination module 701 and a data processing module 702, wherein:

[0132] A mode determination module 701 is used to respond to the model training operation for the target model and determine the data processing mode of the first temporary data generated by the target model during the forward propagation process; the data processing mode is used to indicate: whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the backward propagation process;

[0133] The data processing module 702 is used to perform data processing on the first temporary data based on the data processing method of the first temporary data.

[0134] In an optional embodiment, before responding to the model training operation for the target model, the mode determination module 701 is further used to:

[0135] Record multiple temporary data generated by the target model during the forward propagation process, and their corresponding generation time and usage time; and,

[0136] Performing time consumption tests on multiple temporary data respectively, and obtaining unloading time consumption and reloading time consumption corresponding to the multiple temporary data respectively; wherein each unloading time consumption represents the time required to unload the corresponding temporary data from the target video memory, and each reloading time consumption represents the time required to reload the corresponding temporary data into the target video memory;

[0137] Based on the generation time, use time, unloading time and reloading time respectively corresponding to the plurality of temporary data, the data processing methods respectively corresponding to the plurality of temporary data are determined.

[0138] In an optional embodiment, in the process of recording the generation time and usage time of multiple temporary data generated by the target model during the forward propagation process, the data processing device 700 for model training may further include a data cache module 703, and the data cache module 703 is specifically used to:

[0139] Cache multiple temporary data through the target video memory, and read the cumulative usage of the target video memory in real time;

[0140] When it is determined that the accumulated occupancy reaches a preset occupancy threshold, at least one temporary data not stored in the plurality of temporary data is cached by the standby video memory.

[0141] In an optional embodiment, when determining the data processing modes respectively corresponding to the plurality of temporary data based on the generation time, usage time, unloading time and reloading time respectively corresponding to the plurality of temporary data, the mode determination module 701 is specifically used to:

[0142] Based on the generation time and the use time respectively corresponding to the multiple temporary data, the data cache duration respectively corresponding to the multiple temporary data is obtained, and based on the unloading time and the reloading time respectively corresponding to the multiple temporary data, the data processing duration respectively corresponding to the multiple temporary data is obtained;

[0143] Based on the data cache duration, data processing duration and video memory occupancy respectively corresponding to the multiple temporary data, a data processing strategy for the multiple temporary data is determined; the data processing strategy includes: a data processing method respectively corresponding to the multiple temporary data.

[0144] In an optional embodiment, when determining the data processing method of the first temporary data generated by the target model during the forward propagation process, the method determination module 701 is specifically used to:

[0145] Obtaining a data identifier of the first temporary data; the data identifier is a unique identifier generated according to the data type of the first temporary data;

[0146] The data processing method associated with the data identifier is used as the data processing method of the first temporary data.

[0147] In an optional embodiment, when the first temporary data is processed based on the data processing method of the first temporary data, the data processing module 702 is specifically used to:

[0148] If the data processing mode of the first temporary data is the first processing mode, then in the forward propagation process, according to the generation time of the first temporary data, the first temporary data is unloaded from the target video memory and stored in the preset memory, and in the reverse propagation process, according to the use time of the first data and the reload time determined by the reload time, the first temporary data is reloaded from the preset memory to the target video memory;

[0149] If the data processing method of the first temporary data is the second processing method, the first temporary data is cached in the target video memory, and the first temporary data is called at the usage time of the first temporary data.

[0150] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0151] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0152] An embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0153] See also Figure 8 As shown, the structured block diagram of the electronic device 800 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0154] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0155] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 may be any type of device capable of inputting information to the electronic device 800, and the input unit 806 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 807 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 808 may include but is not limited to a disk, an optical disk. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a worldwide interoperability for microwave access (WiMax) device, a cellular communication device, and / or the like.

[0156] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, CPUs, GPUs, various AI computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above. For example, in some embodiments, the data processing method for the above-mentioned model training may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. In some embodiments, the computing unit 801 may be configured to perform the data processing method for the above-mentioned model training by any other appropriate means (e.g., by means of firmware).

[0157] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0158] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tub (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0161] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0162] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0163] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present application.

Claims

1. A data processing method for model training, characterized in that: include: In response to a model training operation for a target model, determining a data processing method for first temporary data generated by the target model during a forward propagation process; The data processing mode is used to indicate whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the backward propagation process; The first temporary data is processed based on the data processing method of the first temporary data.

2. The method according to claim 1, characterized in that Before the model training operation for the target model is performed, the method further includes: Recording a plurality of temporary data generated by the target model during the forward propagation process, and the corresponding generation time and usage time; and, Performing time consumption tests on the multiple temporary data respectively to obtain unloading time consumptions and reloading time consumptions corresponding to the multiple temporary data respectively; wherein each unloading time consumption represents the time required to unload the corresponding temporary data from the target video memory, and each reloading time consumption represents the time required to reload the corresponding temporary data into the target video memory; Based on the generation time, the use time, the unloading time and the reloading time respectively corresponding to the multiple temporary data, the data processing modes respectively corresponding to the multiple temporary data are determined.

3. The method according to claim 2, characterized in that The process of recording the multiple temporary data generated by the target model during the forward propagation process, and the corresponding generation time and usage time, also includes: caching the plurality of temporary data by using the target video memory, and reading the accumulated occupancy of the target video memory in real time; When it is determined that the accumulated occupancy reaches a preset occupancy threshold, at least one temporary data not stored in the plurality of temporary data is cached by using a standby video memory.

4. The method according to claim 2, characterized in that The determining of the data processing modes respectively corresponding to the plurality of temporary data based on the generation time, the use time, the unloading time and the reloading time respectively corresponding to the plurality of temporary data includes: Based on the generation time and the use time respectively corresponding to the multiple temporary data, obtain the data cache duration respectively corresponding to the multiple temporary data, and based on the unloading time and the reloading time respectively corresponding to the multiple temporary data, obtain the data processing duration respectively corresponding to the multiple temporary data; Based on the data cache duration, the data processing duration and the video memory occupancy respectively corresponding to the multiple temporary data, a data processing strategy for the multiple temporary data is determined; the data processing strategy includes: the data processing methods respectively corresponding to the multiple temporary data.

5. The method according to claim 1, characterized in that The data processing method for determining the first temporary data generated by the target model during the forward propagation process includes: Acquire a data identifier of the first temporary data; the data identifier is a unique identifier generated according to the data type of the first temporary data; The data processing method associated with the data identifier is used as the data processing method of the first temporary data.

6. The method according to any one of claims 1 to 5, characterized in that The performing data processing on the first temporary data in a data processing manner based on the first temporary data includes: If the data processing method of the first temporary data is the first processing method, then in the forward propagation process, according to the generation time of the first temporary data, the first temporary data is unloaded from the target video memory and stored in a preset memory, and in the reverse propagation process, according to the use time of the first data and the reload time determined by the reload time, the first temporary data is reloaded from the preset memory to the target video memory; If the data processing method of the first temporary data is the second processing method, the first temporary data is cached in the target video memory, and the first temporary data is called when the first temporary data is used.

7. A data processing device for model training, characterized in that: include: A mode determination module, for responding to a model training operation for a target model and determining a data processing mode for first temporary data generated by the target model during a forward propagation process; The data processing mode is used to indicate whether to unload the first temporary data from the target video memory of the target model during the forward propagation process, and reload the first temporary data into the target video memory during the backward propagation process; A data processing module is used to perform data processing on the first temporary data based on the data processing method of the first temporary data.

8. The device according to claim 7, characterized in that Before the model training operation is performed in response to the target model, the mode determination module is further used to: Recording a plurality of temporary data generated by the target model during the forward propagation process, and the corresponding generation time and usage time; and, Performing time consumption tests on the multiple temporary data respectively to obtain unloading time consumptions and reloading time consumptions corresponding to the multiple temporary data respectively; wherein each unloading time consumption represents the time required to unload the corresponding temporary data from the target video memory, and each reloading time consumption represents the time required to reload the corresponding temporary data into the target video memory; Based on the generation time, the use time, the unloading time and the reloading time respectively corresponding to the multiple temporary data, the data processing modes respectively corresponding to the multiple temporary data are determined.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

Images