Method, medium, device and program product for controlling data transmission between memories

Abstract

This application provides a method, apparatus, computer-readable storage medium, electronic device, and computer program product for controlling data transfer between memory locations, relating to the field of data processing technology. The method includes: determining the intersection of a j-th feature set and a (j+1)-th feature set to obtain high-frequency features, where the j-th feature set is the feature input into the model during the j-th time period of training the target model, and the (j+1)-th feature set is the feature input into the model during the (j+1)-th time period of training the target model; controlling the high-frequency features to be stored in a first memory location during both the j-th and (j+1)-th time periods; determining a first feature based on features other than the high-frequency features in the j-th feature set; controlling the transfer of the first feature from the first memory to a second memory location; and controlling the transfer of a second feature from the (j+1)-th feature set (excluding the high-frequency feature set) to the first memory location. This application embodiment can save bandwidth and facilitates the rational use of computer resources.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a control method for data transmission between memory, a control device for data transmission between memory, a computer-readable storage medium, an electronic device, and a computer program product. Background Technology

[0002] Graphics Processing Unit (GPU), due to its highly parallelized architecture, is suitable for accelerating the training process of machine learning or deep learning models to handle large-scale matrix operations and floating-point calculations during model training.

[0003] In the process of model training using GPUs, the relevant technologies typically involve swapping in and out all the model training data stored in the GPU's video memory. For example, after one iteration of training is completed, the features (such as embedding vectors or parameters) corresponding to the batch of samples required for that iteration are written back to memory. Then, the features corresponding to the batch of samples required for the next iteration are written from memory to the GPU memory. Iterative training can then be performed based on the features corresponding to the next batch of samples.

[0004] However, using the memory-to-memory data transfer method provided by related technologies for model training is not conducive to the rational use of bandwidth resources. Summary of the Invention

[0005] This application provides a method for controlling data transmission between memory, a device for controlling data transmission between memory, a computer-readable storage medium, an electronic device, and a computer program product, which can save bandwidth and facilitate the rational use of computer resources.

[0006] In a first aspect, embodiments of this application provide a method for controlling data transfer between memory locations. The method includes: determining the intersection of a j-th feature set and a (j+1)-th feature set to obtain high-frequency features, wherein the j-th feature set and the (j+1)-th feature set are respectively features input into the model during the j-th time period and the (j+1)-th time period of training the target model, respectively, and j is a positive integer; controlling the storage of the high-frequency features in a first memory location during the j-th and (j+1)-th time periods; determining a first feature based on features in the j-th feature set other than the high-frequency features; controlling the transfer of the first feature from the first memory to a second memory location; and controlling the transfer of a second feature in the (j+1)-th feature set other than the high-frequency features from the second memory to the first memory location.

[0007] In the inter-memory data transfer control method provided in the first aspect of this application, the intersection of the model input features corresponding to adjacent time periods is taken to obtain the high-frequency features of the adjacent time periods. Furthermore, within the adjacent time periods, the high-frequency features reside in the local memory (i.e., the first memory) of the processor training the target model, without additional communication transmission. Only the non-high-frequency features in the model input features corresponding to the two time periods are transmitted. This ensures accurate updating of the model gradient while reducing feature transmission between different memories, thereby saving bandwidth and improving the utilization of computing resources.

[0008] In an exemplary embodiment, in conjunction with the above scheme, in the first implementation, the processor for training the target model is a graphics processing unit (GPU), and the first memory includes a first cache space and a second cache space; wherein, during the j-th time period, the j-th feature set is stored in the first cache space; controlling the transfer of the second feature in the (j+1)-th feature set, excluding the high-frequency feature set, from the second memory to the first memory includes: during the j-th time period, controlling the transfer of the second feature from the second memory to the second cache space.

[0009] The solution provided in this implementation introduces a dual-buffer mechanism by setting up video memory, combined with pipelined parallel technology, to arrange incremental feature exchange and model training into two stages, achieving parallel execution. Specifically, during the model training process in the previous time period, the aforementioned second feature is transferred in parallel from the aforementioned second memory to the aforementioned second cache space.

[0010] In an exemplary embodiment, in conjunction with the first implementation of the above scheme, in the second implementation, the control of the first feature to be transferred from the first memory to the second memory includes: before the (j+1)th time period, controlling the second feature and the first feature to be swapped by the GPU kernel, so as to transfer the second feature from the second cache space to the first cache space, and to transfer the first feature from the first cache space to the second cache space; and during the (j+1)th time period, controlling the first feature to be moved into the video memory.

[0011] This implementation achieves the effect of parallel execution of incremental feature swapping and model training. Specifically, during model training in the latter time period, the first feature is removed from GPU memory in parallel. Furthermore, the GPU kernel controls the swapping of the second and first features during the interval between the two time periods. Since the swap operation has a short execution time (e.g., a few milliseconds), even if the swap operation is executed between two adjacent time periods, it has little impact on GPU performance.

[0012] In an exemplary embodiment, in conjunction with the first implementation of the above scheme, in the third implementation, controlling the high-frequency feature to be stored in the first memory during the j-th time period and the (j+1)-th time period includes: controlling the high-frequency feature to be stored in the first cache space during the j-th time period and the (j+1)-th time period.

[0013] In this implementation, the aforementioned high-sensitivity features are kept resident in GPU memory during model training in two adjacent time periods, thus eliminating redundant memory-GPU memory transfers. Only features other than these high-frequency features are transferred between memory to ensure smooth model training. This method reduces the order of magnitude of parameter exchanges under multi-level caching, thereby saving bandwidth and improving the utilization of computing resources.

[0014] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the fourth implementation method, the first feature is determined based on the features in the j-th feature set other than the high-frequency features, including: when any feature in the j-th feature set other than the high-frequency features is accessed, its timestamp is updated and the time of the last access is recorded to obtain a feature list sorted by access time; and when the occupancy rate of the first memory is greater than a preset threshold, a preset number of features are determined from the head of the feature list as the first feature.

[0015] In this implementation scheme, in order to improve the memory hit rate and reduce unnecessary memory swapping operations, the Least Recently Used (LRU) strategy is adopted to identify the least recently used features among the non-high-frequency features as the features to be removed from the memory, while retaining the features that have been used recently, thereby increasing the probability that these features will still be in the memory when they are needed again.

[0016] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the fifth implementation method, the j-th feature set includes features corresponding to the first samples of M batches, and the (j+1)-th feature set includes features corresponding to the second samples of N batches, where M and N are integers greater than 1.

[0017] In the solution provided by this implementation, a feature set can contain features corresponding to samples from multiple batches. On the one hand, compared to containing features corresponding to samples from only one batch, this embodiment of the application is beneficial for enabling processors (such as GPUs) to accelerate more efficiently, thereby reducing the frequency of data switching between GPU memory and RAM. On the other hand, the model training time for multiple batches is longer than the time required for feature prefetching under the current multiple batches, thereby ensuring the smooth execution of the model training process.

[0018] In an exemplary embodiment, in conjunction with the fifth implementation of the above scheme, in the sixth implementation, the method further includes: before the (j+1)th time period, obtaining the second samples of the above N batches to obtain a first sample set; deduplicating the first sample set to obtain a second sample set; and obtaining the (j+1)th feature set based on the feature identifiers of the samples in the second sample set.

[0019] This implementation provides a scheme for feature prefetching. Specifically, it swaps the parameters required for model training in multiple iterations into GPU memory in a batch at once, while ensuring that high-frequency parameters remain resident in GPU memory. This avoids frequent and redundant data swapping between memory and GPU, and also ensures that multiple iterations are completed on the GPU based on synchronous training, guaranteeing convergence speed and training accuracy. It can also reduce unnecessary GPU waiting time, which is beneficial to ensuring model training efficiency.

[0020] In an exemplary embodiment, in conjunction with the sixth implementation of the above scheme, in the seventh implementation, the method further includes: determining the cache space of the (j+1)th feature set based on the amount of data in the (j+1)th feature set; wherein, if the amount of data exceeds a preset second threshold, the cache space is determined to include the random access memory (RAM) and the large-capacity storage device in the second memory; if the amount of data does not exceed the preset second threshold, the cache space is determined to include the RAM in the second memory.

[0021] This implementation is based on the multi-level caching provided by this solution, which offers flexible caching methods for feature sets with different data volumes.

[0022] In an exemplary embodiment, in conjunction with the sixth implementation of the above scheme, in the eighth implementation, before determining the intersection between the j-th feature set and the (j+1)-th feature set to obtain the high-frequency feature set, the method further includes: obtaining the first samples of the above M batches to obtain a third sample set; deduplicating the above third sample set to obtain a fourth sample set; and obtaining the above j-th feature set based on the feature identifiers of the samples in the above fourth sample set.

[0023] In the feature set determination scheme provided by this implementation, since multiple batches of samples are obtained at once (e.g., the j-th feature set contains features corresponding to M batches of samples), the samples are deduplicated after the M batches of samples are determined. This can effectively reduce computation and transmission overhead while maintaining low memory usage, and is especially suitable for scenarios involving large-scale sparse embeddings or high-frequency features.

[0024] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the ninth implementation method, the first memory is the local memory of the processor that trains the target model.

[0025] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the tenth implementation method, the above target model is a sparse model.

[0026] Secondly, embodiments of this application provide a control device for inter-memory data transmission, the device comprising: a first determining module, a first control module, a second determining module, a second control module, and a third control module.

[0027] The first determining module is used to determine the intersection between the j-th feature set and the (j+1)-th feature set to obtain high-frequency features, wherein the j-th feature set and the (j+1)-th feature set are the features input into the model during the j-th time period and the (j+1)-th time period, respectively, and j is a positive integer; the first control module is used to control the storage of the high-frequency features in the first memory during the j-th time period and the (j+1)-th time period; the second determining module is used to determine a first feature based on the features in the j-th feature set other than the high-frequency features; the second control module is used to control the transfer of the first feature from the first memory to the second memory; and the third control module is used to control the transfer of the second feature in the (j+1)-th feature set other than the high-frequency feature set from the second memory to the first memory.

[0028] In an exemplary embodiment, in conjunction with the above scheme, in the first implementation, the processor for training the target model is a graphics processing unit (GPU), and the first memory includes a first cache space and a second cache space; wherein, during the j-th time period, the j-th feature set is stored in the first cache space;

[0029] The third control module is specifically used to: control the transfer of the second feature from the second memory to the second cache space during the j-th time period.

[0030] In an exemplary embodiment, in conjunction with the first implementation of the above scheme, in the second implementation, the second control module is specifically used to: before the (j+1)th time period, control the GPU kernel to swap the second feature and the first feature, so as to transfer the second feature from the second cache space to the first cache space, and transfer the first feature from the first cache space to the second cache space; and during the (j+1)th time period, control the first feature to transfer from the second cache space to the second memory.

[0031] In an exemplary embodiment, in conjunction with the first implementation of the above scheme, in the third implementation, the first control module is specifically used to: control the high-frequency features to be stored in the first cache space during the j-th time period and the (j+1)-th time period.

[0032] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the fourth implementation method, the second determining module is specifically used to: when any feature in the j-th feature set other than the high-frequency feature is accessed, update its timestamp and record the time of the last access to obtain a feature list sorted by access time; and when the occupancy rate of the first memory is greater than a preset threshold, determine a preset number of features starting from the head of the feature list as the first feature.

[0033] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the fifth implementation method, the j-th feature set includes features corresponding to the first samples of M batches, and the (j+1)-th feature set includes features corresponding to the second samples of N batches, where M and N are integers greater than 1.

[0034] In an exemplary embodiment, in conjunction with the fifth implementation of the above scheme, in the sixth implementation, the above device further includes: a feature pre-fetching module;

[0035] The aforementioned acquisition module includes: a first acquisition unit, a deduplication unit, and a second acquisition unit; the first acquisition unit is used to: acquire the second samples of the N batches before the (j+1)th time period to obtain a first sample set; the deduplication unit is used to: deduplicate the first sample set to obtain a second sample set; and the second acquisition unit is used to: acquire the (j+1)th feature set based on the feature identifiers of the samples in the second sample set.

[0036] In an exemplary embodiment, in conjunction with the sixth implementation of the above scheme, in the seventh implementation, the above device further includes: a cache module; wherein the cache module is used to: determine the cache space of the (j+1)th feature set based on the amount of data in the (j+1)th feature set; wherein, if the amount of data exceeds a preset second threshold, the cache space is determined to include the random access memory (RAM) and the mass storage device in the second memory; if the amount of data does not exceed the preset second threshold, the cache space is determined to include the RAM in the second memory.

[0037] In an exemplary embodiment, in conjunction with the sixth implementation of the above scheme, in the eighth implementation, the prefetching module is further configured to: obtain the first samples of the above M batches to obtain the third sample set before the first determining module determines the intersection between the j-th feature set and the (j+1)-th feature set to obtain the high-frequency feature set; remove duplicates from the above third sample set to obtain the fourth sample set; and obtain the j-th feature set according to the feature identifiers of the samples in the above fourth sample set.

[0038] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the ninth implementation method, the first memory is the local memory of the processor that trains the target model.

[0039] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the tenth implementation method, the above target model is a sparse model.

[0040] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the inter-memory data transfer control method provided in the first aspect.

[0041] Fourthly, embodiments of this application provide a chip for implementing the inter-memory data transfer control method provided in the first or second aspect. Specifically, the chip includes a processor for retrieving and running a computer program from a memory, causing a device equipped with the chip to execute the inter-memory data transfer control method provided in the first aspect.

[0042] Fifthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to execute the inter-memory data transfer control method provided in the first aspect.

[0043] Sixthly, embodiments of this application provide a computer program product, including computer program instructions, which cause a computer to execute the inter-memory data transfer control method provided in the first aspect.

[0044] In a seventh aspect, embodiments of this application provide a computer program that, when run on a computer, causes the computer to execute the inter-memory data transfer control method provided in the first aspect.

[0045] In summary, the memory data transfer control scheme provided in this application's embodiments obtains the high-frequency features of the adjacent time periods by taking the intersection of the model input features corresponding to each adjacent time period. Furthermore, within the adjacent time periods, the high-frequency features reside in the local memory (i.e., the first memory) of the processor training the target model, without additional communication transmission. Only the low-frequency features of the model input features corresponding to the two time periods are transmitted. This ensures accurate model gradient updates while reducing feature transmission between different memory locations, thereby saving bandwidth and improving computational resource utilization. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating a method for controlling inter-memory data transfer according to an embodiment of this application;

[0048] Figure 2 A schematic diagram of a multi-level cache provided in an embodiment of this application;

[0049] Figure 3This is a schematic diagram illustrating a prefetching of a feature set, provided as an embodiment of this application.

[0050] Figure 4A A schematic diagram illustrating the transfer of data between memory locations during model training for related technologies;

[0051] Figure 4B This application provides a schematic diagram illustrating the transfer of data between memory locations during model training.

[0052] Figure 5 A comparative diagram of the full copy scheme and the incremental copy scheme of the feature set provided in the embodiments of this application;

[0053] Figure 6A A schematic diagram comparing the model training time and the time required for feature prefetching in the case of a batch of samples provided in an embodiment of this application;

[0054] Figure 6B A schematic diagram illustrating the relationship between batch quantity and deduplicated feature quantity provided in this application embodiment;

[0055] Figure 6C A schematic diagram comparing the model training time and the time required for feature prefetching in the case of multiple batches of samples provided in the embodiments of this application;

[0056] Figure 7 A schematic diagram illustrating the parallel execution scheme for memory key output transmission and model training provided in an embodiment of this application;

[0057] Figure 8 A schematic block diagram of a control device for inter-memory data transfer provided in an embodiment of this application;

[0058] Figure 9 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0060] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein. In embodiments of this application, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0061] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0062] Due to factors such as the distribution characteristics of sample data and sample partitioning strategies, there is a high overlap rate between samples in different batches, especially between samples in adjacent batches. Regarding data distribution characteristics, specifically, in many real-world business scenarios, some users or items may be more active than others. For example, in recommendation systems, popular items frequently appear in the interaction records of different users; similarly, active users interact with more items. Therefore, the sample information of these popular items and active users will appear repeatedly in multiple batches, leading to a high overlap rate. Regarding sample partitioning strategies, if the training data is partitioned according to chronological order or other highly continuous logic, then data points in adjacent batches are likely to be closely related. For example, in time-sorted log data, the characteristics (such as behavioral characteristics) of users within adjacent time periods are often similar because user interests do not change drastically in a short period, which leads to sample overlap in adjacent batches.

[0063] For example, analysis of real-world business data from typical scenarios reveals that the feature overlap rate between two adjacent batches is 60%-75%, which translates to approximately 10G of embedding features. Therefore, it can be concluded that in real-world business data, when training on a batch-by-batch basis, the feature overlap rate between adjacent batches is relatively high.

[0064] If the solution provided by the relevant technology is adopted, which involves writing all the features corresponding to the samples required for the current iteration back to memory after each iteration of training, and then writing all the features corresponding to the samples required for the next iteration back to GPU memory, then due to the high overlap between the two batches of samples, these overlapping features will undergo two repeated transfer processes from GPU memory to memory and back again. This repeated communication of overlapping features also wastes bandwidth to some extent, hindering the rational use of computer resources and impeding the improvement of model training efficiency.

[0065] To address the aforementioned issues, the inter-memory data transfer control scheme provided in this application determines the intersection between the j-th feature set and the (j+1)-th feature set, thereby obtaining the high-frequency features from the two feature sets. The j-th feature set is the feature input to the target model during the j-th time period, and similarly, the (j+1)-th feature set is the feature input to the target model during the (j+1)-th period. Further, during the j-th and (j+1)-th time periods, the high-frequency features are controlled to be stored in the local memory (i.e., the first memory) of the processor training the target model, without additional communication transmission. For the non-high-frequency features in the aforementioned feature sets, transmission occurs between the first memory and the second memory to ensure accurate gradient updates for the corresponding time periods. Specifically, for the j-th feature set required for the j-th time period, a first feature is identified from its non-high-frequency features and transmitted to the second memory. For the non-high-frequency features (i.e., the second features) in the (j+1)-th feature set required for the (j+1)-th time period, they are transmitted to the first memory for gradient updates of the model during the (j+1)-th time period.

[0066] As can be seen, in this embodiment, the intersection of the model input features corresponding to adjacent time periods is taken to obtain the high-frequency features of the adjacent time periods. Furthermore, within the adjacent time periods, the high-frequency features reside in the local memory (i.e., the first memory) of the processor training the target model, without additional communication transmission. Only the low-frequency features of the model input features corresponding to the two time periods are transmitted. This ensures accurate updating of the model gradient while reducing feature transmission between different memory locations, thereby saving bandwidth and improving the utilization of computing resources.

[0067] The following describes in detail the control method for inter-memory data transfer according to embodiments of this application. These embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0068] Figure 1 This is a flowchart illustrating a control method P100 for inter-memory data transfer provided in an embodiment of this application. The execution entity of method P100 is the central processing unit (CPU) of a computing device such as a server or terminal.

[0069] In step S110, the intersection between the j-th feature set and the (j+1)-th feature set is determined to obtain high-frequency features. The j-th feature set and the (j+1)-th feature set are the features input into the model during the j-th time period and the (j+1)-th time period, respectively, when training the target model. The value of j is a positive integer.

[0070] To reduce memory usage, improve training efficiency, and enhance model generalization ability by randomizing sample order, training data is typically not provided to the model all at once during training. Instead, it is divided into several batches, and each gradient update of the model processes only one batch of data. In this embodiment, the aforementioned feature set can contain features corresponding to samples from multiple batches. Compared to containing features corresponding to samples from only one batch, this embodiment allows for more efficient acceleration by the processor (such as a GPU), thereby reducing the frequency of data switching between GPU memory and system memory. For example, the j-th feature set includes features corresponding to samples from M batches (denoted as the first sample), and the (j+1)-th feature set includes features corresponding to samples from N batches (denoted as the second feature), where M and N are integers greater than 1.

[0071] As can be seen, the aforementioned time period is related to the duration of gradient updates for the target model. Specifically, if the batch size of samples in the current feature set is M, then the time period corresponding to the current feature set contains M gradient update durations. Specifically, gradient updates are used to optimize model parameters during model training. A single gradient update process can include the following steps: First, a forward propagation process is performed, inputting the samples of the current batch and their corresponding features (such as embedding features) into the model, calculating the model output based on this, and evaluating the loss based on the model output and sample labels; then, a backpropagation process is performed, clearing the gradients and calculating the gradient of each parameter; finally, parameter updates are performed, i.e., updating the model parameters based on the calculated gradients and the optimization algorithm.

[0072] During model training, to improve training efficiency, it is necessary to quickly obtain the embedding vectors or embedding features corresponding to the current batch of input samples. The example target model mentioned above is a sparse model. A sparse model refers to a machine learning or deep learning model designed with a sparse parameter structure, meaning that most weights in the model are set to zero, and only a few non-zero weights participate in the calculation. This type of model optimizes performance, reduces computational complexity, and saves storage space by reducing unnecessary connections and parameters.

[0073] Embedding, as a key technology for transforming sparse features into useful information, plays an indispensable role in building efficient artificial intelligence models, especially in applications that require processing large amounts of sparse data. It not only bridges the gap between raw data and advanced models but also serves as an effective means of overcoming the many challenges posed by sparse data. In this embodiment, caching these vectors can reduce data transmission latency and accelerate training. To cache the embedding vectors corresponding to model training samples, this embodiment provides, as follows: Figure 2 The multi-layered caching mechanism shown is designed to improve caching capabilities.

[0074] Figure 2 This is a schematic diagram of a multi-level cache provided in an embodiment of this application. (Reference) Figure 2 The multi-layered caching mechanism includes: a first-level memory, such as High-Bandwidth Memory (HBM); a second-level memory, such as System RAM; and high-capacity storage devices (such as Solid State Drives (SSDs)). This allows for flexible data storage through a combination of three-level caching and distributed caching. The first-level memory can contain multiple distributed HBMs; for example, the aforementioned high-frequency characteristics are stored in the first-level memory using a distributed hash table. The second-level memory (System RAM) can transfer data to the multiple HBMs in the first-level memory through a distributed synchronous cache.

[0075] To ensure that the features required for each time period can be efficiently and quickly transferred to the first memory HBM, the data volume of the (j+1)th feature set can be estimated. If the data volume exceeds a preset second threshold, the buffer space is determined to be the system memory of the second memory and the large-capacity storage device. If the data volume does not exceed the preset second threshold, the buffer space is determined to be the system memory of the second memory.

[0076] Generally, after determining the i-th batch of samples, the corresponding embedding features are obtained based on the feature keys of each sample in that batch, thereby determining the model input information required for the gradient update of the i-th model. In this embodiment, since multiple batches of samples are obtained at once (e.g., the j-th feature set contains features corresponding to M batches of samples), after determining the M batches of samples, the samples are deduplicated first, which can effectively reduce computation and transmission overhead while maintaining low memory usage, especially suitable for scenarios involving large-scale sparse embeddings or high-frequency features. The specific implementation method for determining the j-th feature set is as shown in steps S11-S13.

[0077] In step S11, the first samples of the above M batches are obtained to obtain a sample set (denoted as the third sample set).

[0078] In step S12, duplicates are removed from the third sample set to obtain the fourth sample set.

[0079] In step S13, the j-th feature set is obtained based on the feature identifier (featherkey) of the samples in the fourth sample set.

[0080] For example, in an ad click prediction scenario, the deduplicated sample set includes Sample 1 and Sample 2. Specifically, Sample 1: For an ad displayed to user C, the feature keys may include advertiser_id = Z (advertiser ID), user_id = C (user ID), and possibly context_time = T (display time), etc. Sample 2: For another ad displayed to user D, the feature keys are advertiser_id = W, user_id = D, and context_time = S. Each sample has its unique feature keys, which are used to index the pre-trained or updated embedding vectors in the multi-level cache. After determining the feature keys of each sample, the corresponding embedding features are obtained from the relevant cache according to the mapping relationship, thus obtaining the above-mentioned j-feature sets.

[0081] The method for determining the (j+1)th feature set can be the same as that for the jth feature set, specifically including steps S21-S23.

[0082] In step S21, the second samples from the above N batches are obtained to obtain the first sample set;

[0083] In step S22, duplicate samples are removed from the first sample set to obtain the second sample set;

[0084] In step S23, the (j+1)th feature set is obtained based on the feature identifiers of the samples in the second sample set.

[0085] In other embodiments, the (j+1)th feature set can also be determined as follows: based on the feature identifiers of the samples in the first sample set, determine the embedded features corresponding to each sample, and further deduplicate the embedded features corresponding to the samples in the first sample set to obtain the (j+1)th feature set.

[0086] It is understandable that each of the above feature sets contains not only the embedded features corresponding to the samples, but also information such as the update of the optimizer state.

[0087] In an exemplary embodiment, to further improve model training efficiency, a strategy of pre-fetching model input information can also be adopted. Specifically, before executing the model training process corresponding to the current time period, the feature set corresponding to that time period is determined. Figure 3 This is a schematic diagram illustrating a prefetching of a feature set, provided as an embodiment of this application. Exemplary Reference Figure 3 The CPU executes a feature prefetching strategy as follows: before the GPU executes the training process for the j-th time period, the j-th feature set is obtained; similarly, before the GPU executes the training process for the (j+1)-th time period, the (j+1)-th feature set is obtained. In this embodiment, the feature prefetching strategy batch-loads the parameters required for future iterations into GPU memory at once, while ensuring that high-frequency parameters remain resident in GPU memory. This avoids frequent and redundant data swapping between memory and GPU, and also ensures that multiple iterations are completed synchronously within the GPU, guaranteeing convergence speed and training accuracy. Furthermore, it reduces unnecessary GPU waiting time, which is beneficial for ensuring model training efficiency.

[0088] Furthermore, after determining the j-th feature set and the (j+1)-th feature set, the overlap information between the two feature sets is further determined, i.e., the high-frequency features are identified. For example, set operations can be used to find common elements between two sets. In Python, for instance, the `set.intersection()` method or the `&` operator can be used to calculate the intersection of two sets.

[0089] Continue to refer to Figure 1 In step S120, during the j-th time period and the (j+1)-th time period, the high-frequency feature is controlled to be stored in the first memory.

[0090] Wherein, the first memory is the local memory of the processor that trains the target model.

[0091] For example, the processor for training the target model can be a GPU, in which case the first memory is Video Random Access Memory (VRAM). VRAM is the GPU's local memory used for temporary data storage to meet the GPU's high throughput requirements; VRAM can be HBM. For example, the processor for training the target model can also be a Neural Processing Unit (NPU), in which case the first memory is on-chip SRAM or on-chip cache, etc. On-chip SRAM or on-chip cache, as the NPU's local memory, can be used to store frequently used data or intermediate results, such as the high-frequency features mentioned above, to reduce reliance on external memory. It is understood that the embodiments of this application do not limit the type of processor used to train the target model; it can be other types of processors in the art, such as Tensor Processing Units (TPUs) or programmable logic devices such as Field Programmable Gate Arrays (FPGAs).

[0092] In this embodiment, during adjacent time periods of model training, the corresponding high-frequency features are kept in the first memory, thereby avoiding repeated transmission of high-frequency features between the first and second memory and reducing bandwidth waste.

[0093] Figure 4A This diagram illustrates the transfer of data between memory locations during model training, providing insights into related technologies. Figure 4B This application provides a schematic diagram illustrating the transfer of data between memory during model training, as illustrated in an embodiment of the present application. (Reference) Figure 4A For each feature set acquired by the CPU, the GPU performs a full copy; however, for feature sets corresponding to two adjacent time periods, overlapping features are moved out of and back into GPU memory, and repeated transmission leads to bandwidth waste and affects model training efficiency.

[0094] refer to Figure 4BFor each feature set acquired by the CPU, the GPU performs incremental copying to save bandwidth and improve model training efficiency. Specifically, for the (j-1)th and jth feature sets, the corresponding high-frequency features 41 can be identified as features 4, 5, and 6. Therefore, when the GPU copies the jth feature set, it only needs to copy the incremental features (features B, C, and D) to acquire all features in the jth feature set, ensuring smooth model training in the jth time interval. Similarly, since the high-frequency features 42 (features B, C, and F) belong to the intersection of the (j+1)th and jth feature sets, the GPU only needs to copy the incremental features (features A, D, and E) when copying the (j+1)th feature set. Obviously, in this implementation, within adjacent time intervals, the feature intersection of the feature sets corresponding to those adjacent time intervals is stored in the first memory without the need for memory transfer, thereby saving bandwidth. Furthermore, the smaller amount of data transferred helps ensure model training efficiency.

[0095] Continue to refer to Figure 1 In step S130, a first feature is determined based on the features in the j-th feature set other than the high-frequency feature. And in step S140, the first feature is controlled to be transferred from the first memory to the second memory.

[0096] After the j-th time period, features other than the aforementioned high-frequency features in the j-th feature set can be removed from the first memory. Specifically, even features not belonging to the aforementioned high-frequency features may still be accessed during model training. This is because, in this embodiment, to improve memory hit rate and reduce unnecessary memory swapping operations, the LRU strategy is used to retain recently used features, increasing the probability that these features will still be in memory when needed again. For example, when any feature in the j-th feature set other than the aforementioned high-frequency features is accessed, its timestamp is updated and the time of the last access is recorded, resulting in a feature list sorted by access time. Further, when the occupancy rate of the first memory is greater than a preset threshold (e.g., greater than 80%), a preset number of recently infrequently used features are determined from the head of the feature list as the aforementioned first features. For example, the aforementioned first features can be removed from memory all at once, or they can be removed from memory in several stages according to actual needs, thereby improving control flexibility.

[0097] Continue to refer to Figure 1 In step S150, the second feature in the (j+1)th feature set, excluding the high-frequency feature set, is controlled to be transferred from the second memory to the first memory.

[0098] Figure 5This diagram illustrates a comparison between a full copy scheme and an incremental copy scheme for the feature set provided in embodiments of this application. (Exemplary reference) Figure 5 In the solutions provided by related technologies, during the two time intervals of model training, the features corresponding to the previous time interval are all moved out of the cache, and then the features corresponding to the next time interval are all moved into the GPU memory, and then the model training process for the next time interval is executed. In the optimization scheme V1 provided in this application, the intersection between the j-th feature set and the (j+1)-th feature set is determined to obtain the high-frequency features. The high-frequency features are kept resident in the GPU memory during the two time intervals mentioned above, and during the two time intervals, the features in the k-th feature set (using LRU) are moved out of the GPU memory, and then the incremental features in the (k+1)-th feature set are moved into the GPU memory. Then, the model training process for the next time interval is executed based on the (j+1)-th feature set.

[0099] As previously stated, the optimization scheme V1 provided in this application reduces the transfer of features between memory locations, saving bandwidth and improving GPU performance. Compared to solutions provided by related technologies, the GPU performance is improved by x% in the optimization scheme V1 provided in this application.

[0100] After introducing the incremental exchange implementation example provided by optimization scheme V1, another technical effect of determining a feature set based on multiple batches and performing the incremental exchange implementation example as described in V1 will be introduced.

[0101] If a single batch is used to determine a feature set and incremental swapping is performed as described in V1 (which can be referred to as "single batch + incremental swapping"), then the performance requirements for each step of feature prefetching are very high, because the incremental features corresponding to the next batch need to be swapped into GPU memory within the training time corresponding to the current single batch, and the incremental features corresponding to the current batch also need to be swapped out of GPU memory. Figure 6A This diagram illustrates a comparison between model training time and feature prefetching time for a single batch of samples provided in this application embodiment. For example, during actual testing with advertising business data, it was found that... Figure 6A As shown: the training time t0 of a single batch is less than the time required for the feature prefetching process of that batch, which is t1. Here, t1 includes the input / output (I / O) time of the current batch's features, the time spent transferring data from the parameter server (PST) to RAM, and the time spent transferring data from RAM to HBM. The t1 time also includes the process of transferring incremental features from the previous batch from HBM to RAM. Therefore, it is typically impossible to complete the feature prefetching process for the samples in a single batch within the training time of a single batch.

[0102] Figure 6B This diagram illustrates the relationship between batch size and the number of deduplicated features, as provided in an embodiment of this application. Analysis of actual business data reveals that as the batch size increases linearly, the number of deduplicated features corresponding to the current batch size increases logarithmically. For example... Figure 6B As shown.

[0103] Since the logarithmic increment is less than the linear increment under the same conditions, it is possible to utilize, for example... Figure 6B The characteristics of the real business data distribution shown indicate that features corresponding to "batch" samples are used for prefetching to control the model training time of batch samples, ensuring it is longer than the time required for feature prefetching across multiple batches. For example, when the batch size increases from 1 to N, the corresponding model training time increases linearly by a factor of N; the corresponding number of deduplicated features increases logarithmically by a factor of less than N. Therefore, the parameter prefetching process can be completed within a relatively relaxed training time.

[0104] Figure 6C This diagram illustrates a comparison between model training time and feature prefetching time when multiple batches of samples are provided in the embodiments of this application. For example, Figure 6C As shown, the training time t0' of N batches is greater than the time required for the feature pre-fetching process of these N batches, which is t1', thus ensuring the smooth execution of the model training process. This is called the "batch + incremental interaction" method.

[0105] In summary, the heterogeneous training system for models provided in this application adopts a "batch + incremental" approach for feature prefetching. By aggregating multiple batches of data, new features are incrementally introduced, and old features are eliminated using an LRU (Least Recently Used) method. From the perspective of adapting to the distribution of business data, it has a more significant performance advantage compared to other frameworks.

[0106] In such Figure 5 In the memory key data transfer control scheme shown, writing the next feature set to video memory depends on writing the relevant data from the previous feature set back to memory. Therefore, "writing video memory to memory" and "writing memory to video memory" are essentially executed serially, and thus the parameter exchange between memory and video memory cannot be executed in parallel with the training process (cannot overlap). Therefore, in order to further improve the model training efficiency, this application embodiment also provides an optimization scheme V2.

[0107] Specifically, in such Figure 5In the related technology and optimization scheme V1 shown, the three stages RAM2HBM->PeriodTrain->HBM2RAM are executed serially within the same stage. This pipeline partitioning mechanism restricts the data exchange (RAM2HBM / HBM2RAM) and periodic training (PeriodTrain) stages from being executed in parallel (overlap), meaning that data exchange can only run at intervals between adjacent time periods. However, when the number of exchanged parameters is large, the data exchange stage takes a long time, resulting in long intervals between adjacent time periods. This fails to fully utilize the GPU's computing resources and is detrimental to model training efficiency.

[0108] The aforementioned stages are specific work units or tasks within the pipeline. Each stage is responsible for a specific part of the work in the pipeline and typically passes data or results sequentially to the next stage.

[0109] For example, in the optimization scheme V2 provided in the embodiments of this application, the strategy for optimizing the pipeline stage division is to no longer bind the two links "second memory to first memory (RAM2HBM)" and "first memory to first memory (HBM2RAM)" with the model training cycle (PeriodTrain) link and execute them serially within one stage, but instead treat RAM2HBM and HBM2RAM as two independent stages in the pipeline.

[0110] Figure 7 This is a schematic diagram illustrating a parallel execution scheme for memory key output transmission and model training provided in an embodiment of this application. To implement the above scheme, refer to... Figure 7 In optimization scheme V2, two cache spaces are provided in the GPU memory (i.e., the first cache space contains both a first cache space and a second cache space). During the j-th time period and the (j+1)-th time period, high-frequency features are stored in the first cache space. Specifically, during the j-th time period, the j-th feature set is stored in the first cache space to implement the model training process for the j-th time period. In parallel, during the j-th time period, the incremental portion of the j-th feature set (i.e., the second feature mentioned above) is moved from the second memory into the second cache space of the GPU memory.

[0111] Furthermore, after the k-th time interval, i.e., after completing the model training for the j-th time interval, a swap operation is performed through the GPU kernel to transfer the second feature from the second cache space of GPU memory to the first cache space, and to transfer the first feature in the j-th feature set from the first cache space of GPU memory to the second cache space. Since the swap operation takes a short time (e.g., a few milliseconds), even if the swap operation is performed between two adjacent time intervals, it has little impact on GPU performance.

[0112] During the (j+1)th time period, the (j+1)th feature set is stored in the first cache space mentioned above to implement the model training process during the (j+1)th time period. In parallel, during the (j+1)th time period, the features determined by the incremental part of the (j+1)th feature set (i.e., the first features mentioned above) are moved from the second cache space of the video memory to the second memory.

[0113] refer to Figure 7 In the optimization scheme V2 provided in this application embodiment, during the j-th time period, the incremental feature (second feature) in the (j+1)-th feature set is moved into the video memory, thereby achieving parallelism between the RAM2HBM stage and the PeriodTrain stage. S2 is executed during the interval between adjacent time periods to swap the first and second features. Since the GPU-Kernel's swap operation is relatively short, even if the swap operation is executed between two adjacent time periods, it has little impact on GPU performance. During the (j+1)-th time period, the first feature determined based on the incremental feature in the j-th feature set is moved out of the video memory, thereby achieving parallelism between the HBM2RAM stage and the PeriodTrain stage. Compared to the solutions provided by related technologies, the optimization scheme V2 provided in this application embodiment improves GPU performance by y%, where y > x. For example, in some embodiments, optimization scheme V1 improves GPU performance by approximately 10% compared to solutions provided by related technologies, and optimization scheme V2 improves GPU performance by approximately 20% compared to solutions provided by related technologies.

[0114] The memory data transfer control scheme provided in this application embodiment performs incremental feature transfer for adjacent feature sets. Specifically, for overlapping features in adjacent feature sets, a strategy is adopted to keep high-frequency features resident in GPU memory within the corresponding adjacent time periods, thus retaining them in GPU memory and avoiding redundant memory-GPU memory transfer. Only features other than the aforementioned high-frequency features are transferred between memory to ensure the smooth execution of the model training process. This method reduces the order of magnitude of parameter exchange under multi-level caching, thereby saving bandwidth and improving the utilization of computing resources.

[0115] Furthermore, a dual-buffer mechanism is introduced into the video memory, combined with pipelined parallelism, to arrange incremental feature swapping and model training into two stages, achieving parallel execution. Specifically, in the first time period, while training the model based on the current feature set in the first memory, the incremental part of the feature set required for the next stage can be transferred to the second cache space of the first memory; in the second time period, while training the model based on the current feature set in the first memory, the incremental part of the feature set required for the previous stage can be moved out of the first memory. During the two time period intervals, the video memory kernel performs a swap operation to exchange the incremental features. Since the execution time of the swap operation is short (e.g., a few milliseconds), even if the swap operation is executed in two adjacent time period intervals, it has little impact on GPU performance. It can be seen that by controlling the parallel execution of feature transfer in memory and model training, low-latency CPU devices and high-throughput GPU devices can be reasonably and efficiently matched to fully utilize the efficiency of the entire system resources.

[0116] The above text combined Figures 1 to 7 The method embodiments of this application are described below, in conjunction with Figure 8 An embodiment of the control device for inter-memory data transfer according to this application is described.

[0117] Figure 8 This is a schematic block diagram of a control device 800 for inter-memory data transmission provided in an embodiment of this application.

[0118] refer to Figure 8 The memory data transfer control device 800 provided in this application embodiment includes: a first determining module 810, a first control module 820, a second determining module 830, a second control module 840, and a third control module 850.

[0119] The first determining module 810 is used to determine the intersection between the j-th feature set and the (j+1)-th feature set to obtain high-frequency features, wherein the j-th feature set and the (j+1)-th feature set are the features input into the model during the j-th time period and the (j+1)-th time period, respectively, and j is a positive integer; the first control module 820 is used to control the storage of the high-frequency features in the first memory during the j-th time period and the (j+1)-th time period; the second determining module 830 is used to determine a first feature based on the features in the j-th feature set other than the high-frequency features; the second control module 840 is used to control the transfer of the first feature from the first memory to the second memory; and the third control module 850 is used to control the transfer of the second feature in the (j+1)-th feature set other than the high-frequency feature set from the second memory to the first memory.

[0120] In an exemplary embodiment, in conjunction with the above scheme, in the first implementation, the processor for training the target model is a graphics processing unit (GPU), and the first memory includes a first cache space and a second cache space; wherein, during the j-th time period, the j-th feature set is stored in the first cache space;

[0121] The third control module 850 is specifically used to: control the transfer of the second feature from the second memory to the second cache space during the j-th time period.

[0122] In an exemplary embodiment, in conjunction with the first implementation of the above scheme, in the second implementation, the second control module 840 is specifically used to: before the (j+1)th time period, control the GPU kernel to swap the second feature and the first feature, so as to transfer the second feature from the second cache space to the first cache space, and transfer the first feature from the first cache space to the second cache space; and during the (j+1)th time period, control the first feature to transfer from the second cache space to the second memory.

[0123] In an exemplary embodiment, in conjunction with the first implementation of the above scheme, in the third implementation, the first control module 820 is specifically used to: control the storage of the high-frequency features in the first cache space during the j-th time period and the (j+1)-th time period.

[0124] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the fourth implementation method, the second determining module 830 is specifically used to: when any feature in the j-th feature set other than the high-frequency feature is accessed, update its timestamp and record the time of the last access to obtain a feature list sorted by access time; and when the occupancy rate of the first memory is greater than a preset threshold, determine a preset number of features starting from the head of the feature list as the first feature.

[0125] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the fifth implementation method, the j-th feature set includes features corresponding to the first samples of M batches, and the (j+1)-th feature set includes features corresponding to the second samples of N batches, where M and N are integers greater than 1.

[0126] In an exemplary embodiment, in conjunction with the fifth implementation of the above scheme, in the sixth implementation, the above device further includes: a feature pre-fetching module;

[0127] The aforementioned acquisition module includes: a first acquisition unit, a deduplication unit, and a second acquisition unit; the first acquisition unit is used to: acquire the second samples of the N batches before the (j+1)th time period to obtain a first sample set; the deduplication unit is used to: deduplicate the first sample set to obtain a second sample set; and the second acquisition unit is used to: acquire the (j+1)th feature set based on the feature identifiers of the samples in the second sample set.

[0128] In an exemplary embodiment, in conjunction with the sixth implementation of the above scheme, in the seventh implementation, the above device further includes: a cache module; wherein the cache module is used to: determine the cache space of the (j+1)th feature set based on the amount of data in the (j+1)th feature set; wherein, if the amount of data exceeds a preset second threshold, the cache space is determined to include the random access memory (RAM) and the mass storage device in the second memory; if the amount of data does not exceed the preset second threshold, the cache space is determined to include the RAM in the second memory.

[0129] In an exemplary embodiment, in conjunction with the sixth implementation of the above scheme, in the eighth implementation, the prefetching module is further configured to: obtain the first samples of the above M batches to obtain the third sample set before the first determining module 810 determines the intersection between the j-th feature set and the (j+1)-th feature set to obtain the high-frequency feature set; remove duplicates from the above third sample set to obtain the fourth sample set; and obtain the j-th feature set according to the feature identifier of the samples in the above fourth sample set.

[0130] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the ninth implementation method, the first memory is the local memory of the processor that trains the target model.

[0131] In an exemplary embodiment, in combination with the above scheme or any one of the first to third implementation methods, in the tenth implementation method, the above target model is a sparse model.

[0132] It should be understood that, as Figure 8 The embodiment of the control device for inter-memory data transfer shown corresponds to the embodiment of the control method for inter-memory data transfer described above, and a similar description can be found in the method embodiment. To avoid repetition, it will not be repeated here. Specifically, through... Figure 8 The information interaction between the various modules in the control device for inter-memory data transfer shown can execute the embodiment of the control method for inter-memory data transfer described above, through, as... Figure 8 The information interaction between the modules in the control device for inter-memory data transmission shown can execute the embodiments of the control method for inter-memory data transmission described above. For the sake of brevity, the method embodiments corresponding to the aforementioned and other operations and / or functions of each module in the device will not be described again here.

[0133] The apparatus provided in this application embodiment can be applied to a heterogeneous training platform for artificial intelligence models. Through a multi-level parameter caching architecture, combined with a high-frequency parameter resident memory strategy and pipeline technology, it can reasonably and efficiently match low-latency CPU devices and high-throughput GPU devices to fully utilize the resource efficiency of the entire heterogeneous system, significantly improve the efficiency and performance of heterogeneous training of artificial intelligence models, and significantly reduce the cost of model training while ensuring model accuracy, thereby improving the efficiency and ceiling of model iteration.

[0134] The above description, in conjunction with the accompanying drawings, describes the operation and maintenance related apparatus of the software agent according to the embodiments of this application from the perspective of functional modules. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0135] This application also provides an electronic device.

[0136] Figure 9 This is a schematic block diagram of an electronic device 900 provided in an embodiment of this application. As described above, the operation and maintenance related devices of the software agent can be deployed in, for example... Figure 9 The electronic device shown can therefore be used to perform the control method for inter-memory data transfer described above.

[0137] like Figure 9 As shown, the electronic device 900 may include:

[0138] The system includes a memory 910 and a processor 920. The memory 910 stores a computer program 930 and transfers the program code 930 to the processor 920. In other words, the processor 920 can retrieve and run the computer program 930 from the memory 910 to implement the methods described in the embodiments of this application.

[0139] For example, the processor 920 can be used to execute the steps in the control method for inter-memory data transfer according to the instructions in the computer program 930, or to execute the steps in the control method for inter-memory data transfer.

[0140] In some embodiments of this application, the processor 920 may include, but is not limited to:

[0141] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0142] In some embodiments of this application, the memory 910 includes, but is not limited to:

[0143] Volatile memory and / or non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM).

[0144] In some embodiments of this application, the computer program 930 may be divided into one or more modules, which are stored in the memory 910 and executed by the processor 920 to complete the inter-memory data transfer control method provided in this application, or to complete the steps in the aforementioned inter-memory data transfer control method. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 930 in the electronic device.

[0145] like Figure 9 As shown, the electronic device 900 may further include:

[0146] Transceiver 940, which can be connected to processor 920 or memory 910.

[0147] The processor 920 can control the transceiver 940 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 940 may include a transmitter and a receiver. The transceiver 940 may further include antennas, and the number of antennas may be one or more.

[0148] It should be understood that the various components in the electronic device 900 are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0149] According to one aspect of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0150] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in the above-described method embodiments.

[0151] In other words, when implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0152] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0154] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0155] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method of data transmission between memories, characterized by, The method includes: Determine the intersection between the j-th feature set and the (j+1)-th feature set to obtain high-frequency features, wherein the j-th feature set and the (j+1)-th feature set are the features input into the model during the j-th time period and the (j+1)-th time period, respectively, and j takes the value of a positive integer; During the j-th time period and the (j+1)-th time period, the high-frequency feature is controlled to be stored in the first memory; The first feature is determined based on the features in the j-th feature set other than the high-frequency features; Control the transfer of the first feature from the first memory to the second memory; Control the transfer of the second feature in the (j+1)th feature set, excluding the high-frequency feature set, from the second memory to the first memory.

2. The method of claim 1, wherein, The processor used to train the target model is a GPU (Graphics Processing Unit), and the first memory includes a first cache space and a second cache space; wherein, during the j-th time period, the j-th feature set is stored in the first cache space; The control of transferring the second feature (excluding the high-frequency feature set) in the (j+1)th feature set from the second memory to the first memory includes: During the j-th time period, the second feature is controlled to be transferred from the second memory to the second cache space.

3. The method of claim 2, wherein, The control of the first feature being transferred from the first memory to the second memory includes: Before the (j+1)th time period, the second feature and the first feature are swapped by the GPU kernel control, so as to transfer the second feature from the second cache space to the first cache space, and transfer the first feature from the first cache space to the second cache space; During the (j+1)th time period, the first feature is controlled to be transferred from the second cache space to the second memory.

4. The method of claim 2, wherein, During the j-th time period and the (j+1)-th time period, controlling the storage of the high-frequency feature in the first memory includes: During the j-th time period and the (j+1)-th time period, the high-frequency features are controlled to be stored in the first cache space.

5. The method according to any one of claims 1 to 4, characterized in that, The step of determining the first feature based on features other than the high-frequency features in the j-th feature set includes: When any feature in the j-th feature set other than the high-frequency feature is accessed, its timestamp is updated and the time of the last access is recorded, resulting in a feature list sorted by access time. When the occupancy rate of the first memory exceeds a preset threshold, a preset number of features are determined starting from the head of the feature list and used as the first feature.

6. The method according to any one of claims 1 to 4, characterized in that, The j-th feature set includes features corresponding to the first samples of M batches, and the (j+1)-th feature set includes features corresponding to the second samples of N batches, where M and N are integers greater than 1.

7. The method of claim 6, wherein, The method further includes: Before the (j+1)th time period, the second samples of the N batches are obtained to obtain the first sample set; The first sample set is deduplicated to obtain the second sample set; Based on the feature identifiers of the samples in the second sample set, the (j+1)th feature set is obtained.

8. The method of claim 7, wherein, The method further includes: The cache space for the (j+1)th feature set is determined based on the amount of data in the (j+1)th feature set. Specifically, if the amount of data exceeds a preset second threshold, the buffer space is determined to include the random access memory (RAM) and large-capacity storage devices in the second memory; if the amount of data does not exceed the preset second threshold, the buffer space is determined to include the RAM in the second memory.

9. The method of claim 7, wherein, Before determining the intersection between the j-th feature set and the (j+1)-th feature set to obtain the high-frequency feature set, the method further includes: Obtain the first samples from the M batches to obtain the third sample set; The third sample set is deduplicated to obtain the fourth sample set; Based on the feature identifiers of the samples in the fourth sample set, the j-th feature set is obtained.

10. The method according to any one of claims 1 to 4, characterized in that, The first memory is the local memory of the processor that trains the target model.

11. The method according to any one of claims 1 to 4, characterized in that, The target model is a sparse model.

12. A control device for data transfer between memories, characterized by The device includes: The first determining module is used to determine the intersection between the j-th feature set and the (j+1)-th feature set to obtain high-frequency features, wherein the j-th feature set and the (j+1)-th feature set are the features input into the model in the j-th time period and the features input into the model in the (j+1)-th time period, respectively, and j takes the value of a positive integer; The first control module is used to control the high-frequency features to be stored in the first memory during the j-th time period and the (j+1)-th time period. The second determining module is used to determine the first feature based on the features in the j-th feature set other than the high-frequency features; The second control module is used to control the transfer of the first feature from the first memory to the second memory; The third control module is used to control the transfer of the second feature in the (j+1)th feature set, excluding the high-frequency feature set, from the second memory to the first memory.

13. A computer-readable storage medium, characterized in that, Used to store computer programs; The computer program causes the computer to perform the control method for inter-memory data transfer as described in any one of claims 1 to 11.

14. An electronic device, comprising: Including processor and memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the control method for inter-memory data transfer as described in any one of claims 1 to 11.

15. A computer program product, characterised in that, It includes computer instructions that, when executed by a processor, implement the control method for inter-memory data transfer as described in any one of claims 1 to 11.

Images