Sample data caching method and system, and computer device and storage medium

Abstract

A sample data caching method and system (600), and a computer device (700) and a storage medium. The method comprises: if decoding sample data matching first sample data in a sample data list is not present in a decoding sample data set of a cache region, determining alternative decoding sample data from the decoding sample data set; if the degree of importance of the first sample data and the degree of importance of the alternative decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoding sample data, performing decoding processing on the first sample data, so as to obtain corresponding target decoding sample data; and updating the decoding sample data set of the cache region.

Description

Sample data caching method, system, computer device and storage medium

[0001] Related applications

[0002] This application claims priority to the Chinese patent application filed on August 16, 2023, with application number 202311029639.8, and invention name “Sample data caching method, system, computer device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of deep neural network technology, and in particular to a sample data caching method, system, computer device, and storage medium. Background Art

[0004] Deep Neural Networks (DNN) are a framework for deep learning. A DNN model consists of multiple layers of computational units, with the output of one layer serving as the input for subsequent layers. DNN model training involves a forward propagation method, which sequentially passes information related to the input data through all model layers to generate predictions. For example, in image recognition applications, image pixel information passes through various model layers to predict the image content. To generate predictions, deep learning (DL) defines a cost / loss function based on the forward propagation output and the true label.

[0005] Large datasets are often used when training deep neural networks because they represent a variety of real-world scenarios. Current research focuses on optimizing the models themselves, such as quantization, pruning, and distillation, or on communication layer optimizations, such as NVLink and InfiniBand, as well as iterative upgrades of accelerator cards that provide higher-performance hardware such as GPUs (Graphics Processing Units) and TPUs (Tensor Processing Units). However, there has been less exploration of bottlenecks in data stream processing, and the IO (Input / Output) capabilities of storage resources have not kept pace with the growth of computing performance. As a result, IO resource consumption accounts for over 85% of the overall training time, especially in certain distributed training scenarios.

[0006] The DNN training process requires multiple iterations of large amounts of data, each called an epoch. During an epoch, all data items are processed only once, with data being read from disk into memory in mini-batches. During training, the I / O process loads data into a queue for preprocessing by the CPU (Central Processing Unit). After processing, the memory data is released, and the same data item must be read from disk into memory again in the next epoch. This adds redundant I / O and preprocessing operations, resulting in low training efficiency for deep neural networks.

[0007] Summary of the Invention

[0008] According to various embodiments of the present application, a sample data caching method, system, computer device, computer-readable storage medium, and computer program product are provided.

[0009] In a first aspect, the present application provides a sample data caching method. The method comprises:

[0010] Get a list of sample data for training;

[0011] If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set;

[0012] If the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, decoding the first sample data to obtain corresponding target decoding sample data;

[0013] The decoded sample data set in the cache area is updated according to the target decoded sample data.

[0014] In one embodiment, if there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set includes:

[0015] If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, obtaining the importance of each decoded sample data in the decoded sample data set;

[0016] The decoded sample with the smallest importance is determined as candidate decoded sample data.

[0017] In one embodiment, before obtaining a list of sample data for training, the method further includes:

[0018] Get the decoding sample dataset;

[0019] The importance of each sample data in the decoded sample data set is determined, and a target data sample set is determined from the decoded sample data set and the target data sample set is cached in a cache area.

[0020] In one embodiment, the decoded sample data set includes a plurality of batches of sample data, and determining the importance of each sample data in the sample data set includes:

[0021] Determining an entropy loss value of a single second sample data in each batch of sample data;

[0022] The individual second sample data in each batch of sample data is sorted according to the entropy loss value to obtain the importance of the individual second sample data; the importance is used to characterize the degree of influence of the second sample data on the accuracy of the deep neural network.

[0023] In one embodiment, sorting the individual second sample data in each batch of sample data according to the entropy loss value to obtain the importance of the individual second sample data includes:

[0024] Determining reference sample data as a reference from each batch of sample data, and obtaining an entropy loss value of the reference sample data;

[0025] By determining whether the entropy loss value of a single second sample data in each batch of sample data is greater than the entropy loss value of the reference sample data, the single second sample data in each batch of sample data is sorted to obtain the importance of the single second sample data.

[0026] In one embodiment, the method further includes: if the importance of the first sample data and the importance of the alternative decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is less than the decoding resource consumption of the alternative decoding sample data, then the decoding sample data set in the cache area is not updated.

[0027] In a second aspect, the present application further provides a sample data cache system, which includes a data layer, wherein the data layer includes: a sample list acquisition module, a candidate decoding sample determination module, a target decoding sample determination module, and a cache update module.

[0028] The sample list acquisition module is used to obtain a list of sample data for training.

[0029] The candidate decoding sample determination module is configured to determine candidate decoding sample data from the decoding sample data set if there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area.

[0030] The target decoding sample determination module is configured to decode the first sample data to obtain corresponding target decoding sample data if the importance of the first sample data and the importance of the candidate decoding sample data meet preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data.

[0031] A cache updating module is used to update the decoding sample data set in the cache area according to the target decoding sample data.

[0032] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0033] Get a list of sample data for training;

[0034] If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set;

[0035] If the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, decoding the first sample data to obtain corresponding target decoding sample data;

[0036] The decoded sample data set in the cache area is updated according to the target decoded sample data.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0038] Get a list of sample data for training;

[0039] If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set;

[0040] If the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, decoding the first sample data to obtain corresponding target decoding sample data;

[0041] The decoded sample data set in the cache area is updated according to the target decoded sample data.

[0042] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0043] Get a list of sample data for training;

[0044] If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set;

[0045] If the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, decoding the first sample data to obtain corresponding target decoding sample data;

[0046] The decoded sample data set in the cache area is updated according to the target decoded sample data.

[0047] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0049] FIG1 is a schematic diagram of resource consumption in various steps of a training process in some embodiments.

[0050] FIG2 is a diagram illustrating an application environment of a sample data caching method in some embodiments.

[0051] FIG3 is a flow chart of a sample data caching method in some embodiments.

[0052] FIG4 is a flowchart of a method for determining the importance of sample data in some embodiments.

[0053] FIG5 is a flow chart of a sample data caching method in some embodiments.

[0054] FIG6 is a structural block diagram of a sample data caching system in some embodiments.

[0055] FIG7 is a diagram illustrating the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] Deep neural networks (DNNs) are a deep learning framework that has been widely used in computer vision, natural language processing, robotics, autonomous driving, biomedicine, and other fields. DNN model training involves a forward propagation method, which sequentially passes information about the input data through all model layers to generate predictions. For example, in image recognition applications, image pixel information passes through various model layers to predict the image content. To generate predictions, DL (deep learning) defines a cost / loss function based on the forward propagation output and the true labels. The DL process aims to improve predictions by increasing or decreasing the weights of the model's intermediate layer outputs to minimize the cost function. This step is called backpropagation. The DL process uses gradient descent optimization techniques to adjust the parameters of the DL model, starting from the outermost layer and working back to the input layer. Gradient descent optimizes the parameters in the opposite direction of the gradient. DL training based on SGD (Stochastic Gradient Descent) simply applies random sampling, or shuffling, to generate a random order at the end of each training cycle, thereby treating all training examples equally. Shuffling refers to randomly shuffling the order of examples in a given dataset to prevent the model from learning sequential patterns in the data.

[0058] The deep neural network training process can be divided into four parts: raw data loading, data decoding, data augmentation, and GPU calculation and gradient update. Deep neural network training consumes resources. Figure 1 shows a schematic diagram of resource consumption at each stage of the training process in some embodiments, including I / O resource consumption, decoding resource consumption, GPU resource consumption, and graphics transformation resource consumption.

[0059] However, the training speed of deep neural networks plays a crucial role in the training process. To improve the training efficiency of deep neural networks, caching raw data can be used to reduce I / O resource consumption and improve data processing performance. However, current methods of improving training efficiency by caching raw data do not meet practical requirements. Therefore, to address the low training efficiency of deep neural networks, a sample data caching method is proposed.

[0060] The sample data caching method provided in the embodiment of the present application can be applied in the application environment shown in Figure 2. The server includes a control layer 202 and a data layer 204. The control layer 202 calculates the importance of the sample data, runs model training, and sends data requests for deep neural network training. The data layer 204 includes a first priority queue (PQ) and a second priority queue (GC) for tracking the currently cached decoded sample data and recording all accessed sample data.

[0061] The data layer 202 responds to the data request for deep neural network training sent by the control layer 204 and obtains a list of sample data for training; if there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area, then the alternative decoding sample data is determined from the decoding sample data set; if the importance of the first sample data and the importance of the alternative decoding sample data meet the preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoding sample data, then the first sample data is decoded to obtain the corresponding target decoding sample data; according to the target decoding sample data, the decoding sample data set in the cache area is updated. Furthermore, in the embodiment of the present application, the sample data caching method can be a cache optimization method based on importance sampling, wherein the access method of importance sampling, that is, the access frequency of sample data, is different. Importance sampling can be understood as the process of finding training samples that are more important than other samples, that is, the process of finding the most important training samples that contribute the most to the loss function. In other words, in the deep learning training process, different data have different effects on the accuracy of the model. Sample data that is hard to learn is usually more important. Increasing the access frequency of samples with high importance during training can accelerate the convergence of the model. Hard to learning sample data can be sample data that is difficult for a machine learning model to learn and classify.

[0062] It is understood that the control layer and data layer can exist on both the terminal and server sides, and can be determined based on the specific system architecture and application scenario. The control layer and data layer can also exist on the terminal and server sides respectively. For example, in some traditional client / server systems, the data layer is usually located on the server side, and the control layer is located on the terminal client. For example, the control layer exists on the terminal and the data layer exists on the server. The control layer calculates the importance of sample data, runs deep neural network training, and sends data requests for deep neural network training; the data layer is used to track the currently cached decoded sample data and record all accessed sample data.

[0063] In one embodiment, as shown in FIG3 , a sample data caching method is provided. The method is described by taking the application environment shown in FIG1 as an example, and includes the following steps:

[0064] Step 302: Obtain a list of sample data for training.

[0065] The sample data list includes at least one sample data identifier for training, and each sample data identifier has corresponding sample data.

[0066] Specifically, before conducting deep neural network training, the original sample data set for training is obtained, and after decoding the original sample data set, a decoded sample data set is obtained and stored on the disk. According to the preset cache conditions, the target decoded sample data set is obtained from the decoded sample data set on the disk and stored in the cache area. The cache area caches the decoded sample data, that is, the tensor data after decoding the original sample data is cached, which can reuse the decoded data and shorten the overall training time. When training the deep neural network, the control layer sends a data request for deep neural network training to the data layer, and the data layer responds to the data request to obtain a list of sample data required for training from the control layer. The cache is searched to see if there is decoded sample data that matches the first sample data in the sample data list. The original sample data can be an image.

[0067] Step 304: If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determine candidate decoded sample data from the decoded sample data set.

[0068] The method for determining candidate decoded sample data can be based on the importance of each decoded sample data in the decoded sample data set in the cache area. Depending on the needs of different scenarios, this can be a preset importance range or a set importance threshold. The importance of decoded sample data can be reflected by the access frequency of the data. For example, the higher the importance of the data, the higher the corresponding access frequency.

[0069] Step 306 : If the importance of the first sample data and the importance of the candidate decoding sample data meet preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, the first sample data is decoded to obtain corresponding target decoding sample data.

[0070] Among them, the preset condition can be that the importance of the first sample data is equal to the importance of the alternative decoding sample data, or that the difference between the importance of the first sample data and the importance of the alternative decoding sample data is within a preset threshold range. This example is explained by taking the preset condition that the importance of the first sample data is equal to the importance of the alternative decoding sample data as an example. It can be understood that if the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoding sample data, it means that the first sample data can be the sample with the highest benefit. The sample with the highest benefit can be the sample in the training data set that is the most difficult to process for the machine learning model and requires more training. The number of first sample data can be one or two, and the number of first samples is not limited here. The method for determining the decoding resource consumption can be determined by the method in the relevant technology, which will not be described here.

[0071] The importance of sample data and the consumption of decoding resources can be recorded by the components in the data layer 202. The components of the data layer include a metadata manager and a distributed memory key-value storage server. The metadata manager uses two heap-based first priority queues PQ and second priority queues (GC, Garbage Collection). PQ is used to track the currently cached decoded sample data, and GC records all accessed samples. PQ and GC store records of sample IDs and metadata.<is,af,at> , where is is the importance of the sample, af is the access frequency, and at is the decoding resource consumption. The cache order of the first priority queue PQ is not first-in, first-out (FIFO), but is determined by the priority of the elements in the queue. In this embodiment, the priority is determined based on the importance of the sample data and the priority of the decoding resource consumption. For example, if 10% of the sample data have the same access frequency, when the importance is the same, the sample data with high decoding overhead will be preferentially cached.

[0072] Step 308: Update the decoded sample data set in the cache area according to the target decoded sample data.

[0073] Within an epoch (also known as a training cycle or number of iterations), a data sample can be accessed in multiple minibatches (also known as the batch size), and the decoded sample data in the cache area will change with each training iteration. Updating the decoded sample data set in the cache area includes deleting candidate decoded sample data from the decoded sample data set in the cache area and adding target decoded sample data to the decoded sample data set in the cache area.

[0074] Specifically, when the cache in the cache area is full, a sample data list is obtained. If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, the importance and decoding resource consumption of the first sample data and the alternative decoded sample data are detected. If the importance of the first sample data is equal to that of the alternative decoded sample data, and the decoding resource consumption of the first sample data is greater than that of the alternative decoded sample data, the first sample data is obtained from the disk. At this time, the first sample data obtained is the encoded data. By decoding the first sample data, the corresponding target decoded sample data is obtained. The alternative decoded sample data is deleted from the decoded sample data set in the cache area, the target decoded sample data is added to the decoded sample data set in the cache area, and the metadata of the target decoded sample data is recorded in the priority queue.

[0075] It is understood that when training a deep neural network, the sample data set used for training is typically divided into multiple batches, each containing a fixed amount of sample data. In one round of training, the model traverses the entire training data set, updating the parameters for each batch until training is complete. This embodiment uses the sample data cache for one round of training as an example. In multiple rounds of training, the above sample data caching steps can be repeated, and difficult-to-learn sample data can be repeatedly used for training, thereby ensuring a good cache hit rate and improving training accuracy.

[0076] The above-mentioned sample data caching method obtains a list of sample data for training in response to a data request for deep neural network training. When there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, alternative decoded sample data is determined from the decoded sample data set, and the importance and decoding resource consumption of the alternative decoded sample data and the first sample data are compared. When the importance of the first sample data and the importance of the alternative decoded sample data meet preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoded sample data, the decoded sample data set in the cache area is updated according to the target decoded sample data of the first sample data. When performing deep neural network training, the two different granularities of sample data importance and decoding resource consumption are considered to optimize the decoded sample data in the cache area, and the most beneficial ones can be cached, eliminating the need to re-read the same data item from disk into memory, and ensuring that more decoded sample data that meets the preset importance conditions are stored in the cache, thereby reducing a lot of redundant I / O and considering the resource consumption of data decoding itself, shortening the overall training time, and thereby improving training efficiency.

[0077] In one embodiment, as shown in FIG4 , a method for determining the importance of sample data is provided, including the following steps:

[0078] Step 402: Obtain a decoded sample data set; wherein the decoded sample data set includes a plurality of batches of sample data.

[0079] Step 404: Determine the entropy loss value of a single second sample data in each batch of sample data.

[0080] Step 406: Sort the individual second sample data in each batch of sample data according to the entropy loss value to obtain the importance of the individual second sample data; the importance is used to characterize the degree of influence of the second sample data on the accuracy of the deep neural network.

[0081] The control layer calculates the entropy loss value, i.e., the importance, of each sample at a fine-grained level. In other words, the importance of each sample data is calculated using the sample-level loss information, i.e., the entropy loss value of a single sample in a small batch. The entropy loss value can be understood as the categorical cross entropy of the sample data. The categorical cross entropy of each sample data indicates the uncertainty of the model's ability to predict the sample category label, and the measurement uncertainty can determine the importance of the sample data. The calculation method of categorical cross entropy can be expressed as:

[0082] Among them, T i represents the hot-encoded true label of the sample under category i, S irepresents the softmax probability of a sample in a minibatch under category i, S i is calculated as follows:

[0083] Among them, r i Indicates the original likelihood that the sample belongs to the i-th class, r j represents the raw likelihood that a sample belongs to class j, with the denominator being a normalization term. As can be seen, using softmax normalization effectively identifies variations in the raw likelihood, assigning importance accordingly, and ensuring a positive exponent. Sample-based entropy loss can be used to identify the most important samples for deep neural network training.

[0084] Furthermore, since softmax is continuously differentiable, the derivative of each weight of the DNN model with respect to the cost function can be calculated. The above method is used to determine the entropy loss of all samples to calculate the average entropy loss. A higher entropy loss value for a sample means that the model will generate multiple predictions for this sample, but only one of them is correct. Therefore, it is difficult for the model to generate an accurate prediction for this sample. Correspondingly, a lower entropy loss value for a sample means that the model can generate a unique prediction for it with high accuracy. Therefore, a lower entropy loss value for a sample means that it does not play a significant role in improving the accuracy of the model in subsequent periods, while a higher entropy loss value indicates the opposite.

[0085] That is to say, the calculation of the entropy loss value of each sample provides a simple method to quantify the importance of different samples, but it cannot explain the different contributions of different samples between minibatches to the model accuracy. Therefore, the single second sample data in each batch of sample data is sorted according to the entropy loss value to obtain the importance of the single second sample data. The method for determining the importance includes: determining the reference sample data as a reference from each batch of sample data, and obtaining the entropy loss value of the reference sample data; by judging whether the entropy loss value of the single second sample data in each batch of sample data is greater than the entropy loss value of the reference sample data, the single second sample data in each batch of sample data is sorted to obtain the importance of the single second sample data.

[0086] The reference sample data may be determined according to actual needs. For example, the first sample data in each batch of sample data may be selected as the reference sample data.

[0087] The importance of sample data can be determined by a logarithmic ranking algorithm, as follows:

[0088] In a small batch containing B samples, the ranking-based importance of the i-th sample data is expressed as rank i . l i and l k Respectively represent the entropy loss of the i-th and k-th sample data, that is, the entropy loss value. b0 is the bias term used to fix the ranking range on the logarithmic scale. I is an identity function. When the condition l i >l k Returns 1 if true, otherwise returns 0. For each k items in batch B, each sample data can be placed in the appropriate ranking.

[0089] For example, two mini-batches, B1 and B2, contain samples <4, 5, 6> and <7, 8, 9>. Assume that the samples in B1 have entropy scores <0.3, 0.5, 0.4>, while the samples in B2 have entropy loss values ​​<0.6, 1.2, 0.8>. Because these entropy loss values ​​are raw, comparing sample importance across mini-batches can be inaccurate. For example, a priority queue-based cache would rank sample 5 in B1 in the lower half of the global ranking when sorting all samples in the two mini-batches, even though sample 5 is the most important sample in B1. Sorting these samples by entropy loss gives a ranking relative to B1 and B2, meaning that samples 5 and 8 are the most important samples in B1 and B2, respectively, and that the top x% of important samples in a mini-batch are always in the top x% of the entire dataset.

[0090] In the above method for determining the importance of sample data, by calculating the entropy loss value of a single sample data in each batch of sample data, the importance of different samples in the corresponding batch of sample data to the accuracy of the model is determined based on the entropy loss value of the single sample data. This can be used to train sample data that is difficult to learn, ensure a good cache hit rate, and improve the accuracy of training.

[0091] In another embodiment, as shown in FIG5 , a sample data caching method is provided, which is described by taking the application environment shown in FIG1 as an example, including the following steps:

[0092] Step 502: Obtain a decoded sample data set.

[0093] Step 504 : determining the importance of each sample data in the decoded sample data set, determining a target data sample set from the decoded sample data set, and caching the target data sample set in a cache area.

[0094] The number of target data sample sets may be determined based on the size of the cache area, or based on the importance of each sample data and the size of the cache area, and there is no limitation on this.

[0095] Step 506: Obtain a list of sample data for training.

[0096] Step 508: If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, obtain the importance of each decoded sample data in the decoded sample data set.

[0097] Step 510: Determine the decoded sample with the least importance as candidate decoded sample data.

[0098] Step 512: If the importance of the first sample data and the importance of the candidate decoding sample data meet the preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, the first sample data is decoded to obtain corresponding target decoding sample data.

[0099] Step 514: Update the decoded sample data set in the cache area according to the target decoded sample data.

[0100] Optionally, in one embodiment, if the importance of the first sample data and the importance of the candidate decoded sample data meet preset conditions and the decoding resource consumption of the first sample data is less than the decoding resource consumption of the candidate decoded sample data, the decoded sample data set in the cache area is not updated.

[0101] Optionally, in one embodiment, the above-mentioned sample data caching method is implemented through PyTorch1.7. Among them, PyTorch includes three main components: Dataset, Sampler and DataLoader. The Dataset class provides an access method for the dataset and exposes a get_item interface for obtaining sample data of a given index and its target label. Sampler provides a subset of the dataset for the training process, using random arrangement. DataLoader uses the information provided by Sampler to load samples in small batches with the help of the working process. In the system, we implement a new class IsDataset by inheriting the PyTorch Dataset class. This class has the function of combining samples and their corresponding class labels so that Dataloader can easily obtain sample data for training from remote storage. This class has the main logic of the cached data item screening strategy.

[0102] In the IsDataset class, we introduce the logic of the data layer by overriding the get_item and len methods. The len method returns the total length of the IsDataset, and the get_item method obtains the index associated with the data sample, allowing the client layer to find the importance and decoding resource consumption at the granularity of each sample. In addition, the get_item method is connected to the memory pool cache, that is, the cache area. Based on the priority queue cache strategy, if it is detected that there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area, then the alternative decoding sample data is determined from the decoding sample data set; if the importance of the first sample data and the importance of the alternative decoding sample data meet the preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoding sample data, then the first sample data is decoded to obtain the corresponding target decoding sample data; and the decoding sample data set in the cache area is updated according to the target decoding sample data.

[0103] The above-mentioned sample data caching method, in response to a data request for deep neural network training, obtains a list of sample data for training; when there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area, by determining alternative decoding sample data from the decoding sample data set, comparing the importance and decoding resource consumption of the alternative decoding sample data with the first sample data, and when the importance of the first sample data and the importance of the alternative decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoding sample data, the decoding sample data set in the cache area is updated according to the target decoding sample data of the first sample data. During training, the two dimensions of sample number importance and decoding resource consumption are considered to optimize the decoding sample data in the cache area, without having to re-read the same data item from the disk into the memory, reducing a lot of redundant IO and considering the resource consumption of the data decoding itself, shortening the overall training time, and thus improving training efficiency. In other words, by combining the data stream processing characteristics of deep learning training with the cache optimization method, the training speed is improved while reducing costs.

[0104] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0105] Based on the same inventive concept, embodiments of the present application also provide a sample data caching system for implementing the sample data caching method described above. The solution provided by this system is similar to the solution described in the method described above. Therefore, the specific limitations in one or more sample data caching system embodiments provided below can be found in the limitations of the sample data caching method described above and will not be further elaborated here.

[0106] In one embodiment, as shown in FIG6 , a sample data cache system 600 is provided. The sample data cache system 600 includes a data layer 204 , wherein the data layer 204 includes a sample list acquisition module 602 , a candidate decoding sample determination module 604 , a target decoding sample determination module 606 , and a cache update module 608 , wherein:

[0107] The sample list acquisition module 602 is used to acquire a sample data list for training.

[0108] The candidate decoding sample determination module 604 is configured to determine candidate decoding sample data from the decoding sample data set if there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area.

[0109] a target decoding sample determination module 606 configured to decode the first sample data to obtain corresponding target decoding sample data if the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data;

[0110] The cache updating module 608 is configured to update the decoded sample data set in the cache area according to the target decoded sample data.

[0111] The above-mentioned sample data caching system obtains a list of sample data for training in response to a data request for deep neural network training; when there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area, by determining alternative decoding sample data from the decoding sample data set, comparing the importance and decoding resource consumption of the alternative decoding sample data and the first sample data, when the importance of the first sample data and the importance of the alternative decoding sample data meet the preset conditions and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the alternative decoding sample data, the decoding sample data set in the cache area is updated according to the target decoding sample data of the first sample data. When training, the two dimensions of sample number importance and decoding resource consumption are taken into consideration to optimize the decoding sample data in the cache area, and there is no need to re-read the same data item from the disk into the memory, thereby reducing a lot of redundant IO and considering the resource consumption of data decoding itself, shortening the overall training time, and thus improving training efficiency.

[0112] In another embodiment, a sample data caching system is provided. The sample data caching system includes a data layer. In addition to a sample list acquisition module 602, a candidate decoding sample determination module 604, a target decoding sample determination module 606, and a cache update module 608, the data layer also includes an importance determination module and a cache module, wherein:

[0113] The candidate decoding sample determination module 604 is further configured to obtain the importance of each decoding sample data in the decoding sample data set if there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area;

[0114] The decoded sample with the smallest importance is determined as candidate decoded sample data.

[0115] The importance determination module is used to obtain a decoded sample data set and determine the importance of each sample data in the sample data set.

[0116] The cache module is used to determine a target data sample set from a sample data set and cache the target data sample set in a cache area.

[0117] In one embodiment, the importance determination module is further configured to determine an entropy loss value of a single second sample data in each batch of sample data;

[0118] The single second sample data in each batch of sample data is sorted according to the entropy loss value to obtain the importance of the single second sample data; the importance is used to characterize the degree of influence of the sample data on the accuracy of the deep neural network.

[0119] In one embodiment, the importance determination module is further configured to determine reference sample data as a reference from each batch of sample data, and obtain an entropy loss value of the reference sample data;

[0120] By determining whether the entropy loss value of the single second sample data in each batch of sample data is greater than the entropy loss value of the reference sample data, the single second sample data in each batch of sample data is sorted to obtain the importance of the single second sample data.

[0121] In one embodiment, the cache update module 608 is further configured to not update the decoded sample data set in the cache area if the importance of the first sample data and the importance of the alternative decoded sample data meet preset conditions and the decoding resource consumption of the first sample data is less than the decoding resource consumption of the alternative decoded sample data.

[0122] Each module in the sample data caching system described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0123] In one embodiment, a computer device 700 is provided. The computer device 700 may be a server, and its internal structure diagram may be shown in FIG7 . The computer device 700 includes a processor 701, a memory 702, an input / output interface 703 (I / O), and a communication interface 704. The processor 701, the memory 702, and the I / O interface 703 are connected via a system bus 705, and the communication interface 704 is connected to the system bus 705 via the I / O interface 703. The processor 701 of the computer device 700 is used to provide computing and control capabilities. The memory 702 of the computer device 700 includes a non-volatile storage medium 7021 and an internal memory 7022. The non-volatile storage medium 7021 stores an operating system, computer programs, and a database. The internal memory 7022 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 7021. The database of the computer device 700 is used to store sample data. The input / output interface 703 of the computer device 700 is used to exchange information between the processor 701 and external devices. The communication interface 704 of the computer device 700 is used to communicate with external terminals via a network connection. When the computer program is executed by the processor 701, a sample data caching method is implemented.

[0124] Those skilled in the art will understand that the structure shown in Figure 7 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 700 to which the solution of the present application is applied. The specific computer device 700 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0125] In one embodiment, a computer device 700 is provided, including a memory 702 and a processor 701. The memory 702 stores a computer program, and the processor 701 implements the steps in the above method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor 701, the steps in the above-mentioned method embodiments are implemented.

[0127] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor 701, the steps in the above method embodiments are implemented.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0129] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory 702, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor 701 involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0130] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A sample data caching method, characterized in that: The method comprises: Get a list of sample data for training; If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set; If the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data, decoding the first sample data to obtain corresponding target decoding sample data; The decoded sample data set in the cache area is updated according to the target decoded sample data.

2. The method according to claim 1, wherein: If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, determining candidate decoded sample data from the decoded sample data set comprises: If there is no decoded sample data matching the first sample data in the sample data list in the decoded sample data set in the cache area, obtaining the importance of each decoded sample data in the decoded sample data set; The decoded sample with the smallest importance is determined as candidate decoded sample data.

3. The method according to claim 1, wherein: Before obtaining the sample data list for training, the method further includes: Get the decoded sample data set; The importance of each sample data in the decoded sample data set is determined, and a target data sample set is determined from the decoded sample data set and the target data sample set is cached in a cache area.

4. The method according to claim 3, wherein: The decoded sample data set includes a plurality of batches of sample data, and determining the importance of each sample data in the sample data set includes: Determine an entropy loss value of a single second sample data in each batch of sample data; The single second sample data in each batch of sample data is sorted according to the entropy loss value to obtain the importance of the single second sample data; the importance is used to characterize the influence of the second sample data on the accuracy of the deep neural network.

5. The method according to claim 4, wherein: The step of sorting a single second sample data in each batch of sample data according to the entropy loss value to obtain the importance of the single second sample data includes: Determine reference sample data as a reference from each batch of sample data, and obtain an entropy loss value of the reference sample data; By determining whether the entropy loss value of a single second sample data in each batch of sample data is greater than the entropy loss value of the reference sample data, the single second sample data in each batch of sample data is sorted to obtain the importance of the single second sample data.

6. The method according to any one of claims 1 to 5, wherein: The method further comprises: If the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is less than the decoding resource consumption of the candidate decoding sample data, the decoding sample data set in the cache area is not updated.

7. [Corrected 03.06.2024 in accordance with Rule 91] A sample data caching system, characterized in that: The system comprises a data layer, wherein the data layer comprises: A sample list acquisition module is used to obtain a sample data list for training; A candidate decoding sample determination module, configured to determine candidate decoding sample data from the decoding sample data set if there is no decoding sample data matching the first sample data in the sample data list in the decoding sample data set in the cache area; a target decoding sample determination module, configured to decode the first sample data to obtain corresponding target decoding sample data if the importance of the first sample data and the importance of the candidate decoding sample data meet a preset condition and the decoding resource consumption of the first sample data is greater than the decoding resource consumption of the candidate decoding sample data; The cache update module is used to update the decoding sample data set in the cache area according to the target decoding sample data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images