A data storage method, medium and device based on data quality

Abstract

The present specification discloses a data storage method, medium and device based on data quality. An initial data set is obtained. For each data in the initial data set, the data is input into each preset annotation model respectively, and the annotation result of each annotation model for the data is obtained as the annotation set corresponding to the data. According to the annotation set corresponding to each data in the initial data set, data is screened in the initial data set to construct a target data set. Data access information of a preset computing node when executing a model training task of a target model is determined, and data characteristic information corresponding to each data in the target data set is determined. According to the data access information and the data characteristic information corresponding to each data contained in the target data set, the storage location of each data in the target data set is determined when the computing node executes the model training task of the target model, and the target data set is stored as a training sample according to the storage location.

Description

Technical Field

[0001] The present specification relates to the field of computer technology, and in particular to a data storage method, medium and device based on data quality. Background Art

[0002] At present, with the rapid development of artificial intelligence, the large models trained in this way have achieved great success in fields such as computer vision, natural language processing or recommendation systems, such as semantic understanding and generation in natural language processing, image recognition and classification in computer vision, etc.

[0003] However, great success is also accompanied by a problem: training large models requires a large amount of training data and computing resources. After obtaining the training data, it is usually stored in the computing nodes that perform the model training task and in the memory outside the computing nodes so that the training data can be read during the model training process to perform the model training task. However, the efficiency of computing nodes in reading training data is low, resulting in reduced efficiency of model training.

[0004] To this end, this specification provides a data storage method, medium and device based on data quality. Summary of the invention

[0005] This specification provides a data storage method, medium and device based on data quality to partially solve the above problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a data storage method based on data quality, including:

[0008] Get the initial data set;

[0009] For each data in the initial data set, the data is input into each preset annotation model respectively, and the annotation result of each annotation model for the data is obtained as the annotation set corresponding to the data;

[0010] According to the annotation set corresponding to each data in the initial data set, data is screened in the initial data set to construct a target data set;

[0011] Determine data access information of a preset computing node when executing a model training task of a target model, and determine data characteristic information corresponding to each data in the target data set, wherein the data access information is used to reflect the usage of various types of training samples by the computing node when executing a model training task for the target model, and for each data included in the target data set, the data characteristic information corresponding to the data is used to reflect the type of the data and the association relationship between the data and other data;

[0012] Based on the data access information and the data characteristic information corresponding to each data contained in the target data set, the storage location of each data in the target data set is determined when the computing node executes the model training task of the target model, and the target data set is stored as a training sample according to the storage location.

[0013] Optionally, for each data in the initial data set, the data is input into each preset annotation model respectively, specifically including:

[0014] Determining data characteristic information of each data in the initial data set;

[0015] Determining, according to the data characteristic information, a classifier that matches the initial data set, the classifier being used to distinguish data quality of each data in the initial data set;

[0016] Input each data in the initial data set into the classifier respectively to obtain a classification result of the data quality corresponding to each data in the initial data set;

[0017] According to the classification result, data with higher than a preset data quality is screened out from the initial data set to construct an intermediate data set;

[0018] For each data in the intermediate data set, the data is input into each preset annotation model respectively.

[0019] Optionally, according to the annotation set corresponding to each data in the initial data set, data screening is performed in the initial data set to construct a target data set, specifically including:

[0020] For each data in the initial data set, according to the annotation set corresponding to the data, the data is combined with each annotation result in the annotation set to obtain each screening data;

[0021] Input each of the screening data into a preset model respectively, and determine each feedback information output by the preset model;

[0022] According to the feedback information, from the screening data, determine the screening data whose feedback information meets the preset expectations;

[0023] Construct a target data set based on the filtered data that meets the preset expectations of the feedback information.

[0024] Optionally, determining, according to the data access information and data characteristic information corresponding to each data contained in the target data set, a storage location of each data in the target data set when the computing node executes the model training task of the target model, and storing the target data set as a training sample according to the storage location, specifically includes:

[0025] Determine, according to the data access information, the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is acquired;

[0026] Determine, from the target data set, the training samples required for each round of model training when the computing node performs the model training task of the target model according to the type of training data required for each round of training and data characteristic information;

[0027] Determine the storage location of the data in the target data set other than the determined training samples in the storage node and store them, and determine the storage location of the training samples required for each round of model training in the cache layer of the computing node and store them according to the acquisition order, wherein the rate at which the computing node acquires data from the cache layer is greater than the rate at which the computing node acquires data from the storage node.

[0028] Optionally, determining, according to the data access information and data characteristic information corresponding to each data contained in the target data set, a storage location of each data in the target data set when the computing node executes the model training task of the target model, and storing the target data set as a training sample according to the storage location, specifically includes:

[0029] Determine, according to the data access information, the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is acquired;

[0030] Determine, from the initial data set, the intermediate data set, and the target data set, the training samples required for each round of model training when the computing node performs the model training task of the target model according to the type of training data required for each round of training and data characteristic information;

[0031] According to the acquisition order, the determined training samples are sequentially stored in the first cache layer of the computing node;

[0032] Determine, from the initial data set, data other than the determined training sample as a first remaining training sample, determine, from the intermediate data set, data other than the determined training sample as a second remaining training sample, and determine, from the target data set, data other than the determined training sample as a third remaining training sample;

[0033] According to the association relationship contained in the data characteristic information, the data in the third remaining training samples are stored in the first cache layer of the computing node, the data in the second remaining training samples are stored in the second cache layer of the computing node, and the data in the first remaining training samples are stored in the storage node, wherein the rates at which the computing node acquires data are, from fast to slow, the rate at which data is acquired from the first cache layer, the rate at which data is acquired from the second cache layer, and the rate at which data is acquired from the storage node.

[0034] Optionally, the method further includes:

[0035] Obtaining intermediate results generated by each round of model training when the computing node executes the model training task of the target model;

[0036] For the intermediate results generated by each round of model training, determine whether the intermediate results of this round of model training are called within a preset time according to the calling rule information of the intermediate results of this round of model training in this round of model training;

[0037] If yes, the intermediate result of the round of model training is stored in the first cache layer of the computing node, so that the computing node can access the intermediate result of the round of model training from the first cache layer;

[0038] If not, the intermediate results of this round of model training are compressed and stored in the second cache layer of the computing node, so that the computing node can access the intermediate results of this round of model training from the second cache layer.

[0039] Optionally, the method further includes:

[0040] When the number of model training rounds of the model training task of the target model executed by the computing node reaches a preset number of rounds, the target model is trained by using the third remaining training samples stored in the first cache layer.

[0041] Optionally, determining, according to the data access information and the data characteristic information corresponding to each data contained in the target data set, a storage location of each data in the target data set when the computing node executes the model training task of the target model, specifically includes:

[0042] Determining, according to the data access information, an initial storage location of each data in the target data set;

[0043] Determining, according to data characteristic information corresponding to each data contained in the target data set, a call efficiency value when each data contained in the target data set is stored according to the initial storage position;

[0044] Taking maximizing the call efficiency value as the optimization goal, the initial storage position of each data in the target data set is adjusted to determine the storage position of each data in the target data set when the computing node executes the model training task of the target model.

[0045] The present specification provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the data storage method based on data quality is implemented.

[0046] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a data storage method based on data quality when executing the program.

[0047] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0048] The data storage method based on data quality provided in this specification obtains an initial data set. For each data in the initial data set, the data is respectively input into each preset annotation model to obtain the annotation result of each annotation model for the data as the annotation set corresponding to the data. According to the annotation set corresponding to each data in the initial data set, data is screened in the initial data set to construct a target data set. Determine the data access information of the preset computing node when executing the model training task of the target model, and determine the data characteristic information corresponding to each data in the target data set. The data access information is used to reflect the use of various types of training samples by the computing node when executing the model training task for the target model. For each data contained in the target data set, the data characteristic information corresponding to the data is used to reflect the type of the data and the association relationship between the data and other data. According to the data access information and the data characteristic information corresponding to each data contained in the target data set, determine the storage location of each data in the target data set when the computing node executes the model training task of the target model, and store the target data set as a training sample according to the storage location.

[0049] By annotating the acquired initial data set, the data quality of the training samples when the computing node performs the model training task is improved. In addition, the storage location of each data in the target data set suitable for the model training task is determined through the data access information of the target model and the data characteristic information of the data required for training when the computing node performs the model training task, so that when the computing node performs the model training task of the target model, it can obtain the training samples from the predetermined reasonable storage location at high speed, increase the efficiency of data supply and model training, and improve the training quality of the target model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0051] Figure 1 A flowchart of a data storage method based on data quality provided in an embodiment of this specification;

[0052] Figure 2 A schematic diagram of a process for constructing a target data set provided in this specification;

[0053] Figure 3 A schematic diagram of a storage framework provided in this specification;

[0054] Figure 4 A schematic diagram of a data storage device based on data quality provided in this specification;

[0055] Figure 5 A method corresponding to the Figure 1 Schematic diagram of the structure of an electronic device. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0058] Figure 1 A flowchart of a training sample storage method provided in an embodiment of this specification includes the following steps:

[0059] S100: Acquire an initial data set.

[0060] In the process of data labeling and training sample storage in this specification, in the embodiment of this specification, the training sample storage process can be performed by a server. Of course, this specification does not limit which device is used to perform the training sample storage process, and data labeling and storage can also be performed by devices such as personal computers and mobile terminals. For the convenience of description, the following description is based on the server as the execution subject.

[0061] In one or more embodiments of the present specification, since training a large model requires a large amount of training data, the server can obtain a large amount of data by crawling the required data through crawler technology or obtaining data through relevant databases, and use these data to construct an initial data set in the specification.

[0062] S102: For each data in the initial data set, the data is input into each preset annotation model respectively to obtain the annotation result of each annotation model for the data as the annotation set corresponding to the data.

[0063] In one or more embodiments of the present specification, after the server obtains the initial data set, in order to make the data in the initial data set more suitable for the model training task, it is necessary to try to improve the data quality of the data in the initial data set, and the most important factor affecting the data quality of a data is the data annotation, and the quality of the data annotation can directly determine the upper limit of the effect of data use. Therefore, for each data in the initial data set, the data can be input into each preset annotation model respectively, and the annotation result of each annotation model for the data can be obtained as the annotation set corresponding to the data.

[0064] In one or more embodiments of the present specification, multiple preset labeling models may be used to label the same data respectively. In order to increase the accuracy of the labeling results, these multiple labeling models may not be the same labeling models, for example, labeling models based on decision trees, support vector machines, random forests, and neural networks.

[0065] In one or more embodiments of the present specification, the initial data set obtained by the server contains a large amount of data, and these data may have various problems. If they are used directly to train the model, they may lead to low model training efficiency and low performance of the trained model, so they cannot be used directly to train the model. These data may have problems, such as some data are missing key parts, and the missing values ​​need to be filled, and some parts of the data are abnormal, and the abnormal values ​​need to be processed. Therefore, before the initial data set is input into the preset annotation models, the data in the initial data set can be preprocessed. In this specification, there is no restriction on the means of preprocessing, which can be set according to actual conditions.

[0066] Of course, further, in order to better utilize the data to train the target model, the data characteristic information of these data can also be understood for targeted use. In this specification, Exploratory Data Analysis (EDA) technology can be used to analyze and process the data in the initial data set to obtain data characteristic information of the data in the initial data set. Data characteristic information may include explicit characteristic information (such as distribution, outliers, and types, etc.) and implicit characteristic information.

[0067] The distribution in explicit characteristic information refers to the frequency of words in a piece of data. Type refers to the type of data determined based on the frequency of words. An outlier is a representation of an abnormal situation in a piece of data, such as data format or data size. Implicit characteristic information refers to the potential association between data, such as whether the data are similar and determining which data have an association relationship.

[0068] In one or more embodiments of the present specification, the server processes a relatively large amount of data in the acquired initial data set through various preset annotation models. The processing time increases with the increase of the amount of data, but the annotation efficiency of the annotation model does not increase with the increase of the amount of data. Therefore, before this, after determining the data characteristic information of each data in the initial data set, the server determines a classifier that matches the initial data set based on the data characteristic information. Then, each data in the initial data set is input into the classifier to obtain the classification result of the data quality corresponding to each data in the initial data set output by the classifier. According to the classification result, data with a quality higher than the preset data quality is screened out from the initial data set to construct an intermediate data set.

[0069] The classifier is used to distinguish the data quality of each data in the initial data set. After that, the data with classification results higher than the preset data quality can be screened out from the initial data set to determine the intermediate data set. Then, for each data in the intermediate data set, the data is input into each preset annotation model to determine the target data set.

[0070] Figure 2 This is a flow chart of constructing a target data set provided in this specification. Figure 2 As shown, the server preprocesses the acquired initial data set, and then determines the classifier that matches the preprocessed initial data set, and classifies it through the classifier. The server can determine the intermediate data set based on the classification results output by the classifier, and then input the intermediate data set into each preset annotation model, and determine the final annotation result from the annotation results of each annotation model, and then construct the target data set.

[0071] S104: According to the annotation set corresponding to each data in the initial data set, data is screened in the initial data set to construct a target data set.

[0072] In one or more embodiments of the present specification, the server filters the initial data set according to the annotation set corresponding to each data in the initial data set to construct a target data set.

[0073] Specifically, the server can use an integrated detection mechanism, such as a voting mechanism, to determine the most frequently appearing annotation result among the annotation results of each annotation model for the same data in the annotation set, and use it as the final annotation result of the data. The data with the final annotation result determined in the initial data set is filtered out to form the target data set.

[0074] Of course, in this specification, the server can also combine each data in the initial data set with each annotation result in the annotation set according to the annotation set corresponding to the data to obtain each screening data. Each screening data is input into the preset model respectively to determine each feedback information output by the preset model. The server determines the screening data whose feedback information meets the preset expectations from each screening data according to each feedback information, and then constructs the target data set according to the screening data whose feedback information meets the preset expectations.

[0075] Among them, the feedback information can be information such as training time or model loss value, which is not limited in this specification. The feedback information is used to reflect the specific performance of the model trained using the screened data. On this basis, the server can use the labeling results in the screened data that meet the preset expectations, that is, the labeling results that determine the specific performance of the training model with better results, as the final labeling results of the data.

[0076] In one or more embodiments of the present specification, after determining the labeling results in the filtered data that meet the preset expectations, the server may train the labeling model that outputs feedback information of the labeling results that do not meet the preset expectations based on the labeling results in the filtered data that meet the preset expectations, and optimize the labeling model so that the labeling results output by the labeling model are more accurate.

[0077] Specifically, the server may determine, from among the screening data, the screening data whose feedback information does not meet the preset expectations based on the feedback information, and determine, based on the screening data whose feedback information does not meet the preset expectations, a labeling model that outputs a labeling result whose feedback information does not meet the preset expectations as the model to be trained. Further, the server may determine a loss value based on the difference between the labeling result in the screening data whose feedback information meets the preset expectations and the labeling result output by the model to be trained, and train the model to be trained based on the loss value, such as minimizing the loss value as an optimization goal, wherein the difference is positively correlated with the loss value.

[0078] S106: Determine data access information of a preset computing node when executing a model training task for a target model, and determine data characteristic information corresponding to each data in the target data set, wherein the data access information is used to reflect the usage of various types of training samples by the computing node when executing a model training task for the target model, and for each data contained in the target data set, the data characteristic information corresponding to the data is used to reflect the type of the data and the association between the data and other data.

[0079] In one or more embodiments of the present specification, after determining the target data set with annotations, the server, in order to better utilize the target data set and give full play to the role of the target data set, enables the computing node to have higher training efficiency when using the target data set to perform the model training task of the target model. The server can determine the data access information of the preset computing node when performing the model training task of the target model. Since the computing node itself is clear about the structure and parameters of the target model and the use of various training samples by the computing node when performing the model training task for the target model when determining to train the target model, when the target model is trained in each round, the training data type required for the next round of training can be determined in advance according to the use of various training samples by the computing node when performing the model training task for the target model, and the order of obtaining the training data, that is, the data access information. In addition, the server can also determine the data characteristic information reflecting the type of the data and the association relationship between the data and other data for each data contained in the target data set, that is, determine the data characteristic information corresponding to each data in the target data set. It is worth noting that at this time, various types of training samples refer to training samples of various data types.

[0080] Therefore, after determining the data access information and the data characteristic information, the server can store the data of the target data set in accordance with the data access information and the data characteristic information so as to facilitate the computing node to call.

[0081] S108: Determine the storage location of each data in the target data set when the computing node executes the model training task of the target model based on the data access information and the data characteristic information corresponding to each data contained in the target data set, and store the target data set as a training sample according to the storage location.

[0082] In one or more embodiments of the present specification, the server determines the storage location of each data in the target data set when the computing node executes the model training task of the target model according to the determined data access information and the data characteristic information corresponding to each data contained in the target data set, and stores the target data set as a training sample according to the storage location.

[0083] Specifically, the server determines the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is obtained, based on the data access information. Based on the type of training data required for each round of training and the data characteristic information, the server determines the training samples required for each round of model training when the computing node performs the model training task of the target model from the target data set. Then, the server can determine the storage location of the data in the target data set other than the determined training samples in the storage node and store them, and determine the storage location of the training samples required for each round of model training in the cache layer of the computing node according to the acquisition order and store them.

[0084] The rate at which the computing node obtains data from the cache layer is greater than the rate at which the computing node obtains data from the storage node. Of course, when storing data other than the determined training samples in the target data set in the storage node, the data with related relationships can be stored together according to the related relationships contained in the data characteristic information.

[0085] In this case, the storage location determined by the server can be included in a preset storage framework, and the storage framework includes two parts, namely, the cache layer in the computing node and the storage node. The cache layer located in the computing node can be composed of high-performance storage devices such as dynamic random access memory (DRAM), high bandwidth memory (HBM) and non-volatile memory devices such as non-volatile memory express (NVMe) solid state drives. The storage node can be a parallel file system composed of conventional large-capacity storage devices such as Serial Advanced Technology Attachment (SATA) / Serial Attached SCSI (SAS) solid state drives and hard disk drives (HDD). SCSI refers to Small Computer System Interface. In the storage framework, the cache layer in the upper computing node can serve as a high-speed cache for the lower storage nodes. They maintain a unified directory tree view through the metadata server and build a unified addressing space between different storage levels. The entire storage framework belongs to the same namespace (i.e., unified addressing space), aggregates and stores data or files, and reasonably utilizes the characteristics of high-performance storage media to accelerate the reading and writing of small files. In addition, the same data usually only needs to be stored in one copy, maximizing data sharing and storage space utilization. Among them, the upper cache layer can be built based on new high-speed interconnection technology. In addition, the construction of the storage framework can reuse existing storage technologies, such as fine-grained distributed lock management, version control and other technologies, to achieve a balance between data security, system performance and cost. This manual does not limit storage technology.

[0086] In addition, in one or more embodiments of the present specification, the server may also determine the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is obtained based on the data access information. The server determines the training samples required for each round of model training when the computing node performs the model training task of the target model from the initial data set, the intermediate data set, and the target data set based on the type of training data required for each round of training and the data characteristic information. Afterwards, the server stores the determined training samples in the first cache layer of the computing node in sequence according to the acquisition order, determines the data other than the determined training samples from the initial data set as the first remaining training samples, determines the data other than the determined training samples from the intermediate data set as the second remaining training samples, and determines the data other than the determined training samples from the target data set as the third remaining training samples.

[0087] Afterwards, the server can store the data in the third remaining training samples to the first cache layer of the computing node, store the data in the second remaining training samples to the second cache layer of the computing node, and store the data in the first remaining training samples to the storage node according to the association relationship contained in the data characteristic information, wherein the rate at which the computing node obtains data is, from fast to slow, the rate at which data is obtained from the first cache layer, the rate at which data is obtained from the second cache layer, and the rate at which data is obtained from the storage node.

[0088] In this case, the storage framework includes the first cache layer and the second cache layer of the computing node, and the storage node, a total of three layers. The first cache layer located at the computing node can be composed of high-performance storage devices such as DRAM and HBM, and the second cache layer located at the computing node can be composed of non-volatile storage devices such as NVMe solid-state drives.

[0089] In one or more embodiments of the present specification, further, the server may obtain the intermediate results generated by each round of model training when the computing node executes the model training task of the target model, temporary data, status information, and activation values ​​generated during the model training process. For the intermediate results generated by each round of model training, according to the calling rule information of the intermediate results of the round of model training in the round of model training, it is determined whether the intermediate results of the round of model training are called within a preset time. Among them, the calling rule information refers to the pattern displayed by the subsequent access and use of the intermediate results generated by each round of model training. The calling rule information can record information such as the time interval from the generation of the intermediate results of a round of model training to the next call of the intermediate results.

[0090] If yes, it means that this intermediate result will be called multiple times during this round of model training. The intermediate result of this round of model training should be stored in the first cache layer of the computing node, so that the computing node can directly access the intermediate result of this round of model training from the first cache layer when it needs to call the intermediate result. After the computing node accesses the intermediate result, the intermediate result is discarded in the first cache layer, but the intermediate result can be stored in the storage node. If no, it means that this intermediate result will not be called multiple times during this round of model training. After this intermediate result is generated, it may take some time before this intermediate result is called. In order to save storage space for storing this intermediate result, the intermediate result of this round of model training can be compressed and stored in the second cache layer of the computing node, so that the computing node can directly access the intermediate result of this round of model training from the second cache layer or storage node when it needs to call the intermediate result. And when this intermediate result is stored in the second cache layer and accessed, it will be discarded from the second cache layer, but this intermediate result needs to be stored in the storage node. Of course, the computing node can access data from the first cache layer, the second cache layer, and any storage location in the storage node according to the Storage Performance Development Kit (SPDK) technology, and this specification does not limit the access technology of the computing node directly accessing data at any storage location.

[0091] It is worth noting that in this specification, before the computing node executes the model training task of the target model, the data is stored in the corresponding storage location according to the data access information and the data characteristic information, and then the server can enable the computing node to execute the model training task of the target model according to the stored data. However, in the later stage of the model training task of the target model (when the number of model training rounds reaches a certain number or the model accuracy meets certain requirements), high-quality data (data in the target data set stored in the first cache layer) is supplied to the computing node as much as possible to accelerate the training model.

[0092] Specifically, when the number of model training rounds of the computing node executing the model training task of the target model reaches a preset number of rounds, the server can enable the computing node to train the target model through the third remaining training samples stored in the first cache layer.

[0093] In one or more embodiments of the present specification, when data in a target data set is stored according to data characteristic information, the storage locations of related data in the target data set may be adjacent. When a computing node calls the stored data of the target data set as a training sample during the execution of a model training task, if the data is read in sequence according to the storage locations where the training samples are stored, the training samples read are not necessarily the training samples that maximize the model training efficiency.

[0094] To this end, the server can determine the initial storage location of each data in the target data set according to the data access information, and determine the call efficiency value when storing each data contained in the target data set according to the initial storage location according to the data characteristic information corresponding to each data contained in the target data set. Among them, the call efficiency value is used to reflect the model training efficiency when the computing node calls the training sample according to the initial storage location. If the call efficiency value is higher, the time required for the computing node to call the training sample according to the initial storage location is less, and the model training efficiency is higher.

[0095] Afterwards, the server can maximize the call efficiency value as the optimization goal, and adjust the initial storage location of each data in the target data set to determine the storage location of each data in the target data set when the computing node executes the model training task of the target model. This process can be understood as a process of constantly exploring the optimal solution (the so-called optimal solution is the optimal storage location of each training sample under the condition of maximizing the model training efficiency). By setting the constraint condition of maximizing the call efficiency value, multiple possible storage locations for each data can be continuously explored, so as to finally determine the storage location that can maximize the model training efficiency, and store each data in the target data set according to this storage location.

[0096] In addition, in one or more embodiments of the present specification, the storage framework not only stores the data required for model training, but also stores the model parameters of the model run by the computing node, and also stores the intermediate results generated during the model operation. Through this storage framework, the computing node can directly access data from the first cache layer and the second cache layer. The first cache layer can store compressed model parameters (model parameters that are being used or will be used), and the second cache layer can store uncompressed model parameters (parameters that are not needed for a period of time), which can balance the data volume, latency and throughput requirements, and the model parameters and data required for training only need to be stored in a certain cache layer, eliminating the waste of storage space caused by multi-layer storage in the traditional hierarchical cache architecture, the need for inter-layer migration and synchronization overhead, and other problems, comprehensively improving the utilization of cache storage space and accelerating the efficiency of large model training.

[0097] It is worth noting that in the model training process of the computing node, by specifying specific precision for some special functions and model structures involved in the model training (for example, when calculating gradients, by using higher precision to ensure the accuracy of model training. Or when calculating matrix multiplication, by using lower precision, the computing power requirements are greatly reduced and the calculation is accelerated), the data range requirements of special functions can be adapted. For unspecified general functions and model structures, an online learning neural network can be used to guide the variable precision that should be used in the next round of operations, thereby achieving high-accuracy data precision adaptive adjustment and reducing the storage space occupied by intermediate results generated during the model training process.

[0098] Figure 3 This is a schematic diagram of a storage framework provided in this manual. Figure 3 As shown, the storage framework includes a first cache layer and a second cache layer located in the computing node, and a large-scale parallel file system located in the storage node. There are several computing cores in the computing node. The computing node performs model training through the computing core. During the training process, the first cache layer can be directly accessed to read and write the training data at high speed to achieve efficient data interaction. The computing core can also access the second cache layer across layers to obtain persistent large-capacity training data stored in the second cache layer, and can also adaptively compress and store the model parameters generated during the model training process to the second cache layer. In the second cache layer, the training data and model parameters that the computing core is about to use can be prepared in advance. Data migration can also be performed between the first cache layer and the second cache layer according to data acquisition requirements. For the large-scale parallel file system located in the storage node, it has a large amount of storage space, can aggregate bandwidth to speed up data access efficiency, and also supports parallel reading and writing of data with high throughput. It can be interconnected with the first cache layer and the second cache layer at high speed, and can send its own stored data to the computing node, and can also receive data sent by the computing node for update or storage.

[0099] The above is a data storage method based on data quality provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding data storage device based on data quality, such as Figure 4 shown.

[0100] Figure 4 A schematic diagram of a data storage device based on data quality provided in this specification specifically includes:

[0101] An acquisition module 400 is used to acquire an initial data set;

[0102] The annotation module 402 is used to input each data in the initial data set into each preset annotation model, and obtain the annotation result of each annotation model for the data as the annotation set corresponding to the data;

[0103] A screening module 404 is used to screen data in the initial data set according to the annotation set corresponding to each data in the initial data set to construct a target data set;

[0104] A determination module 406 is used to determine data access information of a preset computing node when executing a model training task for a target model, and to determine data characteristic information corresponding to each data in the target data set, wherein the data access information is used to reflect the usage of various types of training samples by the computing node when executing a model training task for the target model, and for each data included in the target data set, the data characteristic information corresponding to the data is used to reflect the type of the data and the association relationship between the data and other data;

[0105] The storage module 408 is used to determine the storage location of each data in the target data set when the computing node executes the model training task of the target model based on the data access information and the data characteristic information corresponding to each data contained in the target data set, and store the target data set as a training sample according to the storage location.

[0106] Optionally, the labeling module 402 is used to determine data characteristic information of each data in the initial data set, and determine a classifier that matches the initial data set based on the data characteristic information, the classifier is used to distinguish the data quality of each data in the initial data set, and each data in the initial data set is input into the classifier respectively to obtain a classification result of the data quality corresponding to each data in the initial data set; based on the classification result, data with a higher than preset data quality is screened out from the initial data set to construct an intermediate data set; for each data in the intermediate data set, the data is input into each preset labeling model respectively.

[0107] Optionally, the screening module 404 is used to, for each data in the initial data set, combine the data with each annotation result in the annotation set according to the annotation set corresponding to the data to obtain each screening data, input each screening data into a preset model respectively, determine each feedback information output by the preset model, determine the screening data whose feedback information meets the preset expectations from the each screening data according to the each feedback information, and construct the target data set according to the screening data whose feedback information meets the preset expectations.

[0108] Optionally, the storage module 408 is used to determine, based on the data access information, the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is acquired; determine, from the target data set, the training samples required for each round of model training when the computing node performs the model training task of the target model according to the type of training data required for each round of training and the data characteristic information; determine and store the storage location of the data in the target data set other than the determined training samples in the storage node; determine and store the storage location of the training samples required for each round of model training in the cache layer of the computing node according to the acquisition order, wherein the rate at which the computing node acquires data from the cache layer is greater than the rate at which the computing node acquires data from the storage node.

[0109] Optionally, the labeling module 402 is used to determine, based on the data access information, the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is obtained; based on the type of training data required for each round of training and the data characteristic information, determine, from the initial data set, the intermediate data set and the target data set, the training samples required for each round of model training when the computing node performs the model training task of the target model; store the determined training samples in the first cache layer of the computing node in sequence according to the acquisition order; determine, from the initial data set, data other than the determined training samples as the first remaining training samples; and In the intermediate data set, data other than the determined training samples are determined as second remaining training samples, and from the target data set, data other than the determined training samples are determined as third remaining training samples, and according to the association relationship contained in the data characteristic information, data in the third remaining training samples are stored in the first cache layer of the computing node, data in the second remaining training samples are stored in the second cache layer of the computing node, and data in the first remaining training samples are stored in the storage node, wherein the speed at which the computing node acquires data is, from fast to slow, the speed at which data is acquired from the first cache layer, the speed at which data is acquired from the second cache layer, and the speed at which data is acquired from the storage node.

[0110] Optionally, the labeling module 402 is used to obtain the intermediate results generated by each round of model training when the computing node executes the model training task of the target model. For the intermediate results generated by each round of model training, according to the calling rule information of the intermediate results of this round of model training in this round of model training, it is judged whether the intermediate results of this round of model training are called within a preset time. If so, the intermediate results of this round of model training are stored in the first cache layer of the computing node, so that the computing node accesses the intermediate results of this round of model training from the first cache layer; if not, the intermediate results of this round of model training are compressed and stored in the second cache layer of the computing node, so that the computing node accesses the intermediate results of this round of model training from the second cache layer.

[0111] Optionally, the labeling module 402 is used to train the target model through the third remaining training samples stored in the first cache layer when the number of model training rounds of the model training task of the target model executed by the computing node reaches a preset number of rounds.

[0112] Optionally, the storage module 408 is used to determine the initial storage position of each data in the target data set based on the data access information, and determine the calling efficiency value when storing each data contained in the target data set according to the initial storage position based on the data characteristic information corresponding to each data contained in the target data set, so as to maximize the calling efficiency value as the optimization goal, and adjust the initial storage position of each data in the target data set to determine the storage position of each data in the target data set when the computing node executes the model training task of the target model.

[0113] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A data storage method based on data quality is provided.

[0114] This manual also provides Figure 5 The schematic structure diagram of the electronic device shown in FIG. Figure 5 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The data storage method based on data quality.

[0115] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0116] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0117] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0118] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0119] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0120] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0126] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0127] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0128] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0130] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0131] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A data storage method based on data quality, characterized in that: include: Get the initial data set; For each data in the initial data set, the data is input into each preset annotation model respectively, and the annotation result of each annotation model for the data is obtained as the annotation set corresponding to the data; According to the annotation set corresponding to each data in the initial data set, data is screened in the initial data set to construct a target data set; Determine data access information of a preset computing node when executing a model training task of a target model, and determine data characteristic information corresponding to each data in the target data set, wherein the data access information is used to reflect the usage of various types of training samples by the computing node when executing a model training task for the target model, and for each data included in the target data set, the data characteristic information corresponding to the data is used to reflect the type of the data and the association relationship between the data and other data; Based on the data access information and the data characteristic information corresponding to each data contained in the target data set, the storage location of each data in the target data set is determined when the computing node executes the model training task of the target model, and the target data set is stored as a training sample according to the storage location.

2. The method according to claim 1, characterized in that For each data in the initial data set, the data is input into each preset annotation model, specifically including: Determining data characteristic information of each data in the initial data set; Determining, according to the data characteristic information, a classifier that matches the initial data set, the classifier being used to distinguish data quality of each data in the initial data set; Input each data in the initial data set into the classifier respectively to obtain a classification result of the data quality corresponding to each data in the initial data set; According to the classification result, data with higher than a preset data quality is screened out from the initial data set to construct an intermediate data set; For each data in the intermediate data set, the data is input into each preset annotation model respectively.

3. The method according to claim 1, characterized in that According to the annotation set corresponding to each data in the initial data set, data is screened in the initial data set to construct a target data set, specifically including: For each data in the initial data set, according to the annotation set corresponding to the data, the data is combined with each annotation result in the annotation set to obtain each screening data; Input each of the screening data into a preset model respectively, and determine each feedback information output by the preset model; According to the feedback information, from the screening data, determine the screening data whose feedback information meets the preset expectations; Construct a target data set based on the filtered data that meets the preset expectations of the feedback information.

4. The method according to claim 1, characterized in that Determining, according to the data access information and the data characteristic information corresponding to each data contained in the target data set, the storage location of each data in the target data set when the computing node executes the model training task of the target model, and storing the target data set as a training sample according to the storage location, specifically includes: Determine, according to the data access information, the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is acquired; Determine, from the target data set, the training samples required for each round of model training when the computing node performs the model training task of the target model according to the type of training data required for each round of training and data characteristic information; Determine the storage location of the data in the target data set other than the determined training samples in the storage node and store them, and determine the storage location of the training samples required for each round of model training in the cache layer of the computing node and store them according to the acquisition order, wherein the rate at which the computing node acquires data from the cache layer is greater than the rate at which the computing node acquires data from the storage node.

5. The method according to claim 2, characterized in that Determining, according to the data access information and the data characteristic information corresponding to each data contained in the target data set, the storage location of each data in the target data set when the computing node executes the model training task of the target model, and storing the target data set as a training sample according to the storage location, specifically includes: Determine, according to the data access information, the type of training data required for each round of model training when the computing node performs the model training task of the target model, and the order in which the training data is acquired; Determine, from the initial data set, the intermediate data set, and the target data set, the training samples required for each round of model training when the computing node performs the model training task of the target model according to the type of training data required for each round of training and data characteristic information; According to the acquisition order, the determined training samples are sequentially stored in the first cache layer of the computing node; Determine, from the initial data set, data other than the determined training sample as a first remaining training sample, determine, from the intermediate data set, data other than the determined training sample as a second remaining training sample, and determine, from the target data set, data other than the determined training sample as a third remaining training sample; According to the association relationship contained in the data characteristic information, the data in the third remaining training samples are stored in the first cache layer of the computing node, the data in the second remaining training samples are stored in the second cache layer of the computing node, and the data in the first remaining training samples are stored in the storage node, wherein the rates at which the computing node obtains data are, from fast to slow, the rate at which data is obtained from the first cache layer, the rate at which data is obtained from the second cache layer, and the rate at which data is obtained from the storage node.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining intermediate results generated by each round of model training when the computing node executes the model training task of the target model; For the intermediate results generated by each round of model training, determine whether the intermediate results of this round of model training are called within a preset time according to the calling rule information of the intermediate results of this round of model training in this round of model training; If yes, the intermediate result of the round of model training is stored in the first cache layer of the computing node, so that the computing node can access the intermediate result of the round of model training from the first cache layer; If not, the intermediate results of this round of model training are compressed and stored in the second cache layer of the computing node, so that the computing node can access the intermediate results of this round of model training from the second cache layer.

7. The method according to claim 5, characterized in that The method further comprises: When the number of model training rounds of the model training task of the target model executed by the computing node reaches a preset number of rounds, the target model is trained by using the third remaining training samples stored in the first cache layer.

8. The method according to claim 1, characterized in that Determining, according to the data access information and the data characteristic information corresponding to each data contained in the target data set, the storage location of each data in the target data set when the computing node executes the model training task of the target model, specifically includes: Determining, according to the data access information, an initial storage location of each data in the target data set; Determining, according to data characteristic information corresponding to each data contained in the target data set, a call efficiency value when each data contained in the target data set is stored according to the initial storage position; Taking maximizing the call efficiency value as the optimization goal, the initial storage position of each data in the target data set is adjusted to determine the storage position of each data in the target data set when the computing node executes the model training task of the target model.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images