Method and electronic device for automated data enhancement

Abstract

A method and an electronic device for automatic data enhancement are disclosed. The method for automatic data enhancement includes determining, using a central processing unit (CPU), a plurality of hyper-parameters corresponding to a plurality of sub-models; training the plurality of sub-models based on the plurality of hyper-parameters using a plurality of storage devices and a plurality of graphics processing units (GPUs); at least one sub-model among the trained plurality of sub-models and at least one hyper-parameter corresponding to the updated at least one sub-model are updated using the CPU, the plurality of sub-models are trained for a first predetermined number of rounds, and during each round, the at least one hyper-parameter corresponds to the updated at least one sub-model, and the at least one hyper-parameter corresponds to the updated at least one sub-model. The training step comprises the following steps of: preprocessing an original data set based on the plurality of hyper-parameters by using the plurality of storage devices so as to generate a plurality of preprocessed data sets corresponding to the plurality of hyper-parameters; and training the plurality of sub-models based on the plurality of preprocessed data sets by using the plurality of GPUs.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning, and more particularly to a method and an electronic device for automatic data enhancement. Background Art

[0002] According to some methods for performing deep learning, models can be continuously trained with large data sets to improve generalization capabilities. In the field of computer vision, automatic data augmentation methods can be used to augment data sets. For example, automatic data augmentation methods can include preprocessing operations that perform preprocessing on the original data set. However, there is a need for techniques to improve the efficiency of the preprocessing operations of automatic data augmentation methods and to improve the efficiency of training models. Summary of the invention

[0003] Provided are a method and an electronic device for automatic data enhancement, which can at least improve the efficiency of the preprocessing operation and the efficiency of the training model of the automatic data enhancement method.

[0004] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.

[0005] According to the disclosed aspects, a method for automatic data enhancement includes: using a central processing unit (CPU) to determine multiple hyperparameters corresponding to multiple sub-models; using multiple storage devices and multiple graphics processing units (GPUs) to train the multiple sub-models based on the multiple hyperparameters; using the CPU to update at least one sub-model among the trained multiple sub-models, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the multiple sub-models are trained for a first predetermined number of rounds, and wherein, during each round of the first predetermined number of rounds, the training step includes: using the multiple storage devices to preprocess the original data set based on the multiple hyperparameters to generate multiple preprocessed data sets corresponding to the multiple hyperparameters; using the multiple GPUs to train the multiple sub-models based on the multiple preprocessed data sets.

[0006] Each of the multiple storage devices may include a controller and a field programmable gate array (FPGA), wherein, for each of the multiple hyperparameters, the preprocessing step includes: determining, by a controller of a storage device corresponding to each hyperparameter among the multiple storage devices, a portion of the original data set on which image processing will not be performed based on each hyperparameter, and providing, using a CPU, the portion of the original data set to a GPU among the multiple GPUs for training based on a preprocessed data set corresponding to each hyperparameter, performing image processing on a remaining portion of the original data set based on the hyperparameter to generate processed data through the FPGA of the storage device, and providing the processed data to the GPU using the CPU, wherein the preprocessed data set includes the portion of the original data set and the processed data.

[0007] Each hyperparameter may include: multiple preprocessing operators corresponding to multiple types of image processing, wherein, for each hyperparameter, the preprocessing step also includes: dividing the original data set into multiple sub-data sets in units of a predetermined batch size through the controller, sequentially selecting sub-data sets from the multiple sub-data sets through the controller, and randomly selecting at least one preprocessing operator from the multiple preprocessing operators based on the selected sub-data sets through the controller, wherein the step of determining the portion of the original data set includes: determining a portion of the sub-data set that will not be subjected to image processing based on the at least one preprocessing operator, and wherein the step of generating processed data includes: performing image processing on the remaining portion of the sub-data set based on the at least one preprocessing operator.

[0008] The method may also include: repeating the training step and the updating step until a second predetermined number of rounds is reached, and determining, through the CPU, the hyperparameters corresponding to the best sub-model among the multiple sub-models after training after the repeated training step and the updating step are stopped, wherein the second predetermined number is greater than the first predetermined number.

[0009] The step of repeating training may include: training the multiple sub-models including the updated at least one sub-model for a first predetermined number of rounds, and during each round, using at least one storage device among the multiple storage devices corresponding to the at least one sub-model, preprocessing the original data set based on the updated at least one hyperparameter to generate at least one preprocessed data set corresponding to the updated at least one hyperparameter; using the remaining storage devices among the multiple storage devices, preprocessing the original data set based on the non-updated hyperparameters among the multiple hyperparameters to generate a preprocessed data set corresponding to the non-updated hyperparameters.

[0010] The at least one sub-model may include n worst sub-models among the multiple sub-models after training, wherein the updating step includes: replacing the n worst sub-models with n best sub-models among the multiple sub-models after training, wherein n is an integer greater than or equal to 1 and less than (2N+1) / 2, wherein N represents the total number of the multiple sub-models, and performing perturbations on n hyper-parameters corresponding to the n best sub-models to determine n hyper-parameters corresponding to the replaced n worst sub-models.

[0011] Each of the plurality of storage devices may include a smart solid state drive (SSD).

[0012] The raw data set may include an image data set.

[0013] According to the disclosed aspects, an electronic device includes: a central processing unit (CPU), configured to: determine multiple hyperparameters corresponding to multiple sub-models; multiple storage devices and multiple graphics processing units (GPUs), configured to: train the multiple sub-models based on the multiple hyperparameters, wherein the CPU is also configured to: update at least one sub-model among the multiple sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the multiple sub-models are trained for a first predetermined number of rounds, and wherein, during each round among the first predetermined number of rounds: the multiple storage devices are also configured to perform preprocessing on the original data set based on the multiple hyperparameters to generate multiple preprocessed data sets corresponding to the multiple hyperparameters respectively; wherein, during each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

[0014] Each of the multiple storage devices may include a controller and a field programmable gate array (FPGA), wherein, for each of the multiple hyperparameters, the controller of the storage device corresponding to the hyperparameter among the multiple storage devices is configured to determine, based on the hyperparameter, a portion of the original data set on which image processing will not be performed, and use the CPU to provide the portion of the original data set to a GPU among the multiple GPUs for training based on a preprocessed data set corresponding to the hyperparameter, wherein, for each hyperparameter, the FPGA of the storage device is configured to perform image processing on the remaining portion of the original data set based on the hyperparameter to generate processed data, and use the CPU to provide the processed data to the GPU, wherein the preprocessed data set includes the portion of the original data set and the processed data.

[0015] Each of the multiple hyperparameters may include: multiple preprocessing operators corresponding to multiple types of image processing, respectively, wherein, for each of the hyperparameters, the controller is further configured to: divide the original data set into multiple sub-datasets in units of a predetermined batch size, sequentially select sub-datasets from the multiple sub-datasets, and randomly select at least one preprocessing operator from the multiple preprocessing operators based on the selected sub-datasets, wherein the controller is further configured to: determine the part of the original data set by determining the data in the sub-dataset that will not be subjected to image processing based on the at least one preprocessing operator, wherein the FPGA is further configured to: generate processed data by performing image processing on the remaining part of the sub-dataset based on the at least one preprocessing operator.

[0016] The multiple storage devices and the multiple GPUs can also be configured to: repeat the training steps and the updating steps until a second predetermined number of rounds is reached, wherein the CPU is further configured to: determine the hyperparameters corresponding to the best sub-model among the multiple sub-models after training after the repeated training steps and the updating steps stop, wherein the second predetermined number is greater than the first predetermined number.

[0017] In order to repeat the training steps, the multiple storage devices and the multiple GPUs may also be configured to: train the multiple sub-models including the updated at least one sub-model for a first predetermined number of rounds, wherein, during each round, at least one storage device among the multiple storage devices corresponding to the at least one sub-model is configured to preprocess the original data set based on the updated at least one hyperparameter to generate at least one preprocessed data set corresponding to the updated at least one hyperparameter; wherein, during each round, the remaining storage devices among the multiple storage devices are configured to preprocess the original data set based on the non-updated hyperparameters among the multiple hyperparameters to generate preprocessed data sets corresponding to the non-updated hyperparameters.

[0018] The at least one sub-model may include n worst sub-models among the multiple sub-models after training, wherein n is an integer greater than or equal to 1 and less than (2N+1) / 2, wherein N represents the total number of the multiple sub-models, wherein the CPU is further configured to: update the n worst sub-models by replacing the n worst sub-models with the n best sub-models among the multiple sub-models after training, and determine the n hyper-parameters corresponding to the updated n worst sub-models by performing perturbations on the n hyper-parameters corresponding to the n best sub-models, so as to update the n hyper-parameters corresponding to the updated n worst sub-models.

[0019] Each of the plurality of storage devices may include a smart solid state drive (SSD).

[0020] The raw data set may include an image data set.

[0021] According to the disclosed aspects, a system applying multiple storage devices includes: a main processor; a memory; a storage device; and multiple graphics processing units (GPUs), wherein the main processor is configured to: determine multiple hyperparameters corresponding to multiple sub-models; wherein the storage device and the multiple GPUs are configured to: train the multiple sub-models based on the multiple hyperparameters, wherein the main processor is also configured to: update at least one sub-model among the multiple sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the multiple sub-models are trained for a first predetermined number of rounds, wherein during each round of the first predetermined number of rounds, the storage device is also configured to perform preprocessing on the original data set based on the multiple hyperparameters to generate multiple preprocessed data sets corresponding to the multiple hyperparameters; wherein during each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

[0022] According to the disclosed aspects, a host storage system includes: a host, including a central processing unit (CPU) and multiple graphics processing units (GPUs); and a storage device, wherein the CPU is configured to: determine multiple hyperparameters corresponding to multiple sub-models; wherein the storage device and the multiple GPUs are configured to: train the multiple sub-models based on the multiple hyperparameters, wherein the GPU is also configured to: update at least one sub-model among the multiple sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the multiple sub-models are trained for a first predetermined number of rounds, wherein during each round of the first predetermined number of rounds: the storage device is also configured to perform preprocessing on the original data set based on the multiple hyperparameters to generate multiple preprocessed data sets corresponding to the multiple hyperparameters; wherein, during each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

[0023] According to the disclosed aspects, a universal flash memory UFS system includes: a UFS host, including a central processing unit (CPU) and multiple graphics processing units (GPUs); a UFS device; and a UFS interface, configured to communicate between the UFS host and the UFS device, wherein the CPU is configured to: determine multiple hyperparameters corresponding to multiple sub-models; wherein the UFS device and the multiple GPUs are configured to: train the multiple sub-models based on the multiple hyperparameters, wherein the CPU is also configured to: update at least one sub-model among the multiple sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the multiple sub-models are trained for a first predetermined number of rounds, and during each round of the first predetermined number of rounds: the UFS device is also configured to pre-process the original data set based on the multiple hyperparameters to generate multiple pre-processed data sets corresponding to the multiple hyperparameters; wherein, during each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple pre-processed data sets.

[0024] According to the disclosed aspects, a data center system includes: multiple application servers; and multiple storage servers, wherein at least one of the multiple application servers includes a central processing unit (CPU), multiple storage devices and multiple graphics processing units (GPUs), wherein the CPU is configured to: determine multiple hyperparameters corresponding to multiple sub-models; the multiple storage devices and the multiple GPUs are configured to: train the multiple sub-models based on the multiple hyperparameters, wherein the CPU is also configured to: update at least one sub-model among the multiple sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the multiple sub-models are trained for a first predetermined number of rounds, wherein during each round of the first predetermined number of rounds, the multiple storage devices are also configured to pre-process the original data set based on the multiple hyperparameters to generate multiple pre-processed data sets corresponding to the multiple hyperparameters; wherein during each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple pre-processed data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 is a block diagram illustrating an electronic device according to some example embodiments;

[0027] Figure 2 is a diagram showing some example embodiments Figure 1A block diagram of a storage device;

[0028] Figure 3 is a flowchart illustrating a method for automatic data enhancement according to some example embodiments;

[0029] Figure 4 is a flow chart illustrating a process of training each sub-model for a first predetermined number of rounds according to some example embodiments;

[0030] Figure 5 is a flowchart illustrating a process of training each sub-model during one round according to some example embodiments;

[0031] Figure 6 is a flowchart illustrating a process of updating sub-models and hyperparameters according to some example embodiments;

[0032] Figure 7 is a flow chart illustrating a process of transferring data between a CPU, a storage device, and a GPU according to some example embodiments;

[0033] Figure 8 is a schematic diagram of a system to which a storage device is applied according to an embodiment;

[0034] Fig. 9 is a block diagram illustrating a host storage system according to some example embodiments;

[0035] Fig.10 is a diagram of a UFS system according to an embodiment; and

[0036] Fig.11 FIG. 1 is a diagram of a data center to which a storage device is applied according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following detailed description is provided to help the reader gain a comprehensive understanding of the methods, devices, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, devices, and / or systems described herein will be clear after understanding the disclosure. For example, except for operations that must occur in a particular order, the order of operations described herein is merely an example, and the embodiments are not limited to the specific examples described herein and may be changed as will be clear after understanding the disclosure. In addition, for greater clarity and brevity, descriptions of features known in the art may be omitted.

[0038] The features described herein may be implemented in different forms and should not be construed as being limited to the specific examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein that will be clear after understanding the disclosure of the present application.

[0039] The structural or functional description of the examples disclosed herein below is intended only for the purpose of describing the examples, and the examples may be implemented in various forms. The examples are not intended to be limiting, but various modifications, equivalents, and substitutes are also intended to be included within the scope of the claims.

[0040] Although terms such as "first" and "second" may be used to explain various components, the components are not limited to the terms. These terms should only be used to distinguish one component from another component. For example, within the scope of the present disclosure, a "first" component may be referred to as a "second" component, or similarly, a "second" component may be referred to as a "first" component.

[0041] It will be understood that when a component is referred to as being “connected to” another component, the component may be directly connected or coupled to the other component or intervening components may be present.

[0042] As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. It should also be understood that the terms "include" and / or "comprising" as used herein indicate the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, and do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0043] Unless otherwise defined, all terms (including technical or scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the examples belong. It will also be understood that, unless expressly defined as such herein, terms (such as those defined in general dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense.

[0044] Hereinafter, examples will be described in detail with reference to the accompanying drawings. Regarding reference numerals assigned to elements in the drawings, it should be noted that the same elements will be denoted by the same reference numerals and redundant descriptions thereof will be omitted.

[0045] Figure 1 is a block diagram illustrating an electronic device according to some example embodiments.

[0046] The electronic device 1 according to various exemplary embodiments of the present disclosure may include, for example, a mobile phone, a tablet personal computer (PC), a personal digital assistant (PDA), etc. However, the embodiment is not limited thereto, and the electronic device according to the present disclosure may be any electronic device having an image processing function.

[0047] like Figure 1As shown, the electronic device 1 may include a central processing unit (CPU) 11 , a plurality of storage devices 12 , and a plurality of graphic processing units (GPUs) 13 .

[0048] According to an example embodiment, the CPU 11 may determine a plurality of hyperparameters corresponding to the plurality of sub-models. The hyperparameters may represent a strategy for performing data enhancement on an image.

[0049] For example, each hyperparameter may include multiple pre-processing operators corresponding to multiple types of image processing, and each pre-processing operator may include: a first element indicating one type of image processing among the multiple types of image processing, a second element indicating the ratio of the amount of data on which the one type of image processing will be performed to a predetermined batch size, and a third element indicating the amplitude of performing the one type of image processing.

[0050] For example, various types of image processing may include at least one of cropping, equalization, rotation, translation, inversion, automatic contrast, overexposure, tone separation, sharpening, color change, contrast adjustment, brightness adjustment, cutout, etc., however, the examples are not limited to this, and various types of image processing may also include other image processing.

[0051] As an example, the batch size may represent the size or amount of training data used to make one update to the weights of a sub-model.

[0052] According to example embodiments, the plurality of storage devices 12 and the plurality of GPUs 13 may train the plurality of sub-models based on the plurality of hyperparameters. For example, the plurality of storage devices 12 and the plurality of GPUs 13 may train the plurality of sub-models for a first predetermined number of epochs. For example, this may mean that training the plurality of sub-models may be performed by the plurality of GPUs 13 through a specific number of training epochs, and the specific number may be a first predetermined number.

[0053] According to an example embodiment, the number of the plurality of hyperparameters may be equal to the number of the plurality of sub-models, such that the plurality of preprocessed data sets corresponding to the plurality of hyperparameters can be used to train the plurality of sub-models.

[0054] According to an example embodiment, the number of the plurality of GPUs 13 may be equal to the number of the plurality of sub-models, such that the plurality of GPUs 13 can perform training on the plurality of sub-models in parallel.

[0055] According to an example embodiment, the number of multiple storage devices 12 may be greater than the number of multiple sub-models, so that one or more storage devices 12 can be used to generate a preprocessed data set for training a sub-model, thereby training multiple sub-models in parallel through multiple GPUs.

[0056] For example, each of the plurality of storage devices 12 may include a smart solid state drive (SSD). However, the embodiment is not limited thereto, and the storage device 12 may be other memory capable of performing image processing.

[0057] For example, one round may correspond to or include a complete training of the sub-model using all data included in the preprocessed dataset generated based on the hyperparameters corresponding to the sub-model.

[0058] According to an example embodiment, in each of the first predetermined number of rounds, the plurality of storage devices 12 may perform preprocessing on the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets corresponding to the plurality of hyperparameters, and the plurality of GPUs 13 may train a plurality of sub-models based on the plurality of preprocessed data sets. Hereinafter, the storage device 12 that performs preprocessing based on the hyperparameters corresponding to the sub-model may be referred to as the storage device 12 corresponding to the hyperparameters or the sub-model, and the GPU 13 that performs training based on the preprocessed data set corresponding to the hyperparameters corresponding to the sub-model may be referred to as the GPU 13 corresponding to the hyperparameters or the sub-model.

[0059] According to an example embodiment, the CPU 11 may update at least one sub-model among the trained multiple sub-models and at least one hyperparameter corresponding to the updated at least one sub-model. In an embodiment, the trained multiple sub-models may be referred to as multiple trained sub-models.

[0060] For example, the at least one sub-model may include n worst sub-models among the trained multiple sub-models.

[0061] As an example, the performance of each trained sub-model among the trained multiple sub-models may be determined using the original data set (or a training data set different from the original data set), and the n sub-models with the best performance among the trained multiple sub-models may be determined as the n best sub-models, and the n sub-models with the worst performance among the trained multiple sub-models may be determined as the n worst sub-models, where n may be an integer greater than or equal to 1 and less than (2N+1) / 2, and N may represent the total number of the multiple sub-models. For example, the performance of the sub-model may be determined based on at least one of the number of errors generated by the sub-model and the accuracy of the sub-model.

[0062] As one example, the CPU 11 may update the n worst sub-models by replacing the n worst sub-models with the n best sub-models.

[0063] For example, the CPU 11 may determine the n hyperparameters corresponding to the updated n worst submodels by performing perturbations on the n hyperparameters corresponding to the n best submodels, so as to update the n hyperparameters corresponding to the updated n worst submodels. As an example, performing perturbations on the hyperparameters may include increasing or decreasing at least one third element of at least one preprocessing operator among a plurality of preprocessing operators included in the hyperparameters by a value within a predetermined range.

[0064] According to an example embodiment, the above-mentioned training step and updating step may be repeated until the total number of training rounds applied to the plurality of sub-models is equal to a second predetermined number. For example, the second predetermined number may be greater than the first predetermined number. According to an example embodiment, the second predetermined number may be equal to the product of the first predetermined number and "a value obtained by adding the number of times the training step and the updating step are repeated to two ('2')".

[0065] According to an example embodiment, the step of repeating the training may include training a plurality of sub-models including the updated at least one sub-model for a first predetermined number of rounds. During each round, at least one storage device among the plurality of storage devices 12 corresponding to the at least one sub-model may perform preprocessing on the original data set based on the updated at least one hyperparameter to generate at least one preprocessed data set corresponding to the updated at least one hyperparameter. In addition, during each round, the remaining storage devices except at least one storage device among the plurality of storage devices 12 may perform preprocessing on the original data set based on the non-updated hyperparameters among the plurality of hyperparameters to generate preprocessed data sets respectively corresponding to the non-updated hyperparameters among the plurality of hyperparameters.

[0066] According to an exemplary embodiment, the CPU 11 may determine the hyperparameters corresponding to the best sub-model among the trained multiple sub-models after stopping the repeated training step and the updating step. For example, the determined hyperparameters may represent the best strategy for performing data enhancement on the image.

[0067] Compared with other methods of performing preprocessing through a GPU, the example embodiments of the present disclosure may use a storage device to perform preprocessing, and may therefore allow the GPU to continuously acquire data and perform training, thereby improving the efficiency of the training model, and increasing the batch size (Batch size) used for training, thereby improving the effect of training. Compared with other methods of performing preprocessing through a CPU, the example embodiments of the present disclosure may perform preprocessing through a storage device to allow input and output costs to be reduced, and may allow CPU occupancy to be reduced and the efficiency of preprocessing to be improved. In addition, compared with a GPU and a CPU, a storage device may consume less power when performing preprocessing.

[0068] Figure 2is a diagram showing some example embodiments Figure 1 A block diagram of a storage device.

[0069] Figure 2 The storage device 12 shown in FIG. 1 may be a reference Figure 1 Any one of the described multiple storage devices, and the multiple storage devices have the same configuration as each other.

[0070] like Figure 2 As shown, the storage device 12 may include a first controller 121 , a field programmable gate array (FPGA) 122 , and a solid state drive (SSD) 123 .

[0071] According to example embodiments, the first controller 121 may receive a hyperparameter (eg, one hyperparameter) from the CPU 11 , and may control the FPGA 122 to perform pre-processing on the original data set based on the received hyperparameter.

[0072] According to example embodiments, the first controller 121 may determine part of the raw data within the raw data set on which image processing will not be performed based on the received hyperparameters. According to an embodiment, the part of the raw data may be referred to as a portion of the raw data set.

[0073] According to an example embodiment, the first controller 121 may divide the original data set into a plurality of sub-data sets in units of a predetermined batch size, may sequentially select sub-data sets from among the plurality of sub-data sets, and may randomly select at least one preprocessing operator from among a plurality of preprocessing operators included in the received hyperparameters each time a sub-data set is selected.

[0074] According to example embodiments, the first controller 121 may determine a portion of the original data by determining data in the selected sub-data set that will not be subjected to image processing based on the selected at least one pre-processing operator. According to an embodiment, the data in the selected sub-data set may be referred to as a portion of the selected sub-data set. For example, the first controller 121 may determine a portion of the original data set by determining a portion of the selected sub-data set that will not be subjected to image processing based on the selected at least one pre-processing operator.

[0075] According to an example embodiment, the FPGA 122 may perform image processing on the remaining data except for a portion of the original data in the original data set based on the received hyperparameters (e.g., using the intellectual property (IP) library 124) to generate processed data, and provide the processed data to a GPU (e.g., a GPU corresponding to the received hyperparameters) among the plurality of GPUs 13 for training based on the preprocessed data set corresponding to the received hyperparameters using the CPU 11. According to an embodiment, the remaining data in the original data set may be referred to as a remaining portion in the original data set.

[0076] As an example, the preprocessed data set corresponding to the received hyperparameters may include part of the original data and the processed data.

[0077] According to an example embodiment, FPGA 122 may generate processed data by performing image processing on the remaining data in the selected one sub-data set except for the data on which the image processing is not to be performed based on the selected at least one pre-processing operator. According to an embodiment, the remaining data in the sub-data set may be referred to as the remaining part in the sub-data set.

[0078] For example, when the first controller 121 randomly selects at least one preprocessing operator, the first controller 121 may determine data in a selected sub-data set that will not be subjected to image processing based on at least one second element of the selected at least one preprocessing operator, and the FPGA 122 may perform at least one image processing indicated by the first element of the selected at least one preprocessing operator on the remaining data in the selected sub-data set based on at least one third element of the selected at least one preprocessing operator.

[0079] For example, based on the fact that one preprocessing operator is selected and the first element, second element, and third element of the selected preprocessing operator are rotation, 1 / 5, and 10°, respectively, the first controller 121 may determine that 4 / 5 of the data in the selected sub-data set will not be image processed, and the FPGA 122 may rotate 1 / 5 of the data in the selected sub-data set by 10°. However, the example is not limited thereto, and the preprocessing operator may include other first elements, second elements, and third elements.

[0080] For example, when two or more pre-processing operators are selected, image processing may be performed on the remaining data except for data on which image processing is not to be performed in the selected one sub-data set in a predetermined image processing order.

[0081] According to an embodiment, at least one pre-processing operator may be used to perform image processing on the sub-dataset according to any desired method, and the present invention is not limited in any way.

[0082] For example, the SSD 123 may include an SSD controller 125 and a NAND flash memory 126 .

[0083] For example, NAND flash memory 126 may store raw data sets.

[0084] According to an example embodiment, the SSD controller 125 may provide part of the original data determined by the first controller 121 (eg, data in each sub-data set on which image processing will not be performed) to the GPU 13 corresponding to the received hyperparameters via the CPU 11.

[0085] As an example, the first controller 121 and the SSD controller 125 may be collectively referred to as a controller of the storage device 12 .

[0086] As an example, the first controller 121 may include a logic controller and a preprocessing controller. For example, the logic controller may receive hyperparameters, determine part of the original data, divide the original data set, select a sub-data set and a preprocessing operator. The preprocessing controller may control the FPGA 122 to perform image processing.

[0087] In the exemplary embodiment of the present disclosure, the preprocessing is performed by the FPGA of the storage device 12, so that the efficiency of the preprocessing can be improved.

[0088] In an example embodiment of the present disclosure, before the GPU 13 performs training using data having a batch size (for example, data in a sub-data set that will not be subjected to image processing and data generated by performing image processing on the remaining data in a sub-data set), a storage device may be used to predetermine data having a batch size, which may improve the efficiency of the training model.

[0089] Figure 3 is a flow chart illustrating a method for automatic data augmentation (which may also be referred to as “AutoAugment”) according to some example embodiments.

[0090] like Figure 3 As shown, in operation S110, the CPU 11 may determine a plurality of hyperparameters corresponding to the plurality of sub-models.

[0091] In operation S120 , a plurality of sub-models may be trained based on a plurality of hyperparameters through a plurality of storage devices 12 and a plurality of GPUs 13 .

[0092] According to example embodiments, the step of training the plurality of sub-models may include training the plurality of sub-models for a first predetermined number of rounds.

[0093] During each of the first predetermined number of rounds, the original data set may be preprocessed based on multiple hyperparameters by multiple storage devices 12 to generate multiple preprocessed data sets corresponding to the multiple hyperparameters; multiple sub-models may be trained based on the multiple preprocessed data sets by multiple GPUs 13. Figure 4 An example of a process of training each sub-model for a first predetermined number of rounds is described in detail.

[0094] In operation S130, at least one sub-model among the trained multiple sub-models may be updated through the CPU 11, and at least one hyperparameter corresponding to the updated at least one sub-model may be updated. Figure 6 An example of a process for updating sub-models and hyperparameters is described in detail.

[0095] According to an exemplary embodiment, the method for automatic data enhancement may further include repeating a training step and an updating step until the total number of training rounds of the plurality of sub-models is equal to a second predetermined number, and after stopping the repeating training step and the updating step, determining, by CPU 11, a hyperparameter corresponding to the best sub-model among the plurality of trained sub-models.

[0096] Figure 4 is a flow chart illustrating a process of training each sub-model for a first predetermined number of rounds according to some example embodiments.

[0097] like Figure 4 As shown, in operation S210, a variable "Epoch" may be set to zero ("0").

[0098] In operation S220 , a hyperparameter corresponding to the sub-model may be determined by a controller of the storage device 12 corresponding to the sub-model or the hyperparameter.

[0099] As an example, when the controller receives a hyperparameter corresponding to a sub-model from CPU 11, the controller may use the received hyperparameter as the hyperparameter corresponding to the sub-model; when the controller does not receive a hyperparameter corresponding to the sub-model from CPU 11, the controller may use the hyperparameter previously received from CPU 11 as the hyperparameter corresponding to the sub-model.

[0100] In operation S230 , the storage device 12 may perform preprocessing on the original data set based on the hyperparameter to generate a preprocessed data set corresponding to the hyperparameter.

[0101] In operation S240, the sub-model may be trained based on the preprocessed data set by the GPU 13 corresponding to the hyperparameters.

[0102] In operation S250 , the variable “Epoch” may be incremented (eg, increased by one (“1”)).

[0103] In operation S260, it may be determined whether the variable "Epoch" is equal to M (e.g., a first predetermined number). Based on the variable "Epoch" being less than M, the process may proceed to operation S230 to perform the next round of training on the sub-model. Based on the variable "Epoch" being greater than or equal to M (e.g., based on the number of rounds of training the sub-model reaching the first predetermined number), the process of training the sub-model for the first predetermined number of rounds may end.

[0104] Figure 5 is a flowchart illustrating a process of training each sub-model during one round according to some example embodiments.

[0105] like Figure 5As shown, in operation S310, a variable "Batch" may be set to zero ("0").

[0106] In operation S320, a hyper parameter corresponding to the sub model may be determined by a controller of a storage device corresponding to the sub model or the hyper parameter.

[0107] As described above, when the controller receives a hyperparameter corresponding to a sub-model from CPU 11, the controller may use the received hyperparameter as the hyperparameter corresponding to the sub-model; when the controller does not receive a hyperparameter corresponding to the sub-model from CPU 11, the controller may use the hyperparameter previously received from CPU 11 as the hyperparameter corresponding to the sub-model.

[0108] In operation S330, the original data set may be divided into a plurality of sub-original data sets in units of a predetermined batch size by the controller.

[0109] In operation S340, a controller may select a sub-dataset from a plurality of sub-datasets (eg, sequentially or non-repeatedly). In addition, in operation S341, a controller may randomly select at least one pre-processing operator from a plurality of pre-processing operators included in the hyperparameters.

[0110] In operation S350, the controller can determine the data in the selected sub-data set that will not be image processed based on the selected at least one pre-processing operator; and the FPGA of the storage device can perform image processing on the remaining data in the selected sub-data set based on the selected at least one pre-processing operator to generate processed data.

[0111] In operation S360, the sub-model may be trained by the GPU 13 corresponding to the hyperparameters using the data on which image processing is not to be performed and the processed data in the selected one of the sub-data sets.

[0112] In operation S370 , the variable “Batch” may be increased by one (“1”).

[0113] In operation S380 , it may be determined whether a variable “Batch” is less than L (eg, the number of sub-data sets).

[0114] When the variable Batch is less than L, the process may return to operations S340 and S350 to perform training on the sub-model using the next sub-original data set. When the variable Batch is greater than or equal to L, the process of training the sub-model for one round may end.

[0115] Figure 6 is a flow chart illustrating a process of updating sub-models and hyperparameters according to some example embodiments.

[0116] like Figure 6 As shown, in operation S410, a plurality of hyper parameters corresponding to a plurality of sub-models may be set by the CPU 11, and a variable "Epoch" and a variable "Stage" may be set to zero ("0"), respectively.

[0117] In operation S420 , the original data set may be preprocessed based on a plurality of hyperparameters through a plurality of storage devices 12 to generate a plurality of preprocessed data sets.

[0118] In operation S430, multiple sub-models may be trained based on multiple pre-processed data sets through multiple GPUs 13.

[0119] When one round of training of the plurality of sub-models based on the plurality of preprocessed data sets is completed, in operation S440 , the variable “Epoch” is incremented or increased by one (“1”).

[0120] In operation S450 , it may be determined whether the variable “Epoch” is less than M. Based on the variable Epoch being less than M, the process may return to operation S420 .

[0121] Based on the variable Epoch being greater than or equal to M, in operation S460 , the variable “Stage” may be incremented or increased by one (“1”), and “Epoch” may be set to zero (“0”).

[0122] In operation S470, it may be determined whether the variable "Stage" is less than S. For example, S may be associated with the number of times the steps of updating the sub-model and the hyper-parameters are repeated. As an example, the number of times the steps of updating the sub-model and the hyper-parameters are repeated may be S-2.

[0123] Based on the variable "Stage" being greater than or equal to S, the process of updating the sub-model and hyperparameters (or, the method for automatic data augmentation) may end.

[0124] Based on the variable "Stage" being less than S, in operation S480, the n worst sub-models may be replaced by the n best sub-models among the multiple trained sub-models through the CPU 11, and in operation S490, the n hyper-parameters corresponding to the n best sub-models may be perturbed respectively through the CPU 11, and the perturbed n hyper-parameters may be updated to the n hyper-parameters corresponding to the replaced n worst sub-models. Then, the process may proceed to operation S420.

[0125] Figure 7 is a flow chart illustrating transferring data between a CPU, a storage device, and a GPU according to some example embodiments.

[0126] like Figure 7As shown, in operation S510 , the CPU 11 may set a plurality of hyper parameters and send the plurality of hyper parameters to a plurality of storage devices 12 .

[0127] In operation S520 , the plurality of storage devices 12 may perform preprocessing on the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets respectively corresponding to the plurality of hyperparameters, and send the plurality of preprocessed data sets to the CPU 11 .

[0128] In operation S530 , the CPU 11 may send the plurality of pre-processed data sets to the plurality of GPUs 13 .

[0129] In operation S540, the plurality of GPUs 13 may train the plurality of sub-models for a first predetermined number of rounds based on the plurality of preprocessed data sets.

[0130] In operation S550 , the plurality of storage devices 12 (or one of the plurality of storage devices 12 ) may transmit the original data set to the CPU 11 .

[0131] In operation S560 , the CPU 11 may send the original data set to the plurality of GPUs 13 .

[0132] In operation S570 , the plurality of GPUs 13 may determine a plurality of errors corresponding to the trained plurality of sub-models based on the original data set, and send the plurality of errors to the CPU 11 .

[0133] In operation S580, the CPU 11 may determine the performance of the plurality of sub-models after training based on the plurality of errors, and determine the n sub-models with the best performance among the plurality of sub-models after training as the n best sub-models, and determine the n sub-models with the worst performance among the plurality of sub-models after training as the n worst sub-models. In addition, the CPU 11 may replace the n worst sub-models with the n best sub-models by replacing the weights of the n worst sub-models with the weights of the n best sub-models, and send the weights of the n best sub-models to the n GPUs 13 corresponding to the n worst sub-models.

[0134] In operation S590, the CPU 11 may perturb the n hyperparameters corresponding to the n best sub-models, respectively, and update the perturbed n hyperparameters to the n hyperparameters corresponding to the replaced n worst sub-models. In addition, the CPU 11 may send the updated n hyperparameters to the n storage devices corresponding to the updated n hyperparameters in the plurality of storage devices 12.

[0135] In operation S591 , the plurality of storage devices 12 may preprocess the original data set based on a plurality of hyperparameters including the updated n hyperparameters to generate a plurality of preprocessed data sets respectively corresponding to the plurality of hyperparameters, and send the plurality of preprocessed data sets to the CPU 11 .

[0136] In operation S592 , the CPU 11 may send the plurality of pre-processed data sets to the plurality of GPUs 13 .

[0137] In operation S593, multiple GPUs 13 may train multiple sub-models including the replaced n sub-models based on multiple preprocessed data sets for a first predetermined number of rounds.

[0138] Thereafter, operations S550 to S593 may be repeated until the total number of rounds of training the plurality of sub-models is equal to the second predetermined number.

[0139] Figure 8 FIG. 1 is a schematic diagram of a system 1000 to which a storage device may be applied according to an embodiment. Figure 8 The system 1000 may be a mobile system, such as a portable communication terminal (e.g., a mobile phone), a smart phone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IOT) device. However, Figure 8 The system 1000 is not necessarily limited to a mobile system, and may be a PC, a laptop, a server, a media player, or an automotive device (eg, a navigation device).

[0140] Reference Figure 8 , the system 1000 may include a main processor 1100, a memory (e.g., memory 1200a and memory 1200b), and a storage device (e.g., storage device 1300a and storage device 1300b). In addition, the system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply device 1470, and a connection interface 1480.

[0141] The main processor 1100 may control all operations of the system 1000, and more specifically, may control operations of other components included in the system 1000. The main processor 1100 may be implemented as a general-purpose processor, a dedicated processor, an application processor, or the like.

[0142] The main processor 1100 may include at least one central processing unit (CPU) core 1110, and also include a controller 1120 for controlling memories 1200a and 1200b and / or storage devices 1300a and 1300b. As an example, the main processor 1100 may include the above-mentioned CPU 11. In some embodiments, the main processor 1100 may further include an accelerator 1130, which is a dedicated circuit for high-speed data operations such as artificial intelligence (AI) data operations. The accelerator 1130 may include a GPU, a neural processing unit (NPU), and / or a data processing unit (DPU), etc., and is implemented as a chip physically separated from other components of the main processor 1100.

[0143] The memories 1200a and 1200b may be used as main storage devices of the system 1000. Although the memories 1200a and 1200b may include volatile memories, such as static random access memory (SRAM) and / or dynamic random access memory (DRAM), etc., the memories 1200a and 1200b may include non-volatile memories, such as flash memory, phase change random access memory (PRAM), and / or resistive random access memory (RRAM), etc. The memories 1200a and 1200b may be implemented in the same package as the main processor 1100.

[0144] The storage devices 1300a and 1300b may be used as non-volatile storage devices, which are configured to store data regardless of whether they are powered and have a larger storage capacity than the memories 1200a and 1200b. The storage devices 1300a and 1300b may include memory controllers (shown as "STRG CTRL") 1310a and 1310b and non-volatile memories (NVM) 1320a and 1320b, respectively, which are configured to store data via the control of the memory controllers 1310a and 1310b. Although the NVMs 1320a and 1320b may include V-NAND flash memories having a two-dimensional (2D) or three-dimensional (3D) structure, the NVMs 1320a and 1320b may include other types of NVMs, such as PRAM and / or RRAM, etc.

[0145] The storage devices 1300a and 1300b may be physically separated from the main processor 1100 and included in the system 1000, or may be implemented in the same package as the main processor 1100. In addition, the storage devices 1300a and 1300b may be of the type of SSD or memory card, and may be removably combined with other components of the system 1000 through an interface such as the connection interface 1480 described later. The storage devices 1300a and 1300b may be devices to which standard protocols such as universal flash memory (UFS), embedded multimedia card (eMMC) or NVMe may be applied, but are not limited thereto. As an example, the storage devices 1300a and 1300b may include the above-mentioned multiple storage devices 12.

[0146] The image capture device 1410 can capture still images or moving images. The image capture device 1410 may include a camera, a camcorder, and / or a webcam, etc.

[0147] The user input device 1420 may receive various types of data input by a user of the system 1000 and includes a touch pad, a keypad, a keyboard, a mouse, a microphone, and the like.

[0148] The sensor 1430 may detect various types of physical quantities that may be obtained from the outside of the system 1000 and convert the detected physical quantities into electrical signals. The sensor 1430 may include a temperature sensor, a pressure sensor, an illumination sensor, a position sensor, an acceleration sensor, a biosensor, and / or a gyro sensor, etc.

[0149] The communication device 1440 may transmit and receive signals between other devices outside the system 1000 according to various communication protocols. The communication device 1440 may include an antenna, a transceiver, a modem, or the like.

[0150] Display 1450 and speaker 1460 may be used as output devices configured to output visual information and auditory information, respectively, to a user of system 1000 .

[0151] The power supply device 1470 may appropriately convert power supplied from a battery embedded in the system 1000 and / or an external power source, and supply the converted power to each component of the system 1000 .

[0152] The connection interface 1480 may provide a connection between the system 1000 and an external device that is connected to the system 1000 and is capable of transmitting and receiving data to and from the system 1000. The connection interface 1480 may be implemented by using various interface schemes, for example, Advanced Technology Attachment (ATA), Serial ATA (SATA), External Serial ATA (e-SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI express (PCIe), NVMe, IEEE 1394, Universal Serial Bus (USB) interface, Secure Digital (SD) card interface, Multimedia Card (MMC) interface, Embedded Multimedia Card (eMMC) interface, UFS interface, Embedded UFS (eUFS) interface, Compact Flash (CF) card interface, and the like.

[0153] Those skilled in the art should understand that, although not shown, the system 1000 may further include the aforementioned GPU.

[0154] As an example, a system (e.g., system 1000) using multiple storage devices is provided, including: a main processor (e.g., main processor 1100); a memory (e.g., memory 1200a and 1200b); a storage device (e.g., storage device 1300a and storage device 1300b); and a GPU, wherein the host device includes at least the main processor (e.g., main processor 1100) and the memory (e.g., memory 1200a and memory 1200b).

[0155] Fig. 9 is a block diagram illustrating a host storage system 10 according to some example embodiments.

[0156] The host storage system 10 may include a host 100 and a storage device 200. In addition, the storage device 200 may include a memory controller 210 and an NVM 220. According to an exemplary embodiment of the present invention, the host 100 may include a host controller 110 and a host memory 120. The host memory 120 may be used as a buffer memory configured to temporarily store data to be sent to or received from the storage device 200. For example, the host 100 may further include the above-mentioned CPU 11 and GPU 13.

[0157] The storage device 200 may include a storage medium configured to store data in response to a request from the host 100. As an example, the storage device 200 may include at least one of an SSD, an embedded memory, and a removable external memory. When the storage device 200 is an SSD, the storage device 200 may be a device that complies with the NVMe standard. When the storage device 200 is an embedded memory or an external memory, the storage device 200 may be a device that complies with the UFS standard or the eMMC standard. Both the host 100 and the storage device 200 may generate a packet and send the packet according to the adopted standard protocol.

[0158] When the NVM 220 of the storage device 200 includes flash memory, the flash memory may include a 2D NAND memory array or a 3D (or vertical) NAND (VNAND) memory array. As another example, the storage device 200 may include various other types of NVM. For example, the storage device 200 may include a magnetic random access memory (MRAM), a spin transfer torque MRAM, a conductive bridge RAM (CBRAM), a ferroelectric RAM (FRAM), a PRAM, a RRAM, and various other types of memory. For example, the storage device 200 may include the above-mentioned multiple storage devices 12.

[0159] According to an embodiment, the host controller 110 and the host memory 120 may be implemented as separate semiconductor chips. Alternatively, in some embodiments, the host controller 110 and the host memory 120 may be integrated in the same semiconductor chip. As an example, the host controller 110 may be any one of a plurality of modules included in an application processor (AP). The AP may be implemented as a system on chip (SoC). In addition, the host memory 120 may be an embedded memory included in the AP or a memory module external to the AP.

[0160] The host controller 110 may manage an operation of storing data (eg, write data) of the buffer area of ​​the host memory 120 in the NVM 220 or an operation of storing data (eg, read data) of the NVM 220 in the buffer area.

[0161] like Fig. 9 As shown in , the memory controller 210 may include a host interface (IF) 211, a memory interface 212, and a CPU 213. In addition, the memory controller 210 may further include a flash translation layer (FTL), a packet manager 215 (shown as "PCKMNG"), a buffer memory 216 (shown as "BUF MEM"), an error correction code engine 217 (shown as "ECC ENG"), and an advanced encryption standard engine 218 (shown as "AES ENG").

[0162] According to an embodiment of the present disclosure, a host storage system (eg, host storage system 10 ) is provided, including: a host (eg, host 100 ); and a storage device (eg, storage device 200 ).

[0163] Fig.10 FIG. 2 is a diagram of a UFS system 2000 according to an embodiment. The UFS system 2000 may be a system conforming to the UFS standard announced by the Joint Electronic Engineering Design Development Council (JEDEC), and may include a UFS host 2100, a UFS device 2200, and a UFS interface 2300. Fig.10 To the extent that the description conflicts, Figure 8 The above description of system 1000 may also be applied to Fig.10 UFS system 2000.

[0164] Reference Fig.10 , the UFS host 2100 may be connected to the UFS device 2200 through the UFS interface 2300. Figure 8 When the main processor 1100 is an application processor, the UFS host 2100 may be implemented as a part of the application processor. The UFS host controller 2110 and the host memory 2140 may correspond to Figure 8 The UFS device 2200 may correspond to the controller 1120 of the main processor 1100 and the memories 1200a and 1200b. Figure 8 The storage devices 1300a and 1300b, and the UFS device controller 2210 and the NVM 2220 may correspond to Figure 8 For example, the UFS device 2200 may include the plurality of storage devices 12 described above.

[0165] The UFS host 2100 may include a UFS host controller 2110, an application 2120, a UFS driver 2130, a host memory 2140, and a UFS interconnect (UIC) layer 2150. The UFS device 2200 may include a UFS device controller 2210, an NVM 2220, a storage interface 2230, a device memory 2240, a UIC layer 2250, and a regulator 2260. The NVM 2220 may include a plurality of storage elements 2221. Although each storage element 2221 may include a V-NAND flash memory having a 2D structure or a 3D structure, each storage element 2221 may include another NVM, such as a PRAM and / or RRAM, etc. The UFS device controller 2210 may be connected to the NVM 2220 via a storage interface 2230. The storage interface 2230 may be set to comply with a standard protocol such as Toggle or ONFI. For example, the UFS host 2100 may also include the above-mentioned CPU 11 and GPU 13.

[0166] The application 2120 may refer to a program that wants to communicate with the UFS device 2200 to use the functions of the UFS device 2200. The application 2120 may send an input-output request (IOR) to the UFS driver 2130 for an input / output (I / O) operation on the UFS device 2200. The IOR may refer to a data read request, a data storage (or write) request, and / or a data erase (or discard) request, but is not limited thereto.

[0167] The UFS driver 2130 may manage the UFS host controller 2110 through a UFS-Host Controller Interface (UFS-HCI). The UFS driver 2130 may convert the IOR generated by the application 2120 into a UFS command defined by the UFS standard, and send the UFS command to the UFS host controller 2110. One IOR may be converted into a plurality of UFS commands. Although the UFS command may be basically defined by the SCSI standard, the UFS command may be a command dedicated to the UFS standard.

[0168] The UFS host controller 2110 may transmit the UFS command converted by the UFS driver 2130 to the UIC layer 2250 of the UFS device 2200 through the UIC layer 2150 and the UFS interface 2300. During transmission of the UFS command, the UFS host register 2111 of the UFS host controller 2110 may function as a command queue (CQ).

[0169] The UIC layer 2150 on the UFS host 2100 side may include a Mobile Industry Processor Interface (MIPI) M-PHY 2151 and a MIPI UniPro 2152 , and the UIC layer 2150 on the UFS device 2200 side may also include a MIPI M-PHY 2251 and a MIPI UniPro 2252 .

[0170] The UFS interface 2300 may include a line configured to transmit a reference clock signal REF_CLK, a line configured to transmit a hardware reset signal RESET_n of the UFS device 2200 , a pair of lines configured to transmit a pair of differential input signals DIN_t and DIN_c, and a pair of lines configured to transmit a pair of differential output signals DOUT_t and DOUT_c.

[0171] The UFS device controller 2210 may manage the NVM 2220 by using a logical unit (LU) 2211, which is a logical data storage unit. The number of the LU 2211 may be 8, but is not limited thereto.

[0172] Voltages VCC, VCCQ1−, VCCQ2 , and the like may be applied to the UFS device 2200 as power supply voltages.

[0173] According to an embodiment of the present disclosure, a UFS system (e.g., UFS system 2000) is provided, including: a UFS host (e.g., UFS host 2100); a UFS device (e.g., UFS device 2200); and a UFS interface (UFS interface 2300) for communication between the UFS host and the UFS device.

[0174] Fig.11 FIG. 3 is a diagram of a data center 3000 to which a storage device may be applied according to an embodiment of the present invention.

[0175] Reference Fig.11 , the data center 3000 may be a facility that collects various types of data and provides services, and is referred to as a data storage center. The data center 3000 may be a system for operating a search engine and a database, and may be a computing system used by a company (such as a bank) or a government agency. The data center 3000 may include application servers 3100 to 3100n and storage servers 3200 to 3200m. According to an embodiment, the number of applications 3100 to 3100n and the number of storage servers 3200 to 3200m may be selected differently. The number of application servers 3100 to 3100n and the number of storage servers 3200 to 3200m may be different from each other.

[0176] The application server 3100 or the storage server 3200 may include at least one of the processors 3110 and 3210 and the memories 3120 and 3220. Now, the storage server 3200 will be described as an example. The processor 3210 may control all operations of the storage server 3200, access the memory 3220, and execute instructions and / or data loaded into the memory 3220. The memory 3220 may be a double data rate synchronous DRAM (DDR SDRAM), a high bandwidth memory (HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM), an Optane DIMM, or a non-volatile DIMM (NVMDIMM). In some embodiments, the number of processors 3210 and memories 3220 included in the storage server 3200 may be selected differently. In one embodiment, the processor 3210 and the memory 3220 may provide a processor-memory pair. In one embodiment, the number of processors 3210 and the number of memories 3220 may be different from each other. The processor 3210 may include a single-core processor or a multi-core processor. The above description of the storage server 3200 may be similarly applied to the application server 3100. In some embodiments, the application server 3100 may not include the storage device 3150. In some embodiments, the switch 3130 may be connected between the processor 3110 and the storage device 3150. The storage server 3200 may include at least one storage device 3250. In some embodiments, the switch 3230 may be connected between the processor 3210 and the storage device 3250. In some embodiments, the storage device 3250 may include a controller 3251 (shown as "CTRL"), a flash memory device 3252 (shown as "NAND"), a DRAM 3253, and an interface 3254 (shown as "I / F"). According to embodiments, the number of storage devices 3250 included in the storage server 3200 may be selected differently.

[0177] The application servers 3100 to 3100n can communicate with the storage servers 3200 to 3200m through the network 3300 (e.g., via network interface cards (NICs) 3140 to 3140n and 3240 to 3240m). The network 3300 can be implemented by using a fiber channel (FC) or Ethernet. In this case, FC can be a medium for relatively high-speed data transmission, and an optical switch with high performance and high availability can be used. According to the access method of the network 3300, the storage servers 3200 to 3200m can be set as file storage devices, block storage devices, or object storage devices.

[0178] In one embodiment, the network 3300 may be a network dedicated to storage, such as a storage area network (SAN). For example, the SAN may be an FC-SAN, which uses an FC network and is implemented according to the FC protocol (FCP). As another example, the SAN may be an Internet Protocol (IP)-SAN, which uses a transmission control protocol (TCP) / IP network and is implemented according to the SCSI or Internet SCSI (iSCSI) protocol over TCP / IP. In another embodiment, the network 3300 may be a general network, such as a TCP / IP network. For example, the network 3300 may be implemented according to protocols such as FC over Ethernet (FCoE), network attached storage (NAS), and NVMe over fabric (NVMe-oF).

[0179] According to an embodiment of the present disclosure, a data center system (e.g., data center system 3000) is provided, comprising: a plurality of application servers (3100 to 3100n); and a plurality of storage servers (e.g., storage servers 3200 to 3200m), wherein at least one of the plurality of application servers comprises the above-mentioned CPU 11, a plurality of storage devices 12, and a plurality of GPUs 13.

[0180] The equipment, unit, module and other components described herein are implemented by hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components in the hardware components that perform the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer may be implemented by one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond and execute instructions in a limited manner to achieve the desired result). As an example, a processor or computer may include or may be connected to one or more memories storing instructions or software executed by a processor or computer. The hardware components implemented by a processor or a computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create and store data in response to the execution of instructions or software. For simplicity, the singular term "processor" or "computer" can be used in the description of the examples described herein, but in other examples, multiple processors or computers can be used, or the processor or computer can include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component, or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components can be implemented by one or more processors, or a processor and a controller, and one or more other hardware components can be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller can implement a single hardware component, or two or more hardware components. The hardware components may have any one or more of different processing configurations, examples of which include: a single processor, independent processors, parallel processors, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0181] The method for performing the operations described herein may be performed by computing hardware (e.g., by one or more processors or computers), which is implemented as executing instructions or software as described above to perform the operations performed by the method described in this application. For example, a single operation, or two or more operations may be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be performed by one or more processors, or a processor and a controller, and one or more other operations may be performed by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller may perform a single operation, or two or more operations.

[0182] The instructions or software for controlling a processor or computer to implement a hardware component and perform the method as described above may be written as a computer program, a code segment, an instruction, or any combination thereof, to individually or collectively instruct or configure a processor or computer to operate as a machine or a special-purpose computer to perform operations performed by the hardware components and methods as described above. As an example, the instructions or software may include machine code (such as machine code generated by a compiler) directly executed by a processor or computer. In another example, the instructions or software may include high-level code executed by a processor or computer using an interpreter. An ordinary programmer in the art can easily write instructions or software based on the block diagrams and flow charts shown in the accompanying drawings and the corresponding descriptions in the specification, and the block diagrams and flow charts shown in the accompanying drawings and the corresponding descriptions in the specification disclose algorithms for performing operations performed by the hardware components and methods as described above.

[0183] The instructions or software for controlling a processor or computer to implement the hardware components and perform the methods described above, and any associated data, data files, and data structures are recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage device, hard disk drive (HDD), solid state drive (SSD), flash memory, card memory (such as, multimedia card or micro card (for example, Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store instructions or software and any associated data, data files and data structures in a non-temporary manner and provide the instructions or software and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the instructions.

[0184] While the present disclosure has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the following claims.

Claims

1. A method for automatic data augmentation, include: Using a central processing unit (CPU), determining a plurality of hyperparameters corresponding to the plurality of sub-models; Using multiple storage devices and multiple graphics processing units (GPUs), training the multiple sub-models based on the multiple hyperparameters; Using the CPU, updating at least one sub-model among the plurality of sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the plurality of sub-models are trained for a first predetermined number of rounds, and Wherein, during each round of the first predetermined number of rounds, the step of training comprises: Using the plurality of storage devices, preprocessing the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets corresponding to the plurality of hyperparameters; The multiple sub-models are trained based on the multiple pre-processed data sets using the multiple GPUs.

2. The method according to claim 1, in, Each of the plurality of storage devices includes a controller and a field programmable gate array FPGA, Wherein, for each of the multiple hyperparameters, the preprocessing step includes: Determine, by a controller of a storage device corresponding to each hyperparameter among the plurality of storage devices, a portion of the original data set on which image processing will not be performed based on each hyperparameter, and provide, using a CPU, the portion of the original data set to a GPU among the plurality of GPUs for training based on a preprocessed data set corresponding to each hyperparameter, performing image processing on the remaining portion of the original data set based on the hyperparameters through the FPGA of the storage device to generate processed data, and providing the processed data to the GPU using the CPU, The preprocessed data set includes the part of the original data set and the processed data.

3. The method according to claim 2, in, Each hyperparameter includes: multiple preprocessing operators corresponding to various types of image processing, Among them, for each hyperparameter, the preprocessing steps also include: By means of the controller, the original data set is divided into a plurality of sub-data sets in units of a predetermined batch size, sequentially selecting, by the controller, a sub-data set from among the plurality of sub-data sets, By means of the controller, at least one preprocessing operator is randomly selected from the plurality of preprocessing operators based on the selection sub-data set, The step of determining the portion of the original data set comprises: determining a portion of the sub-data set that will not be subjected to image processing based on the at least one pre-processing operator, The step of generating processed data includes: performing image processing on the remaining part of the sub-data set based on the at least one pre-processing operator.

4. The method according to claim 1, further comprising: include: Repeat the training step and the updating step until a second predetermined number of rounds is reached, After the repeated training step and the updating step are stopped, the CPU determines a hyperparameter corresponding to the best sub-model among the trained sub-models. The second predetermined number is greater than the first predetermined number.

5. The method according to claim 4, in, The steps of repeated training include: training the plurality of sub-models including the updated at least one sub-model for a first predetermined number of rounds, and During each round, Using at least one storage device corresponding to the at least one sub-model among the plurality of storage devices, preprocessing the original data set based on the at least one updated hyperparameter to generate at least one preprocessed data set corresponding to the at least one updated hyperparameter; Using the remaining storage devices among the plurality of storage devices, the original data set is preprocessed based on the non-updated hyperparameters among the plurality of hyperparameters to generate a preprocessed data set corresponding to the non-updated hyperparameters.

6. The method according to claim 1, in, The at least one sub-model comprises n worst sub-models among the plurality of sub-models after training, The updating steps include: replacing the n worst sub-models with the n best sub-models among the plurality of sub-models after training, wherein n is an integer greater than or equal to 1 and less than (2N+1) / 2, wherein N represents the total number of the plurality of sub-models, and Perturbations are performed on the n hyperparameters corresponding to the n best sub-models to determine n hyperparameters corresponding to the replaced n worst sub-models.

7. The method according to claim 1, in, Each storage device of the plurality of storage devices includes a smart solid state drive (SSD).

8. The method according to claim 1, in, The original dataset includes the image dataset.

9. An electronic device, include: A central processing unit CPU is configured to: determine a plurality of hyperparameters corresponding to the plurality of sub-models; A plurality of storage devices and a plurality of graphics processing units (GPUs) are configured to: train the plurality of sub-models based on the plurality of hyperparameters, The CPU is further configured to: update at least one sub-model among the plurality of sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the plurality of sub-models are trained for a first predetermined number of rounds, and wherein, during each round of the first predetermined number of rounds, the plurality of storage devices are further configured to perform preprocessing on the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets respectively corresponding to the plurality of hyperparameters; During each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

10. A system using a storage device, include: Main processor; Memory; Storage device; as well as Multiple Graphics Processing Units (GPUs), Wherein, the main processor is configured to: determine a plurality of hyperparameters corresponding to the plurality of sub-models; The storage device and the multiple GPUs are configured to: train the multiple sub-models based on the multiple hyperparameters, The main processor is further configured to: update at least one sub-model among the multiple sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the plurality of sub-models are trained for a first predetermined number of rounds, wherein, during each round in the first predetermined number of rounds, the storage device is further configured to perform preprocessing on the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets corresponding to the plurality of hyperparameters; During each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

11. A host storage system, include: A host computer, including a central processing unit (CPU) and multiple graphics processing units (GPUs); as well as storage device, The CPU is configured to: determine a plurality of hyperparameters corresponding to the plurality of sub-models; The storage device and the multiple GPUs are configured to: train the multiple sub-models based on the multiple hyperparameters, The CPU is further configured to: update at least one sub-model among the plurality of sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the plurality of sub-models are trained for a first predetermined number of rounds, wherein, during each round in the first predetermined number of rounds, the storage device is further configured to perform preprocessing on the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets corresponding to the plurality of hyperparameters respectively; During each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

12. A universal flash memory UFS system, include: A UFS host, including a central processing unit (CPU) and multiple graphics processing units (GPUs); UFS device; as well as A UFS interface is configured to communicate between the UFS host and the UFS device, The CPU is configured to: determine a plurality of hyperparameters corresponding to the plurality of sub-models; The UFS device and the plurality of GPUs are configured to: train the plurality of sub-models based on the plurality of hyperparameters, The CPU is further configured to: update at least one sub-model among the plurality of sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the plurality of sub-models are trained for a first predetermined number of rounds, wherein, during each round of the first predetermined number of rounds, the UFS device is further configured to preprocess the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets corresponding to the plurality of hyperparameters; During each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

13. A data center system, include: Multiple application servers; as well as Multiple storage servers, At least one of the multiple application servers includes a central processing unit (CPU), multiple storage devices, and multiple graphics processing units (GPUs). The CPU is configured to: determine a plurality of hyperparameters corresponding to the plurality of sub-models; The plurality of storage devices and the plurality of GPUs are configured to: train the plurality of sub-models based on the plurality of hyperparameters, The CPU is further configured to: update at least one sub-model among the plurality of sub-models after training, and at least one hyperparameter corresponding to the at least one updated sub-model, wherein the plurality of sub-models are trained for a first predetermined number of rounds, wherein, during each round of the first predetermined number of rounds, the plurality of storage devices are further configured to preprocess the original data set based on the plurality of hyperparameters to generate a plurality of preprocessed data sets corresponding to the plurality of hyperparameters; During each round, the multiple GPUs are also configured to train the multiple sub-models based on the multiple preprocessed data sets.

Images