Storage device, host and data processing method of storage device and host

Abstract

The invention provides a storage device, a host and a data processing method of the storage device and the host. The data processing method of the storage device may include: the storage device receiving input data to be processed by the storage device; in response to a parameter update request in model training of the host, the storage device calculates deep neural network model parameters by using the input data; and the storage device sends at least a part of the calculated deep neural network model parameters to a host.

Description

Technical Field

[0001] The present disclosure relates to data processing, and more particularly, to a storage device, a host, and data processing methods of the storage device and the host. Background Art

[0002] At present, increasing the size of the Deep Neural Network (DNN) model is crucial to improving the accuracy of the DNN model. However, as the size of the DNN model has gradually increased in recent years (for example, increasing by about 10 times per year), the parameter data in the training process of the DNN model requires more storage resources. If the model training only relies on the memory of the Graphics Processing Unit (GPU), the Out Of Memory (OOM) problem or the Memory Wall problem will occur during the data processing process, thereby affecting the data processing efficiency of the model training. Summary of the invention

[0003] In order to at least solve the above problems and / or disadvantages, embodiments of the present disclosure provide a storage device, a host, and data processing methods of the storage device and the host.

[0004] According to a first aspect of at least one embodiment of the present disclosure, a data processing method of a storage device is provided, the data processing method comprising: the storage device receiving input data to be processed by the storage device; in response to a parameter update request in model training of a host, the storage device calculating deep neural network (DNN) model parameters using the input data; and the storage device sending at least a portion of the calculated DNN model parameters to the host.

[0005] Optionally, the storage device includes a non-volatile memory, wherein the data processing method further includes: the storage device compresses the input data and stores the compressed input data in the non-volatile memory, wherein, in response to a parameter update request in a model training of the host, the step of the storage device using the input data to calculate the DNN model parameters includes: in response to the parameter update request, decompressing the compressed input data stored in the non-volatile memory, and calculating the DNN model parameters using the decompressed input data, and wherein the data processing method further includes: in response to a storage request of the host, the storage device determines the calculated DNN model parameters as updated DNN model parameters, compresses the updated DNN model parameters, and stores the compressed DNN model parameters in the non-volatile memory.

[0006] Optionally, the step of receiving input data to be processed by the storage device includes: receiving the input data in units of data blocks, wherein compression rates of at least some of the data blocks satisfy a compression rate threshold condition.

[0007] Optionally, in response to a parameter update request in model training of the host, the step of the storage device calculating the DNN model parameters using the input data includes: calculating the DNN model parameters using the input data and gradient data received from the host.

[0008] Optionally, the step of calculating the DNN model parameters using the input data and the gradient data received from the host includes: in response to the parameter update request, iteratively calculating the DNN model parameters using the input data and the gradient data received from the host, so that, in the tth iterative calculation, the storage device calculates the tth DNN model parameter using the tth iteration data and the tth gradient data received from the host, wherein t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, wherein, when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

[0009] Optionally, the tth iteration data includes momentum, momentum variance and weight, wherein the step of calculating the tth DNN model parameters using the tth iteration data and the tth gradient data received from the host includes: obtaining the momentum and momentum variance in the tth DNN model parameters by performing calculation operations using the tth gradient data and the momentum and momentum variance included in the tth iteration data; and obtaining the weights in the tth DNN parameters by performing calculation operations using the momentum and momentum variance in the tth DNN model parameters and the weights in the tth iteration data.

[0010] According to a second aspect of at least one embodiment of the present disclosure, a data processing method of a host is provided, the data processing method comprising: the host sends input data to be processed by the storage device to a storage device; in response to starting to train a deep neural network (DNN) model, the host sends a parameter update request to the storage device, wherein the parameter update request is configured to request the storage device to calculate DNN model parameters using the input data; the host receives at least a portion of the calculated DNN model parameters from the storage device.

[0011] Optionally, the data processing method further includes: in response to determining that the model training is completed, sending a storage request for the DNN model parameters to the storage device, wherein the storage request is configured to initiate the storage device to determine the calculated DNN model parameters as updated DNN model parameters, compress the updated DNN model parameters and store the compressed DNN model parameters in the non-volatile memory of the storage device.

[0012] Optionally, the data processing method also includes: the host divides the data of the DNN model parameters into multiple data blocks; the host determines the compression rate of each data block; and the host allocates the data included in the data blocks whose compression rates meet the compression rate threshold condition as input data.

[0013] Optionally, the step of the host sending input data to be processed by the storage device to the storage device includes: the host adds data blocks whose compression rates satisfy a compression rate threshold condition to a candidate queue; and in response to determining that the usage rate of the host's central processing unit CPU satisfies a CPU usage threshold condition and / or the usage rate of the host's memory satisfies a memory usage threshold condition, the host sends the input data in the candidate queue to the storage device in units of data blocks in sequence.

[0014] Optionally, the data processing method further includes: in response to starting to perform training on the DNN model, the host calculates gradient data and sends the gradient data to the storage device, wherein the gradient data is configured to enable the storage device to calculate DNN model parameters based on the input data.

[0015] Optionally, the host calculates gradient data and sends the gradient data to the storage device, comprising: the host calculates the tth gradient data and sends the tth gradient data to the storage device, wherein the tth gradient data is used by the storage device to calculate the tth DNN model parameter based on the tth iteration data in the tth iteration training calculation, wherein t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, and wherein, when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

[0016] According to a third aspect of at least one embodiment of the present disclosure, a storage device is provided, comprising: a receiving unit configured to receive input data to be processed by the storage device; a processing unit configured to calculate deep neural network (DNN) model parameters using the input data in response to a parameter update request in model training of a host; and a sending unit configured to send at least a portion of the calculated DNN model parameters to the host.

[0017] Optionally, the storage device includes a non-volatile memory, wherein the processing unit is further configured to: compress input data and store the compressed input data in the non-volatile memory, wherein, in response to a parameter update request from the host, the processing unit is configured to calculate DNN model parameters using the input data by: decompressing the compressed input data stored in the non-volatile memory and calculating the DNN model parameters using the decompressed input data, and wherein the processing unit is further configured to: in response to a storage request from the host, determine the calculated DNN model parameters as updated DNN model parameters, compress the updated DNN model parameters and store the compressed DNN model parameters in the non-volatile memory.

[0018] Optionally, the receiving unit is configured to receive input data to be processed by the storage device through the following steps: receiving the input data in units of data blocks, wherein compression rates of at least some of the data blocks meet a compression rate threshold condition.

[0019] Optionally, the processing unit is configured to calculate DNN model parameters using input data in response to a parameter update request in model training of the host by performing the following operations: calculating DNN model parameters using input data and gradient data received from the host.

[0020] Optionally, the processing unit is configured to respond to the parameter update request of the host by performing the following operations to calculate the DNN model parameters using the input data and the gradient data received from the host: iteratively calculating the DNN model parameters using the input data and the gradient data received from the host, wherein, in the tth iteration calculation, the processing unit is configured to calculate the tth DNN model parameter using the tth iteration data and the tth gradient data received from the host, wherein t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, wherein, when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

[0021] Optionally, the tth iteration data includes momentum, momentum variance and weight, wherein the processing unit is configured to calculate the tth DNN model parameters using the tth iteration data and the tth gradient data received from the host through the following operations: obtaining the momentum and momentum variance in the tth DNN model parameters by performing calculation operations using the tth gradient data and the momentum and momentum variance included in the tth iteration data; obtaining the weight in the tth DNN model parameters by performing calculation operations using the momentum and momentum variance in the tth DNN model parameters and the weight in the tth iteration data.

[0022] According to a fourth aspect of at least one embodiment of the present disclosure, a host is provided, comprising: a sending unit configured to send input data to be processed by the storage device to a storage device and, in response to starting to train a deep neural network (DNN) model, send a parameter update request to the storage device, wherein the parameter update request is configured to request the storage device to calculate DNN model parameters using the input data; and a receiving unit configured to receive at least a portion of the calculated DNN model parameters from the storage device.

[0023] Optionally, the sending unit is further configured to: in response to determining that the model training is completed, send a storage request for the DNN model parameters to the storage device, wherein the storage request is configured to initiate the storage device to determine the calculated DNN model parameters as updated DNN model parameters, compress the updated DNN model parameters and store the compressed DNN model parameters in the non-volatile memory of the storage device.

[0024] Optionally, the host also includes a data unit, which is configured to: divide the data of the DNN model parameters into multiple data blocks; determine the compression rate of each data block; and allocate the data included in the data blocks whose compression rates meet the compression rate threshold condition as input data.

[0025] Optionally, the sending unit is also configured to send input data to be processed by the storage device to the storage device through the following operations: adding data blocks whose compression rates satisfy a compression rate threshold condition to a candidate queue; in response to determining that the usage rate of the host's central processing unit CPU satisfies a CPU usage threshold condition and / or the usage rate of the host's memory satisfies a memory usage threshold condition, sending the input data in the candidate queue to the storage device in units of data blocks in sequence.

[0026] Optionally, the host also includes a processing unit, which is configured to: calculate gradient data in response to starting training of the DNN model, wherein the sending unit is configured to send the gradient data to the storage device, and wherein the gradient data is configured to enable the storage device to calculate DNN model parameters based on input data.

[0027] Optionally, the processing unit and the sending unit are configured to calculate gradient data and send the gradient data to the storage device through the following operations: the processing unit calculates the tth gradient data, and the sending unit sends the tth gradient data to the storage device, wherein the tth gradient data is used by the storage device to calculate the tth DNN model parameter based on the tth iteration data in the tth iteration training calculation, wherein t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, and wherein, when t is equal to 1, the tth iteration data is input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

[0028] According to a fifth aspect of at least one embodiment of the present disclosure, a system using a storage device is provided, the system comprising: a main processor; a memory; and a storage device, wherein the main processor is configured to control the storage device to execute the data processing method as described above, and / or the main processor is configured to execute the data processing method as described above.

[0029] According to a sixth aspect of at least one embodiment of the present disclosure, there is provided a host storage system, comprising: a storage device, and a host configured to control the storage device according to the data processing method as described above.

[0030] According to a seventh aspect of at least one embodiment of the present disclosure, there is provided a storage system, the storage system comprising: a storage device; and a memory controller configured to control the storage device to execute the data processing method as described above.

[0031] According to an eighth aspect of at least one embodiment of the present disclosure, a universal flash memory UFS system is provided, the UFS system comprising: a UFS host; a UFS device; and a UFS interface connecting the UFS host with the UFS device, wherein the UFS device is configured to execute the data processing method as described above, and / or the UFS host is configured to execute the data processing method as described above.

[0032] According to a ninth aspect of at least one embodiment of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to execute the data processing method as described above.

[0033] According to a tenth aspect of at least one embodiment of the present disclosure, a computer-readable storage medium is provided, wherein, when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the data processing method as described above.

[0034] According to at least one embodiment of the present disclosure, the storage device, the host, and the data processing method of the storage device and the host have at least the following technical effects:

[0035] Parameter calculations in DNN model training are performed on the storage device side, only necessary data is transmitted between the host and the storage device, and a large amount of data is stored in the storage device, which can greatly reduce unnecessary data transmission overhead and host memory usage. Therefore, unprecedented model scale can be achieved on limited resources, reducing CPU and GPU computing overhead and data movement, and saving host storage space, thereby solving OOM problems or memory wall problems. In addition, the host can identify sparse tensor data to store it in the storage device, saving storage resources without affecting data quality and reducing unnecessary storage space usage.

[0036] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0038] Figure 1 is a diagram showing an implementation of a related data processing method;

[0039] Figure 2 is a flowchart illustrating a data processing method of a storage device according to at least one embodiment of the present disclosure;

[0040] Figure 3 is a diagram illustrating an example process of a data processing method according to at least one embodiment of the present disclosure;

[0041] Figure 4 is a diagram illustrating an iterative computation process according to at least one embodiment of the present disclosure;

[0042] Figure 5 is a flowchart illustrating a data processing method of a host according to at least one embodiment of the present disclosure;

[0043] Figure 6 is a diagram illustrating an example implementation of a data processing method of a host according to at least one embodiment of the present disclosure;

[0044] Figure 7 is a diagram showing tensor data in an example Visual Geometry Group 16 network;

[0045] Figure 8 is a diagram illustrating sending input data according to at least one embodiment of the present disclosure;

[0046] Fig. 9 is a block diagram illustrating a storage device according to at least one embodiment of the present disclosure;

[0047] Fig.10 is a block diagram illustrating a host according to at least one embodiment of the present disclosure;

[0048] Fig.11 is a schematic diagram of a system 1000 to which a storage device is applied according to at least one embodiment;

[0049] Fig.12 is a block diagram of a host storage system 10 according to an exemplary embodiment;

[0050] Fig.13 is a block diagram of a storage system 15 according to at least one embodiment of the present invention;

[0051] Fig.14 is a diagram of a UFS system 2000 according to at least one embodiment;

[0052] Fig.15 is a graph showing memory usage statistics for various networks; and

[0053] Fig.16 is a diagram illustrating a comparison between a related data processing method and a data processing method according to at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0054] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0055] The terms "include" or "comprising" used in this specification illustrate the existence of the stated features, wholes, steps, operations, elements, components and / or combinations thereof, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof. It should be understood that although the terms "first", "second", "third", etc. are used to describe various information, the information should not be limited by these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of the present disclosure, the first information may be referred to as the second information; and similarly, the second information may also be referred to as the first information. As used herein, the term "in response to..." may be understood to mean "when...", "at..." or "if", depending on the context.

[0056] In addition, the phrase "at least one of..." that appears in the present disclosure includes three types of parallel situations: "any one of...", "any combination of...", and "the whole of..." For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" includes the following three parallel situations: (1) performing step 1; (2) performing step 2; (3) performing step 1 and step 2.

[0057] Unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as those generally understood by ordinary technicians in the field to which the present disclosure belongs based on the understanding of the disclosure of this application, and should not be interpreted in an idealized or overly formal way. Here, the use of the term "may" with respect to examples or embodiments (for example, "... may include") indicates that there is at least one example or embodiment that includes or implements such features, and all examples are not limited to this. Unless otherwise clearly defined, terms in the singular also include plural forms.

[0058] Unless otherwise defined, the functional elements used in the description or drawings in the specification (including functional elements modified by terms such as "unit", "module", "... device / user") and / or the functional blocks shown in the drawings may be implemented in the form of processing circuits (including software, hardware or a combination thereof configured to perform specific functions). For example, the processing circuit may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), etc., and / or may include active and / or passive electrical components (such as transistors, resistors, capacitors, etc.) and / or an electronic circuit including one or more of the components. The division of functions for the processing circuit may be provided to specific functional elements, as described in further detail below.

[0059] In addition, according to example embodiments, examples of computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in electronic devices such as clients, hosts, proxy devices, servers, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0060] In order to solve the out of memory (OOM) problem or memory wall problem in the data processing of model training (especially the data processing of ultra-large model training), the relevant technology proposes two typical attempts on the Zero Redundancy Optimizer (Zero): ZeRO-Offload technology and ZeRO-Infinity technology. The idea of ​​ZeRO-Offload technology is to offload parameter data and calculations to the central processing unit (CPU) so that the CPU performs the functions of the optimizer (Optimizer) and calculator. Therefore, model training not only uses the storage resources of the memory on the CPU side, but also uses the computing resources on the CPU side, thereby reducing the memory usage of the GPU and the computing pressure of the GPU. Similarly, the idea of ​​ZeRO-Infinity technology is to use the entire memory capacity to store parameter data, that is, to use all heterogeneous memories (GPU, CPU and non-volatile memory (NVMe). Although these two technologies can partially solve the OOM problem and / or the memory wall problem, they have many defects, such as the capacity of dynamic random access memory (DRAM) does not meet the exponentially increasing capacity requirements of model training, and / or the efficiency of data processing is reduced due to factors such as bus competition on DRAM (for example, the throughput of large model training is reduced due to bus competition), frequent and repeated data transmission between the host and NVMe, and the input / output (I / O) delay is extended due to the lower interface bandwidth of NVMe than DRAM. Refer to the following Figure 1 The relevant data processing methods are described in detail.

[0061] Figure 1 It is a diagram showing an implementation of a related data processing method.

[0062] Reference Figure 1 When training a deep neural network (DNN) model, the GPU on the host side needs to perform training operations such as weight initialization, forward (FWD) propagation, backward (BWD) propagation, gradient calculation, and weight update. The memory overhead of the model parameters during the calculation period of the GPU executing the DNN model training is relatively large.

[0063] In the related data processing method, by using the memory of the GPU on the host side and the memory of the solid state disk (SSD) on the storage device side (for example, Figure 1The flash memory (Flash) in the storage device side stores the parameter data (such as weight W, momentum M and momentum variance V) of the DNN model, and the optimizer (optimizer) of the CPU on the host side uses the gradient data (such as gradient G) received from the GPU and the parameter data received from the SSD on the storage device side to perform calculation operations. Specifically, in response to starting to train the Deep Neural Network (DNN) model, the GPU performs a gradient calculation operation to determine the gradient data, and sends the gradient data to the CPU for parameter calculation processing. The CPU can receive gradient data from the GPU, read the parameter data stored in the flash memory (Flash) of the SSD on the storage device side, and perform parameter calculation operations using the received gradient data (such as gradient G) and the read data (such as weight W, momentum M and momentum variance V). In the iterative process of training the DNN model, the CPU needs to continuously read the parameter data from the SSD and / or store the parameter data to the SSD. In addition, the CPU sends the calculation results (such as the updated weight W') to the GPU for subsequent operations. Through such a data processing method, during the training process of the DNN model, the host side and the storage device side frequently perform data transmission, and the host side needs to consume a large amount of computing resources. Therefore, an effective solution that can solve the above memory problem to improve data processing efficiency is needed.

[0064] Therefore, in order to at least solve the various problems and / or defects mentioned above, a storage device, a host, and a data processing method for the storage device and the host are currently proposed, which can achieve unprecedented model scale on limited resources by using the storage device to perform parameter calculations in DNN model training, reduce the computing overhead and data movement of the CPU and GPU, and save the storage space of the host, thereby solving the OOM problem or memory wall problem. Although a GPU is provided as an example, the example embodiments are not limited thereto; for example, parallel processors of other DNN configurations may be included. The following will refer to Figures 2 to 14 The technical solution according to the embodiments of the present disclosure is described in detail.

[0065] Figure 2 is a flowchart illustrating a data processing method of a storage device according to at least one embodiment of the present disclosure. Figure 3 right Figure 2 The data processing method is described in detail. Figure 3 is a diagram illustrating an example process of a data processing method according to at least one embodiment of the present disclosure.

[0066] Reference Figure 2In operation S201, the storage device receives input data to be processed by the storage device. Data may be received from a host. For example, the storage device unloads data of initial DNN model parameters (such as weight W, momentum M, and momentum variance V) to be processed by the storage device from the host. Here, although at least one embodiment only describes that the DNN model parameters include weight W, momentum M, and momentum variance V, the present disclosure is not limited thereto, and the DNN model parameters of some embodiments according to at least one embodiment may include various other parameters (such as activation values, etc.) for DNN model training, and may even include gradient G in some embodiments.

[0067] According to at least one embodiment, the storage device receives input data in units of data blocks, wherein the compression rate of at least some of the data blocks satisfies the compression rate threshold condition. Specifically, since the data for DNN model training may include a large amount of sparse tensor data, the sparsity of the tensor data may be represented by the compression rate. For example, the compression rate may correspond to the proportion of zero elements (or non-zero elements) in the data to all elements, the number of zero elements (or non-zero elements), etc. The details about the tensor data will be described below in conjunction with Figure 7 Give a description.

[0068] As an example, as the proportion or number of zero elements in the data increases, the compression rate increases. When the compression rate of the data block meets the compression rate threshold condition (for example, the compression rate of the data block is greater than or equal to the preset compression rate threshold), the data included in the data block can be processed and / or compressed by the storage device. Therefore, the compression rate of the data block of the input data received by the storage device meets the compression rate threshold condition.

[0069] According to at least one embodiment, a storage device (such as an SSD) includes a nonvolatile memory, such as a flash memory (Flash), etc. Although only an embodiment in which the storage device includes an SSD is shown according to at least one embodiment of the present disclosure, the present disclosure is not limited thereto, and the storage device may include an SSD, an embedded memory, and a removable external memory, etc. Although only an embodiment in which the nonvolatile memory includes a flash memory is shown according to at least one embodiment of the present disclosure, the present disclosure is not limited thereto, and the nonvolatile memory may include a phase change random access memory (PRAM), a resistive random access memory (RRAM), etc.

[0070] According to at least one embodiment, the data processing method may further include: compressing the input data by the storage device, and storing the compressed input data in the non-volatile memory. Specifically, since the input data may include sparse data, when storing the input data, the storage device may compress the input data and store the compressed input data in the flash memory, thereby saving storage space and bus bandwidth.

[0071] When the storage device stores input data (e.g., initial DNN model parameters) and / or after the storage device stores input data (e.g., initial DNN model parameters), in operation S202, in response to a parameter update request in model training of the host, the storage device calculates the DNN model parameters using the input data.

[0072] According to at least one embodiment, since the storage device stores compressed data, in response to a parameter update request from a host, the step of the storage device calculating the DNN model parameters using the input data includes: in response to the parameter update request, decompressing the compressed input data stored in the non-volatile memory, and calculating the DNN model parameters using the decompressed input data.

[0073] As described above, when the host performs model training, it is necessary to update the DNN model parameters. According to at least one embodiment, the host requests the storage device to calculate the updated DNN model parameters. Figure 3 Describe this step in detail.

[0074] According to at least one embodiment, first, referring to Figure 3 In operation S301, the host stores data in a storage device (such as Figure 3 In some embodiments, the host may allocate a buffer for the DNN model parameters (e.g., including weight W, momentum M, and momentum variance V) in the SSD in the host. In some embodiments, the host may also allocate a buffer for the gradient data G. For example, a predetermined buffer may be allocated for each of the weight W, momentum M, momentum variance V, and gradient data G. Optionally, the allocated buffer may be mapped to the host's memory so that the host can use the buffer of the storage device in the same manner as the host's memory is used.

[0075] In operation S302, the host sends a read request to the storage device so that the storage device reads input data from the nonvolatile memory of the storage device to the buffer. Specifically, since the storage device has stored the input data in the nonvolatile memory, the host can instruct the storage device to read the input data from the nonvolatile memory.

[0076] In operation S303, the storage device sends the input data from the volatile memory to the buffer. Specifically, the storage device decompresses the input data and stores the decompressed input data in the buffer for use.

[0077] In operation S304, a data computing request is received from the host. Specifically, in response to the data computing request, the storage device may add the data to be processed (eg, the data of the DNN model parameters in the buffer) to a computing processing queue to perform computing operations on the data to be processed.

[0078] Despite Figure 3FIG. 4 shows that the storage device receives a read request and a data calculation request from the host in operation S302 and operation S304, respectively. However, the storage device may also receive only a parameter update request and execute Figure 3 The present disclosure is not limited to the various operations of the present disclosure. According to at least one embodiment, in response to a parameter update request in a model training of a host, the step of the storage device calculating the DNN model parameters using input data includes: in response to the parameter update request of the host, calculating the DNN model parameters using the input data and the gradient data received from the host.

[0079] In operation S305, a computing operation is performed using the input data and the computed data (eg, computed DNN parameters) is stored in a buffer.

[0080] According to at least one embodiment, the following reference Figure 4 The operation of calculating DNN model parameters using input data and gradient data received from a host is described in detail.

[0081] Figure 4 is a diagram illustrating an iterative calculation process according to at least one embodiment of the present disclosure.

[0082] Reference Figure 4 In response to the parameter update request, the storage device iteratively calculates the DNN model parameters using the input data and the gradient data received from the host. Figure 4 Where t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training.

[0083] According to at least one embodiment, when the host executes DNN model training, the DNN training framework determines whether the model training is completed (i.e., the model training is ended). For example, during the model training process, when the error of the output of the DNN model meets a predetermined error condition (e.g., less than or equal to a predetermined error condition), the DNN training framework can determine that the model training is completed, and accordingly, the iterative calculation of the model training is ended. For another example, during the model training process, when the number of training iterations of the DNN model reaches a predetermined number of iterations, the DNN training framework can determine that the model training is completed. The determination of the number of iterations of model training may not be limited to the above method, and this is not described in detail.

[0084] Reference Figure 4 When the storage device performs the first iteration calculation, as described above, the input data is moved from the non-volatile memory to the buffer. Then, the input data is used as the first iteration data (including the initial momentum m 0 , initial momentum variance v 0 and the initial weight w 0 ), receiving the first gradient data g from the host 1 , and using the first iteration data and the first gradient data g1 Calculate the first DNN model parameters.

[0085] According to at least one embodiment, the first DNN model parameter may include a first momentum m 1 、First momentum variance v 1 and the first weight w 1 , and the first DNN model parameters can be used as second iteration data in the second iteration calculation.

[0086] Specifically, refer to Figure 4 The storage device can utilize the first gradient data g 1 and the initial momentum m included in the first iteration data 0 Perform calculation operations to obtain the first momentum m in the first DNN model parameters 1 The storage device can utilize the first gradient data g 1 and the initial momentum variance v included in the first iteration data 0 Perform calculation operations to obtain the first momentum variance v in the first DNN model parameters 1 The storage device may utilize the first momentum m in the first DNN model parameter 1 and the first momentum variance v 1 and the initial weight w in the first iteration data 0 Perform calculation operations to obtain the first weight w in the first DNN model parameter 1 .

[0087] According to at least one embodiment, the storage device may store the first DNN model parameters in one or more buffers. For example, the first momentum m included in the first DNN model parameters 1 、First momentum variance v 1 and the first weight w 1 They are stored in the momentum buffer, momentum variance buffer and weight buffer respectively for subsequent iterative calculations.

[0088] By analogy, in the tth iteration calculation, the storage device calculates the tth DNN model parameter using the tth iteration data and the tth gradient data received from the host.

[0089] Reference Figure 4 , the storage device can be used by using the tth gradient data g t and the t-1th momentum m included in the tth iteration data t-1 Perform calculation operations to obtain the tth momentum m in the tth DNN model parameter t The storage device can utilize the t-th gradient data g t and the t-1th momentum variance v included in the tth iteration data t-1Perform calculation operations to obtain the t-th momentum variance v in the t-th DNN model parameters t The storage device can be used to store the t-th momentum m in the t-th DNN model parameter. t and the tth momentum variance v t and the t-1th weight w in the tth iteration data t-1 Perform calculation operations to obtain the tth weight w in the tth DNN model parameter t .

[0090] According to at least one embodiment, the storage device may store the tth DNN model parameter in one or more buffers. For example, the tth momentum m included in the tth DNN model parameter t , the tth momentum variance v t and the tth weight w t They are stored in the momentum buffer, momentum variance buffer and weight buffer respectively for subsequent iterative calculations.

[0091] The storage device repeats this iterative calculation operation until an integer number of iterations of the host's model training is reached, or until a storage request from the host or an indication of completion of training is received.

[0092] Then, return to the reference Figure 2 , in operation S203, the storage device sends at least a portion of the calculated DNN model parameters to the host.

[0093] According to at least one embodiment, in each iteration calculation, the storage device receives the gradient data of the iteration calculation from the host, stores all the calculated DNN model parameters in the buffer, and stores at least a portion of the calculated DNN model parameters (for example, the weight w in the tth iteration calculation) in the buffer. t ) is sent to the host.

[0094] Return to reference Figure 3 In step S306, the storage device sends the calculated DNN model parameters to the host.

[0095] In step S307, when a storage request (e.g., a request to write updated DNN model parameters) is received from the host, the storage device may store the calculation results in a non-volatile memory. For example, the storage device may store the DNN model parameters of the current iteration calculation results when the storage request is received in a non-volatile memory. That is, the storage device may determine the DNN model parameters calculated for the last time as the updated DNN model parameters and store the updated DNN model parameters. For example, after the Tth iteration calculation, if the storage device receives a storage request, the storage device stores the Tth DNN model parameters as the updated DNN model parameters.

[0096] According to at least one embodiment, in response to a storage request from a host, a storage device determines the calculated DNN model parameters as updated DNN model parameters, compresses the updated DNN model parameters and stores the compressed DNN model parameters in a non-volatile memory (such as flash memory).

[0097] Like this, parameter calculations in DNN model training are performed on the storage device side, only necessary data is transmitted between the host and the storage device, and a large amount of data is stored in the storage device, which can greatly reduce unnecessary data transmission overhead and host memory usage. Therefore, unprecedented model scale can be achieved with limited resources, reducing the computing overhead and data movement of the CPU and GPU, and saving the host's storage space.

[0098] The above describes the data processing method of the storage device. Figures 5 to 8 The data processing method of the corresponding host is described in detail.

[0099] Figure 5 is a flowchart illustrating a data processing method of a host according to at least one embodiment of the present disclosure. Figure 6 is a diagram illustrating an example implementation of a data processing method of a host according to at least one embodiment of the present disclosure.

[0100] Reference Figure 5 , in operation S501, the host sends input data to be processed by the storage device to the storage device.

[0101] According to at least one embodiment, when the host determines to perform training of the DNN model, the DNN model parameters may be initialized. The host may initialize the DNN model parameters by using any appropriate method, which will not be described in detail herein.

[0102] The following is a detailed description of examples of tensor data. As DNN models grow in size and complexity, sparsity becomes the most critical dimension to explore efficiency and scalability. Sparsity in DNN models includes introducing some sparsity on tensor data, for example, quantizing some tensor data with lower precision (e.g., 16 bits to 8 bits), and / or pruning DNN model parameters by setting some or all of the values ​​of some tensor data to zero (e.g., block sparsity or fine-grained sparsity).

[0103] According to at least one embodiment of the present disclosure, the tensor data of the DNN model parameters may have sparsity, and the computation and storage overhead may be reduced by utilizing the sparsity. Figure 7 Describes tensor data.

[0104] Figure 7is a diagram showing tensor data in an example Visual Geometry Group (VGG) 16 network.

[0105] Tensor data can include multi-dimensional data. Figure 7 In the example, the tensor data in the form of a two-dimensional matrix is ​​taken as an example. Although the present disclosure only shows the tensor data in the form of two-dimensional matrix, the present disclosure is not limited thereto. Figure 7 , the data set [3,10] as the sparse tensor data of the VGG16 network includes only two non-zero elements, namely, [0,3]=10 and [2,4]=20. Figure 7 , other squares without numerical values ​​represent zero elements. In this case, for example, the compression ratio of the tensor data can be calculated as the number of zero elements divided by the number of all elements, that is, 14 / 15.

[0106] According to statistics, the non-zero elements in the sparse tensor data in the VGG16 network account for only about 20%. Therefore, according to an example, the sparse tensor data can be compressed to be stored in a storage device.

[0107] According to at least one embodiment, for a large amount of tensor data of a host's DNN model, only sparse tensor data (e.g., sparse vectors) are offloaded to a storage device for processing. Figure 8 This is described in detail.

[0108] Figure 8 is a diagram illustrating sending input data according to at least one embodiment of the present disclosure.

[0109] Reference Figure 8 , the host divides the data of the DNN model parameters into multiple data blocks.

[0110] Herein, a data block may represent a collection of data, for example, a batch of data.

[0111] For example, in Figure 8 In the example, each data block includes 6 sub-data blocks, and each sub-data block can represent one data (for example, a data matrix) or multiple data. For example, the first data block includes two weight sub-data blocks W 0 and W 1 , two dynamic sub-data blocks M 0 and M 1 and two momentum variance sub-data blocks V 0 and V 1 , the second data block includes two weight sub-data blocks W 2 and W 3 , two dynamic sub-data blocks M 2 and M 3 and two momentum variance sub-data blocks V2 and V 3 , the third data block includes two weight sub-data blocks W 4 and W 5 , two dynamic sub-data blocks M 4 and M 5 and two momentum variance sub-data blocks V 4 and V 5 Similarly, a large amount of data of DNN model parameters may be divided into a plurality of data blocks including one or more sub-data blocks. Although the present disclosure shows that each data block includes various parameters, the present disclosure is not limited thereto, and the data blocks may be divided in any manner.

[0112] According to at least one embodiment, the host may determine a compression ratio ECR for each data block.

[0113] For example, the host may determine the compression ratio of each sub-data block by determining the ratio of zero elements in each sub-data block (eg, each data matrix).

[0114] For example, the host may determine the compression rate of the data block based on compression rates of sub-data blocks included in the data block, for example, may determine the compression rate of the data block based on the sum of the compression rates of the sub-data blocks.

[0115] For example, the host may determine the compression rate of the data block based on the number of sub-data blocks with higher compression rates among the sub-data blocks. For example, the host may determine the sub-data blocks with compression rates greater than or equal to a predetermined sub-data block compression rate as sub-data blocks with higher compression rates (or referred to as sub-data blocks suitable for compression), and determine the ratio of the sub-data blocks with higher compression rates to all sub-data blocks as the compression rate of the data block.

[0116] According to at least one embodiment, the host determines (or allocates) data included in a data block whose compression rate satisfies a compression rate threshold condition as input data. For example, the host may identify (or determine, allocate) data in a data block whose compression rate is greater than or equal to a predetermined data block compression rate as input data.

[0117] Reference Figure 8 , the host can set the ECR to be greater than or equal to the predetermined compression rate threshold ECR thres The data included in the data block is identified (or determined, assigned) as input data.

[0118] For example, when the compression rate of the sub-data block is high (for example, greater than or equal to a predetermined compression rate threshold), the sub-data block is determined to be suitable for compression. For example, the host may determine the data block as a data block to be sent to the storage device based on the number of sub-data blocks suitable for compression being greater than a predetermined number, that is, determine the data included in the data block as input data.

[0119] For example, refer to Figure 8 , the number of sub-data blocks suitable for compression of the first data block is 1, and the compression rate of the first data block can be expressed as one-sixth (1 / 6); the number of sub-data blocks suitable for compression of the second data block is 3, and the compression rate of the second data block can be expressed as one-half (1 / 2); the number of sub-data blocks suitable for compression of the third data block is 5, and the compression rate of the third data block can be expressed as five-sixths (5 / 6). Assuming the predetermined ECR thres is 2 / 3, then only the data in the third data block can be determined as input data.

[0120] When the data blocks to be sent are determined, according to at least one embodiment, the host adds the data blocks whose compression rates meet the compression rate threshold condition to the candidate queue. According to at least one embodiment, the data blocks that are not selected can be processed according to the previous processing method of the host.

[0121] Reference Figure 8 , candidate data blocks B1, B2 and B3 already exist in the candidate queue. When the third data block meets the conditions, it is added to the candidate queue as candidate data block B4.

[0122] Although there are data blocks in the candidate queue, if the current host can complete the computing process without using the storage device, the data blocks in the candidate queue are not sent to the storage device.

[0123] According to at least one embodiment, in response to the CPU usage rate %CPU of the host satisfying the CPU usage threshold condition and / or the memory usage rate %MEM of the host satisfying the memory usage threshold condition, the host sequentially sends the input data in the candidate queue to the storage device in units of data blocks. Figure 8 When the CPU usage% is greater than the CPU usage threshold thres When the usage %MEM is greater than the threshold MEM usage %MEM thres , the candidate data blocks B1, B2, B3 and B4 in the candidate queue are unloaded to a storage device (such as an SSD) in sequence.

[0124] According to at least one embodiment, the host may periodically check the CPU and / or memory usage, or the host may monitor the CPU and / or memory usage in real time, and once a threshold condition is met, trigger an operation to send or unload input data.

[0125] In operation S502 , in response to starting to perform training on the DNN model, the host sends a parameter update request to the storage device, wherein the parameter update request is configured to request the storage device to calculate DNN model parameters using input data.

[0126] According to at least one embodiment, in response to starting to perform training on the DNN model, the host calculates gradient data and sends the gradient data to the storage device. According to at least one embodiment, the gradient data is configured to enable the storage device to calculate DNN model parameters based on the input data. Figure 6 Describe this step.

[0127] exist Figure 6 As described above, the host unloads the DNN model parameters to the storage device. When the host starts to train the DNN model through the DNN training framework, the GPU of the host performs FWD operations and / or BWD operations and calculates the gradient data G. The GPU of the host can send the calculated gradient data G to the CPU and the storage device.

[0128] According to at least one embodiment, the CPU of the host may perform parameter update processing of some DNN model parameters that cannot be sent to the storage device, that is, the CPU calculates the weight W' based on the gradient data and sends W' to the GPU.

[0129] According to at least one embodiment, the storage device performs an iterative computation process using input data and gradient data.

[0130] According to at least one embodiment, in the t-th iteration calculation, the host calculates the t-th gradient data and sends the t-th gradient data to the storage device, wherein the t-th gradient data is used by the storage device to calculate the t-th DNN model parameter based on the t-th iteration data in the t-th iteration training calculation.

[0131] In operation S503 , the host receives at least a portion of the calculated DNN model parameters from the storage device.

[0132] According to at least one embodiment, in each iteration calculation, the host sends the gradient data of the iteration calculation to the storage device, and receives the weight calculated in the iteration from the storage device (for example, in the tth iteration calculation, the weight w is received). t ).

[0133] Like this, parameter calculations in DNN model training are performed on the storage device side, only necessary data is transmitted between the host and the storage device, and a large amount of data is stored in the storage device, which can greatly reduce unnecessary data transmission overhead and host memory usage, thereby achieving unprecedented model scale on limited resources, reducing CPU and GPU computing overhead and data movement, and saving host storage space. In addition, the host identifies sparse tensor data to store it in the storage device, saving storage resources without affecting data quality and reducing unnecessary storage space usage.

[0134] A storage device and a host according to at least one embodiment of the present disclosure are described in detail below.

[0135] Fig. 9 is a block diagram illustrating a storage device according to at least one embodiment of the present disclosure.

[0136] Reference Fig. 9 , the storage device 900 may include a receiving unit 901, a processing unit 902, and a sending unit 903. Each of the receiving unit 901, the processing unit 902, and / or the sending unit 903 may communicate with any or all other elements described with reference to the storage device 900. For example, each of the receiving unit 901, the processing unit 902, and the sending unit 903 may be combined with each other in a manner (such as, in a serial and / or parallel manner) in a unidirectional communication and / or a bidirectional communication and / or a broadcast communication to transfer and / or exchange and / or receive information (such as but not limited to data and / or commands). The information may be in various formats encoded, such as, an analog format and / or a digital format.

[0137] According to at least one embodiment, the receiving unit 901 may be configured to receive input data to be processed by a storage device, the processing unit 902 may be configured to calculate DNN model parameters using the input data in response to a parameter update request in model training of a host, and the sending unit 903 may be configured to send at least a portion of the calculated DNN model parameters to the host.

[0138] That is, the receiving unit 901 may perform the same Figure 2 The processing unit 902 may perform the corresponding operation of step S201 of the data processing method as described above. Figure 2 The corresponding operation of step S202 of the data processing method described above, the sending unit 903 can perform the same Figure 2 The data processing method described in step S203 is a corresponding operation. Accordingly, the details of the various descriptions of the data processing method described above for the storage device are applicable to the storage device 900.

[0139] According to at least one embodiment, the storage device 900 includes a non-volatile memory (not shown).

[0140] According to at least one embodiment, the processing unit 902 is further configured to: compress the input data, and store the compressed input data in a non-volatile memory.

[0141] According to at least one embodiment, in response to a parameter update request in a model training of a host, the processing unit 902 is configured to calculate DNN model parameters using input data by the following operations: in response to the parameter update request, decompressing the compressed input data stored in a non-volatile memory, and calculating the DNN model parameters using the decompressed input data, and wherein the processing unit 902 is further configured to: in response to a storage request of the host, determine the calculated DNN model parameters as updated DNN model parameters, compress the updated DNN model parameters and store the compressed DNN model parameters in the non-volatile memory.

[0142] According to at least one embodiment, the receiving unit 901 is configured to receive input data to be processed by the storage device by the following steps: receiving the input data in units of data blocks, wherein the compression rates of at least some of the data blocks satisfy a compression rate threshold condition.

[0143] According to at least one embodiment, the processing unit 902 is configured to calculate DNN model parameters using input data in response to a parameter update request in model training of the host by performing the following operations: In response to a parameter update request of the host, calculate DNN model parameters using input data and gradient data received from the host.

[0144] According to at least one embodiment, the processing unit 902 is configured to respond to a parameter update request of the host and calculate the DNN model parameters using the input data and the gradient data received from the host by the following operations: in response to the parameter update request, iteratively calculate the DNN model parameters using the input data and the gradient data received from the host, wherein, in the tth iteration calculation, the processing unit 902 is configured to calculate the tth DNN model parameter using the tth iteration data and the tth gradient data received from the host, wherein t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, wherein when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

[0145] According to at least one embodiment, the tth iteration data includes momentum, momentum variance and weight, wherein the processing unit 902 is configured to calculate the tth DNN model parameters using the tth iteration data and the tth gradient data received from the host through the following operations: obtaining the momentum and momentum variance in the tth DNN model parameters by performing calculation operations using the tth gradient data and the momentum and momentum variance included in the tth iteration data; obtaining the weight in the tth DNN model parameters by performing calculation operations using the momentum and momentum variance in the tth DNN model parameters and the weight in the tth iteration data.

[0146] Fig.10is a block diagram illustrating a storage device according to at least one embodiment of the present disclosure.

[0147] Reference Fig.10 , the host 1010 may include a transmitting unit 1001 and a receiving unit 1002. Each of the transmitting unit 1001 and / or the receiving unit 1002 may communicate with any or all other elements described with reference to the host 1010. For example, each of the transmitting unit 1001 and / or the receiving unit 1002 may be combined with each other in a manner (such as, in a serial and / or parallel manner) in a unidirectional communication and / or a bidirectional communication and / or a broadcast communication to transfer and / or exchange and / or receive information (such as but not limited to data and / or commands). The information may be in various formats encoded, such as, an analog format and / or a digital format.

[0148] According to at least one embodiment, the sending unit 1001 may be configured to send input data to be processed by the storage device to the storage device and, in response to starting training of the DNN model, send a parameter update request to the storage device, wherein the parameter update request is configured to request the storage device to calculate the DNN model parameters using the input data, and the receiving unit 1002 may be configured to receive at least a portion of the calculated DNN model parameters from the storage device.

[0149] That is, the sending unit 1001 can execute the same Figure 5 The receiving unit 1002 may perform the corresponding operation of step S501 of the data processing method as described above. Figure 5 The data processing method of step S502 is a corresponding operation. Accordingly, the details of the various descriptions of the data processing method of the host described above are applicable to the host 1010.

[0150] According to at least one embodiment, the sending unit 1001 is also configured to: in response to determining that the model training is completed, send a storage request for the DNN model parameters to the storage device, wherein the storage request is configured to initiate the storage device to determine the calculated DNN model parameters as updated DNN model parameters, compress the updated DNN model parameters and store the compressed DNN model parameters in the non-volatile memory of the storage device.

[0151] According to at least one embodiment, the host 1010 may also include a data unit, which is configured to: divide the data of the DNN model parameters into multiple data blocks; determine the compression rate of each data block; and allocate the data included in the data blocks whose compression rates meet the compression rate threshold condition as input data.

[0152] According to at least one embodiment, the sending unit 1001 is also configured to send input data to be processed by the storage device to the storage device through the following operations: adding data blocks whose compression rates satisfy a compression rate threshold condition to a candidate queue; in response to determining that the usage rate of the host's central processing unit CPU satisfies a CPU usage threshold condition and / or the usage rate of the host's memory satisfies a memory usage threshold condition, sending the input data in the candidate queue to the storage device in units of data blocks in sequence.

[0153] According to at least one embodiment, the host 1010 also includes a processing unit, which is configured to: calculate gradient data in response to starting training of the DNN model, and the sending unit 1001 is configured to send the gradient data to a storage device, wherein the gradient data is configured to enable the storage device to calculate DNN model parameters based on the input data.

[0154] According to at least one embodiment, the processing unit and the sending unit 1001 are configured to calculate gradient data and send the gradient data to a storage device in response to starting to perform training on a DNN model through the following operations: the processing unit calculates the tth gradient data, and the sending unit 1001 sends the tth gradient data to the storage device, wherein the tth gradient data is used by the storage device to calculate the tth DNN model parameter based on the tth iteration data in the tth iteration training calculation, wherein t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the model training of the host, wherein when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

[0155] In some embodiments, the storage device 900 may correspond to Figures 2 to 6 and Figure 8 The host 1010 may correspond to the storage device in Figure 3 , Figures 5 to 8 The specific manner in which each unit in at least one of the above embodiments performs operations has been described in detail in the embodiments of the relevant data processing method, so it will not be elaborated here.

[0156] In addition, it should be understood that the various units in the storage device 900 and the host 1010 according to the exemplary embodiment of the present disclosure may be implemented by hardware components and / or software components. Those skilled in the art may implement the processing circuits of the various units, for example, using a field programmable gate array (FPGA) and / or an application specific integrated circuit (ASIC), according to the processing performed by the various units defined.

[0157] According to at least one embodiment of the present disclosure, a system using a storage device may also be provided, the system comprising: a main processor; a memory; and a storage device, wherein the main processor is configured to control the storage device to execute the data processing method of the storage device as described above, and / or the main processor is configured to execute the data processing method of the host as described above.

[0158] Fig.11 is a schematic diagram of a system 1000 to which a storage device is applied according to at least one embodiment.

[0159] Fig.11 The system 1000 may basically be a mobile system, such as a portable communication terminal (e.g., a mobile phone), a smart phone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IOT) device. However, Fig.11 The system 1000 need not be limited to a mobile system, and may be a PC, a laptop computer, a server, a media player, or an automotive device (e.g., a navigation device). In at least some embodiments, the system 1000 may be configured to be trained to perform speech recognition, image recognition, image classification, image processing, automatic response, autonomous driving, etc. using a DNN. Accordingly, in at least some embodiments, data used for a data processing method according to at least one embodiment of the present disclosure may include data related to speech recognition, image recognition, image classification, image processing, automatic response, autonomous driving, etc.

[0160] Reference Fig.11 , the system 1000 may include a main processor 1100, a memory (e.g., 1200a and 1200b), and a storage device (e.g., 1300a and 1300b). In addition, the system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply device 1470, and a connection interface 1480.

[0161] The main processor 1100 may control all operations of the system 1000, and more specifically, may control operations of other components included in the system 1000. The main processor 1100 may be implemented as a general-purpose processor, a dedicated processor, an application processor, or the like.

[0162] The main processor 1100 may include at least one central processing unit (CPU) core 1110, and also include a controller 1120 for controlling memories 1200a and 1200b and / or storage devices 1300a and 1300b. In some embodiments, the main processor 1100 may further include an accelerator 1130, which is a dedicated circuit for high-speed data operations such as artificial intelligence (AI) data operations. The accelerator 1130 may include a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU), etc., and is implemented as a chip physically separated from other components of the main processor 1100.

[0163] The memories 1200a and 1200b may be used as main storage devices of the system 1000. Although the memories 1200a and 1200b may respectively include volatile memories, such as static random access memory (SRAM) and / or dynamic random access memory (DRAM), etc., the memories 1200a and 1200b may respectively include non-volatile memories, such as flash memory, phase change random access memory (PRAM) and / or resistive random access memory (RRAM), etc. The memories 1200a and 1200b may be implemented in the same package as the main processor 1100.

[0164] The storage devices 1300a and 1300b may be used as non-volatile storage devices, which are configured to store data regardless of whether they are powered on and have a larger storage capacity than the memories 1200a and 1200b. The storage devices 1300a and 1300b may include memory controllers (STRG CTRL) 1310a and 1310b and non-volatile memories (NVM) 1320a and 1320b, respectively, which are configured to store data via the control of the memory controllers 1310a and 1310b. Although the NVMs 1320a and 1320b may include V-NAND flash memories having a two-dimensional (2D) or three-dimensional (3D) structure, the NVMs 1320a and 1320b may include other types of NVMs, such as PRAM and / or RRAM, etc.

[0165] The storage devices 1300a and 1300b may be physically separated from the main processor 1100 and included in the system 1000, or may be implemented in the same package as the main processor 1100. In addition, the storage devices 1300a and 1300b may have a type of solid-state devices (SSDs) or memory cards, and may be removably combined with other components of the system 100 through an interface such as a connection interface 1480 to be described later. The storage devices 1300a and 1300b may be devices to which a standard protocol such as Universal Flash Storage (UFS), Embedded Multimedia Card (eMMC), or NVMe is applied, but are not limited thereto.

[0166] The connection interface 1480 may provide a connection between the system 1000 and an external device that is connected to the system 1000 and is capable of transmitting and receiving data to and from the system 1000. The connection interface 1480 may be implemented by using various interface schemes, for example, Advanced Technology Attachment (ATA), Serial ATA (SATA), External Serial ATA (e-SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI express (PCIe), NVMe, IEEE 1394, Universal Serial Bus (USB) interface, Secure Digital (SD) card interface, Multimedia Card (MMC) interface, Embedded Multimedia Card (eMMC) interface, UFS interface, Embedded UFS (eUFS) interface, Compact Flash (CF) card interface, and the like.

[0167] In some embodiments, the memories 1200a and 1200b and / or the storage devices 1300a and 1300b may correspond to Figures 2 to 6 and Figure 8 storage device or Fig. 9 The storage device 900, the main processor 1100 may correspond to Figure 3 , Figures 5 to 8 Host or Fig.10 Host 1010.

[0168] According to at least one embodiment of the present disclosure, a host storage system may be provided. The host storage system includes: a storage device, and a host configured to control the storage device according to the host data processing method as described above.

[0169] Fig.12 is a block diagram of a host storage system 10 according to at least one example embodiment.

[0170] The host storage system 10 may include a host 100 and a storage device 200. In addition, the storage device 200 may include a memory controller 210 and an NVM 220. According to an exemplary embodiment of the present invention, the host 100 may include a host controller 110 and a host memory 120. The host memory 120 may be used as a buffer memory configured to temporarily store data to be transmitted to or received from the storage device 200.

[0171] The storage device 200 may include a storage medium configured to store data in response to a request from the host 100. As an example, the storage device 200 may include at least one of an SSD, an embedded memory, and a removable external memory. When the storage device 200 is an SSD, the storage device 200 may be a device that complies with the NVMe standard. When the storage device 200 is an embedded memory or an external memory, the storage device 200 may be a device that complies with the UFS standard or the eMMC standard. Both the host 100 and the storage device 200 may generate a packet and send the packet according to the adopted standard protocol.

[0172] When the NVM 220 of the storage device 200 includes a flash memory, the flash memory may include a 2D NAND memory array or a 3D (or vertical) NAND (VNAND) memory array. As another example, the storage device 200 may include various other types of NVM. For example, the storage device 200 may include a magnetic random access memory (MRAM), a spin transfer torque MRAM, a conductive bridge RAM (CBRAM), a ferroelectric RAM (FRAM), a PRAM, a RRAM, and various other types of memory.

[0173] According to at least one embodiment, the host controller 110 and the host memory 120 may be implemented as separate semiconductor chips. Alternatively, in some embodiments, the host controller 110 and the host memory 120 may be integrated in the same semiconductor chip. As an example, the host controller 110 may be any one of a plurality of modules included in an application processor (AP). The AP may be implemented as a system on chip (SoC). In addition, the host memory 120 may be an embedded memory included in the AP or a memory module external to the AP.

[0174] The host controller 110 may manage an operation of storing data (eg, write data) of the buffer area of ​​the host memory 120 in the NVM 220 or an operation of storing data (eg, read data) of the NVM 220 in the buffer area.

[0175] The memory controller 210 may include a host interface 211, a memory interface 212, and a CPU 213. In addition, the memory controller 210 may further include a flash translation layer (FTL), a packet manager 215, a buffer memory 216, an error correction code (ECC) engine 217, and an advanced encryption standard (AES) engine 218. The memory controller 210 may further include a working memory (not shown) in which the FTL 214 is loaded. The CPU 213 may execute the FTL 214 to control data write and read operations on the NVM 220.

[0176] The host interface 211 may transmit and receive packets to and from the host 100. The packets transmitted from the host 100 to the host interface 211 may include commands or data to be written to the NVM 220, etc. The packets transmitted from the host interface 211 to the host 100 may include responses to commands or data read from the NVM 220, etc. The memory interface 212 may transmit data to be written to the NVM 220 to the NVM 220 or receive data read from the NVM 220. The memory interface 212 may be configured to comply with a standard protocol such as Toggle or Open NAND Flash Interface (ONFI).

[0177] The FTL 214 may perform various functions, such as an address mapping operation, a wear leveling operation, and a garbage collection operation. The address mapping operation may be an operation of converting a logical address received from the host 100 into a physical address for actually storing data in the NVM 220. The wear leveling operation may be a technique for preventing excessive degradation of a specific block by allowing the blocks of the NVM 220 to be used evenly. As an example, the wear leveling operation may be implemented by using a firmware technique that balances the erase counts of physical blocks. The garbage collection operation may be a technique for ensuring available capacity in the NVM 220 by erasing an existing block after copying valid data of the existing block to a new block.

[0178] The packet manager 215 may generate a packet according to a protocol agreed to an interface of the host 100, or parse various types of information from a packet received from the host 100. In addition, the buffer memory 216 may temporarily store data to be written to or read from the NVM 220. Although the buffer memory 216 may be a component included in the memory controller 210, the buffer memory 216 may be outside the memory controller 210.

[0179] The ECC engine 217 may perform error detection and correction operations on read data read from the NVM 220. More specifically, the ECC engine 217 may generate parity bits for write data to be written to the NVM 220, and the generated parity bits may be stored together with the write data in the NVM 220. During reading of data from the NVM 220, the ECC engine 217 may correct errors in the read data by using the read data and the parity bits read from the NVM 220, and output the error-corrected read data.

[0180] The AES engine 218 may perform at least one of an encryption operation and a decryption operation on data input to the memory controller 210 by using a symmetric key algorithm.

[0181] In some embodiments, the storage device 200 may correspond to Figures 2 to 6 and Figure 8 storage device or Fig. 9 The storage device 900, the host 100 may correspond to Figure 3 , Figures 5 to 8 Host or Fig.10 Host 1010.

[0182] According to at least one embodiment of the present disclosure, a storage system may be provided, the storage system comprising: a storage device; and a memory controller. The memory controller is configured to control the storage device to execute the data processing method of the storage device as described above.

[0183] Fig.13 is a block diagram of a storage system 15 in accordance with at least one embodiment of the present invention.

[0184] Reference Fig.13 , the memory system 15 may include a memory device 17 and a memory controller 16. The memory system 15 may support a plurality of channels CH1 to CHm, and the memory device 17 may be connected to the memory controller 16 through the plurality of channels CH1 to CHm. For example, the memory system 15 may be implemented as a memory device such as an SSD.

[0185] The memory device 17 may include a plurality of NVM devices NVM11 to NVMmn. Each of the NVM devices NVM11 to NVMmn may be connected to one of the plurality of channels CH1 to CHm through a path corresponding thereto. For example, the NVM devices NVM11 to NVM1n may be connected to the first channel CH1 through paths W11 to W1n, and the NVM devices NVM21 to NVM2n may be connected to the second channel CH2 through paths W21 to W2n. In an exemplary embodiment, each of the NVM devices NVM11 to NVM1n may be implemented as an arbitrary storage element that may be operated according to a separate command from the memory controller 16. For example, each of the NVM devices NVM11 to NVM1n may be implemented as a chip or a die, but the present invention is not limited thereto.

[0186] The memory controller 16 may send and receive signals to and from the memory device 17 through the plurality of channels CH1 to CHm. For example, the memory controller 16 may send commands CMDa to CMDm, addresses ADDRa to ADDRm, and data DATAa to DATAm to the memory device 17 through the channels CH1 to CHm, or receive data DATAa to DATAm from the memory device 17.

[0187] The memory controller 16 may select one from among the NVM devices NVM11 to NVMmn connected to each of the channels CH1 to CHm by using a corresponding one of the channels CH1 to CHm, and transmit and receive signals to and from the selected NVM device. For example, the memory controller 16 may select the NVM device NVM11 from among the NVM devices NVM11 to NVM1n connected to the first channel CH1. The memory controller 16 may transmit a command CMDa, an address ADDRa, and data DATAa to the selected NVM device NVM11 through the first channel CH1, or receive data DATAa from the selected NVM device NVM11.

[0188] The memory controller 16 may transmit and receive signals to and from the memory device 17 in parallel through channels different from each other. For example, the memory controller 16 may transmit a command CMDa to the memory device 17 through the first channel CH1 while transmitting a command CMDb to the memory device 17 through the second channel CH2. For example, the memory controller 16 may receive data DATAa from the memory device 17 through the first channel CH1 while receiving data DATAb from the memory device 17 through the second channel CH2.

[0189] The memory controller 16 may control all operations of the memory device 17. The memory controller 16 may send signals to the channels CH1 to CHm and control each of the NVM devices NVM11 to NVMmn connected to the channels CH1 to CHm. For example, the memory controller 16 may send a command CMDa and an address ADDRa to the first channel CH1 and control one selected from the NVM devices NVM11 to NVM1n.

[0190] Each of the NVM devices NVM11 to NVMmn may operate via the control of the memory controller 16. For example, the NVM device NVM11 may program data DATAa based on a command CMDa, an address ADDRa, and data DATAa provided to the first channel CH1. For example, the NVM device NVM21 may read data DATAb based on a command CMDb and an address ADDb provided to the second channel CH2, and transmit the read data DATAb to the memory controller 16.

[0191] although Fig.13 An example is shown in which the storage device 17 communicates with the memory controller 16 through m channels and includes n NVM devices corresponding to each channel, but the number of channels and the number of NVM devices connected to one channel may be changed.

[0192] In some embodiments, the storage device 17 may correspond to Figures 2 to 6 and Figure 8 storage device or Fig. 9 A storage device 900.

[0193] According to at least one embodiment of the present disclosure, a universal flash memory (UFS) system may also be provided, the UFS system comprising: a UFS host; a UFS device; and a UFS interface connecting the UFS host with the UFS device. The UFS device is configured to execute the data processing method of the storage device as described above, and / or the UFS host is configured to execute the data processing method of the host as described above.

[0194] Fig.14 2 is a diagram of a UFS system 2000 according to at least one embodiment. The UFS system 2000 may be a system conforming to the UFS standard announced by the Joint Electronic Engineering Design Development Council (JEDEC), and includes a UFS host 2100, a UFS device 2200, and a UFS interface 2300. Fig.14 To the extent that the description conflicts, Fig.11 The above description of system 1000 can also be applied to Fig.14 UFS system 2000.

[0195] Reference Fig.14 , the UFS host 2100 may be connected to the UFS device 2200 through the UFS interface 2300. Fig.11 When the main processor 1100 is an application processor, the UFS host 2100 may be implemented as a part of the application processor. The UFS host controller 2110 and the host memory 2140 may correspond to Fig.11 The UFS device 2200 may correspond to the main processor 1100 and the controller 1120 and the memories 1200a and 1200b. Fig.11 The storage devices 1300a and 1300b, and the UFS device controller 2210 and the NVM 2220 may correspond to Fig.11 Memory controllers 1310a and 1310b and NVMs 1320a and 1320b.

[0196] The UFS host 2100 may include a UFS host controller 2110, an application 2120, a UFS driver 2130, a host memory 2140, and a UFS interconnect (UIC) layer 2150. The UFS device 2200 may include a UFS device controller 2210, an NVM 2220, a storage interface 2230, a device memory 2240, a UIC layer 2250, and a regulator 2260. The NVM 2220 may include a plurality of storage elements 2221. Although each storage element 2221 may include a V-NAND flash memory having a 2D structure or a 3D structure, each storage element 2221 may include another NVM, such as a PRAM and / or an RRAM, etc. The UFS device controller 2210 may be connected to the NVM 2220 through a storage interface 2230. The storage interface 2230 may be set to comply with a standard protocol such as Toggle or ONFI.

[0197] The application 2120 may refer to a program that wants to communicate with the UFS device 2200 to use the functions of the UFS device 2200. The application 2120 may send an input-output request (IOR) to the UFS driver 2130 for an input / output (I / O) operation on the UFS device 2200. The IOR may refer to a data read request, a data storage (or write) request, and / or a data erase (or discard) request, but is not limited thereto.

[0198] The UFS driver 2130 may manage the UFS host controller 2110 through a UFS-Host Controller Interface (UFS-HCI). The UFS driver 2130 may convert the IOR generated by the application 2120 into a UFS command defined by the UFS standard, and send the UFS command to the UFS host controller 2110. One IOR may be converted into a plurality of UFS commands. Although the UFS command may be basically defined by the SCSI standard, the UFS command may be a command dedicated to the UFS standard.

[0199] The UFS host controller 2110 may transmit the UFS command converted by the UFS driver 2130 to the UIC layer 2250 of the UFS device 2200 through the UIC layer 2150 and the UFS interface 2300. During transmission of the UFS command, the UFS host register 2111 of the UFS host controller 2110 may function as a command queue (CQ).

[0200] The UIC layer 2150 on the UFS host 2100 side may include a Mobile Industry Processor Interface (MIPI) M-PHY 2151 and a MIPI UniPro 2152 , and the UIC layer 2150 on the UFS device 2200 side may also include a MIPI M-PHY 2251 and a MIPI UniPro 2252 .

[0201] The UFS interface 2300 may include a line configured to transmit a reference clock signal REF_CLK, a line configured to transmit a hardware reset signal RESET_n of the UFS device 2200 , a pair of lines configured to transmit a pair of differential input signals DIN_t and DIN_c, and a pair of lines configured to transmit a pair of differential output signals DOUT_t and DOUT_c.

[0202] The frequency of the reference clock signal REF_CLK provided from the UFS host 2100 to the UFS device 2200 may be one of 19.2 MHz, 26 MHz, 38.4 MHz, and 52 MHz, but is not limited thereto. The UFS host 2100 may change the frequency of the reference clock signal REF_CLK during operation, that is, during a data transmission / reception operation between the UFS host 2100 and the UFS device 2200. The UFS device 2200 may generate a clock signal having various frequencies from the reference clock signal REF_CLK provided by the UFS host 2100 by using a phase-locked loop (PLL). And, the UFS host 2100 may set the data rate between the UFS host 2100 and the UFS device 2200 by using the frequency of the reference clock signal REF_CLK. That is, the data rate may be determined according to the frequency of the reference clock signal REF_CLK.

[0203] The UFS interface 2300 may support multiple channels, each of which may be implemented as a pair of differential lines. For example, the UFS interface 2300 may include at least one receiving channel and at least one transmitting channel. Fig.14 In the embodiment, a pair of lines configured to transmit a pair of differential input signals DIN_T and DIN_C may constitute a receiving channel, and a pair of lines configured to transmit a pair of differential output signals DOUT_T and DOUT_C ​​may constitute a transmitting channel. Fig.14 One transmission channel and one reception channel are shown in FIG. 1 , but the number of transmission channels and the number of reception channels may be changed.

[0204] The receiving channel and the transmitting channel may transmit data based on a serial communication scheme. Due to the structure in which the receiving channel is separated from the transmitting channel, full-duplex communication between the UFS host 2100 and the UFS device 2200 may be enabled. That is, while receiving data from the UFS host 2100 through the receiving channel, the UFS device 2200 may transmit data to the UFS host 2100 through the transmitting channel. In addition, control data (e.g., commands) from the UFS host 2100 to the UFS device 2200 and user data to be stored in the NVM 2220 of the UFS device 2200 or read from the NVM 2220 by the UFS host 2100 may be transmitted through the same channel. Therefore, between the UFS host 2100 and the UFS device 2200, in addition to a pair of receiving channels and a pair of transmitting channels, there is no need to further provide a separate channel for data transmission.

[0205] The UFS device controller 2210 of the UFS device 2200 may control all operations of the UFS device 2200. The UFS device controller 2210 may manage the NVM 2220 by using a logical unit (LU) 2211, which is a logical data storage unit. The number of LUs 2211 may be 8, but is not limited thereto. The UFS device controller 2210 may include an FTL, and convert a logical data address (e.g., a logical block address (LBA)) received from the UFS host 2100 into a physical data address (e.g., a physical block address (PBA)) by using address mapping information of the FTL. A logical block configured to store user data in the UFS system 2000 may have a size within a predetermined range. For example, the minimum size of a logical block may be set to 4Kbyte.

[0206] When a command from the UFS host 2100 is applied to the UFS device 2200 through the UIC layer 2250 , the UFS device controller 2210 may perform an operation in response to the command and transmit a completion response to the UFS host 2100 when the operation is completed.

[0207] As an example, when the UFS host 2100 initiates storing user data in the UFS device 2200, the UFS host 2100 may send a data storage command to the UFS device 2200. When receiving a response (“Ready to Transfer” response) from the UFS device 2200 indicating that the UFS host 2100 is ready to receive user data (Ready to Transfer), the UFS host 2100 may send the user data to the UFS device 2200. The UFS device controller 2210 may temporarily store the received user data in the device memory 2240, and store the user data temporarily stored in the device memory 2240 in a selected location of the NVM 2220 based on the address mapping information of the FTL.

[0208] As another example, when the UFS host 2100 initiates a read operation of user data stored in the UFS device 2200, the UFS host 2100 may send a data read command to the UFS device 2200. The UFS device controller 2210, which has received the command, may read the user data from the NVM 2220 based on the data read command, and temporarily store the read user data in the device memory 2240. During the read operation, the UFS device controller 2210 may detect and correct errors in the read user data by using an ECC engine (not shown) embedded therein. More specifically, the ECC engine may generate parity bits for write data to be written to the NVM 2220, and the generated parity bits may be stored in the NVM 2220 together with the write data. During the reading of data from the NVM 2220, the ECC engine may correct errors in the read data by using the read data and the parity bits read from the NVM 2220, and output the error-corrected read data.

[0209] In addition, the UFS device controller 2210 may transmit the user data temporarily stored in the device memory 2240 to the UFS host 2100. Also, the UFS device controller 2210 may further include an AES engine (not shown). The AES engine may perform at least one of an encryption operation and a decryption operation on the data transmitted to the UFS device controller 2210 by using a symmetric key algorithm.

[0210] The UFS host 2100 may sequentially store commands to be sent to the UFS device 2200 in the UFS host register 2111, which may be used as a common queue, and sequentially send the commands to the UFS device 2200. In this case, even when the previously sent command is still being processed by the UFS device 2200, that is, even before receiving a notification that the previously sent command has been processed by the UFS device 2200, the UFS host 2100 may send the next command on standby in the CQ to the UFS device 2200. Therefore, during the processing of the previously sent command, the UFS device 2200 may also receive the next command from the UFS host 2100. The maximum number of commands (or queue depth) that can be stored in the CQ may be, for example, 32. Also, the CQ may be implemented as a circular queue in which the start and end of the command line stored in the queue are indicated by a head pointer and a tail pointer.

[0211] Each of the plurality of storage elements 2221 may include a storage cell array (not shown) and a control circuit (not shown) configured to control the operation of the storage cell array. The storage cell array may include a 2D storage cell array or a 3D storage cell array. The storage cell array may include a plurality of storage cells. Although each storage cell is a single-layer cell (SLC) configured to store 1 bit of information, each storage cell may be a cell configured to store 2 or more bits of information, such as a multi-layer cell (MLC), a triple-layer cell (TLC), and a quad-layer cell (QLC). The 3D storage cell array may include a vertical NAND string in which at least one storage cell is vertically oriented and located on another storage cell.

[0212] Voltages VCC, VCCQ, and VCCQ2, etc., may be applied to the UFS device 2200 as power supply voltages. The voltage VCC may be a main power supply voltage for the UFS device 2200, and may be in the range of 2.4V to 3.6V. The voltage VCCQ may be a power supply voltage mainly used to supply a low voltage to the UFS device controller 2210, and may be in the range of 1.14V to 1.26V. The voltage VCCQ2 may be a power supply voltage mainly used to supply a voltage lower than the voltage VCC and higher than the voltage VCCQ to an I / O interface such as the MIPI M-PHY 2251, and may be in the range of 1.7V to 1.95V. The power supply voltage may be provided to the various components of the UFS device 2200 by the regulator 2260. The regulator 2260 may be implemented as a group of unit regulators, which are respectively connected to different power supply voltages among the above-mentioned power supply voltages.

[0213] In some embodiments, the UFS device 2200 may correspond to Figures 2 to 6 and Figure 8 storage device or Fig. 9 The storage device 900, the UFS host 2100 may correspond to Figure 3 , Figures 5 to 8 Host or Fig.10 Host 1010.

[0214] According to at least one embodiment of the present disclosure, there is also provided an electronic device, comprising: at least one processor; and at least one memory storing computer executable instructions. When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the data processing method as described above.

[0215] According to at least one embodiment of the present disclosure, the electronic device may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above-mentioned instructions (or instruction sets) individually or jointly. The electronic device may also be part of an integrated control system or a system manager, or may be configured as a portable electronic device interconnected with an interface locally or remotely (e.g., via wireless transmission).

[0216] In electronic devices, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0217] The processor can execute instructions or codes stored in the memory, wherein the memory can also store data. Instructions and data can also be sent and received through the network via the network interface device, wherein the network interface device can adopt any known transmission protocol.

[0218] The memory may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor or the like. In addition, the memory may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the memory.

[0219] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0220] According to at least one embodiment of the present disclosure, a computer-readable storage medium may be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the data processing method as described above.

[0221] Fig.15 is a graph showing memory usage statistics for various networks.

[0222] Reference Fig.15 It can be seen that by using at least one embodiment according to the present disclosure, the upper limit of the storage capacity in the original network training process can be improved. For example, the dotted line shows the upper limit of the storage capacity of applying at least one embodiment of the present disclosure.

[0223] Fig.16 is a diagram illustrating a comparison between a related data processing method and a data processing method according to at least one embodiment of the present disclosure.

[0224] Reference Fig.16 In the relevant data processing method, when the host performs iterative calculations on DNN model parameters during the DNN model training process, it is necessary to read the DNN model parameters (such as weight W, momentum M, and momentum variance V) from the flash memory of the storage device (such as SSD) multiple times, and write the calculated DNN model parameters (such as weight W, momentum M, and momentum variance V) to the flash memory of the storage device (such as SSD) multiple times. Such repeated data transmission consumes a lot of resources.

[0225] In at least one embodiment according to the present disclosure, the host only needs to unload the input data of the DNN model parameters (such as weight W, momentum M, and momentum variance V) to a non-volatile memory (such as flash memory) in a storage device (such as SSD) in the initial stage, and during subsequent iterative calculations, only gradient data needs to be sent and weight data needs to be received without the need for large amounts of data transmission. The host and storage device can work together to improve the efficiency of DNN model parameter calculation, thereby improving the efficiency of the overall DNN model training.

[0226] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and at least one embodiment are to be considered as examples only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0227] It should be understood that the present disclosure is not limited to the example structures that have been described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims.

Claims

1. A data processing method for a storage device, comprising: The storage device receives input data to be processed by the storage device; In response to a parameter update request in a model training of a host, the storage device calculates a deep neural network (DNN) model parameter using input data; as well as The storage device sends at least a portion of the calculated DNN model parameters to the host.

2. The data processing method according to claim 1, wherein: The storage device includes a non-volatile memory, Wherein, the data processing method further includes: The storage device compresses the input data, and Store the compressed input data into non-volatile memory, In response to a parameter update request in a model training of a host, the step of the storage device calculating the DNN model parameters using input data includes: In response to the parameter update request, decompressing the compressed input data stored in the non-volatile memory, and Compute the DNN model parameters using the decompressed input data, and Wherein, the data processing method further includes: In response to a storage request from the host, the storage device determines the calculated DNN model parameters as updated DNN model parameters, Compress the updated DNN model parameters, and Store the compressed DNN model parameters into non-volatile memory.

3. The data processing method according to claim 1, wherein: The step of receiving input data to be processed by the storage device comprises: Input data is received in units of data blocks, wherein compression rates of at least some of the data blocks satisfy a compression rate threshold condition.

4. The data processing method according to claim 1, wherein: In response to a parameter update request in a model training of a host, the step of the storage device calculating the DNN model parameters using input data comprises: Compute DNN model parameters using input data and gradient data received from the host.

5. The data processing method according to claim 4, wherein: The steps to calculate the DNN model parameters using the input data and the gradient data received from the host include: In response to the parameter update request, iteratively calculate the DNN model parameters using the input data and the gradient data received from the host, So that, in the t-th iteration calculation, the storage device calculates the t-th DNN model parameter using the t-th iteration data and the t-th gradient data received from the host, Where t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, Wherein, when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

6. The data processing method according to claim 5, wherein: The t-th iteration data includes momentum, momentum variance and weight, The step of calculating the tth DNN model parameter using the tth iteration data and the tth gradient data received from the host comprises: Obtaining momentum and momentum variance in a t-th DNN model parameter by performing a calculation operation using the t-th gradient data and the momentum and momentum variance included in the t-th iteration data; and The weights in the t-th DNN model parameters are obtained by performing a calculation operation using the momentum and momentum variance in the t-th DNN model parameters and the weights in the t-th iteration data.

7. A storage device, comprising: a receiving unit configured to receive input data to be processed by the storage device; A processing unit configured to calculate a deep neural network (DNN) model parameter using input data in response to a parameter update request in model training of a host; The sending unit is configured to send at least a portion of the calculated DNN model parameters to the host.

8. The storage device according to claim 7, wherein: The storage device includes a non-volatile memory, The processing unit is further configured to: compress the input data and store the compressed input data in a non-volatile memory. In response to a parameter update request in a model training of a host, the processing unit is configured to calculate the DNN model parameters using the input data by performing the following operations: decompressing compressed input data stored in non-volatile memory, and Compute the DNN model parameters using the decompressed input data, and Wherein, the processing unit is further configured to: In response to a storage request from the host, determining the calculated DNN model parameters as updated DNN model parameters, Compress the updated DNN model parameters, and Store the compressed DNN model parameters into non-volatile memory.

9. The storage device according to claim 7, wherein: The receiving unit is configured to receive input data to be processed by the storage device through the following steps: Input data is received in units of data blocks, wherein compression rates of at least some of the data blocks satisfy a compression rate threshold condition.

10. The storage device according to claim 7, wherein: The processing unit is configured to respond to a parameter update request in model training of the host by using input data to calculate the DNN model parameters by: Compute DNN model parameters using input data and gradient data received from the host.

11. The storage device according to claim 10, wherein: The processing unit is configured to calculate the DNN model parameters using the input data and the gradient data received from the host in response to the parameter update request of the host by performing the following operations: In response to the parameter update request, iteratively calculate the DNN model parameters using the input data and the gradient data received from the host, In the t-th iteration calculation, the processing unit is configured to calculate the t-th DNN model parameter using the t-th iteration data and the t-th gradient data received from the host, Where t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, and Wherein, when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

12. The storage device according to claim 11, wherein: The t-th iteration data includes momentum, momentum variance and weight, The processing unit is configured to calculate the tth DNN model parameter using the tth iteration data and the tth gradient data received from the host by the following operations: Obtaining momentum and momentum variance in a t-th DNN model parameter by performing a calculation operation using the t-th gradient data and the momentum and momentum variance included in the t-th iteration data; The weights in the t-th DNN model parameters are obtained by performing a calculation operation using the momentum and momentum variance in the t-th DNN model parameters and the weights in the t-th iteration data.

13. A host, comprising: A sending unit configured to send input data to be processed by the storage device to the storage device and, in response to starting to perform training on the deep neural network DNN model, send a parameter update request to the storage device, wherein the parameter update request is configured to request the storage device to calculate the DNN model parameters using the input data A receiving unit is configured to receive at least a portion of the calculated DNN model parameters from the storage device.

14. The host according to claim 13, wherein: The sending unit is further configured to: In response to determining that the model training is completed, sending a storage request for the DNN model parameters to the storage device, Wherein, the storage request is configured to initiate the storage device to determine the calculated DNN model parameters as updated DNN model parameters, compress the updated DNN model parameters and store the compressed DNN model parameters in the non-volatile memory of the storage device.

15. The host according to claim 13, wherein: The host also includes a data unit, which is configured to: Divide the data of DNN model parameters into multiple data blocks; determining a compression ratio for each data block; and Data included in a data block whose compression rate satisfies the compression rate threshold condition is allocated as input data.

16. The host according to claim 15, wherein: The sending unit is configured to send input data to be processed by the storage device to the storage device by the following operations: Adding data blocks whose compression rates meet the compression rate threshold condition to the candidate queue; as well as In response to determining that the usage rate of the host's central processing unit CPU meets the CPU usage threshold condition and / or the usage rate of the host's memory meets the memory usage threshold condition, the input data in the candidate queue is sent to the storage device in units of data blocks.

17. The host according to claim 13, wherein: The host also includes a processing unit, which is configured to: In response to starting to perform training on the DNN model, calculating gradient data, Wherein, the sending unit is configured to send the gradient data to the storage device, and Wherein, the gradient data is configured to enable the storage device to calculate DNN model parameters based on the input data.

18. The host according to claim 17, wherein: The processing unit and the sending unit are configured to calculate the gradient data through the following operations, and send the gradient data to the storage device: The processing unit calculates the t-th gradient data, and the sending unit sends the t-th gradient data to the storage device, The t-th gradient data is used by the storage device to calculate the t-th DNN model parameter based on the t-th iteration data in the t-th iteration training calculation. Where t is an integer greater than or equal to 1 and less than or equal to the number of iterations of the host's model training, Wherein, when t is equal to 1, the tth iteration data is the input data, and when t is not equal to 1, the tth iteration data is the t-1th DNN model parameter.

Images