Data processing method and data processing system

Abstract

The invention provides a data processing method and a data processing system. The data processing method executed by the near-memory processing apparatus includes: receiving a data processing request about an object from a host; obtaining data of the object to be processed from the host based on the data processing request; obtaining result data of the object by processing the data of the object; sending a message indicating completion of data processing to the host; and transmitting the result data of the object to a storage device corresponding to the object.

Description

Technical Field

[0001] The present disclosure relates to data storage, and more particularly, to a data processing method and a data processing system. Background Art

[0002] Ceph is an open source distributed storage system designed to meet the block, file, and object storage needs of modern enterprises. Ceph's highly scalable architecture makes it a new norm for block storage, object storage, and data lakes. In Ceph, each object storage device (OSD) daemon of object storage corresponds to an OSD node, and a storage device (e.g., a solid-state device (SSD)) can be set for each OSD node as the actual storage area. In the case of object storage, since Ceph's read or write operations all adopt the master-slave mode, when a Ceph-based client writes or reads data, it can only initiate a request to the primary OSD node corresponding to the object to operate on the data of the primary OSD node and other OSD nodes corresponding to the object. However, when Ceph writes and recovers data in erasure code mode, it is necessary to move known data to and store it in a volatile memory controlled by the central processing unit (CPU), perform calculations using the data stored in the volatile memory, and then write the calculation results to the storage device corresponding to the OSD node. In the processing in erasure code mode, a large amount of data movement and context switching between the memory and the CPU is required. Summary of the invention

[0003] Provided are a data processing method and a data processing system.

[0004] According to one aspect of the present disclosure, a data processing method performed by a near memory processing (PNM) device includes: receiving a data processing request regarding an object from a host; based on the data processing request, obtaining data of the object to be processed from the host; obtaining result data of the object by processing the data of the object; sending a message indicating completion of data processing to the host; and sending the result data of the object to a storage device corresponding to the object.

[0005] The PNM device may include a memory, and the step of obtaining the data of the object to be processed from the host based on the data processing request may include: based on the data processing request, the memory receives the data of the object from the host and stores the data of the object in the memory, and the step of sending the result data of the object to the storage device corresponding to the object may include: in response to the completion of data processing, sending the result data of the object from the memory to the storage device.

[0006] The data processing request may include a data write request for the object or a data restore request for the object initiated by an object storage device (OSD) daemon corresponding to the object in an erasure coding mode, and the storage device corresponding to the object may be a storage device corresponding to the OSD daemon.

[0007] The step of obtaining result data of the object by processing the data of the object may include: obtaining the result data by encoding the data of the object based on that the data processing request is a data write request initiated by the OSD daemon in the erasure code mode, and obtaining the result data by decoding the data of the object based on that the data processing request is a data restore request initiated by the OSD daemon in the erasure code mode.

[0008] The data processing method may further include: writing intermediate data generated during encoding or decoding into a memory; and reading the intermediate data from the memory.

[0009] The PNM device may be a Universal Flash Storage (UFS) device, the host may be a UFS host, and the PNM device and the host may be connected via a UFS interface.

[0010] According to one aspect of the present disclosure, a data processing method performed by a host and a near memory processing (PNM) device includes: a computing agent of the host sends a data processing request regarding an object to the PNM device; based on the PNM device receiving the data processing request, prompting the PNM device to obtain data of the object to be processed from the host; the PNM device obtains result data of the object by processing the data of the object; a message indicating that the data processing is completed is sent from the PNM device to the host; and in response to the computing agent receiving a message indicating that the data processing is completed from the PNM device, prompting the PNM device to send the result data to a storage device corresponding to the object.

[0011] The PNM device may include a memory, and based on the PNM device receiving the data processing request, the step of prompting the PNM device to obtain the data of the object to be processed from the host may include: based on the PNM device receiving the data processing request, the memory receives the data of the object from the host and stores the data of the object in the memory, and in response to the computing agent receiving a message indicating that the data processing is completed from the PNM device, the step of prompting the PNM device to send the result data of the object to the storage device corresponding to the object may include: sending the result data from the memory to the computing agent.

[0012] The data processing request may include a data write request for the object or a data restore request for the object initiated by the computing agent for the object storage device OSD daemon corresponding to the object in an erasure coding mode, and the storage device corresponding to the object may be a storage device corresponding to the OSD daemon.

[0013] The step of obtaining result data of the object by the PNM device through processing the data of the object may include: obtaining the result data by encoding the data of the object based on the data processing request being a data write request initiated by the OSD daemon in the erasure code mode, and obtaining the result data by decoding the data of the object based on the data processing request being a data restore request initiated by the OSD daemon in the erasure code mode.

[0014] The data processing method may further include: writing intermediate data generated during encoding or decoding into a memory; and reading the intermediate data from the memory.

[0015] The PNM device may be a universal flash memory (UFS) device, the host may be a UFS host, and the PNM device and the host may be connected via a UFS interface.

[0016] According to one aspect of the present disclosure, a data processing system may include: a host storage system including:

[0017] At least one host processor; and at least one host memory storing one or more host instructions; a storage device; and a near memory processing (PNM) device, comprising: at least one PNM processor; and at least one PNM memory storing one or more PNM instructions, wherein the at least one host processor can be configured to execute the one or more host instructions and cause the host storage system to perform the following operations: send a data processing request about an object to the PNM device, wherein the at least one PNM processor can be configured to execute the one or more PNM instructions and cause the PNM device to perform the following operations: based on the data processing request, obtain data of the object to be processed from the host storage system, obtain result data of the object by processing the data of the object, and send a message indicating that the data processing is completed to the host storage system, and wherein the at least one host processor can be configured to execute the one or more host instructions and cause the host storage system to perform the following operations: in response to receiving a message indicating that the data processing is completed from the PNM device, cause the PNM device to send the result data to the storage device, wherein the storage device corresponds to the object.

[0018] The at least one PNM processor may be configured to execute the one or more PNM instructions and cause the PNM device to store the data of the object in the at least one PNM memory.

[0019] The host storage system may also include a computing agent, and the data processing request may include a data write request for the object or a data restore request for the object initiated by the computing agent for an object storage device (OSD) daemon corresponding to the object in an erasure coding mode, and the storage device may be a storage device corresponding to the OSD daemon.

[0020] The at least one PNM processor may be configured to execute the one or more PNM instructions and cause the PNM device to perform the following operations: based on the data processing request being a data write request initiated by the OSD daemon in the erasure code mode, obtaining the result data by encoding the data of the object; and based on the data processing request being a data restoration request initiated by the OSD daemon in the erasure code mode, obtaining the result data by decoding the data of the object.

[0021] The at least one PNM processor may be configured to execute the one or more PNM instructions and cause the PNM device to: write intermediate data generated during encoding or decoding into the at least one PNM memory; and read the intermediate data from the at least one PNM memory.

[0022] The PNM device may be a Universal Flash Storage (UFS) device, the host storage system may be a UFS host, and the PNM device and the host storage system may be connected via a UFS interface.

[0023] The memory device may include a memory controller and at least one nonvolatile memory.

[0024] According to the near memory processing device and host and data processing method thereof of the embodiments of the present disclosure, data processing and temporary storage can be performed inside the PNM device, thereby avoiding a large amount of data transmission between the CPU on the host side and the main memory. In addition, by offloading the computational processing including encoding and decoding from the CPU on the host side to the PNM device, the CPU utilization rate can be reduced and computing resources can be saved. In addition, by using defined functions, the data writing and data restoration processing in the erasure code mode of Ceph is simplified, thereby improving the data processing performance of Ceph in the erasure code mode.

[0025] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other aspects and features of certain embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0027] Figure 1 and Figure 2 is a diagram showing an example of a data processing method of the related art;

[0028] Figure 3 is a diagram illustrating an example of a related art and a data processing method according to one or more embodiments of the present disclosure;

[0029] Figure 4 is a flowchart illustrating a data processing method according to one or more embodiments of the present disclosure;

[0030] Figure 5 is a diagram illustrating an example of a data writing process according to one or more embodiments of the present disclosure;

[0031] Figure 6 is a diagram illustrating an example of a data restoration process according to one or more embodiments of the present disclosure;

[0032] Fig. 7A and Figure 7B is a diagram showing an example of data of data processing according to one or more embodiments of the present disclosure;

[0033] Figure 8 is a flowchart illustrating an example of a data writing process according to one or more embodiments of the present disclosure;

[0034] Fig. 9 is a flowchart illustrating an example of a data restoration process according to one or more embodiments of the present disclosure;

[0035] Fig.10 is a flow chart of a data processing method according to one or more embodiments of the present disclosure;

[0036] Fig.11 is a block diagram illustrating a PNM device according to one or more embodiments of the present disclosure;

[0037] Fig.12 is a block diagram illustrating a host according to one or more embodiments of the present disclosure;

[0038] Fig.13 is a schematic diagram of a system to which a storage device is applied according to one or more embodiments of the present disclosure;

[0039] Fig.14 is a block diagram of a host storage system according to one or more embodiments of the present disclosure;

[0040] Fig.15 is a block diagram of a storage system according to one or more embodiments of the present disclosure;

[0041] Fig.16 is a diagram of a universal flash storage (UFS) system according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] In some aspects, embodiments may provide an efficient solution that can implement data processing of an OSD daemon while reducing data transfer between memory and the CPU.

[0043] In order to enable persons of ordinary skill in the art to better understand the present disclosure, one or more embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0044] The terms "include" or "comprising" used in this specification indicate the existence of the stated features, wholes, steps, operations, elements, components and / or combinations thereof, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof. It should be understood that although the terms "first", "second", "third", etc. are used to describe various information, the information should not be limited by these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of the present disclosure, the first information may be referred to as the second information; and similarly, the second information may also be referred to as the first information. As used herein, the term "in response to..." may be understood to mean "when...", "at..." or "if," depending on the context.

[0045] In addition, the phrase "at least one of..." that appears in the present disclosure includes three types of parallel situations: "any one of...", "any combination of...", and "the whole of..." For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. For another example, "performing at least one of step 1 and step 2" includes the following three parallel situations: (1) performing step 1; (2) performing step 2; (3) performing step 1 and step 2.

[0046] Unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as those generally understood by ordinary technicians in the field to which the present disclosure belongs based on the understanding of the disclosure of this application, and should not be interpreted in an idealized or overly formal way. Here, the use of the term "may" with respect to examples or embodiments (for example, "... may include") indicates that there is at least one example or embodiment that includes or implements such features, and all examples are not limited to this. Unless otherwise clearly defined, terms in the singular also include plural forms.

[0047] First refer to Figure 1 and Figure 2 The data writing process and data restoring process of the related art are described. Figure 1 and Figure 2 is a diagram illustrating an example of a data processing method of the related art.

[0048] Reference Figure 1 , shows an example of data writing processing of an OSD daemon in erasure coding mode in the related art.

[0049] In operation S110, in response to a write demand for data ABCD about an object, a write request is initiated by the client to the primary OSD node corresponding to the object to request that the data about the object be written to the storage device OSD1 corresponding to the OSD node. In this case, in addition to OSD1 corresponding to the primary OSD node to which data will be written, there are storage devices corresponding to other OSD nodes to which data about the object will be written, such as OSD2 and OSD3.

[0050] In operation S120, data ABCD regarding an object to be written is moved from OSD1 to a memory such as a volatile memory.

[0051] In operation S130, data ABCD regarding an object to be written is moved from the memory to the CPU.

[0052] In operation S140, original data AB and original data CD in the form of a data block and check data XY in the form of a check block are calculated by the CPU based on data ABCD on an object to be written.

[0053] In operation S150, original data AB in the form of data blocks is moved from the CPU to the memory.

[0054] In operation S160, original data CD in the form of data blocks is moved from the CPU to the memory.

[0055] In operation S170, check data XY in the form of a check block is moved from the CPU to a memory.

[0056] In operation S180, original data AB and original data CD in the form of data blocks and check data XY in the form of check blocks are moved from the memory to OSD1.

[0057] In operation S190, OSD1 stores original data AB, and moves and stores original data CD and check data XY to OSD2 and OSD3, respectively.

[0058] Although only four data transfer operations are described in the above operations S130 to S170, in the calculation process of the CPU, the CPU must load data from the memory or temporarily store the calculation data to the memory through multiple read and / or write operations, which results in a large amount of data movement. In addition, more CPU calculation resources are occupied and a large amount of context switching is caused.

[0059] In erasure coding mode, when part of the original data stored in the storage device corresponding to the OSD node is missing, the original data can be restored by using the check data. Figure 2, shows an example of data restoration processing of an OSD daemon in erasure coding mode in the related art.

[0060] When the OSD daemon corresponding to OSD2 goes offline, the original data CD in OSD2 is lost. In this case, a data restoration request in erasure coding mode can be initiated.

[0061] In operation S210, original data AB about the object is moved from OSD1 to a memory such as a volatile memory in response to a restore request of data about the object.

[0062] In operation S220, the check data XY on the object for restoration is moved from OSD1 to the memory. Here, operation S220 also includes moving the check data XY on the object for restoration from OSD3 to OSD1.

[0063] In operation S230, original data AB about the object is moved from the memory to the CPU.

[0064] In operation S240, check data XY about the object is moved from the memory to the CPU.

[0065] In operation S250, new restored data CD in the form of a data block is calculated by the CPU based on the original data AB about the object and the check data XY about the object.

[0066] In operation S260, the restored data CD in the form of data blocks is moved from the CPU to the memory.

[0067] In operation S270, restored data CD in the form of data blocks is moved from the storage to the OSD1.

[0068] In operation S280, the restored data CD in the form of data blocks is moved from OSD1 and stored in the new storage device OSD4.

[0069] Similar to the data writing process, although only three data transfer operations are described in the above operations S230 to S260, in the CPU's calculation process, the CPU must load data from the memory or temporarily store the calculation data to the memory through multiple read and / or write operations, which results in a large amount of data movement. In addition, more CPU calculation resources are occupied and a large amount of context switching is caused.

[0070] In summary, in the related art, a large amount of data movement and context switching will lead to poor performance of write processing and recovery processing, and computing and processing a large amount of data in the CPU will lead to high CPU utilization and high energy consumption.

[0071] Therefore, in order to at least solve the above-mentioned various problems or defects, the present disclosure proposes a near memory processing (PNM) device and a host and a data processing method thereof, which can reduce the CPU's computing overhead and the data movement between the CPU and the memory by using the PNM device instead of the CPU and the memory to perform data processing. Figures 3 to 16 Embodiments according to the present disclosure are described in detail.

[0072] First refer to Figures 3 to 9 A data processing method according to an embodiment of the present disclosure is described. Figure 3 is a diagram illustrating an example of a related art and a data processing method according to an embodiment of the present disclosure. Figure 4 is a flowchart illustrating a data processing method according to an embodiment of the present disclosure. Figure 5 is a diagram illustrating an example of a data write process according to an embodiment of the present disclosure. Figure 6 FIG. 7 is a diagram illustrating an example of data restoration processing according to an embodiment of the present disclosure. FIG. 7 is a diagram illustrating an example of data of data processing according to an embodiment of the present disclosure.

[0073] Hereinafter, in order to distinguish between the modules of the host and the PNM device, the processor of the host may be referred to as a "main processor (such as a CPU)" or a "host processor", and the memory corresponding to the CPU on the host side may be referred to as a "main memory" or a "host memory", and accordingly, the processor included in the PNM device may be referred to as a "PNM processor", and the memory included in the PNM device may be referred to as a "PNM memory". It should be understood that the terms used here are only for the purpose of distinguishing different elements, and do not impose any limitations on the functions of the embodiments of the present disclosure, and any other suitable terms may be used to distinguish them.

[0074] Reference Figure 3 In the related art, as described above, there is a large amount of data transmission between the main processor and the main memory, while in the present disclosure, by using a PNM device including a PNM processor and a PNM memory to implement data processing, it can be ensured that the calculation processing and the data movement in the calculation processing occur inside the PNM device. Figures 4 to 9 The embodiments of the present disclosure are described in detail.

[0075] Reference Figure 4 , in operation S410, a data processing request regarding an object is received from a host.

[0076] According to an embodiment, the data processing request includes a data write request or a data restore request for an object initiated by an OSD daemon corresponding to the object in erasure coding mode. Figure 5 and Figure 6 An example of a data write process corresponding to a data write request and an example of a data restore process corresponding to a data restore request are described separately.

[0077] Reference Figure 5 In response to a write demand for data ABCD of an object, a client initiates a write request of the client to a primary OSD node corresponding to the object to request writing the data of the object to a storage device OSD1 corresponding to the OSD node. In this case, the data processing request includes a data write request. In operation S410, a PNM device (e.g., a PNM processor included in the PNM device) receives a data write request for data ABCD of an object from a host.

[0078] According to the embodiment, the data write request may be sent by calling the write_request function (or interface). For example, the declaration of the write_request function may be as shown in Table 1 below:

[0079] Table 1

[0080] function statement write_request int write_request(*obj,k,m,*data,*parity)

[0081] The description of each parameter in the above Table 1 can be shown in the following Table 2:

[0082] Table 2

[0083]

[0084] In Table 2, the address pointer of obj may include a location pointer where the data of the OSD daemon's object is stored in the main memory before being written, and the data pointers of data and parity may include a location pointer where the data of the OSD daemon's object will be written to a storage device (such as an SSD) corresponding to the OSD node.

[0085] For example, refer to Fig. 7A According to the embodiment, the parameters of the data write request include obj, data, parity, k and m, wherein the parameter value in data before the data write process is empty.

[0086] Reference Figure 6 , when the OSD daemon corresponding to OSD2 is offline, it is necessary to perform restoration processing on the data of the object. In this case, the data processing request includes a data restoration request. In operation S410, the PNM device (e.g., a PNM processor included in the PNM device) receives a data restoration request on the data ABCD of the object from the host (e.g., a computing agent included in the host).

[0087] According to an embodiment, a data restoration request may be sent by calling a restore_request function (or interface). For example, a declaration of the restore_request function may be as shown in Table 3 below:

[0088] Table 3

[0089] function statement restore_request int restore_request(*data,data_size,*parity,parity_size,k,m)

[0090] The description of each parameter in the above Table 3 can be shown in the following Table 4:

[0091] Table 4

[0092]

[0093] Similarly, in Table 4, the data pointers of data and parity may include a location pointer of a storage device (such as an SSD) corresponding to the OSD node where data of the object of the OSD daemon is written.

[0094] For example, refer to Figure 7B According to the embodiment, the parameters of the data restoration request include data, parity, data_size, parity_size, k and m, wherein some data in data is missing.

[0095] Return to reference Figure 4 In operation S420, based on the data processing request, data of the object to be processed is obtained from the host. For example, based on the address pointer (such as the above-mentioned obj, data and / or parity) and other parameters (such as k, m, data_size and / or parity_size) included in the data processing request, the data of the object to be processed is obtained from the host.

[0096] According to an embodiment, the step of obtaining the data of the object to be processed from the host based on the data processing request includes: based on the data processing request, the PNM memory receives and stores the data of the object to be processed from the host.

[0097] For example, refer to Figure 5 , when the data processing request includes a data writing request, the PNM memory receives initial data ABCD to be written about the object from the host. As another example, referring to Figure 6 When the data processing request includes a data restoration request, the PNM memory receives original data AB and check data XY for a restoration operation on the object from the host.

[0098] In operation S430, result data of the object is obtained by processing the data of the object to be processed.

[0099] According to an embodiment, the data processing may include a data writing process and / or a data restoring process. In addition, the data processing according to an embodiment of the present disclosure may also include any other data processing that can be implemented by a PNM device, and the present disclosure is not limited thereto.

[0100] According to an embodiment, the step of obtaining result data of the object by processing the data of the object to be processed includes: based on the data processing request being a data write request initiated by the OSD daemon process in the erasure coding mode, encoding the data of the object to be processed, and obtaining encoded data when the encoding is completed as the result data. That is, when the data processing request is a data write request, the PNM device performs encoding and obtains the encoded data.

[0101] For example, refer to Figure 5 In the erasure code mode, the PNM device performs encoding on the data ABCD about the object, and obtains the encoded data AB, CD and XY when the encoding is completed as the result data (such as, Figure 5 The data blocks and check blocks in Fig. 7A According to an embodiment, the encoding may include erasure code encoding. According to an embodiment, a calculation algorithm (such as an erasure code encoding and decoding algorithm) may be preset and / or built-in in the PNM device for calculation processing. According to an embodiment, the encoded data AB, CD and XY may include original data AB and original data CD in the form of data blocks and check data XY in the form of check blocks, but the data form of the present disclosure is not limited thereto.

[0102] According to an embodiment, intermediate data while encoding is being performed is written into and read from the PNM memory.

[0103] Figure 8 Detailed description of the invention is a flowchart showing an example of a data writing process according to an embodiment of the present disclosure. Figure 8 In operation S810, the PNM device receives a write request from the host, in operation S820, receives data to be encoded from the PNM memory, in operation S830, encodes the data to be encoded, and in operation S840, the processed data obtained in operation S830 may be sent to and stored in the PNM memory.

[0104] In operation S850, it is determined whether the encoding is completed. When the encoding is completed, the current encoded data may be determined as the result data, and subsequent processing may be performed. When the encoding is not completed, operations S820 to S840 may be repeatedly performed until the encoding is determined to be completed. The intermediate data obtained during the repeated execution of operations S820 to S840 are all sent to and / or stored in the PNM memory for subsequent use. Such direct data transmission between the PNM processor and the PNM memory for a large amount of processing data can greatly reduce the data transmission between the main processor and the main memory. Here, in order to avoid unnecessary details that obscure the present disclosure, a detailed description of the details of the encoding process and the processing of the encoding completion judgment is omitted.

[0105] According to an embodiment, the step of obtaining result data of an object by processing the data of the object to be processed includes: based on the data processing request being a data restoration request initiated by the OSD daemon process in the erasure code mode, by decoding the data of the object to be processed, obtaining decoded data when the decoding is completed as result data, that is, when the data processing request is a data restoration request, the PNM device performs decoding and obtains decoded data.

[0106] For example, refer to Figure 6 In the erasure code mode, the PNM device performs decoding on the original data AB and the check data XY about the object, and obtains the decoded data CD when the decoding is completed as the result data (such as, Figure 6 New data blocks in or Figure 7B According to an embodiment, the decoding may include erasure code decoding. According to an embodiment, the decoded data CD may include new restored data CD in the form of a data block, but the data form of the present disclosure is not limited thereto.

[0107] According to an embodiment, intermediate data while decoding is being performed is written into and read from the PNM memory.

[0108] Fig. 9 is a flowchart showing an example of a data restoration process according to an embodiment of the present disclosure. Fig. 9 In operation S910, the PNM device receives a restoration request from a host, receives data to be decoded from a PNM memory in operation S920, decodes the data to be decoded in operation S930, and transmits and stores the processed data obtained in operation S930 to the PNM memory in operation S940.

[0109] In operation S950, it is determined whether the decoding is completed. When the decoding is completed, the current decoded data may be determined as the result data, and subsequent processing may be performed. When the decoding is not completed, operations S920 to S940 may be repeatedly performed until it is determined that the decoding is completed. The intermediate data obtained during the repeated execution of operations S920 to S940 are all sent to and / or stored in the PNM memory for subsequent use. Such direct data transmission between the PNM processor and the PNM memory for a large amount of processing data can greatly reduce the data transmission between the main processor and the main memory. Here, in order to avoid unnecessary details that obscure the present disclosure, a detailed description of the details of the decoding process and the processing of the decoding completion judgment is omitted.

[0110] return Figure 4 , in operation S440, a message indicating that the data processing is completed is sent to the host.

[0111] According to an embodiment, when the data processing request is a data writing request, referring to Figure 8 In operation S860, a message indicating that the encoding is completed may be sent to the host, and when the data processing request is a data restoration request, refer to Fig. 9 , in operation S960, a message indicating that the decoding is completed may be sent to the host.

[0112] In operation S450, the result data of the object is transmitted to a storage device corresponding to the object.

[0113] The step of sending the result data of the object to the storage device corresponding to the object includes: in response to the completion of data processing, sending the result data of the object from the PNM memory to the storage device corresponding to the object.

[0114] According to an embodiment, the storage device corresponding to the object includes a storage device corresponding to the OSD daemon, for example, the storage device corresponding to the object may include OSD1 corresponding to the primary OSD node as described above. According to an embodiment, the storage device corresponding to the object may include various forms of non-volatile memory (such as SSD), but the present disclosure is not limited thereto. For example, the result data of the object may be sent from the PNM memory to the address of the storage device indicated by the address pointer of the data block and the check block.

[0115] According to an embodiment, when the data processing request is a data writing request, referring to Figure 5 , the data block and the check block can be sent to OSD1, and when the data processing request is a data restoration request, refer to Figure 6, a new data block may be sent to OSD1. When data is sent to OSD1, OSD1 may send or store the data to storage devices of other OSD nodes, such as OSD2 and OSD3. In order to avoid unnecessary details to obscure the present disclosure, a detailed description of the details of the distributed storage of OSD is omitted here.

[0116] According to the data processing method performed by PNM in the embodiment of the present disclosure, data processing and temporary storage can be performed inside the PNM device, avoiding a large amount of data transmission between the CPU on the host side and the main memory. In addition, by offloading the computational processing including encoding and decoding from the CPU on the host side to the PNM device, the CPU utilization rate can be reduced and computing resources can be saved. In addition, by using defined functions, the data writing and data restoration processing in the erasure code mode of Ceph is simplified, thereby improving the data processing performance of Ceph in the erasure code mode.

[0117] The following will refer to Figure 5 and Figure 6 , Figures 8 to 10 The data processing method performed by the host and the PNM device is described in detail. The details of the data processing method performed by the PNM device according to the embodiment of the present disclosure described above are all applicable to the data processing method performed by the host according to the embodiment of the present disclosure. Fig.10 is a flowchart of a data processing method according to an embodiment of the present disclosure.

[0118] In the present disclosure, a computing agent included in a host is proposed, which can communicate with the PNM device as described above. In addition, the computing agent can also be used to control operations related to the OSD.

[0119] Reference Fig.10 In operation S1010, the computing agent sends a data processing request for an object to the PNM device, wherein the data processing request is used to request the PNM device to process the data of the object to be processed. That is, the PNM device performs a reference Figure 4 Operation S410 is described.

[0120] According to an embodiment, a data processing request includes a data write request or a data restore request for an object initiated by a computing agent for an OSD daemon corresponding to the object in an erasure code mode. The details of the data processing request described above are applicable here. Here, although only one computing agent is shown according to an embodiment of the present disclosure, the computing agent may be multiple, and the computing agent may include an encoding agent and a decoding agent for respectively performing encoding processing corresponding to the data write request and decoding processing corresponding to the data restore request, but the present disclosure is not limited thereto.

[0121] For example, refer to Figure 5 In response to a write request for data ABCD of an object from a client, the computing agent may encapsulate various parameters as shown in Table 2 into a data write request or write_request function (or structure), and send the data write request to the PNM device by calling the write_request function (or structure), thereby starting the data write process. In this case, the computing agent may also be referred to as an encoding agent.

[0122] For example, refer to Figure 6 In response to a restore request for data ABCD of an object, the computing agent may encapsulate various parameters as shown in Table 4 into a data restore request or write them into a restore_request function (or structure), and send the data restore request to the PNM device by calling the restore_request function (or structure), thereby starting the data restore process. In this case, the computing agent may also be referred to as a decoding agent.

[0123] Return to reference Fig.10 In operation S1020, based on the PNM device receiving the data processing request, the PNM device is prompted to obtain the data of the object to be processed from the host. Figure 4 Operation S420 is described.

[0124] In operation S1030, the PNM device obtains result data of the object by processing the data of the object. Figure 4 Operation S430 is described.

[0125] In operation S1040, a message indicating that the data processing is completed is sent from the PNM device to the host. Figure 4 Operation S440 is described.

[0126] In operation S1050, in response to receiving a message indicating that data processing is completed from the PNM device by the computing agent, the computing agent stores the result data of the object from the PNM device to the storage device corresponding to the object, that is, causes the PNM device to send the result data to the storage device corresponding to the object. According to an embodiment, the storage device corresponding to the object includes a storage device corresponding to the OSD daemon, such as OSD1. That is, the PNM performs reference Figure 4 For example, referring to operation S450 Figure 8 In operation S860, the computing agent may receive a message from the PNM device indicating that the encoding is complete. Fig. 9 In operation S960, the computing agent may receive a message indicating that decoding is complete from the PNM device.

[0127] Reference Figure 5 After receiving the message indicating that the encoding is completed, the computing agent may control the data storage of OSD1. For example, the computing agent moves the original data AB, the original data CD, and the verification data XY obtained by encoding from the PNM memory to OSD1, and then stores the original data AB in OSD1, stores the original data CD in OSD2, and stores the verification data XY in OSD3.

[0128] Reference Figure 6 After receiving the message indicating that the decoding is complete, the computing agent may control the data storage of OSD1. For example, the computing agent moves the restored data CD obtained by decoding from the PNM storage to OSD1, and then stores the restored data CD to a new storage device OSD4. According to an embodiment, when a certain OSD daemon goes offline, the data in the corresponding storage device cannot be used, and the host will determine a new OSD daemon and a corresponding storage device (such as OSD4) for storing the restored data.

[0129] According to the data processing method performed by the host in the embodiment of the present disclosure, data processing and temporary storage can be performed by an external PNM device, avoiding a large amount of data transmission between the CPU on the host side and the main memory. In addition, by unloading the calculation processing including encoding and decoding from the CPU on the host side to the PNM device, the CPU utilization rate can be reduced and computing resources can be saved. In addition, by using the defined function, the data writing and data restoration processing in the erasure code mode of Ceph is simplified, thereby improving the data processing performance of Ceph in the erasure code mode.

[0130] Fig.11 is a block diagram illustrating a PNM device according to an embodiment of the present disclosure.

[0131] Reference Fig.11 , a PNM device 1111 includes a processor 1112 .

[0132] According to an embodiment, the processor 1112 is configured to: receive a data processing request regarding an object from a host; obtain data of the object to be processed from the host based on the data processing request; obtain result data of the object by processing the data of the object to be processed; send a message indicating completion of the data processing to the host; and send the result data of the object to a storage device corresponding to the object.

[0133] That is, the processor 1112 according to an embodiment of the present disclosure may execute or control other modules (or units) of the PNM device 1111 to execute the data processing method performed by the PNM device as described above, and therefore, the details of the above-mentioned data processing method may be applicable to the PNM device 1111 according to an embodiment of the present disclosure.

[0134] According to an embodiment, the PNM device may further include a memory 1113. The memory 1113 according to an embodiment of the present disclosure may correspond to the PNM memory described above, that is, the operation performed by the PNM memory in the data processing method performed by the PNM device described above may be applicable to the memory 1113.

[0135] According to an embodiment, the processor 1112 is configured to obtain data of the object to be processed from the host based on the data processing request by the following operations: based on the data processing request, causing the memory 1113 to receive and store the data of the object to be processed from the host.

[0136] According to an embodiment, the processor 1112 is configured to send the result data of the object to the storage device corresponding to the object by the following operations: in response to the data processing being completed, sending the result data of the object from the memory 1113 to the storage device corresponding to the object.

[0137] According to an embodiment, the data processing request includes a data write request or a data restore request for an object initiated by an object storage device OSD daemon corresponding to the object in an erasure coding mode.

[0138] According to an embodiment, the storage device corresponding to the object includes a storage device corresponding to the OSD daemon.

[0139] According to an embodiment, the processor 1112 is configured to obtain result data of the object by processing the data of the object to be processed through the following operations: based on the data processing request being a data write request initiated by the OSD daemon process in the erasure code mode, encoding the data of the object to be processed, and obtaining the encoded data when the encoding is completed as the result data; and based on the data processing request being a data restoration request initiated by the OSD daemon process in the erasure code mode, decoding the data of the object to be processed, and obtaining the decoded data when the decoding is completed as the result data,

[0140] According to an embodiment, intermediate data when encoding or decoding is performed is written into the memory 1113 and read from the memory 1113 .

[0141] Fig.12 is a block diagram illustrating a host according to an embodiment of the present disclosure.

[0142] Reference Fig.12, a host 1211 includes a computing agent 1212 and a processor 1213 for communicating with a PNM device. According to an embodiment, the processor 1113 is configured to: cause the computing agent 1212 to send a data processing request about an object to the PNM device, wherein the data processing request is used to request the PNM device to process the data of the object to be processed; in response to receiving a message indicating that the data processing is completed from the PNM device by the computing agent 1212, cause the computing agent 1212 to store the result data of the object from the PNM device to a storage device corresponding to the object. That is, the processor 1113 according to the embodiment is configured to cause the computing agent 1212 to execute the data processing method executed by the host as described above, and therefore, the details of the above-mentioned data processing method may be applicable to the host 1211 according to an embodiment of the present disclosure.

[0143] According to an embodiment, the data processing request includes a data write request or a data restore request for an object initiated by a compute agent 1212 for an OSD daemon corresponding to the object in an erasure coding mode, and wherein the storage device corresponding to the object includes a storage device corresponding to the OSD daemon.

[0144] In addition, it should be understood that the various modules or units in the PNM device 1111 and the host 1211 according to the exemplary embodiment of the present disclosure may be implemented as hardware components and / or software components. Those skilled in the art may implement various modules or units, for example, using a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), according to the processing performed by the defined various modules or units.

[0145] According to an embodiment of the present disclosure, a data processing system including a host storage system, a storage device, and a near memory processing (PNM) device is also provided. The host storage system may include at least one host processor and at least one host memory storing one or more host instructions. The PNM device may include at least one PNM processor and at least one PNM memory storing one or more PNM instructions.

[0146] According to an embodiment, the at least one host processor is configured to execute the one or more host instructions and cause the host storage system to perform the following operations: send a data processing request regarding an object to the PNM device.

[0147] According to an embodiment, the at least one PNM processor is configured to execute the one or more PNM instructions and cause the PNM device to perform the following operations: based on the data processing request, obtain the data of the object to be processed from the host storage system, obtain the result data of the object by processing the data of the object, and send a message indicating that the data processing is completed to the host storage system.

[0148] According to an embodiment, the at least one host processor is configured to execute the one or more host instructions and cause the host storage system to perform the following operations: in response to receiving a message indicating that data processing is completed from the PNM device, cause the PNM device to send the result data to a storage device, wherein the storage device corresponds to the object.

[0149] That is, the PNM device may be configured to execute the data processing method executed by the PNM device as described above and the host storage system may be configured to execute the data processing method executed by the host as described above, which will not be described repeatedly here.

[0150] According to an embodiment of the present disclosure, a system using a storage device may also be provided, comprising: a main processor; a memory; and a storage device, wherein the main processor is configured to control the storage device to execute the data processing method performed by the PNM device as described above, and / or the main processor is configured to execute the data processing method performed by the host as described above.

[0151] Fig.13 FIG. 1 is a schematic diagram of a system 1000 to which a storage device is applied according to an embodiment. Fig.13 The system 1000 may basically be a mobile system, such as a portable communication terminal (e.g., a mobile phone), a smart phone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IOT) device. However, Fig.13 The system 1000 is not necessarily limited to a mobile system, and may be a PC, a laptop, a server, a media player, or an automotive device (eg, a navigation device).

[0152] Reference Fig.13 , the system 1000 may include a main processor 1100, a memory (e.g., 1200a and 1200b), and a storage device (e.g., 1300a and 1300b). In addition, the system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply device 1470, and a connection interface 1480.

[0153] The main processor 1100 may control all operations of the system 1000, and more specifically, may control operations of other components included in the system 1000. The main processor 1100 may be implemented as a general-purpose processor, a dedicated processor, an application processor, etc. The main processor 1100 may be implemented as, for example, a main processor of a host as described above (such as, Figure 1 and 2 The CPU in Figure 3 The main processor in the memory device may be configured to control the storage device to execute the data processing method executed by the PNM device as described above, and / or may be configured to execute the data processing method executed by the host as described above.

[0154] The main processor 1100 may include at least one central processing unit (CPU) core 1110, and also include a controller 1120 for controlling memories 1200a and 1200b and / or storage devices 1300a and 1300b. In one or more embodiments, the main processor 1100 may further include an accelerator 1130, which is a dedicated circuit for high-speed data operations such as artificial intelligence (AI) data operations. The accelerator 1130 may include a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU), etc., and is implemented as a chip physically separated from other components of the main processor 1100.

[0155] The memories 1200a and 1200b may be used as main storage devices of the system 1000. Although the memories 1200a and 1200b may respectively include volatile memories, such as static random access memories (SRAM) and / or dynamic random access memories (DRAM), etc., the memories 1200a and 1200b may respectively include non-volatile memories, such as flash memory, phase change random access memories (PRAM) and / or resistive random access memories (RRAM), etc. The memories 1200a and 1200b may be implemented in the same package as the main processor 1100. The memories 1200a and 1200b may be implemented as, for example, the main memories of the host as described above (such as, Figure 1 and 2 Memory in Figure 3 main memory in .

[0156] The storage devices 1300a and 1300b may be used as non-volatile storage devices, which are configured to store data regardless of whether they are powered on, and have a larger storage capacity than the memories 1200a and 1200b. The storage devices 1300a and 1300b may include memory controllers (STRG CTRL) 1310a and 1310b and non-volatile memories (NVM) 1320a and 1320b, respectively, which are configured to store data via the control of the memory controllers 1310a and 1310b. Although the NVMs 1320a and 1320b may include V-NAND flash memories having a two-dimensional (2D) or three-dimensional (3D) structure, the NVMs 1320a and 1320b may include other types of NVMs, such as PRAM and / or RRAM, etc. The storage devices 1300a and 1300b may be implemented, for example, as storage devices corresponding to the OSD daemons as described above (such as, Figures 1 to 6 In addition or alternatively, the storage devices 1300a and 1300b may also be implemented as PNM devices as described above (such as, Figures 3 to 11 PNM device in the embodiment), and can be configured to execute the data processing method performed by the PNM device as described above.

[0157] The storage devices 1300a and 1300b may be physically separated from the main processor 1100 and included in the system 1000, or may be implemented in the same package as the main processor 1100. In addition, the storage devices 1300a and 1300b may have a type of solid-state devices (SSDs) or memory cards, and may be removably combined with other components of the system 100 through an interface such as a connection interface 1480 to be described later. The storage devices 1300a and 1300b may be devices to which a standard protocol such as Universal Flash Storage (UFS), Embedded Multimedia Card (eMMC), or NVMe is applied, but are not limited thereto.

[0158] The communication device 1440 may transmit and receive signals between other devices outside the system 1000 according to various communication protocols. The communication device 1440 may include an antenna, a transceiver, a modem, or the like.

[0159] According to an embodiment of the present disclosure, a host storage system is also provided, including: a storage device, and a host, wherein the storage device is configured to execute the data processing method performed by the PNM device as described above, and / or the host is configured to execute the data processing method performed by the host as described above.

[0160] Fig.14 is a block diagram of a host storage system 10 according to an exemplary embodiment.

[0161] The host storage system 10 may include a host 100 and a storage device 200. In addition, the storage device 200 may include a memory controller 210 and an NVM 220. According to an exemplary embodiment of the present invention, the host 100 may include a host controller 110 and a host memory 120. The host memory 120 may be used as a buffer memory configured to temporarily store data to be sent to or received from the storage device 200. According to an embodiment, the host 100 may be implemented as, for example, the host as described above, and may be configured to execute the data processing method performed by the host as described above. According to an embodiment, the storage device 200 may be implemented as, for example, a PNM device (such as, Figures 3 to 11 PNM device in the embodiment), and can be configured to execute the data processing method performed by the PNM device as described above.

[0162] The storage device 200 may include a storage medium configured to store data in response to a request from the host 100. As an example, the storage device 200 may include at least one of an SSD, an embedded memory, and a removable external memory. When the storage device 200 is an SSD, the storage device 200 may be a device that complies with the NVMe standard. When the storage device 200 is an embedded memory or an external memory, the storage device 200 may be a device that complies with the UFS standard or the eMMC standard. Both the host 100 and the storage device 200 may generate a packet and send the packet according to the adopted standard protocol.

[0163] When the NVM 220 of the storage device 200 includes a flash memory, the flash memory may include a 2D NAND memory array or a 3D (or vertical) NAND (VNAND) memory array. As another example, the storage device 200 may include various other types of NVM. For example, the storage device 200 may include a magnetic random access memory (MRAM), a spin transfer torque MRAM, a conductive bridge RAM (CBRAM), a ferroelectric RAM (FRAM), a PRAM, a RRAM, and various other types of memory. According to an embodiment, the NVM 220 may be implemented as, for example, a PNM memory as described above (such as, Figures 3 to 9 PNM memory in the .

[0164] According to an embodiment, the host controller 110 and the host memory 120 may be implemented as separate semiconductor chips. Alternatively, in one or more embodiments, the host controller 110 and the host memory 120 may be integrated in the same semiconductor chip. As an example, the host controller 110 may be any one of a plurality of modules included in an application processor (AP). The AP may be implemented as a system on chip (SoC). In addition, the host memory 120 may be an embedded memory included in the AP or a memory module external to the AP.

[0165] The host controller 110 may manage an operation of storing data (e.g., write data) of the buffer area of ​​the host memory 120 in the NVM 220 or an operation of storing data (e.g., read data) of the NVM 220 in the buffer area. According to an embodiment, the host controller 110 may be implemented as, for example, a computing agent of the host as described above (such as, Figure 5 and Figure 6 , Figures 8 to 10 ,as well as Fig.12 Calculation agent in ).

[0166] The memory controller 210 may include a host interface 211, a memory interface 212, and a CPU 213. In addition, the memory controller 210 may further include a flash translation layer (FTL), a packet manager 215, a buffer memory 216, an error correction code (ECC) engine 217, and an advanced encryption standard (AES) engine 218. The memory controller 210 may further include a working memory in which the FTL 214 is loaded. The CPU 213 may execute the FTL 214 to control data write and read operations on the NVM 220. According to an embodiment, the memory controller 210 may be implemented as, for example, a PNM processor as described above (such as, Figures 3 to 9 PNM processor in the .

[0167] The host interface 211 may transmit and receive packets to and from the host 100. The packets transmitted from the host 100 to the host interface 211 may include commands or data to be written to the NVM 220, etc. The packets transmitted from the host interface 211 to the host 100 may include responses to commands or data read from the NVM 220, etc. The memory interface 212 may transmit data to be written to the NVM 220 to the NVM 220 or receive data read from the NVM 220. The memory interface 212 may be configured to comply with a standard protocol such as Toggle or Open NAND Flash Interface (ONFI).

[0168] The FTL 214 may perform various functions, such as an address mapping operation, a wear leveling operation, and a garbage collection operation. The address mapping operation may be an operation of converting a logical address received from the host 100 into a physical address for actually storing data in the NVM 220. The wear leveling operation may be a technique for preventing excessive degradation of a specific block by allowing the blocks of the NVM 220 to be used evenly. As an example, the wear leveling operation may be implemented by using a firmware technique that balances the erase counts of physical blocks. The garbage collection operation may be a technique for ensuring available capacity in the NVM 220 by erasing an existing block after copying valid data of the existing block to a new block.

[0169] The packet manager 215 may generate a packet according to a protocol agreed to an interface of the host 100, or parse various types of information from a packet received from the host 100. In addition, the buffer memory 216 may temporarily store data to be written to or read from the NVM 220. Although the buffer memory 216 may be a component included in the memory controller 210, the buffer memory 216 may be outside the memory controller 210.

[0170] The ECC engine 217 may perform error detection and correction operations on read data read from the NVM 220. More specifically, the ECC engine 217 may generate parity bits for write data to be written to the NVM 220, and the generated parity bits may be stored together with the write data in the NVM 220. During reading of data from the NVM 220, the ECC engine 217 may correct errors in the read data by using the read data and the parity bits read from the NVM 220, and output the error-corrected read data.

[0171] The AES engine 218 may perform at least one of an encryption operation and a decryption operation on data input to the memory controller 210 by using a symmetric key algorithm.

[0172] According to an embodiment, there is further provided a storage system, including: a storage device; and a memory controller configured to control the storage device to execute the data processing method performed by the PNM device as described above.

[0173] Fig.15 FIG. 1 is a block diagram of a storage system 15 according to an embodiment of the present disclosure. Fig.15, the storage system 15 may include a storage device 17 and a memory controller 16. The storage system 15 may support a plurality of channels CH1 to CHm, and the storage device 17 may be connected to the memory controller 16 through the plurality of channels CH1 to CHm. For example, the storage system 15 may be implemented as a storage device such as an SSD. According to an embodiment, the storage system 15 may be implemented as, for example, a PNM device as described above (such as, Figures 3 to 11 PNM device in the embodiment), and can be configured to execute the data processing method performed by the PNM device as described above.

[0174] The storage device 17 may include a plurality of NVM devices NVM11 to NVMmn. Each of the NVM devices NVM11 to NVMmn may be connected to one of the plurality of channels CH1 to CHm through a path (way) corresponding thereto. For example, the NVM devices NVM11 to NVM1n may be connected to the first channel CH1 through paths W11 to W1n, and the NVM devices NVM21 to NVM2n may be connected to the second channel CH2 through paths W21 to W2n. In an exemplary embodiment, each of the NVM devices NVM11 to NVM1n may be implemented as an arbitrary storage element that may be operated according to a separate command from the memory controller 16. For example, each of the NVM devices NVM11 to NVM1n may be implemented as a chip or a die, but the present disclosure is not limited thereto. According to an embodiment, the storage device 17 may be implemented as, for example, a PNM memory (such as, Figures 3 to 9 PNM memory in the .

[0175] The memory controller 16 may send and receive signals to and from the memory device 17 through a plurality of channels CH1 to CHm. For example, the memory controller 16 may send commands CMDa to CMDm, addresses ADDRa to ADDRm, and data DATAa to DATAm to the memory device 17 through the channels CH1 to CHm, or receive data DATAa to DATAm from the memory device 17. According to an embodiment, the memory controller 16 may be implemented as, for example, a PNM processor as described above (such as, Figures 3 to 9 PNM processor in the device), and can be configured to execute the data processing method performed by the PNM device as described above.

[0176] The memory controller 16 may select one from among the NVM devices NVM11 to NVMmn connected to each of the channels CH1 to CHm by using a corresponding one of the channels CH1 to CHm, and transmit and receive signals to and from the selected NVM device. For example, the memory controller 16 may select the NVM device NVM11 from among the NVM devices NVM11 to NVM1n connected to the first channel CH1. The memory controller 16 may transmit a command CMDa, an address ADDRa, and data DATAa to the selected NVM device NVM11 through the first channel CH1, or receive data DATAa from the selected NVM device NVM11.

[0177] The memory controller 16 may transmit and receive signals to and from the memory device 17 in parallel through channels different from each other. For example, the memory controller 16 may transmit a command CMDa to the memory device 17 through the first channel CH1 while transmitting a command CMDb to the memory device 17 through the second channel CH2. For example, the memory controller 16 may receive data DATAa from the memory device 17 through the first channel CH1 while receiving data DATAb from the memory device 17 through the second channel CH2.

[0178] The memory controller 16 may control all operations of the memory device 17. The memory controller 16 may send signals to the channels CH1 to CHm and control each of the NVM devices NVM11 to NVMmn connected to the channels CH1 to CHm. For example, the memory controller 16 may send a command CMDa and an address ADDRa to the first channel CH1 and control one selected from the NVM devices NVM11 to NVM1n.

[0179] Each of the NVM devices NVM11 to NVMmn may operate via the control of the memory controller 16. For example, the NVM device NVM11 may program data DATAa based on a command CMDa, an address ADDRa, and data DATAa provided to the first channel CH1. For example, the NVM device NVM21 may read data DATAb based on a command CMDb and an address ADDb provided to the second channel CH2, and transmit the read data DATAb to the memory controller 16.

[0180] although Fig.15 An example is shown in which the storage device 17 communicates with the memory controller 16 through m channels and includes n NVM devices corresponding to each channel, but the number of channels and the number of NVM devices connected to one channel may be changed.

[0181] According to an embodiment, a universal flash memory (UFS) system is also provided, including: a UFS host; a UFS device; and a UFS interface, connecting the UFS host with the UFS device, wherein the UFS device is configured to execute the data processing method performed by the PNM device as described above, and / or the UFS host is configured to execute the data processing method performed by the host as described above.

[0182] Fig.16 FIG. 2 is a diagram of a UFS system 2000 according to an embodiment. The UFS system 2000 may be a system conforming to the UFS standard announced by the Joint Electronic Engineering Design Development Council (JEDEC), and includes a UFS host 2100, a UFS device 2200, and a UFS interface 2300. Fig.16 To the extent that the description conflicts, Fig.13 The above description of system 1000 may also be applied to Fig.16 UFS system 2000.

[0183] Reference Fig.16 , the UFS host 2100 may be connected to the UFS device 2200 through the UFS interface 2300. Fig.13 When the main processor 1100 is an application processor, the UFS host 2100 may be implemented as a part of the application processor. The UFS host controller 2110 and the host memory 2140 may correspond to Fig.13 The UFS device 2200 may correspond to the controller 1120 of the main processor 1100 and the memories 1200a and 1200b. Fig.13 The storage devices 1300a and 1300b, and the UFS device controller 2210 and the NVM 2220 may correspond to Fig.13 Memory controllers 1310a and 1310b and NVMs 1320a and 1320b. According to an embodiment, the UFS host 2100 may be implemented as, for example, the host as described above, and may be configured to control the UFS device 2200 to perform the data processing method performed by the PNM device as described above, and / or may be configured to perform the data processing method performed by the host as described above. According to an embodiment, the UFS device 2200 may be implemented as, for example, the PNM device as described above (such as, Figures 3 to 11 PNM device in the embodiment), and can be configured to execute the data processing method performed by the PNM device as described above.

[0184] The UFS host 2100 may include a UFS host controller 2110, an application 2120, a UFS driver 2130, a host memory 2140, and a UFS interconnect (UIC) layer 2150. The UFS device 2200 may include a UFS device controller 2210, an NVM 2220, a storage interface 2230, a device memory 2240, a UIC layer 2250, and a regulator 2260. The NVM 2220 may include a plurality of storage elements 2221. Although each storage element 2221 may include a V-NAND flash memory having a 2D structure or a 3D structure, each storage element 2221 may include another NVM, such as a PRAM and / or an RRAM, etc. The UFS device controller 2210 may be connected to the NVM 2220 through a storage interface 2230. The storage interface 2230 may be set to comply with a standard protocol such as Toggle or ONFI.

[0185] The application 2120 may refer to a program that wants to communicate with the UFS device 2200 to use the functions of the UFS device 2200. The application 2120 may send an input-output request (IOR) to the UFS driver 2130 for an input / output (I / O) operation on the UFS device 2200. The IOR may refer to a data read request, a data storage (or write) request, and / or a data erase (or discard) request, but is not limited thereto.

[0186] The UFS device controller 2210 of the UFS device 2200 may control all operations of the UFS device 2200. The UFS device controller 2210 may manage the NVM 2220 by using a logical unit (LU) 2211, which is a logical data storage unit. The number of LUs 2211 may be 8, but is not limited thereto. The UFS device controller 2210 may include an FTL, and convert a logical data address (e.g., a logical block address (LBA)) received from the UFS host 2100 into a physical data address (e.g., a physical block address (PBA)) by using address mapping information of the FTL. A logical block configured to store user data in the UFS system 2000 may have a size within a predetermined range. For example, the minimum size of a logical block may be set to 4Kbyte.

[0187] When a command from the UFS host 2100 is applied to the UFS device 2200 through the UIC layer 2250 , the UFS device controller 2210 may perform an operation in response to the command and transmit a completion response to the UFS host 2100 when the operation is completed.

[0188] As an example, when the UFS host 2100 intends to store user data in the UFS device 2200, the UFS host 2100 may send a data storage command to the UFS device 2200. When receiving a response (“Ready to Transfer” response) indicating that the UFS host 2100 is ready to receive user data (Ready to Transfer) from the UFS device 2200, the UFS host 2100 may send the user data to the UFS device 2200. The UFS device controller 2210 may temporarily store the received user data in the device memory 2240, and store the user data temporarily stored in the device memory 2240 in a selected location of the NVM 2220 based on the address mapping information of the FTL.

[0189] As another example, when the UFS host 2100 intends to read user data stored in the UFS device 2200, the UFS host 2100 may send a data read command to the UFS device 2200. The UFS device controller 2210, which has received the command, may read the user data from the NVM 2220 based on the data read command, and temporarily store the read user data in the device memory 2240. During the read operation, the UFS device controller 2210 may detect and correct errors in the read user data by using an ECC engine embedded therein. More specifically, the ECC engine may generate parity bits for write data to be written to the NVM 2220, and the generated parity bits may be stored in the NVM 2220 together with the write data. During the reading of data from the NVM 2220, the ECC engine may correct errors in the read data by using the read data and the parity bits read from the NVM 2220, and output the error-corrected read data.

[0190] In addition, the UFS device controller 2210 may transmit user data temporarily stored in the device memory 2240 to the UFS host 2100. Also, the UFS device controller 2210 may further include an AES engine. The AES engine may perform at least one of an encryption operation and a decryption operation on data transmitted to the UFS device controller 2210 by using a symmetric key algorithm.

[0191] The UFS host 2100 may sequentially store commands to be sent to the UFS device 2200 in the UFS host register 2111, which may be used as a common queue, and sequentially send the commands to the UFS device 2200. In this case, even when the previously sent command is still being processed by the UFS device 2200, that is, even before receiving a notification that the previously sent command has been processed by the UFS device 2200, the UFS host 2100 may send the next command on standby in the CQ to the UFS device 2200. Therefore, during the processing of the previously sent command, the UFS device 2200 may also receive the next command from the UFS host 2100. The maximum number of commands (or queue depth) that can be stored in the CQ may be, for example, 32. Also, the CQ may be implemented as a circular queue in which the start and end of the command line stored in the queue are indicated by a head pointer and a tail pointer.

[0192] Each of the plurality of storage elements 2221 may include a storage cell array and a control circuit configured to control the operation of the storage cell array. The storage cell array may include a 2D storage cell array or a 3D storage cell array. The storage cell array may include a plurality of storage cells. Although each storage cell is a single-layer cell (SLC) configured to store 1 bit of information, each storage cell may be a cell configured to store 2 or more bits of information, such as a multi-layer cell (MLC), a triple-layer cell (TLC), and a quad-layer cell (QLC). The 3D storage cell array may include a vertical NAND string in which at least one storage cell is vertically oriented and located on another storage cell.

[0193] According to an embodiment of the present disclosure, an electronic device is also provided, the electronic device comprising: at least one processor; at least one memory storing computer executable instructions. When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the data processing method as described above.

[0194] According to an embodiment of the present disclosure, the electronic device may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above-mentioned instruction (or instruction set) individually or jointly. The electronic device may also be part of an integrated control system or a system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0195] In electronic devices, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0196] The processor can execute instructions or codes stored in the memory, wherein the memory can also store data. Instructions and data can also be sent and received through the network via the network interface device, wherein the network interface device can adopt any known transmission protocol.

[0197] The memory may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor or the like. In addition, the memory may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the memory.

[0198] In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0199] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the data processing method as described above.

[0200] According to an exemplary embodiment of the present disclosure, examples of computer-readable storage media include read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in electronic devices such as clients, hosts, proxy devices, servers, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0201] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0202] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims.

Claims

1. A data processing method performed by a near memory processing (PNM) device, comprising: receiving a data processing request regarding an object from a host; Based on the data processing request, obtaining data of the object to be processed from the host; Obtaining result data of the object by processing the data of the object; sending a message to the host indicating completion of data processing; and The result data of the object is sent to a storage device corresponding to the object.

2. The data processing method according to claim 1, in, The PNM device comprises a memory, Wherein, based on the data processing request, the step of obtaining the data of the object to be processed from the host comprises: based on the data processing request, receiving the data of the object from the host by a memory, and storing the data of the object in the memory; and The step of sending the result data of the object to a storage device corresponding to the object includes: in response to completion of data processing, sending the result data of the object from a memory to the storage device.

3. The data processing method according to claim 1 or 2, wherein: The data processing request includes a data write request for the object or a data restore request for the object initiated by an object storage device OSD daemon corresponding to the object in an erasure coding mode, and Wherein, the storage device corresponding to the object includes a storage device corresponding to the OSD daemon process.

4. The data processing method according to claim 3, wherein: The step of obtaining result data of the object by processing the data of the object comprises: Based on the data processing request being a data write request initiated by the OSD daemon in erasure coding mode, obtaining the result data by encoding the data of the object, and Based on the fact that the data processing request is a data restoration request initiated by the OSD daemon in erasure coding mode, the result data is obtained by decoding the data of the object.

5. The data processing method according to claim 4, further comprising: Writing intermediate data generated during the encoding or decoding into a memory; as well as The intermediate data is read from the memory.

6. The data processing method according to claim 1, wherein: The PNM device includes a universal flash memory (UFS) device, the host includes a UFS host, and the PNM device and the host are connected via a UFS interface.

7. A data processing method performed by a host and a near memory processing (PNM) device, the data processing method comprising: Sending a data processing request about an object to the PNM device by a computing agent of the host; Based on the PNM device receiving the data processing request, causing the PNM device to obtain data of the object to be processed from the host; The PNM device obtains result data of the subject by processing the data of the subject; sending a message from the PNM device to the host indicating completion of data processing; as well as In response to receiving, by the computation agent, a message from the PNM device indicating completion of data processing, the PNM device is caused to send the result data to a storage device corresponding to the object.

8. The data processing method according to claim 7, in, The PNM device comprises a memory, The step of causing the PNM device to obtain the data of the object to be processed from the host based on the PNM device receiving the data processing request comprises: based on the PNM device receiving the data processing request, receiving the data of the object from the host by a memory, and storing the data of the object in the memory, and In response to the computing agent receiving a message indicating completion of data processing from the PNM device, the step of prompting the PNM device to send the result data to the storage device corresponding to the object includes: sending the result data from the storage to the computing agent.

9. The data processing method according to claim 7, wherein: The data processing request includes a data write request for the object or a data restore request for the object initiated by the compute agent for the object storage device OSD daemon corresponding to the object in erasure coding mode, and The storage device corresponding to the object includes a storage device corresponding to the OSD daemon process.

10. The data processing method according to claim 9, in, The step of obtaining result data of the object by processing the data of the object by the PNM device comprises: Based on the data processing request being a data write request initiated by the OSD daemon in erasure coding mode, obtaining the result data by encoding the data of the object, and Based on the fact that the data processing request is a data restoration request initiated by the OSD daemon in erasure coding mode, the result data is obtained by decoding the data of the object.

11. The data processing method according to claim 10, further comprising: Writing intermediate data generated during encoding or decoding to memory; as well as The intermediate data is read from the memory.

12. The data processing method according to claim 7, wherein: The PNM device includes a universal flash memory (UFS) device, the host includes a UFS host, and the PNM device and the host are connected via a UFS interface.

13. A data processing system comprising: Host storage system, including: at least one host processor; and at least one host memory storing one or more host instructions; storage device; and Near memory processing (PNM) device, comprising: at least one PNM processor; and at least one PNM memory storing one or more PNM instructions, The at least one host processor is configured to execute the one or more host instructions and cause the host storage system to perform the following operations: sending a data processing request regarding an object to the PNM device, The at least one PNM processor is configured to execute the one or more PNM instructions and cause the PNM device to perform the following operations: Based on the data processing request, obtaining data of the object to be processed from the host storage system, obtaining result data of the object by processing the data of the object, and sending a message to the host storage system indicating that data processing is complete, and The at least one host processor is configured to execute the one or more host instructions and cause the host storage system to perform the following operations: In response to receiving a message from the PNM device indicating that data processing is complete, the PNM device is caused to send the result data to a storage device, wherein the storage device corresponds to the object.

Images