Data processing method and related equipment

Abstract

The invention relates to a data processing method and related equipment, and the method comprises the steps: pulling all stripe data related to target data processing from a memory of a central processing unit of a host associated with near-memory computing equipment, and storing all stripe data in a cache module; and pulling the strip data one by one from the cache module through the near storage calculation module and executing target data processing. In this way, the cache module is arranged in the near-memory computing equipment, so that the near-memory computing module can directly pull the strip data from the cache module one by one and execute target data processing. As the distance between the near memory calculation module and the cache module is smaller than the distance between the central processing unit and the memory of the central processing unit, compared with a mode of reading data from the memory of the central processing unit through the central processing unit, the time consumed for reading the data from the cache module through the near memory calculation module is shorter, and the data reading efficiency is improved. And the overall data processing efficiency can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more particularly, to a data processing method and related devices. Background Art

[0002] Redundant Array of Independent Disks (RAID) stores the same data in different locations on multiple storage devices, such as different locations on Solid State Disks (SSDs), to protect the data when some storage devices fail. It can also improve overall storage performance by aggregating the capabilities of multiple storage devices.

[0003] In the related art, data is mainly stored in the memory of the central processing unit (CPU). Each time data is processed, it is necessary to read the data from the host's memory, that is, the CPU's memory, and then the RAID on the CPU performs operations on the read data. However, the path for the CPU to access the host's memory is long, that is, the distance between the CPU and the host's memory is relatively far. At this time, it takes a long time for the CPU to read data from the host's memory, resulting in low overall data processing efficiency. Summary of the invention

[0004] The present disclosure provides a data processing method and related devices to at least solve the problem in the above-mentioned related technologies that the overall data processing efficiency is low because it takes a long time for the CPU to read data from the memory of the host.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a data processing method, which is applied to a near memory computing device, wherein the near memory computing device comprises a near memory computing module and a cache module, and the data processing method comprises: pulling all stripe data related to target data processing from the memory of a central processing unit of a host associated with the near memory computing device, and storing all the stripe data in the cache module; pulling stripe data from the cache module one by one through the near memory computing module and executing the target data processing.

[0006] Optionally, in the case where the target data processing is independent disk redundant array intensive calculation, the data processing method according to the present disclosure is executed, wherein the independent disk redundant array intensive calculation is a calculation that requires a central processing unit computing resource size exceeding a preset resource size.

[0007] Optionally, the independent redundant array of disks intensive calculation includes at least one of data verification calculation, data cloning calculation and data modification calculation.

[0008] Optionally, in the case where the target data processing is the data verification calculation, executing the target data processing includes: performing data verification calculation on data blocks contained in the stripe data pulled one by one, and obtaining verification data blocks corresponding to the stripe data pulled one by one.

[0009] Optionally, the data processing method also includes: receiving, through the near memory computing module, an algorithm type for verification calculation sent by the central processing unit; performing data verification calculation on the data blocks contained in the stripe data pulled one by one, and obtaining verification data blocks corresponding to the stripe data pulled one by one, including: performing data verification calculation on the data blocks contained in the stripe data pulled one by one based on the algorithm type for verification calculation, and obtaining verification data blocks corresponding to the stripe data pulled one by one.

[0010] Optionally, in the case that the target data processing is the data clone calculation, executing the target data processing includes: performing data clone calculation on the target data blocks contained in the stripe data pulled one by one, and obtaining clone data blocks corresponding to the target data blocks.

[0011] Optionally, the data processing method also includes: determining whether there is expired stripe data in the cache module through the recent storage calculation module, wherein the expired stripe data is stripe data that has not been accessed within a preset first time period; when it is determined that the expired stripe data exists, deleting the expired stripe data from the cache module, and retaining the verification data block corresponding to the expired stripe data in the cache module.

[0012] Optionally, the cache module includes a first cache area and a second cache area, the first cache area is used to store stripe data and its verification data block, and the second cache area is used to store the verification data block of the deleted stripe data; wherein, deleting the expired stripe data from the cache module and retaining the verification data block corresponding to the expired stripe data in the cache module includes: deleting the expired stripe data from the first cache area, and moving the verification data block of the expired stripe data from the first cache area to the second cache area for storage.

[0013] Optionally, the data processing method also includes: determining whether there is an expired verification data block in the second cache area through the recent memory calculation module, wherein the expired verification data block is a verification data block that has not been accessed within a preset second time period; if it is determined that the expired verification data block exists, deleting the expired verification data block from the second cache area.

[0014] Optionally, the data processing method also includes: packaging all the stripe data and all data processing results thereof through the near storage computing module to obtain packaged data, and sending the packaged data to the central processing unit; writing the packaged data into the storage device of the host associated with the near storage computing device through the central processing unit.

[0015] According to a second aspect of an embodiment of the present disclosure, there is provided a data processing apparatus, which is applied to a near memory computing device, and the data processing apparatus comprises: a pulling module, configured to pull all stripe data related to target data processing from the memory of the central processing unit of a host associated with the near memory computing device, and store all the stripe data in a cache module; a data processing module, configured to pull the stripe data one by one from the cache module through the near memory computing module and perform the target data processing.

[0016] Optionally, in the case where the target data processing is independent disk redundant array intensive calculation, the pulling module and the data processing module work, wherein the independent disk redundant array intensive calculation is a calculation that requires a central processing unit computing resource size exceeding a preset resource size.

[0017] Optionally, the independent redundant array of disks intensive calculation includes at least one of data verification calculation, data cloning calculation and data modification calculation.

[0018] Optionally, in the case where the target data processing is the data verification calculation, the data processing module is configured to: perform data verification calculation on the data blocks contained in the stripe data pulled one by one, and obtain the verification data blocks corresponding to the stripe data pulled one by one.

[0019] Optionally, the data processing device also includes: a receiving module, configured to receive the algorithm type for verification calculation sent by the central processing unit through the near storage calculation module; the data processing module is configured to: perform data verification calculation on the data blocks contained in the stripe data pulled one by one based on the algorithm type for verification calculation, and obtain the verification data blocks corresponding to the stripe data pulled one by one.

[0020] Optionally, in the case where the target data processing is the data cloning calculation, the data processing module is configured to: perform data cloning calculation on the target data blocks contained in the stripe data pulled one by one, and obtain the cloned data blocks corresponding to the target data blocks.

[0021] Optionally, the data processing module is further configured to: determine whether there is expired stripe data in the cache module through the recent storage calculation module, wherein the expired stripe data is stripe data that has not been accessed within a preset first time period; if it is determined that the expired stripe data exists, delete the expired stripe data from the cache module, and retain the verification data block corresponding to the expired stripe data in the cache module.

[0022] Optionally, the cache module includes a first cache area and a second cache area, the first cache area is used to store stripe data and its verification data block, and the second cache area is used to store the verification data block of deleted stripe data; the data processing module is also configured to: delete the expired stripe data from the first cache area, and move the verification data block of the expired stripe data from the first cache area to the second cache area for storage.

[0023] Optionally, the data processing module is further configured to: determine whether there is an expired verification data block in the second cache area through the recent storage calculation module, wherein the expired verification data block is a verification data block that has not been accessed within a preset second time period; and if it is determined that the expired verification data block exists, delete the expired verification data block from the second cache area.

[0024] Optionally, the data processing device also includes: a packaging module, configured to package all the stripe data and all data processing results thereof through the near storage computing module, obtain packaged data, and send the packaged data to the central processing unit; a writing module, configured to write the packaged data into the storage device of the host associated with the near storage computing device through the central processing unit.

[0025] According to a third aspect of an embodiment of the present disclosure, a near memory computing device is provided, comprising: a near memory computing module; a cache module; wherein the cache module is used to store all stripe data related to target data processing pulled by the near memory computing module from the memory of the central processing unit of a host associated with the near memory computing device; the near memory computing module is used to pull stripe data from the cache module one by one and execute the target data processing.

[0026] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a data processing method according to the present disclosure.

[0027] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform a data processing method according to the present disclosure.

[0028] According to a sixth aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the data processing method according to the present disclosure is implemented.

[0029] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0030] In the present disclosure, by setting a cache module in a near-memory computing device, the near-memory computing module can directly pull stripe data from the cache module one by one and perform target data processing. Since the distance between the near-memory computing module and the cache module is smaller than the distance between the central processing unit and the memory of the central processing unit, compared with reading data from the host's memory through the central processing unit, the time taken to read data from the cache module through the near-memory computing module will be shorter, thereby improving the overall data processing efficiency. Furthermore, since the time-consuming and intensive data processing process is transferred from the central processing unit to the near-memory computing module, the occupancy rate of the central processing unit can be reduced to release the computing power of the central processing unit.

[0031] According to an exemplary embodiment of the present disclosure, when it is determined that a certain stripe data stored in the stripe cache is expired, the stripe data can be deleted from the stripe cache, thereby preventing invalid stripe data that has not been accessed for a long time from occupying too much of the stripe cache. Furthermore, after deleting the stripe data, the cache bitmap value corresponding to the deleted stripe data can be synchronously updated, so that in the subsequent query process, it is possible to directly determine whether the corresponding stripe data exists in the stripe cache based on the updated cache bitmap value, which can improve the query efficiency of the stripe data.

[0032] According to an exemplary embodiment of the present disclosure, compared with reading data from an SSD, it takes less time to read data from a cache module. Therefore, by moving the check data block corresponding to the deleted stripe data to the redundant cache, it is convenient to directly read the check data block from the redundant cache included in the cache module in subsequent data writing scenarios, even in subsequent data modification scenarios, thereby avoiding the problem of long reading time caused by directly reading the check data block from the SSD, and improving the reading efficiency of the check data block.

[0033] According to an exemplary embodiment of the present disclosure, compared with directly caching stripe data in the stripe cache, caching only the check data blocks corresponding to the stripe data in the redundant cache can greatly reduce the occupation of cache space; and, if a cache item in the redundant cache becomes cold, the cache item can be directly deleted, which can avoid the excessive occupation of the redundant cache by invalid check data blocks that have not been accessed for a long time. Furthermore, since the check data blocks are stored separately in the redundant cache, it is convenient to read the check data blocks directly from the redundant cache later, that is, the cache hit rate of the check data blocks that can be read directly from the redundant cache is improved, thereby avoiding the problem of long reading time caused by reading the check data blocks directly from the SSD, and the reading efficiency of the check data blocks can be improved. It can be seen that the redundant cache set in the cache module of the present disclosure preferably achieves a balance between cache space occupation and cache hit rate.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0036] Figure 1 It is a schematic diagram showing the structure of a redundant array of independent disks RAID in the related art;

[0037] Figure 2 is a schematic diagram showing the structure of CXL-PNM in the related art;

[0038] Figure 3 is a schematic diagram showing an implementation process of calculating redundant data blocks through RAID5 in the related art;

[0039] Figure 4 is a schematic diagram showing an implementation process of performing data mirroring calculation through RAID1 in the related art;

[0040] Figure 5 is a schematic diagram showing an implementation process of a write operation through RAID5 in the related art;

[0041] Figure 6 It is a schematic diagram showing various problems existing in data processing using RAID in the related art;

[0042] Figure 7 is a flowchart illustrating a data processing method according to an exemplary embodiment of the present disclosure;

[0043] Figure 8is a schematic structural diagram showing a host in the related art and a data processing device according to an exemplary embodiment of the present disclosure;

[0044] Fig. 9 is a schematic diagram showing three agents included in a CPU and three computing cores included in a near-memory computing module according to an exemplary embodiment of the present disclosure;

[0045] Fig.10 is a schematic diagram showing the structure of a computational redundancy core according to an exemplary embodiment of the present disclosure;

[0046] Fig.11 is a schematic diagram showing the structure of a mirror core according to an exemplary embodiment of the present disclosure;

[0047] Fig.12 is a schematic diagram showing the structure of a cache management core according to an exemplary embodiment of the present disclosure;

[0048] Fig.13 is a schematic diagram illustrating a read IO / write IO hit / miss cache according to an exemplary embodiment of the present disclosure;

[0049] Fig.14 is a flowchart showing a specific implementation of a redundancy calculation process according to an exemplary embodiment of the present disclosure;

[0050] Fig.15 is a flowchart showing a specific implementation of a data mirroring calculation process according to an exemplary embodiment of the present disclosure;

[0051] Fig.16 is a flowchart showing a specific implementation of a cache management process according to an exemplary embodiment of the present disclosure;

[0052] Fig.17 is a schematic diagram illustrating a near storage computing device according to an exemplary embodiment of the present disclosure;

[0053] Fig.18 is a block diagram illustrating a data processing apparatus according to an exemplary embodiment of the present disclosure;

[0054] Fig.19 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0057] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0058] RAID can divide the device storage space into multiple blocks, and the blocks in the same storage location of different devices can form a stripe. Figure 1 1 is a schematic diagram showing the structure of a redundant array of independent disks RAID in the related art. Figure 1 , Redundant Array of Independent Disks RAID can include 4 solid-state drives SSD, namely SSD1, SSD2, SSD3, and SSD4. D0, D1, D2, D3, D4, D5, D6, D7, D8, D9, Da, Db, Dc, Dd, De, and Df are data blocks. Among them, the 8 data blocks D0, D1, D2, D3, D4, D5, D6, and D7 form data stripe strip1; the 8 data blocks D8, D9, Da, Db, Dc, Dd, De, and Df form data stripe strip2.

[0059] Striping technology can distribute user data to multiple storage devices SSD of RAID, and can use the aggregated bandwidth of multiple storage devices for parallel storage; mirroring technology can copy user data to multiple storage devices to ensure data security. RAID technology is widely used in data centers. For example, server operating systems can run on RAID1, data can be established on RAID10, RAID5 / RAID6 can build large-scale high-reliability storage systems, and so on.

[0060] Near Memory Processing (PNM) is a technology that integrates logic chips into advanced integrated circuit packages, which can use memory for data calculation to reduce data movement between CPU and memory. Furthermore, the open interconnect standard (Compute Express Link, CXL) can be used with PNM to facilitate the expansion of memory capacity. In actual tests, PNM solutions based on the CXL interface have been proven to improve application performance by more than two times.

[0061] Figure 2 Schematic diagram showing the structure of CXL-PNM in the related art. Figure 2 The CXL controller (CXL CTRL) may include a PNM and a memory controller (MC) of the PNM. There is data interaction between the CXL controller and the storage device (Device Memory), and there is also data interaction between the CXL controller and the CPU. The components associated with the CPU mainly include the host memory controller MC and the double data rate synchronous dynamic random access memory (Double Data Rate, DDR).

[0062] It should be noted that RAID is a technology closely related to computing and storage. Against the backdrop of explosive data growth, the improvement in CPU computing power has gradually slowed down. With the rapid development of high-performance, large-capacity storage devices, such as SSDs that comply with the Non-Volatile Memory express (NVMe) standard, existing RAID technology is struggling to keep up with the demand, which is mainly reflected in high CPU occupancy and high input / output (IO) latency.

[0063] The implementation process of “data verification calculation”, “data mirror calculation” and “data modification calculation” and their respective corresponding problems will be specifically introduced below with reference to the accompanying drawings.

[0064] Data verification calculation

[0065] During the striping process, the RAID5 / 6 algorithm can perform data verification calculations, generate redundant data blocks of a specific size for each stripe, and fill the redundant data blocks into corresponding positions to recover lost user data.

[0066] For example, RAID5 can generate parity using an exclusive OR algorithm (XOR). Specifically, in each stripe, all data blocks can be XORed with each other to generate redundant data blocks of the same size, which is a process that consumes a lot of computing resources. Figure 31 is a schematic diagram showing the implementation process of calculating redundant data blocks by RAID5 in the related art. Figure 3 , Figure 3 The left figure shows the structure of RAID5, which contains 4 SSDs, namely SSD1, SSD2, SSD3, and SSD4. Among them, the redundant data block obtained by performing XOR operation on data blocks A1, A2, and A3 is Ap; the redundant data block obtained by performing XOR operation on data blocks B1, B2, and B4 is Bp; the redundant data block obtained by performing XOR operation on data blocks C1, C3, and C4 is Cp. Figure 3 In the figure on the right, for a 12KB user data (User Data), 512 XOR operations (Calculate XOR) and 4096 memory accesses (Access mem) are required.

[0067] For another example, RAID6 can generate two redundant blocks. First, it can generate the first redundant block like RAID5; second, it can multiply each original data block by a coefficient and then perform an XOR operation to generate another redundant data block. It can be seen that the data verification calculation amount of RAID6 has more than doubled compared to RAID5.

[0068] Data mirroring calculation

[0069] "Data mirroring" can also be called "data cloning". Cloning large segments of data is a very time-consuming process. In the RAID1 / 10 algorithm, the mirroring algorithm uses cloning technology to traverse each stripe, generate a copy for it, and then store the original data and the cloned copy on different storage devices. RAID1 can be composed of two storage devices, one storage device is used to store raw data (raw data), and the other storage device is used to store mirror data (mirror data).

[0070] Figure 4 1 is a schematic diagram showing the implementation process of data mirroring calculation by RAID1 in the related art. Figure 4 , Figure 4The left figure shows the structure of RAID1. RAID1 contains two SSDs, namely SSD1 and SSD2. Among them, data cloning calculation can be performed on data block A1 stored on SSD1 to obtain copy A1, and the copy A1 can be stored on SSD2; data cloning calculation can be performed on data block B1 stored on SSD1 to obtain copy B1, and the copy B1 can be stored on SSD2; data cloning calculation can be performed on data block C1 stored on SSD1 to obtain copy C1, and the copy C1 can be stored on SSD2. Figure 4 In the figure on the right, for a 4MB user data segment, 1024 data clone calculations (Clone) and 2048 memory accesses are required. RAID10 takes into account both data protection and write performance, but like RAID1, it also requires multiple clones.

[0071] Data modification calculation

[0072] It should be noted that in the scenario of writing data, that is, in the scenario of modifying data, there is still a problem of "random write amplification" in the related technology. Specifically, as mentioned above, RAID is an array composed of multiple storage devices, such as SSDs, which performs read and write operations in stripes. Therefore, small-size random write IOs need to simultaneously update the redundant data located in another storage device in the stripe. For example, in RAID5, if the length of the written data is smaller than the stripe block, that is, the size of the data block, two additional read IOs and one additional write IO are required.

[0073] Figure 5 Schematic diagram showing the implementation process of writing operation through RAID5 in the related art. Figure 5 RAID5 contains 3 SSDs, namely SSD1, SSD2, and SSD3. The data block D1 stored on SSD1 and the data block D2 stored on SSD3 are XORed to obtain the redundant data block P1; the data block D4 stored on SSD1 and the data block D5 stored on SSD2 are XORed to obtain the redundant data block P2.

[0074] If the user wants to modify the data block D2, five steps need to be performed: (1) Read the original redundant data block P1 from SSD2; (2) Read the original data block D2 from SSD3, and modify the read original data block D2 to obtain the modified data block D2′; (3) Based on the read original redundant data block P1 and the modified data block D2′, calculate the new redundant data block P1′: P1 XOR D2′=P1′; (4) Write the modified data block D2′ to SSD3, that is, use the modified data block D2′ to overwrite the original data block D2 previously stored on SSD3; (5) Write the new redundant data block P1′ to SSD2, that is, use the new redundant data block P1′ to overwrite the original redundant data block P1 previously stored on SSD2.

[0075] In the data modification calculation process of the above modified data block, the original data block and the original redundant data block need to be read out first, and then the modified data block and the new redundant data block need to be written back to the SSD, which generates a lot of additional read IO and write IO. For RAID6, more IO will be generated in the process of modifying the data block.

[0076] It can be seen that in the related art, when processing data, the main problems are time-consuming calculation, high latency, and write amplification. In addition, the number of PCIe slots on the host is limited. In the related art, if you want to implement the "expanded memory" function, you need to insert a card separately on the host; and if you want to implement the "data verification calculation, data cloning calculation, data modification calculation and cache management" functions, you also need to insert a RAID card separately on the host. At this time, two independent card slots are required to insert the above two cards. Therefore, in the related art, more card slots on the host need to be occupied.

[0077] Figure 6 Schematic diagram showing various problems existing in data processing using RAID in the related art. Figure 6 In the related art, when using RAID for data processing, there are mainly problems such as time-consuming calculation, high latency and limited resources.

[0078] In order to solve the above-mentioned problems existing in the related art, the data processing method and related devices provided by the present disclosure, by setting a cache module in the near-memory computing device, enable the near-memory computing module to directly pull stripe data from the cache module one by one and perform target data processing. Since the distance between the near-memory computing module and the cache module is smaller than the distance between the central processing unit and the memory of the central processing unit, compared with the method of reading data from the host's memory through the central processing unit, the time taken to read data from the cache module through the near-memory computing module will be shorter, thereby improving the overall data processing efficiency. Furthermore, since the time-consuming and intensive data processing process is transferred from the central processing unit to the near-memory computing module, the occupancy rate of the central processing unit can be reduced to release the computing power of the central processing unit.

[0079] Figure 7 1 is a flow chart showing a data processing method according to an exemplary embodiment of the present disclosure. The data processing method can be applied to a near memory computing device (CXL-PNM Device), which may include a near memory computing module (PNM) and a cache module (Memory).

[0080] Reference Figure 7 In step 701, all stripe data related to target data processing can be pulled from the memory of the central processing unit of the host associated with the near storage computing device, and all stripe data can be stored in the cache module.

[0081] It should be noted that the user can hand over the data to be processed to the host, and then the CPU on the host can perform striping processing on the data to be processed given by the user, that is, the data to be processed given by the user can be divided into multiple small data blocks to obtain stripe data. Then, the CPU can write the stripe data obtained by the division into the host's memory. Next, the CPU can send the storage address of the stripe data in the host's memory to the near memory computing module. Then, the near memory computing module can pull the stripe data from the host's memory to the cache module based on the storage address sent by the CPU. It should be noted that the "host memory" here can also be referred to as "CPU memory."

[0082] According to an exemplary embodiment of the present disclosure, the above data processing method can be performed when the above-mentioned "target data processing" is an independent disk redundant array intensive calculation. The "independent disk redundant array intensive calculation" can be a calculation that requires a central processing unit computing resource size exceeding a preset resource size.

[0083] According to an exemplary embodiment of the present disclosure, the above-mentioned "independent disk redundant array intensive calculation" may include at least one of "data verification calculation", "data cloning calculation" and "data modification calculation". Among them, "data verification calculation" may refer to the calculation of performing XOR operation on the data blocks included in the stripe data to obtain the verification data block; "data cloning calculation" may refer to the calculation of copying the data blocks included in the stripe data to obtain the mirror copy; "data modification calculation" may refer to the calculation of modifying the data blocks included in the stripe data to obtain the new data block. In addition, "data verification calculation" may also be called "redundancy calculation"; "data cloning calculation" is the "data mirror calculation" mentioned above.

[0084] Figure 8 1 is a schematic diagram showing the structure of a host in the related art and a data processing device according to an exemplary embodiment of the present disclosure. Figure 8 , Figure 8 The left figure shows the structure of the host in the related art, and the host in the related art may include a CPU and the host's memory. In addition, the RAID logic can be implemented on the CPU, that is, the related art mainly uses the CPU for "data verification calculation", "data cloning calculation", "data modification calculation" and "cache management", which will lead to an excessively high CPU occupancy rate.

[0085] Figure 8 The figure on the right side of shows the structure of a data processing device according to an exemplary embodiment of the present disclosure. In the present disclosure, the data processing device may include a CPU and a near memory computing device, and the near memory computing device may include a near memory computing module and a cache module. Among them, the near memory computing module is mainly used to implement RAID logic, that is, in the present disclosure, the time-consuming intensive computing process is unloaded from the CPU to the near memory computing module to reduce the CPU occupancy rate, thereby releasing the CPU computing power.

[0086] Furthermore, in the present disclosure, the CPU may include three proxies, namely, a redundant computing proxy "CalcParity proxy", a mirror proxy "Mirror proxy" and a cache proxy "Mgt Cache proxy". Moreover, the near memory computing module may include three computing cores, namely, a redundant computing core "Calc Parity core", a mirror core "Mirrorcore" and a cache management core "Mgt Cache core". Fig. 9 is a schematic diagram showing three agents included in a CPU and three computing cores included in a near-memory computing module according to an exemplary embodiment of the present disclosure.

[0087] Among them, the above three agents in the CPU are mainly responsible for communicating with the computing core on the PNM, starting calculations, generating and transmitting parameters, collecting calculation results and subsequent logical processing. For the "redundant computing agent": it is responsible for processing the verification code generation, validity verification, and data recovery functions of the stripe data according to the status of the RAID stripe, and generating and sending calculation requests to the computing redundancy core; for the "mirror agent": it is responsible for generating and sending mirror data requests to the mirror core according to the status of the RAID stripe to complete the generation of cloned stripes; for the "cache agent": it is responsible for maintaining cache validity and write cache block solidification according to the cache policy, and requests the cache management core to regularly clean up invalid caches and encapsulate write cache data into larger data blocks.

[0088] It should be noted that each of the above computing cores can be composed of a control unit (ctrl) and multiple computing units. Among them, the control unit (ctrl) is mainly responsible for interacting with the CPU-side agent, executing the calculation process, accessing the device memory, and calling the computing unit. In addition, each computing core can have a built-in single algorithm.

[0089] Fig.10 is a schematic diagram showing the structure of a computing redundancy core according to an exemplary embodiment of the present disclosure. Fig.10 , the computational redundancy core can be composed of a control unit (ctrl) and a computational unit. Among them, the "control unit (ctrl)" is mainly used to traverse the stripe and execute the corresponding redundancy algorithm according to the RAID level; the computational unit may include a "redundancy unit Parity" and a "recovery unit Recover". The "redundancy unit Parity" is internally provided with a stripe redundancy algorithm, which is mainly used to perform redundancy calculations on the data blocks contained in the stripe data; the "recovery unit Recover" is mainly used to recover the lost data blocks. Fig.10 , a data stripe Strip0 is shown, and the data stripe Strip0 may include two data blocks, namely D1 and D2. Moreover, the "redundancy unit Parity" may perform redundancy calculation, namely data verification calculation, on the two data blocks, and then obtain the verification data block P1 corresponding to the data stripe Strip0.

[0090] Fig.11 is a schematic diagram showing the structure of a mirror core according to an exemplary embodiment of the present disclosure. Fig.11 , the mirror core can be composed of a control unit (ctrl) and a computing unit. Among them, the "control unit (ctrl)" is mainly used to execute the stripe block cloning task in batches; the computing unit may include a "clone unit Clone", and the "clone unit Clone" is mainly used to implement the stripe cloning algorithm. Fig.11In FIG. 1 , a data strip Strip1 is shown, and the data strip Strip1 may include a data block D3. Furthermore, the “clone unit Clone” may perform data mirroring calculation, namely data cloning calculation, on the data block D3, and then obtain cloned data blocks D3′ and D3″ corresponding to the data block D3.

[0091] Fig.12 is a schematic diagram showing the structure of a cache management core according to an exemplary embodiment of the present disclosure. Fig.12 The cache management core can be composed of a control unit (ctrl) and a computing unit. The "control unit (ctrl)" is mainly used to periodically check the stripes, and evict and merge the cache; the computing unit may include an "eviction unit (Evict)" and a "packaging unit Pack". The "eviction unit (Evict)" has a cache strategy set inside to maintain cache validity; the "packaging unit Pack" is mainly used to organize the stripe data according to the strategy, and then pack it into data blocks suitable for storage device SSD reading and writing.

[0092] In step 702, the near storage computing module may pull stripe data from the cache module one by one and perform target data processing. Exemplarily, the near storage computing module may pull stripe data from the cache module one by one and perform "data verification calculation", "data cloning calculation", "data modification calculation" or "cache management" for each stripe data pulled out.

[0093] According to an exemplary embodiment of the present disclosure, when the target data processing is data verification calculation, the stripe data can be pulled from the cache module one by one through the near memory computing module and data verification calculation can be performed on the data blocks contained in the stripe data pulled one by one to obtain the verification data blocks corresponding to the stripe data pulled one by one.

[0094] Exemplarily, the near memory computing module can read all data blocks contained in the stripe data, assuming that a certain stripe data contains two data blocks in total, namely data block D1 and data block D2. The near memory computing module can perform data verification calculations on the data blocks D1 and data blocks D2 contained in the read stripe data. For example, an XOR operation can be performed to obtain the verification data block P1: D1 XOR D2 = P1. Next, the near memory computing module can send the verification data block P1 to the cache module, and the cache module can associate the received verification data block P1 with the stripe data read this time for storage.

[0095] Furthermore, the near memory computing module can also send the obtained verification data block to the CPU, and then the CPU can write the received verification data block and the stripe data corresponding to the verification data block into the SSD together, that is, the CPU can write the received verification data block and the original data block used to obtain the verification data block into the SSD together.

[0096] It should be noted that, while sending the check data block to the CPU, the near memory computing module can also send the corresponding stripe data to the CPU, and then the CPU writes the received check data block and the corresponding stripe data together to the SSD; or, the near memory computing module can also only send the check data block to the CPU, and the CPU itself obtains the corresponding stripe data from the host's memory, and then the CPU writes the received check data block and the corresponding stripe data together to the SSD.

[0097] According to an exemplary embodiment of the present disclosure, the algorithm type for verification calculation sent by the central processing unit can also be received through the near storage calculation module. Among them, the "algorithm type for verification calculation" can be the "algorithm type of independent disk redundant array". Exemplarily, the "algorithm type of independent disk redundant array" can be but not limited to: RAID0, RAID1, RAID5, RAID6, RAID10, etc., and the redundancy algorithms corresponding to the algorithm types of different independent disk redundant arrays may be different. For example, for RAID5, it only needs to calculate one redundant data block, namely the verification data block P Parity; while for RAID6, it needs to calculate two redundant data blocks P Parity and Q Parity.

[0098] Next, the stripe data can be pulled from the cache module one by one through the near memory computing module, and based on the verification calculation algorithm type sent by the CPU, that is, the algorithm type of the independent disk redundant array, data verification calculation can be performed on the data blocks contained in the stripe data pulled one by one to obtain the verification data blocks corresponding to the stripe data pulled one by one.

[0099] According to an exemplary embodiment of the present disclosure, when the target data processing is data cloning calculation, the near storage calculation module can pull stripe data from the cache module one by one and perform data cloning calculation on the target data blocks contained in the stripe data pulled one by one to obtain the clone data blocks corresponding to the target data blocks, that is, to obtain the replica data blocks corresponding to the target data blocks. In addition, the near storage calculation module can also send the clone data blocks to the cache module, and then the cache module can associate the received clone data blocks with the stripe data read this time for storage.

[0100] Furthermore, the near memory computing module can also send the obtained cloned data block to the CPU, and then the CPU can write the received cloned data block and the stripe data corresponding to the cloned data block into the SSD together, that is, the CPU can write the received cloned data block and the original data block used to obtain the cloned data block into the SSD together.

[0101] It should be noted that, when the near memory computing module sends the cloned data block to the CPU, it can also send the corresponding stripe data to the CPU, and then the CPU writes the received cloned data block and the corresponding stripe data together to the SSD; or, the near memory computing module can also only send the cloned data block to the CPU, and the CPU obtains the corresponding stripe data from the host's memory, and then the CPU writes the received cloned data block and the corresponding stripe data together to the SSD.

[0102] According to an exemplary embodiment of the present disclosure, it can be determined by a near memory calculation module whether there is expired stripe data in the cache module. The expired stripe data may be stripe data that has not been accessed within a preset first time period. If it is determined that there is expired stripe data, the expired stripe data may be deleted from the cache module, and the verification data block corresponding to the expired stripe data may be retained in the cache module.

[0103] According to an exemplary embodiment of the present disclosure, the cache bitmap value of the deleted stripe data may be modified from a first value to a second value. Exemplarily, the first value may be 1; the second value may be 0. The first value may be used to indicate that the stripe data exists in the stripe cache, and the second value may be used to indicate that the stripe data does not exist in the stripe cache.

[0104] In this way, when it is determined that a certain stripe data stored in the cache module is expired, the stripe data can be deleted from the cache module, thereby preventing the invalid stripe data that has not been accessed for a long time from occupying too much of the cache module. Furthermore, after deleting the stripe data, the cache bitmap value corresponding to the deleted stripe data can be synchronously updated, so that in the subsequent query process, it is possible to directly determine whether the corresponding stripe data exists in the stripe cache based on the updated cache bitmap value, which can improve the query efficiency of the stripe data.

[0105] According to an exemplary embodiment of the present disclosure, referring back to Figure 8The cache module may include a first cache area and a second cache area. The first cache area may be used to store stripe data and its check data block, and the second cache area may be used to store the check data block of the deleted stripe data. Moreover, the "first cache area" may also be referred to as a stripe cache (StripCache), and the "second cache area" may also be referred to as a redundant cache (Parity Cache).

[0106] Expired stripe data can be deleted from the first cache area, and the check data blocks of the expired stripe data can be moved from the first cache area to the second cache area for storage. That is, the check data blocks corresponding to the deleted stripe data stored in the stripe cache (StripCache) can be moved to the redundant cache through the near storage calculation module. That is, in the stripe cache (StripCache), if the cache content heat decreases, only the redundant data blocks of the cache content can be retained and the redundant data blocks of the cache content can be expelled to the redundant cache contained in the cache module.

[0107] It should be noted that compared with reading data from the SSD, reading data from the cache module takes less time. Therefore, by moving the check data block corresponding to the deleted stripe data to the redundant cache, it is convenient to directly read the check data block from the redundant cache contained in the cache module in subsequent data writing scenarios, even in subsequent data modification scenarios, thereby avoiding the problem of long reading time caused by directly reading the check data block from the SSD, and improving the reading efficiency of the check data block.

[0108] According to an exemplary embodiment of the present disclosure, it is also possible to determine whether there is an expired verification data block in the second cache area through a near storage calculation module. The expired verification data block may be a verification data block that has not been accessed within a preset second time period. In the case where it is determined that there is an expired verification data block, the expired verification data block may be deleted from the second cache area. That is, in a redundant cache (Parity Cache), if a cache item stored therein becomes cold, the cache item may be directly deleted.

[0109] It should be noted that in the present disclosure, relatively hot stripe data can be stored in the stripe cache; for stripe data that is not so hot, that is, relatively "warm", only the check data blocks corresponding to the relatively "warm" stripe data can be stored in the redundant cache (Parity Cache); and for those relatively cold stripe data, they can be directly stored in the SSD.

[0110] In this way, compared with the method of directly caching stripe data in the stripe cache, only caching the check data blocks corresponding to the stripe data in the redundant cache can greatly reduce the occupation of cache space; and, if a cache item in the redundant cache becomes cold, the cache item can be directly deleted, which can avoid the excessive occupation of the redundant cache by invalid check data blocks that have not been accessed for a long time. Furthermore, since the check data blocks are stored separately in the redundant cache, it is convenient to read the check data blocks directly from the redundant cache later, that is, the cache hit rate of the check data blocks that can be read directly from the redundant cache is improved, thereby avoiding the problem of long reading time caused by reading the check data blocks directly from the SSD, and the reading efficiency of the check data blocks can be improved. It can be seen that the redundant cache set in the cache module of the present disclosure preferably achieves a balance between cache space occupation and cache hit rate.

[0111] As mentioned above, the modification scenario corresponding to “data modification calculation” can also be called a data writing scenario. Fig.13 is a schematic diagram showing a read IO / write IO hit / miss cache according to an exemplary embodiment of the present disclosure. Fig.13 , a total of 5 IOs are shown, namely IO1, IO2, IO3, IO4, and IO5. Among them, IO1, IO2, and IO3 belong to User Random Write Small IOs; IO4 and IO5 belong to User Read IOs. Fig.13 Four processes are also shown, namely: (1) reading the parity data block from the SSD (Rd Parity); (2) reading the data block contained in the stripe data from the SSD (Rd Strip); (3) writing the new parity data block and the modified data block to the SSD (New Parity, Wr Strip); (4) reading the stripe data from the SSD (Rd Strip).

[0112] As mentioned above, compared with reading data from the SSD, it takes less time to read data from the cache module, that is, compared with reading data directly from the SSD, it takes less time to read data from the stripe cache (Strip Cache) / redundant cache (Parity Cache).

[0113] For writing IO1, it includes three processes (1), (2) and (3), that is, writing IO1 needs to read the original data blocks and original verification data blocks contained in the stripe data from the SSD. Then, writing IO1 needs to modify the original data blocks read from the SSD to obtain modified data blocks. Next, writing IO1 also needs to perform an XOR operation on the modified data blocks and the original verification data blocks read from the SSD to obtain new verification data blocks. Then, writing IO1 needs to write the modified data blocks and the new verification data blocks back to the SSD, that is, writing IO1 needs to use the modified data blocks and the new verification data blocks to overwrite the original data blocks and the original verification data blocks previously stored in the SSD.

[0114] For writing IO2, it includes two processes (2) and (3). Specifically, writing IO2 needs to read the original data blocks contained in the stripe data from the SSD, and writing IO2 can read the original verification data blocks from the redundant cache (Parity Cache). Next, writing IO2 needs to modify the original data blocks read from the SSD to obtain modified data blocks. Next, writing IO2 also needs to perform an XOR operation on the modified data blocks and the original verification data blocks read from the redundant cache (Parity Cache) to obtain new verification data blocks. Then, writing IO2 needs to write the modified data blocks and the new verification data blocks to the SSD again, that is, writing IO2 needs to use the modified data blocks and the new verification data blocks to overwrite the original data blocks and the original verification data blocks previously stored in the SSD. It can be seen that compared with writing IO1, writing IO2 hits the redundant cache, so the overall writing efficiency of writing IO2 is higher than the overall writing efficiency of writing IO1.

[0115] For Write IO3, Write IO3 can read the original data block and the original verification data block from the Strip Cache. Next, Write IO3 needs to modify the original data block read from the Strip Cache to obtain a modified data block. Next, Write IO3 also needs to perform an XOR operation on the modified data block and the original verification data block read from the Strip Cache to obtain a new verification data block. It can be seen that whether it is the process of reading the original data block or the process of reading the original verification data block, Write IO3 does not access the storage device (SSDs), that is, Write IO3 hits the Strip Cache, thereby eliminating write amplification, that is, the overall write efficiency of Write IO3 is higher than the overall write efficiency of Write IO1 and Write IO2.

[0116] For reading IO5, it includes the process (4). Specifically, reading IO5 needs to read stripe data from the SSD; for reading IO4, reading IO4 can read stripe data from the strip cache. It can be seen that reading IO4 hits the strip cache. Therefore, the overall reading efficiency of reading IO4 is higher than the overall reading efficiency of reading IO5.

[0117] From the above analysis, it can be seen that in the present disclosure, by expanding the cache, that is, by setting up stripe cache (StripCache) and redundant cache (Parity Cache), the cache hit rate (Hit Cache) can be improved, and then the RAID read and write delay can be effectively reduced, that is, write amplification is effectively avoided and data processing efficiency is improved.

[0118] According to an exemplary embodiment of the present disclosure, all stripe data and all data processing results thereof can be packaged by the near storage computing module to obtain packaged data, and the packaged data can be sent to the central processing unit. Then, the packaged data can be written to the storage device of the host associated with the near storage computing device by the central processing unit, for example, it can be written to the SSD of the host. That is, in a write scenario, the cache stripe can be packaged into a larger data block and written to the storage device SSD of the host.

[0119] It should be noted that the return reference Figure 1 , Figure 1 Two data stripes are shown, namely strip1 and strip2. If the packing operation is not performed on strip1 and strip2, when the data blocks D0 and D1 contained in the data stripe strip1, and the data blocks D8 and D9 contained in the data stripe strip2 are written to SSD1, a write operation needs to be performed for each data block, that is, a total of 4 write operations need to be performed.

[0120] If the packing operation is performed on strip1 and strip2, the data blocks D0 and D1 contained in the data stripe strip1, and the data blocks D8 and D9 contained in the data stripe strip2 will be packed into a larger data block. At this time, only the larger data block needs to be written to SSD1, that is, only one write operation needs to be performed on the larger data block. It can be seen that compared with the method of not packing the data stripes, packing the data stripes and then writing them to the SSD can greatly reduce the number of executions of the write operation, that is, it can greatly improve the write efficiency.

[0121] It should be noted that in the related art, if you want to implement the "expanded memory" function, you need to insert a card into the host separately; and if you want to implement the "data verification calculation, data cloning calculation, data modification calculation and cache management" functions, you also need to insert a RAID card into the host separately. In this case, two independent card slots on the host are required to insert the above two cards.

[0122] In the present disclosure, since the near memory computing module (PNM) and the cache module (memory) are set in the same device (CXL-PNM Device), the present disclosure only needs one slot on the host to insert the CXL-PNM Device, and can realize the "expanded memory" function, as well as the "data verification calculation, data cloning calculation, data modification calculation and cache management" functions. It can be seen that compared with the related art, the present disclosure can save the occupation of the card slot on the host.

[0123] Furthermore, if the host includes different types of SSDs, in the related art, a corresponding RAID card needs to be inserted into the host for each type of SSD to provide RAID services for that type of SSD. For example, assuming that the host includes 3 types of SSDs, namely PCIe type SSD, SARS type SSD and SATA type SSD, then in the related art, a corresponding RAID card needs to be inserted into the host for each type of SSD in the 3 types of SSDs to provide RAID services for that type of SSD.

[0124] In the present disclosure, no matter how many different types of SSDs are included in the host, only one CXL-PNM Device needs to be inserted into the host to provide RAID services for different types of SSDs on the host, which can effectively save the use of RAID cards and save hardware costs.

[0125] Fig.14 is a flowchart showing a specific implementation of a redundancy calculation process according to an exemplary embodiment of the present disclosure.

[0126] Reference Fig.14 In step 1401, the CPU on the host sends the storage addresses (strip-id) of multiple stripe data and the algorithm type (RAID-level) of the redundant array of independent disks to the near memory computing module (PNM) of the CXL-PNM Device.

[0127] Specifically, the user can hand over the data to be processed to the host, and then the CPU on the host can perform striping processing on the data to be processed given by the user, that is, the data to be processed given by the user can be split into multiple small data blocks to obtain strip data. Then, the CPU can write the obtained strip data into the memory of the host. Next, the CPU can send the storage address of the strip data in the memory of the host to the near-memory computing module. Then, based on the storage address sent by the CPU, the near-memory computing module can pull the strip data from the memory of the host into the cache module (Memory).

[0128] In step 1402, the near-memory computing module pulls strip data from the cache module one by one.

[0129] Specifically, for strip data strip[i], where i < M, the near-memory computing module can read the N data blocks strip-block[j] contained in the strip data strip[i] one by one from the cache module, where j < N. Here, M is the total number of strip data stored in the cache module.

[0130] In step 1403, based on the algorithm type (RAID-level) of the redundant array of independent disks sent by the CPU on the host, the near-memory computing module performs redundancy calculation on the N data blocks contained in the strip data strip[i] to obtain the redundant data block corresponding to the strip data strip[i], that is, the parity data block.

[0131] Exemplarily, assuming the above RAID-level is RAID6, the near-memory computing module can calculate two redundant data blocks corresponding to the strip data strip[i], namely P Parity and Q Parity.

[0132] In step 1404, the near-memory computing module can send the calculated redundant data block to the cache module. In this way, the cache module can store the received redundant data block and the strip data strip[i] used to obtain the redundant data block in an associated manner.

[0133] In step 1405, the near-memory computing module PNM sends a completion message to the redundancy calculation proxy "CalculateParity Proxy" on the CPU of the host.

[0134] In this way, in the present disclosure, since the time-consuming data verification calculation process is transferred from the central processing unit to the near-memory computing module, the frequency of the CPU accessing the memory of the host is reduced, the occupancy rate of the central processing unit can be reduced, and the computing power of the central processing unit can be released.

[0135] Fig.15It is a specific implementation flowchart showing the data mirroring calculation process according to an exemplary embodiment of the present disclosure.

[0136] Referring to Fig.15 , in step 1501, the CPU on the host sends the storage addresses (strip - id) of multiple strip data to the near - memory computing module (PNM) of the CXL - PNM Device.

[0137] Specifically, the user can hand the data to be processed to the host. Then, the CPU on the host can perform striping processing on the data to be processed given by the user, that is, the data to be processed given by the user can be divided into multiple small data blocks to obtain strip data. Then, the CPU can write the obtained strip data into the memory of the host. Next, the CPU can send the storage address of the strip data in the host's memory to the near - memory computing module. Then, the near - memory computing module can pull the strip data from the host's memory to the cache module (Memory) based on the storage address sent by the CPU.

[0138] In step 1502, the near - memory computing module pulls the strip data from the cache module one by one.

[0139] Specifically, for the strip data strip[i], i < M, the near - memory computing module can read each of the N data blocks strip - block[j], j < N, included in the strip data strip[i] from the cache module. Where M is the total number of strip data stored in the cache module.

[0140] In step 1503, the near - memory computing module performs data cloning calculation on each data block strip - block[j] included in the strip data strip[i] to obtain the cloned data block corresponding to the data block strip - block[j], that is, the cloned copy.

[0141] In step 1504, the near - memory computing module sends the cloned data block corresponding to each data block strip - block[j] to the cache module. In this way, the cache module can store the received cloned data block in an associated manner with the strip data strip[i] used to obtain the cloned data block.

[0142] In step 1505, the near - memory computing module PNM sends a completion message to the mirror proxy "Mirror Proxy" on the CPU of the host.

[0143] In this way, in the present disclosure, since the time - consuming data cloning process is transferred from the central processing unit to the near - memory computing module, the frequency of the CPU accessing the host's memory is reduced, the occupancy rate of the central processing unit can be reduced, and the computing power of the central processing unit can be released.

[0144] Fig.16 It is a specific implementation flowchart showing the cache management process according to an exemplary embodiment of the present disclosure.

[0145] Referring to Fig.16 , in step 1601, the CPU on the host sends cache management indication information to the near-memory computing module PNM. It should be noted that cache management can be triggered by the CPU, or the near-memory computing module PNM can automatically perform cache management every preset time period. The present disclosure does not make specific limitations on this.

[0146] In step 1602, the near-memory computing module PNM pulls cache entries cache-entry[i] one by one from the cache module (Memory), where i < M. Here, M is the total number of cache entries stored in the cache module.

[0147] In step 1603, the near-memory computing module PNM determines whether the pulled cache entry cache-entry[i] has expired. And, in the case where it is determined that the pulled cache entry cache-entry[i] has expired, the cache entry cache-entry[i] is deleted (Evict) from the cache module. Among them, if the cache entry stored in the cache module has not been accessed for a long time, it can be considered that the cache entry has expired.

[0148] In step 1604, the near-memory computing module PNM modifies the cache bitmap value of the deleted cache entry from the first value to the second value.

[0149] Among them, the first value can be used to indicate that the cache entry exists in the cache module, and the second value can be used to indicate that the cache entry does not exist in the cache module. Exemplarily, the first value can be 1 and the second value can be 0.

[0150] In step 1605, the near-memory computing module PNM can pack multiple cache entries stored in the cache module and their data processing results to obtain packed data, and can send the packed data to the central processing unit so that the central processing unit writes the packed data into the storage device SSD of the host associated with the near-memory computing device (CXL-PNM Device).

[0151] In step 1606, the near-memory computing module PNM sends a completion message to the cache management agent (Mgt CacheProxy) on the CPU of the host.

[0152] Thus, in the present disclosure, when it is determined that a cache entry stored in the stripe cache is expired, the cache entry can be deleted from the stripe cache, thereby preventing invalid cache entries that have not been accessed for a long time from occupying the stripe cache excessively. Furthermore, after deleting the cache entry, the cache bitmap value corresponding to the deleted cache entry can be synchronously updated, so that in subsequent query processes, it is possible to directly determine whether the corresponding cache entry exists in the stripe cache based on the updated cache bitmap value, thereby improving the query efficiency of the cache entry. Furthermore, the data stripes can be packaged and then written to the SSD, which can greatly reduce the number of executions of the write operation, that is, the write efficiency can be greatly improved.

[0153] Fig.17 1 is a schematic diagram showing a near storage computing device 1700 according to an exemplary embodiment of the present disclosure. Fig.17 , the near storage computing device 1700 may include a near storage computing module 1701 and a cache module 1702 .

[0154] The cache module 1702 is used to store all stripe data related to the target data processing pulled by the near storage computing module 1701 from the memory of the central processing unit of the host associated with the near storage computing device 1700. The near storage computing module 1701 is used to pull the stripe data from the cache module 1702 one by one and perform the target data processing.

[0155] Specifically, the near memory calculation module 1701 can be used to perform data verification calculation (Calc Parity), that is, redundancy calculation; data mirror calculation (Mirror), that is, data clone calculation (Clone); data modification calculation; and cache management (Mgt Cache). The cache module 1702 can include a stripe cache (Strip Cache) and a redundant cache (ParityCache). The stripe cache is used to cache stripe data and the parity data blocks of the stripe data; the redundant cache is only used to cache the parity data blocks of the stripe data.

[0156] When the near memory calculation module 1701 determines that a certain stripe data in the stripe cache is expired, the stripe data can be deleted from the stripe cache, and the check data block corresponding to the deleted stripe data can be moved to the redundant cache. Alternatively, when the near memory calculation module 1701 determines that a certain check data block in the redundant cache is expired, the check data block can be deleted from the redundant cache.

[0157] According to an exemplary embodiment of the present disclosure, in a modification / reading scenario, the near memory calculation module 1701 can also be used to determine whether the stripe data to be modified / read currently hits the stripe cache, and to determine whether the check data block to be read currently hits the redundant cache. In the case of hitting the stripe cache, the stripe data and check data blocks read from the stripe cache can be directly processed; or, in the case of hitting the redundant cache, the check data blocks read from the redundant cache can be directly processed. Compared with the method of directly reading stripe data and check data blocks from the SSD, reading stripe data and check data blocks from the cache module takes less time and has higher overall data processing efficiency.

[0158] According to an exemplary embodiment of the present disclosure, in a modification / reading scenario, if the cache is not hit, that is, if the stripe cache / redundant cache is not hit, the near memory computing module 1701 needs to read the stripe data and the check data block from the SSD for subsequent processing.

[0159] Thus, in the present disclosure, since the time-consuming and intensive data processing process is transferred from the central processing unit to the near-memory computing module of the near-memory computing device, the occupancy rate of the central processing unit can be reduced to release the computing power of the central processing unit.

[0160] Fig.18 1 is a block diagram showing a data processing device 1800 according to an exemplary embodiment of the present disclosure. The data processing device 1800 may be applied to a near storage computing device. The data processing device 1800 may include a pull module 1801 and a data processing module 1802.

[0161] The pulling module 1801 may pull all stripe data related to target data processing from the memory of the central processing unit of the host associated with the near-storage computing device, and may store all stripe data in the cache module.

[0162] According to an exemplary embodiment of the present disclosure, when the above-mentioned "target data processing" is a redundant array of independent disks intensive calculation, the data processing device 1800 can perform an operation, that is, can perform the above-mentioned data processing method. Wherein, the "redundant array of independent disks intensive calculation" can be a calculation that requires a central processing unit computing resource size exceeding a preset resource size.

[0163] According to an exemplary embodiment of the present disclosure, the above-mentioned "independent disk redundant array intensive calculation" may include at least one of "data verification calculation", "data cloning calculation" and "data modification calculation". Among them, "data verification calculation" may refer to the calculation of performing XOR operation on the data blocks included in the stripe data to obtain the verification data block; "data cloning calculation" may refer to the calculation of copying the data blocks included in the stripe data to obtain the mirror copy; "data modification calculation" may refer to the calculation of modifying the data blocks included in the stripe data to obtain the new data block. In addition, "data verification calculation" is the "redundancy calculation" mentioned above; "data cloning calculation" is the "data mirror calculation" mentioned above.

[0164] The data processing module 1802 can pull stripe data from the cache module one by one through the near storage computing module and perform target data processing. Exemplarily, the near storage computing module can pull stripe data from the cache module one by one, and perform "data verification calculation", "data cloning calculation" or "data modification calculation" on each pulled stripe data.

[0165] According to an exemplary embodiment of the present disclosure, when the target data processing is data verification calculation, the data processing module 1802 can perform data verification calculation on the data blocks contained in the stripe data pulled one by one to obtain the verification data blocks corresponding to the stripe data pulled one by one.

[0166] According to an exemplary embodiment of the present disclosure, the above-mentioned data processing device 1800 may further include a receiving module, which may be configured to receive an algorithm type regarding verification calculation sent by the central processing unit through the near-storage calculation module. Among them, the "algorithm type regarding verification calculation" may be an "algorithm type for an independent disk redundant array". Exemplarily, the "algorithm type for an independent disk redundant array" may be, but not limited to: RAID0, RAID1, RAID5, RAID6, RAID10, etc., and the redundancy algorithms corresponding to the algorithm types of different independent disk redundant arrays may be different. For example, for RAID5, it only needs to calculate one redundant data block, namely the verification data block P Parity; while for RAID6, it needs to calculate two redundant data blocks P Parity and Q Parity.

[0167] Next, the data processing module 1802 can pull the stripe data one by one from the cache module through the near memory computing module, and can perform data verification calculations on the data blocks contained in the stripe data pulled one by one based on the verification calculation algorithm type sent by the CPU, that is, the algorithm type of the independent disk redundant array, to obtain the verification data blocks corresponding to the stripe data pulled one by one.

[0168] According to an exemplary embodiment of the present disclosure, when the target data processing is data cloning calculation, the data processing module 1802 can pull the stripe data from the cache module one by one through the near storage calculation module and can perform data cloning calculation on the target data blocks contained in the stripe data pulled one by one, and obtain the clone data block corresponding to the target data block, that is, obtain the replica data block corresponding to the target data block. In addition, the near storage calculation module can also send the clone data block to the cache module, and then the cache module can associate the received clone data block with the stripe data read this time for storage.

[0169] According to an exemplary embodiment of the present disclosure, the data processing module 1802 can determine whether there is expired stripe data in the cache module through the near storage calculation module. The expired stripe data can be stripe data that has not been accessed within a preset first time period. When the data processing module 1802 determines that there is expired stripe data, it can delete the expired stripe data from the cache module and retain the check data block corresponding to the expired stripe data in the cache module.

[0170] According to an exemplary embodiment of the present disclosure, the cache bitmap value of the deleted stripe data may be modified from a first value to a second value. Exemplarily, the first value may be 1; the second value may be 0. The first value may be used to indicate that the stripe data exists in the stripe cache, and the second value may be used to indicate that the stripe data does not exist in the stripe cache.

[0171] In this way, when it is determined that a certain stripe data stored in the cache module is expired, the stripe data can be deleted from the cache module, thereby preventing the invalid stripe data that has not been accessed for a long time from occupying too much of the cache module. Furthermore, after deleting the stripe data, the cache bitmap value corresponding to the deleted stripe data can be synchronously updated, so that in the subsequent query process, it is possible to directly determine whether the corresponding stripe data exists in the stripe cache based on the updated cache bitmap value, which can improve the query efficiency of the stripe data.

[0172] According to an exemplary embodiment of the present disclosure, the cache module may include a first cache area and a second cache area. The first cache area may be used to store stripe data and its check data blocks, and the second cache area may be used to store check data blocks of deleted stripe data. Furthermore, the "first cache area" may also be referred to as a stripe cache, and the "second cache area" may also be referred to as a parity cache.

[0173] The above-mentioned data processing module 1802 can delete the expired stripe data from the first cache area, and can move the verification data block of the expired stripe data from the first cache area to the second cache area for storage. That is, the verification data block corresponding to the deleted stripe data stored in the stripe cache can be moved to the redundant cache through the near storage calculation module. That is, in the stripe cache, if the cache content heat decreases, only the redundant data block of the cache content can be retained and the redundant data block of the cache content can be expelled to the redundant cache contained in the cache module.

[0174] It should be noted that compared with reading data from the SSD, reading data from the cache module takes less time. Therefore, by moving the check data block corresponding to the deleted stripe data to the redundant cache, it is convenient to directly read the check data block from the redundant cache contained in the cache module in subsequent data writing scenarios, even in subsequent data modification scenarios, thereby avoiding the problem of long reading time caused by directly reading the check data block from the SSD, and improving the reading efficiency of the check data block.

[0175] According to an exemplary embodiment of the present disclosure, the above-mentioned data processing module 1802 can also determine whether there is an expired verification data block in the second cache area through the near storage calculation module. Among them, the expired verification data block can be a verification data block that has not been accessed within a preset second time period. In the case of determining that there is an expired verification data block, the expired verification data block can be deleted from the second cache area. That is, in the redundant cache (Parity Cache), if a cache item stored therein becomes cold, the cache item can be directly deleted.

[0176] It should be noted that in the present disclosure, relatively hot stripe data can be stored in the stripe cache; for stripe data that is not so hot, that is, relatively "warm", only the check data blocks corresponding to the relatively "warm" stripe data can be stored in the redundant cache (Parity Cache); and for those relatively cold stripe data, they can be directly stored in the SSD.

[0177] In this way, compared with the method of directly caching stripe data in the stripe cache, only caching the check data blocks corresponding to the stripe data in the redundant cache can greatly reduce the occupation of cache space; and, if a cache item in the redundant cache becomes cold, the cache item can be directly deleted, which can avoid the excessive occupation of the redundant cache by invalid check data blocks that have not been accessed for a long time. Furthermore, since the check data blocks are stored separately in the redundant cache, it is convenient to read the check data blocks directly from the redundant cache later, that is, the cache hit rate of the check data blocks that can be read directly from the redundant cache is improved, thereby avoiding the problem of long reading time caused by reading the check data blocks directly from the SSD, and the reading efficiency of the check data blocks can be improved. It can be seen that the redundant cache set in the cache module of the present disclosure preferably achieves a balance between cache space occupation and cache hit rate.

[0178] According to an exemplary embodiment of the present disclosure, the data processing device 1800 may further include a packaging module and a writing module.

[0179] The packing module can pack all the stripe data and all the data processing results thereof through the near storage computing module to obtain the packed data, and can send the packed data to the central processing unit. Then, the writing module can write the packed data to the storage device of the host associated with the near storage computing device through the central processing unit, for example, it can be written to the SSD of the host. That is, in the writing scenario, the cache stripes can be packed into larger data blocks and written to the storage device SSD of the host.

[0180] Fig.19 is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment of the present disclosure.

[0181] Reference Fig.19 The electronic device 1900 includes at least one memory 1901 and at least one processor 1902. The at least one memory 1901 stores instructions. When the instructions are executed by the at least one processor 1902, the data processing method according to the exemplary embodiment of the present disclosure is executed.

[0182] As an example, the electronic device 1900 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instructions. Here, the electronic device 1900 is not necessarily a single electronic device, but may also be any device or circuit collection capable of executing the above instructions (or instruction sets) individually or in combination. The electronic device 1900 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0183] In the electronic device 1900, the processor 1902 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0184] The processor 1902 may execute instructions or codes stored in the memory 1901, wherein the memory 1901 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0185] The memory 1901 may be integrated with the processor 1902, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 1901 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 1901 and the processor 1902 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 1902 can read files stored in the memory.

[0186] In addition, the electronic device 1900 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 1900 may be connected to each other via a bus and / or a network.

[0187] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned data processing method. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0188] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including a computer program, and when the computer program is executed by a processor, the data processing method according to the present disclosure is implemented.

[0189] According to the data processing method and related devices disclosed in the present invention, by setting a cache module in the near-memory computing device, the near-memory computing module can directly pull stripe data from the cache module one by one and perform target data processing. Since the distance between the near-memory computing module and the cache module is smaller than the distance between the central processing unit and the memory of the central processing unit, compared with the method of reading data from the memory of the central processing unit through the central processing unit, the time taken to read data from the cache module through the near-memory computing module will be shorter, thereby improving the overall data processing efficiency. Furthermore, since the time-consuming and intensive data processing process is transferred from the central processing unit to the near-memory computing module, the occupancy rate of the central processing unit can be reduced to release the computing power of the central processing unit.

[0190] According to an exemplary embodiment of the present disclosure, when it is determined that a certain stripe data stored in the stripe cache is expired, the stripe data can be deleted from the stripe cache, thereby preventing invalid stripe data that has not been accessed for a long time from occupying too much of the stripe cache. Furthermore, after deleting the stripe data, the cache bitmap value corresponding to the deleted stripe data can be synchronously updated, so that in the subsequent query process, it is possible to directly determine whether the corresponding stripe data exists in the stripe cache based on the updated cache bitmap value, which can improve the query efficiency of the stripe data.

[0191] According to an exemplary embodiment of the present disclosure, compared with reading data from an SSD, it takes less time to read data from a cache module. Therefore, by moving the check data block corresponding to the deleted stripe data to the redundant cache, it is convenient to directly read the check data block from the redundant cache included in the cache module in subsequent data writing scenarios, even in subsequent data modification scenarios, thereby avoiding the problem of long reading time caused by directly reading the check data block from the SSD, and improving the reading efficiency of the check data block.

[0192] According to an exemplary embodiment of the present disclosure, compared with directly caching stripe data in the stripe cache, caching only the check data blocks corresponding to the stripe data in the redundant cache can greatly reduce the occupation of cache space; and, if a cache item in the redundant cache becomes cold, the cache item can be directly deleted, which can avoid the excessive occupation of the redundant cache by invalid check data blocks that have not been accessed for a long time. Furthermore, since the check data blocks are stored separately in the redundant cache, it is convenient to read the check data blocks directly from the redundant cache later, that is, the cache hit rate of the check data blocks that can be read directly from the redundant cache is improved, thereby avoiding the problem of long reading time caused by reading the check data blocks directly from the SSD, and the reading efficiency of the check data blocks can be improved. It can be seen that the redundant cache set in the cache module of the present disclosure preferably achieves a balance between cache space occupation and cache hit rate.

[0193] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0194] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method, applied to a near-storage computing device, It is characterized in that The near-storage computing device includes a near-storage computing module and a cache module, and the data processing method includes: Pulling all stripe data related to target data processing from the memory of the central processing unit of the host associated with the near-storage computing device, and storing all the stripe data in the cache module; The near storage computing module pulls stripe data from the cache module one by one and executes the target data processing.

2. The data processing method according to claim 1, It is characterized in that In the case where the target data processing is independent disk redundant array intensive calculation, the data processing method as claimed in claim 1 is executed, wherein the independent disk redundant array intensive calculation is a calculation that requires a central processing unit computing resource size exceeding a preset resource size.

3. The data processing method according to claim 2, It is characterized in that The independent redundant array of disks intensive calculation includes at least one of data verification calculation, data cloning calculation and data modification calculation.

4. The data processing method according to claim 3, It is characterized in that In a case where the target data processing is the data verification calculation, the performing of the target data processing includes: Data verification calculation is performed on the data blocks included in the stripe data pulled one by one to obtain verification data blocks corresponding to the stripe data pulled one by one.

5. The data processing method according to claim 4, It is characterized in that The data processing method further includes: Receiving, through the near-storage computing module, an algorithm type for verification computing sent by the central processing unit; The step of performing data verification calculation on the data blocks contained in the stripe data pulled one by one to obtain the verification data blocks corresponding to the stripe data pulled one by one includes: Based on the algorithm type of the verification calculation, data verification calculation is performed on the data blocks contained in the stripe data pulled one by one, so as to obtain the verification data blocks corresponding to the stripe data pulled one by one.

6. The data processing method according to claim 3, It is characterized in that In a case where the target data processing is the data clone calculation, the performing of the target data processing includes: Data cloning calculation is performed on the target data blocks contained in the stripe data pulled one by one to obtain the cloned data blocks corresponding to the target data blocks.

7. The data processing method according to claim 1, It is characterized in that Also includes: Determine, by the near storage calculation module, whether there is expired stripe data in the cache module, wherein the expired stripe data is stripe data that has not been accessed within a preset first time period; In the case where it is determined that the expired stripe data exists, the expired stripe data is deleted from the cache module, and the check data block corresponding to the expired stripe data is retained in the cache module.

8. The data processing method according to claim 7, It is characterized in that The cache module includes a first cache area and a second cache area, the first cache area is used to store stripe data and its check data block, and the second cache area is used to store the check data block of the deleted stripe data; The step of deleting the expired stripe data from the cache module and retaining the check data block corresponding to the expired stripe data in the cache module includes: The expired stripe data is deleted from the first cache area, and the check data block of the expired stripe data is moved from the first cache area to the second cache area for storage.

9. The data processing method according to claim 8, It is characterized in that The data processing method further includes: Determine, by the near memory calculation module, whether there is an expired verification data block in the second cache area, wherein the expired verification data block is a verification data block that has not been accessed within a preset second time period; When it is determined that the expired verification data block exists, the expired verification data block is deleted from the second cache area.

10. The data processing method according to claim 1, It is characterized in that The data processing method further includes: Packing all the stripe data and all the data processing results thereof by the near storage computing module to obtain packed data, and sending the packed data to the central processing unit; The packaged data is written into a storage device of a host associated with the near storage computing device by the central processor.

11. A data processing device, applied to a near-storage computing device, It is characterized in that The data processing device comprises: A pulling module configured to pull all stripe data related to target data processing from a memory of a central processing unit of a host associated with the near-storage computing device, and store all stripe data in a cache module; The data processing module is configured to pull the stripe data from the cache module one by one through the near storage computing module and perform the target data processing.

12. A near-storage computing device, It is characterized in that include: Near memory computing module; Cache module; Wherein, the cache module is used to store all stripe data related to target data processing pulled by the near storage computing module from the memory of the central processing unit of the host associated with the near storage computing device; The near memory computing module is used to pull stripe data from the cache module one by one and execute the target data processing.

13. An electronic device, It is characterized in that include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the data processing method according to any one of claims 1 to 10.

14. A computer-readable storage medium, It is characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data processing method as claimed in any one of claims 1 to 10.

15. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

Images