Memory system, data processing method, device, storage medium, and program product

Abstract

The present invention discloses a memory system, a data processing method, a device, a storage medium, and a program product, which relate to the field of storage technologies. Among them, the method includes determining a plurality of data writing methods for writing the data to be stored into a remote memory according to the check data determination method and / or the writing condition corresponding to the data to be stored. According to the storage parameter information of the target storage space of the data to be stored in the remote memory system, the number of read / write operations and the read / write cost of each data writing method, the storage cost of each data writing method is determined. The data to be stored is stored in the remote memory system by using the target data writing method with the lowest storage cost. The present invention can solve the problem that the related technology is limited by the distance between the memory node and the computing node and cannot guarantee the optimal memory performance in different situations, and can realize the storage and read / write processing of highly reliable remote memory data.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular, to a data processing method, an electronic device, a computer-readable storage medium, a computer program product, and a remote memory system. Background Art

[0002] Remote memory expansion technology allows a computer system to utilize remotely connected memory resources via a network. To achieve highly reliable remote memory data storage and read / write, related technologies selectively protect a subset of the memory space, and achieve memory protection at the cache page level through batch write-back and set associative mapping. However, this method has requirements for the distance between the memory node and the computing node, and the set associative mapping optimization method cannot guarantee optimal memory performance in different scenarios. Summary of the Invention

[0003] The present invention provides a data processing method, an electronic device, a computer-readable storage medium, a computer program product, and a remote memory system, which can achieve highly reliable remote memory data storage and read / write, and ensure optimal memory performance in different scenarios.

[0004] To solve the above technical problems, the present invention provides the following technical solutions:

[0005] The present invention provides a data processing method, including:

[0006] Determine multiple data writing methods for storing the data to be stored in the remote memory system according to the check data determination method corresponding to the data to be stored and / or the writing condition for writing the data to be stored into the remote memory system; respectively count the number of read / write operations of each data writing method, and determine the storage cost of each data writing method according to the storage parameter information of the target storage space corresponding to the data to be stored in the remote memory system, the number of read / write operations of each data writing method, and the corresponding read / write cost; use the target data writing method with the lowest storage cost to store the data to be stored in the remote memory system; wherein, at least one of the check data determination method and the writing condition of different data writing methods is different, and the storage parameter information includes at least the storage capacity of the target storage space and the data volume associated with the data to be stored.

[0007] The present invention further provides an electronic device, including a memory and a processor, and the processor is configured to implement the steps of any of the above data processing methods when executing the computer program stored in the memory.

[0008] The present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is configured to implement the steps of any of the above data processing methods when executed by a processor.

[0009] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of any of the above data processing methods.

[0010] Finally, the present invention also provides a remote memory system, which at least includes a remote memory, a buffer, a read / write processor, and a program memory; the memory is connected to the buffer, and the read / write processor is connected to the buffer and the program memory; the remote memory at least includes one stripe, and the stripe is used to store corresponding data to be stored; the buffer serves as the memory space of the read / write processor, including a read operation buffer space and / or a write operation buffer space, and the occupied capacities of the read operation buffer space and the write operation buffer space are determined according to the current application requirements and load characteristics; the read / write processor, when executing the computer program stored in the program memory, implements the steps of any of the above data processing methods.

[0011] The advantages of the technical solution provided by the present invention are as follows: for the data to be stored on the remote memory system, the remote memory system replaces the computing node to complete the verification calculation, write cost evaluation, and write method selection of the data to be written. In this way, the read / write performance and memory performance of the remote memory system will not be limited by the distance between the computing node and the remote memory system, thereby realizing remote memory expansion in any distance scenario. From the two aspects of verification data calculation and write timing, the influence of the storage situation and service operation situation of the remote memory system in different scenarios on the write method is reflected. By evaluating the costs of different write methods, the data write method that best matches the current storage performance can be determined, so as to ensure that the memory performance of the remote memory system reaches the optimal level in different scenarios, and the high-reliability remote memory can realize efficient data storage and read / write functions. In addition, the present invention also provides corresponding implementation electronic devices, computer-readable storage media, computer program products, and remote memory systems for the data processing method, further making the method more practical, and the electronic devices, computer-readable storage media, computer program products, and remote memory systems have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 It is a schematic diagram of the hardware composition framework applicable to the data processing method provided by the present invention;

[0014] Figure 2Flow schematic diagram of a data processing method provided by the present invention;

[0015] Figure 3 Data interaction schematic diagram corresponding to various data writing methods provided by the present invention;

[0016] Figure 4 Structural framework diagram under an exemplary embodiment of a data processing device provided by the present invention;

[0017] Figure 5 Structural framework diagram of an exemplary embodiment of a remote memory system provided by the present invention;

[0018] Figure 6 Data interaction schematic diagram between a remote memory system and a computing node under an exemplary scenario provided by the present invention. Detailed implementation manners

[0019] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. Among them, the terms "first", "second", "third", "fourth", etc. in the specification and the above drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. The term "exemplary" means "serving as an example, embodiment or illustration". Any embodiment described as "exemplary" here does not have to be construed as superior or better than other embodiments.

[0020] With the rapid development of big data analysis, HPC (High Performance Computing), and AI (Artificial Intelligence) technologies, memory-intensive applications have increasingly higher requirements for the memory capacity, memory access performance, and flexibility of computer systems. In order to support the needs of large-scale data processing and computing tasks and improve the overall performance of computer systems, a memory system based on remote memory expansion technology has replaced the traditional memory system and is widely used in memory-intensive applications.

[0021] Remote memory expansion technology can dynamically adjust memory resources according to application requirements through memory allocation technology, avoiding a large one-time investment in local memory. It not only achieves flexible resource management but also better controls costs. In addition, remote memory expansion can share data among multiple computing units and maintain data consistency, which reasonably improves the data availability of the computing system. Although remote memory expansion technology can increase memory capacity and improve the performance of computer systems, since this technology uses the memory resources of multiple storage nodes or computing nodes based on network connections, once there is a network failure or a node failure, the corresponding memory resources cannot be accessed, affecting the stability and reliability of the entire computer system. Secondly, remote memory access has higher latency and lower bandwidth and is not suitable for application scenarios with high requirements for latency and bandwidth, such as memory-sensitive applications. In addition, multiple computing nodes simultaneously reading and writing remote memory is very likely to cause a wider spread of faults within the computer system, which means that a small number of memory faults may cause errors or crashes of a large amount of computing resources.

[0022] To address the above risks and achieve highly reliable remote memory data storage and high-performance remote memory reading and writing, related technologies usually adopt remote memory fault tolerance strategies, which selectively protect a subset of the memory space and achieve memory protection at the cache page level through batch write-back and set-associative mapping. Among them, set-associative mapping includes memory interleaved mapping and cache line interleaved mapping. However, this method involves data exchange between the memory and the last-level cache of the CPU (Central Processing Unit), has requirements for the distance between the memory node and the computing node, and is not suitable for remote memory expansion scenarios where the two are far apart. In addition, various set-associative mapping optimization methods cannot guarantee that the memory performance is optimal in different situations.

[0023] In view of this, the present invention builds a processing unit with a check calculation function, a write cost evaluation function, and a write method selection function in the remote memory system. For the data to be stored that needs to be stored thereon, multiple data write methods for writing it to the remote memory are determined according to the check data determination method and / or the write timing corresponding to the data to be stored. According to the storage parameter information of the target storage space of the data to be stored in the remote memory system, the number of read and write operations and the read and write costs of each data write method, the storage costs of each data write method are determined. The data to be stored is stored in the remote memory system using the target data write method with the lowest storage cost, thereby ensuring that the memory performance of the remote memory system reaches the optimal level in different scenarios and enabling highly reliable and efficient data storage and reading and writing functions for remote memory. Combining with the specific application environment architecture or specific hardware architecture on which the execution of the data processing method depends, the specific application environment architecture or specific hardware architecture is described herein. Next, in combination with Figure 1Some possible application scenarios related to the technical solution of the present invention are introduced by way of example, which may include the following:

[0024] The server 11 is connected to the remote memory system 12 via a network. The remote memory system 12 at least includes a processing unit 121 and a memory 122. The memory 122 serves as the remote memory space of the remote memory system 12 and provides remote memory resources to the server 11, so that the server 11 can use the remote memory resources of the remote memory system 12 via the network. When the server 11 needs to store data in the remote memory system 12, it can send a data write request to the remote memory system 12. The data write request carries the data to be stored or data storage location information. The remote memory system 12 receives the data write request and reads the data to be stored. The processing unit 121 determines various data writing methods for storing the data to be stored in the remote memory system according to the check data determination method corresponding to the data to be stored and / or the writing timing of the data to be stored written into the remote memory system 12; respectively counts the read and write operation times of each data writing method, and determines the storage cost of each data writing method according to the storage parameter information of the target storage space of the data to be stored in the remote memory system, the read and write operation times of each data writing method, and the corresponding read and write costs; finally, uses the target data writing method with the lowest storage cost to store the data to be stored in the remote memory system, which can achieve efficient data writing while ensuring the high reliability of the remote memory system 12, and further realizes the efficient data storage and read / write functions of the highly reliable remote memory.

[0025] It should be noted that the above application scenarios are only shown for the convenience of understanding the idea and principle of the present invention, and the embodiments of the present invention are not limited in this regard. On the contrary, the embodiments of the present invention can be applied to any applicable scenario. After introducing the technical solution of the present invention, the various non-limiting embodiments of the present invention will be described in detail below with reference to the drawings and specific embodiments. First, please refer to Figure 2 , the data processing method provided in this embodiment may include the following:[[]]END]]

[0026] S201: Determine various data writing methods for storing the data to be stored in the remote memory system according to the check data determination method corresponding to the data to be stored and / or the writing condition of the data to be stored written into the remote memory system.

[0027] Among them, the data to be stored is the data that needs to be stored in the remote memory by a computing node with remote memory expansion through a network. The computing node can be, for example, a general server, an artificial intelligence server, various types of host ends and user ends, etc. When writing the data to be stored into the remote memory system, since the data to be stored will be associated with corresponding verification data, in order to ensure the consistency of the data stored in the remote memory space of the remote memory system and its verification data, each write not only needs to update the data to be written, that is, the data to be stored, but also needs to update the corresponding verification data. There are many calculation methods for verification data, and different calculation methods for verification data have different impacts on write performance and data reliability. The verification data determination method is the method for calculating verification data, and the write condition refers to the write timing for writing the data to be stored back to the remote memory system. For example, whenever the data to be stored is received, it is immediately written to the corresponding storage location, or a certain specific condition is reached and written to the remote memory space together with other data. A data write method includes at least one of a verification data determination method or a write condition. By arbitrarily combining multiple verification data determination methods and multiple write conditions, multiple data write methods can be obtained, that is, the verification data determination methods and write conditions of different data write methods are at least one different.

[0028] S202: Count the read and write operation times of each data write method respectively. According to the storage parameter information of the target storage space of the data to be stored in the remote memory system, the read and write operation times of each data write method, and the corresponding read and write costs, determine the storage cost of each data write method.

[0029] It can be understood that different data writing methods have their own advantages, and the impact on the memory performance of the remote memory system is also different. While ensuring the reliability of the data to be stored, reducing the impact on the writing performance, and ensuring that the memory performance of the remote memory system is the best under various conditions, this step will dynamically select the data writing method that best matches the current conditions from the multiple data writing methods determined in S201 in combination with the storage parameter information of the remote memory system. The matching measurement rule is based on the storage cost or the writing cost. Among them, the target storage space refers to the location in the remote memory system used to store the data to be stored. Considering that the remote memory system usually consists of multiple memory devices to provide rich remote memory resources, such as multiple memory modules or multiple memory cards, the target storage space refers to the storage space corresponding to the memory device where the data to be stored is located. Of course, if the remote memory system has only one memory device, the target storage space is the remote memory space of the remote memory system. The storage parameter information includes at least the storage capacity of the target storage space and the amount of data associated with the data to be stored. Associated data such as dirty data or clean data in the same storage page as the data to be stored, and data in other storage pages related to this storage page in the target storage space. The number of read and write operations includes the number of read operations and the number of write operations, that is, from the time the remote memory system receives the data to be stored until the data to be stored is written into the target storage space, the number of data read operations and the number of data write operations involved in different data writing methods. The read and write cost includes the read cost and the write cost, which refers to the resource consumption or cost required when reading or writing data from the remote memory system, measured from multiple dimensions such as time delay, energy consumption, hardware wear, and economic cost (such as the life of the storage medium).

[0030] S203: Store the data to be stored in the remote memory system by using the target data writing method with the lowest storage cost.

[0031] After calculating the storage costs corresponding to different data writing methods in the previous step, the data writing method with the lowest storage cost is the data writing method that best matches or is optimal for the current conditions. This step is defined as the target data writing method. Using the target data writing method to write the data to be stored into the remote memory system can not only ensure the reliability of the data, but also ensure that the data is efficiently written while minimizing the impact on the storage performance of the remote storage system, that is, it can ensure that the remote storage system can maintain good memory performance throughout the process.

[0032] In the technical solution provided in this embodiment, for the data to be stored that needs to be stored on the remote memory system, the remote memory system replaces the computing node to complete the verification calculation, write cost evaluation, and write method selection of the data to be written. In this way, the read and write performance and memory performance of the remote memory system will not be limited by the distance between the computing node and the remote memory system, thereby realizing remote memory expansion in any distance scenario. From the two aspects of verification data calculation and write timing, the influence of the storage situation of the remote memory system in different scenarios and the business operation situation on the write method is reflected. By evaluating the costs of different write methods, the data write method that best matches the current storage performance can be determined, so as to ensure that the memory performance of the remote memory system reaches the optimal level in different scenarios, and the efficient data storage and read and write functions of the highly reliable remote memory can be realized.

[0033] It should be noted that there is no strict order of execution between the steps in the present invention. As long as it conforms to the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 2 This is just a schematic way and does not mean that it can only be such an execution order.

[0034] In the above embodiment, the calculation method of the verification data is not limited. In order to improve the write efficiency of the data to be written, this embodiment also provides two verification value calculation methods to achieve the efficient write of the data to be stored, which may include the following content:

[0035] As an exemplary verification value calculation method, in this method, the remote storage space of the remote storage system is composed of multiple stripes, that is, the data to be stored is stored in the stripes, the target storage space is the stripe, the data to be stored is the dirty data to be written, and each target associated data belonging to the same stripe as the dirty data to be written is obtained; the exclusive OR sum between the dirty data to be written and each target associated data is calculated as the verification value of the dirty data to be written. Among them, the target associated data is the dirty data or clean data located in the same stripe as the dirty data to be written, and the exclusive OR sum refers to the exclusive OR operation between the dirty data to be written and each target associated data. For the sake of description, the verification data determination method recorded in this embodiment can be defined as the stripe calculation verification value method, that is, the stripe calculation verification value method is to use the dirty data to be written and other data belonging to the same stripe to calculate the exclusive OR sum together as the new verification value and write it into the remote memory system together.

[0036] Another way to calculate the check value parallel to the above embodiments: According to the target memory address to which the dirty data to be written is correspondingly written into the remote memory system, read the old data stored in the target memory address at the current time and before, and the old check data of the old data from the remote memory system; calculate the difference between the dirty data to be written and the old data, and calculate the exclusive OR value of the difference and the old check data as the check value of the dirty data to be written. Wherein, the target memory address refers to the address corresponding to the storage location of the data to be stored in the remote memory system, that is, the address to be written. The old data refers to the memory data stored before the address to be written. For the convenience of description, the check data determination method described in this embodiment can be defined as the differential calculation check value method, that is, the differential calculation check value method is to use the dirty data to be written, the old memory data at the write location and the corresponding old check value data, calculate the difference at the write location and add it to the old check value to obtain a new check value, and write them into the memory together.

[0037] As can be seen from the above, this embodiment provides two ways to determine the check data, which can efficiently implement the data writing in this process, ensure the reliability of the data to be stored through the check value, and thus reduce the impact on the writing performance while ensuring the data reliability.

[0038] For the consistency of the written data and the check data, the present invention stores the corresponding check data while storing the data, which makes the memory data writing operation not only need to write the original data, that is, the data to be stored, but also need to update the relevant check data. Therefore, additional check value calculations are required. For the remote memory scenario, the distance between the computing node and the remote memory system is relatively far. The check value calculation, long-distance transmission of the check value, memory write latency and time, etc. at the computing node will all affect the I / O (Input / Output) performance of the remote memory system. Based on this, on the basis of deploying and executing the above check value calculation, cost evaluation and dynamic selection of the writing method in the remote memory system in the above embodiment, in order to further improve the read and write performance of the remote memory, a cache can also be built in the remote memory system near the remote memory node. Through the cache device, a cache space is provided, and check value calculation, data pre-reading, etc. are carried out in the cache space, which may include the following content:

[0039] Calculating the check value in the cache space may include that the strip calculation check value method may read all other relevant data of the strip where the dirty data to be written in the remote memory system is located into the cache space, calculate the check value together with the dirty data to be written, and then write the dirty data and the obtained check value back to the remote storage space of the remote memory system. The differential calculation check value method may read the corresponding old check value of the old data at the address to be written in the remote memory system into the cache space, calculate the difference between the old data at the address to be written and the dirty data to be written, add it to the old check value to obtain a new check value, and then write the dirty data and the new check value back to the remote storage space of the remote memory system.

[0040] To further improve the data writing flexibility, based on the embodiment, according to the different timings of the remote storage space of the remote memory system, the writing conditions may include the direct writing method and the group write-back method based on the cache space. For the strip calculation check value method, the process of the direct writing method may include: for the scenario of storing the dirty data to be written into a pre-constructed cache space; based on the target memory address in the remote memory system where the dirty data to be written is written, determine each target data belonging to the same strip as the dirty data to be written in the remote memory system, and read each target data into the cache space. Of course, if the target data is already in the cache space, there is no need to write it again. Calculate the first exclusive OR sum between the dirty data to be written and each target data; write the dirty data to be written to the target memory address, and write the first exclusive OR sum to the first memory check address in the remote memory system. For the scenario where the dirty data to be written is not stored in the pre-constructed cache space, based on the target memory address in the remote memory system where the dirty data to be written is written, determine each target data belonging to the same strip as the dirty data to be written in the remote memory system, calculate the first exclusive OR sum between the dirty data to be written and each target data; write the dirty data to be written to the target memory address, and write the first exclusive OR sum to the first memory check address in the remote memory system. Among them, the first memory check address refers to the storage location in the remote memory system for storing the first exclusive OR sum, that is, the check value of the data to be stored under the data writing method determined by the strip calculation check value method and the direct writing method. For example, when a remote memory write request occurs, the dirty data T to be written and all other relevant data in the same strip, that is, each target data , n represents the number of target data. If in the form of cache pages, n represents the number of mutually associated data pages in the strip. Through the relational expression calculate the first exclusive OR sum of these data as the check value P1 of the dirty data to be written. Write the dirty data T to be written to the target memory address, and write the new check value P1 to the first memory check address to complete all steps of the strip calculation check value method for writing a single dirty data back to the remote memory system.

[0041] For the method of calculating the check value of stripes, the existence of the cache allows the write-back operations of associated addresses in the same stripe to share a single check value update. The process of the group write-back method based on the cache space may include: storing the dirty data to be written into a pre-constructed cache space; when the group write-back condition is met, writing the dirty data to be written to the target memory address in the remote memory system, determining the target data in the remote memory space of the remote memory system that belongs to the same stripe as the dirty data to be written, and reading the target data into the cache space; determining the target dirty data to be written and the target clean data in the cache space that belong to the same stripe as the dirty data to be written according to the target memory address; calculating the second exclusive OR sum among the dirty data to be written, the target dirty data to be written, and the target clean data as the check value of the dirty data to be written and the target dirty data to be written; writing the dirty data to be written and the target dirty data to be written to the corresponding memory addresses in the remote memory system respectively, and writing the second target exclusive OR sum to the second memory check address in the remote memory system. Among them, the second memory check address refers to the storage location in the remote memory system for storing the second exclusive OR sum, that is, under the data writing method determined by the method of calculating the check value based on stripes and the group write-back method, the check values of the data to be stored and each target dirty data to be written. The group write-back condition may include, for example, after a preset t clock cycles, the occupancy of the cache space allocated for the write operation reaches or exceeds a preset ratio. For example, store the dirty data to be written into the cache close to the remote memory and wait until the following preset conditions are met: i. Do not wait and directly proceed to the next step (this method is the direct write-back method); ii. After a preset t clock cycles; iii. The occupancy of the cache space allocated for the write operation reaches or exceeds a preset ratio. Then, according to the target memory address of the dirty data to be written, search for the dirty data to be written and the clean data pre-read into the cache at other addresses in the same stripe in the cache space. Then search for other relevant addresses in the same stripe that are not in the cache space and read the data at these addresses from the remote memory system and store them in the cache space. Finally, according to the dirty data to be written, each target dirty data to be written and each target clean data , where d is the number of other dirty data in the cache space that belong to the same stripe as the dirty data to be written, that is, the total number of target dirty data to be written, and n represents the number of mutually associated data pages in the stripe. Based on the relational expression calculate the second exclusive OR sum of these data as the check value P2 of the dirty data to be written and each target dirty data to be written. Write the dirty data to be written and each target dirty data to be written Write to the corresponding addresses respectively, and write P2 to the second memory check address. In addition, the above write operation does not necessarily wait until the group write-back condition is satisfied to trigger subsequent steps. It may also cause the write operation of the dirty data to be written in advance due to the dirty data write operation of other related addresses, which initiates the corresponding group write-back condition.

[0042] As can be seen from the above, in this embodiment, by combining the cache space and two writing opportunities, while realizing efficient data writing by means of strip calculation of check values, the flexibility and practicality are improved through various writing methods, and the efficiency of the write operation of the remote memory system is effectively improved by means of group write-back. It is beneficial to realize the efficient data storage and read-write functions of a highly reliable remote memory, and can be directly applied to various computing system scenarios with memory expansion requirements and high reliability requirements.

[0043] Similarly, for the differential calculation check value method, the write conditions may include the direct write method and the group write-back method based on the cache space. That is, the differential calculation check value method + the direct write method constitute a data write method, and the differential calculation check value method + the group write-back method based on the cache space constitute a data write method. Among them, for the differential calculation check value method + the direct write method constituting a data write method: for the scenario of storing the dirty data to be written in a pre-constructed cache space, based on the target memory address of the dirty data to be written into the remote memory system, in the remote memory system, read the old data and the corresponding check data of the target memory address, and read the old data and the corresponding check data into the cache space. Of course, if the old data and the corresponding check data are already in the cache space, there is no need to write again. Calculate the difference between the dirty data T to be written and the old data and use the exclusive OR value of the difference and the old check value as the new check value P3. For example, the new check value can be calculated through Finally, write the dirty data to be written to the target memory address, and write P3 to the third memory check address of the remote memory system to complete the steps of the differential calculation check value method for writing back a single dirty data to the remote memory system.

[0044] For the data writing method composed of the differential calculation check value method + the group write-back method based on the cache space: store the dirty data to be written into the pre-constructed cache space; when the group write-back condition is met, write the dirty data to be written into the first target memory address of the remote memory system, and determine in the cache space the target dirty data to be written that belongs to the same stripe as the dirty data to be written, and the second target memory address corresponding to the target dirty data to be written into the remote memory system; read in the remote memory space of the remote memory system the old data stored at the first target memory address and the second target memory address at the current moment and before, and the corresponding old check data; calculate the difference between the dirty data to be written and the corresponding old data, the difference between the target dirty data to be written and the corresponding old data, and the third exclusive OR sum of the old check data, as the check value of the dirty data to be written and the target dirty data to be written; write the dirty data to be written into the first target memory address, write the target dirty data to be written into the second target memory address, and write the third exclusive OR sum into the corresponding fourth memory check address of the remote memory system.

[0045] For example, when a remote memory write request occurs, the write-back operations of related addresses in the same stripe share one check value update. To improve the write operation efficiency and reduce the write cost, the steps of the differential calculation check value method for the write-back operations of related addresses in the same stripe to the remote memory system include: A1. Store the dirty data to be written into the cache space of the cache device close to the remote memory. A2. Wait until one of the following pre-set conditions is met: after a pre-set t clock cycles; or the occupancy of the cache space allocated for the write operation reaches or exceeds the pre-set ratio. A3. According to the first target memory address of the dirty data to be written, find each target dirty data to be written of the related other addresses belonging to the same stripe located in the cache space. . A4. Read from the remote memory system the dirty data to be written T and each target dirty data to be written , the old data in the corresponding memory addresses , and read the old check value data in the corresponding check address , and store them in the high cache space. A5. According to the dirty data to be written, each target dirty data to be written and the corresponding old data , use the relational expression to calculate the new check value P4 corresponding to the dirty data to be written and each target dirty data to be written, where d represents the total number of target dirty data to be written. Finally, write the target dirty data to be written and each target dirty data to be written Write them to the corresponding addresses in the remote memory system respectively, and write the new check value P4 to the corresponding fourth memory check address. Among them, the first memory check address, the second memory check address, the third memory check address, and the fourth memory check address are used to distinguish the storage locations of the check values calculated under different scenarios. For the actual situation, when the data to be stored is written to the remote memory system, only the check value corresponding to the optimal data writing method is stored. Similarly, in A2, the write operation does not necessarily wait until the condition is met to trigger the subsequent steps. It may also be that the subsequent A4 is initiated due to the write operation of the dirty data to be written at other related addresses, resulting in the premature completion of the current write operation of the dirty data to be written.

[0046] Under different data writing methods, the data interaction between the cache space and the remote memory space can be as Figure 3 shown. In Figure 3 , for the convenience of description, the data writing method composed of the strip calculation check value method + the direct writing method is defined as Strategy A, the data writing method composed of the strip calculation check value method + the group write-back method is defined as Strategy B, the data writing method composed of the differential calculation check value method + the direct writing method is defined as Strategy C, and the data writing method composed of the differential calculation check value method + the group write-back method is defined as Strategy D. The first row of data represents all the data in the same strip, T represents the dirty data to be written, D represents other dirty data in the cache space, that is, the target dirty data to be written, C represents the clean data in the cache space, the dotted line represents the data not in the cache space, P represents the strip check data, and the upward arrow in each strategy represents the data read from the remote memory system to the cache space, and the downward arrow represents the data written from the cache space to the remote memory space of the remote memory system.

[0047] As can be seen from the above, in this embodiment, by combining the cache space and two writing opportunities, while realizing efficient data writing through the strip calculation check value method, the flexibility and practicality are improved through multiple writing methods, and the efficiency of the write operation of the remote memory system is effectively improved through the group write-back method, which is beneficial to realizing the efficient data storage and read-write functions of the highly reliable remote memory, and can be directly applied to various computing system scenarios with memory expansion requirements and high reliability requirements.

[0048] Furthermore, in order to improve the effective utilization rate of the cache space and further enhance the data reading and writing efficiency, based on the above embodiments, the cache space provided by the cache can be divided into a read operation cache space and a write operation cache space. The read operation cache space is used to support the prefetching of the remote memory system. For example, other cache lines of the cache page to which the target cache line to be read belongs can be prefetched into the read operation cache space, and other cache pages of the stripe where the cache page to which the target cache line to be read belongs can be prefetched into the read operation cache space. The write operation cache space is used to support the remote memory write-back as described in the above embodiments. For example, the data to be stored can be temporarily stored in the write operation cache space and wait for a fixed clock cycle to complete the write operation of writing the data back to the remote memory space; the data to be stored can be temporarily stored in the write operation cache space, and when the occupancy of the write operation cache space reaches or exceeds a fixed ratio, the write operation of writing the data back to the remote memory space is performed. This embodiment may include the following content:

[0049] For the data reading scenario: Due to spatial locality, subsequent data reading requests are more likely to directly hit in the cache space. In order to reduce memory addressing operations and shorten the response time of data read requests, when a data read request is received, determine the target data cache page where the data to be read corresponding to the data read request is located, and determine the target stripe to which the target data cache page belongs; prefetch and store the target data cache page and other data cache pages in the target stripe except the target data cache page into the pre-constructed read operation cache space. In this embodiment, when there is a data read request to the remote memory system, data in other memory spaces with a high spatial locality with the target cache line space can be prefetched into the cache. For example, cache lines belonging to the same page and other pages belonging to the same stripe can not only reduce memory addressing operations, effectively shorten the response time of data read requests, and thus improve the data reading speed of the remote memory system, but also, even considering eviction in the cache, the probability of data belonging to the same stripe existing in the cache is still relatively high, which is beneficial to quickly calculating the check value during the write operation.

[0050] For the data writing scenario: When a data write request is received, the data to be written corresponding to the data write request is stored in a pre-constructed write operation cache space; when the data writing condition is met, the data in the write operation cache space is written to the remote memory space; wherein, the write operation cache space and the remote memory space belong to the remote memory system. In this embodiment, the remote memory space is the actual storage location of the data to be stored, that is, the memory space of the remote memory system. When there is a data write request to the remote memory system, the data to be written is temporarily stored in the cache. When a specific condition for data writing is met, such as after a certain number of clock cycles or when the amount of data to be written reaches a certain level, the data in the write operation cache space is written to the remote memory space. In this way, for scenarios where the possibility and quantity of dirty data to be written belonging to the same stripe are relatively large, since the operation of updating the check value is shared when writing dirty data of the same stripe, the writing of multiple dirty data can share the overhead cost of check value calculation, thereby reducing the write cost. While improving the data write speed, it can also ensure the high performance of the remote storage system.

[0051] As can be seen from the above, in this embodiment, a cache hardware structure is built near the remote memory node. On the one hand, it can be used as a cache space for completing computing tasks. On the other hand, it can be used as a cache space for writing data to the remote memory system, enhancing the performance of remote memory data writing. On the third hand, it can also be used as a cache space for data prefetching, improving the data reading performance of the remote memory system, which is conducive to realizing the efficient data storage and read / write functions of a highly reliable remote memory system.

[0052] Based on the above embodiments, the remote memory system can provide better memory expansion capabilities and more flexible memory allocation and scheduling capabilities for computing nodes. However, once the remote memory system fails or data is lost, the impact on the computing nodes will be very serious. In view of this, in this embodiment, according to the principle of erasure code, an independent redundant memory array data storage method is established in the remote memory system, enabling the remote memory system to have the ability to recover or reconstruct data when a hardware failure occurs, improving the reliability of data storage, which may include the following content:

[0053] In this embodiment, the remote memory space of the remote memory system is composed of stripes. The remote memory space includes multiple stripes. The target storage space is a stripe. The process of storing the data to be stored into the corresponding stripe includes: dividing the data to be stored into multiple data blocks to be stored, and storing each data block to be stored into the corresponding first memory space; calculating the exclusive-or checksum between the data blocks to be stored, and storing the exclusive-or checksum into the second memory space; combining each first memory space and the second memory space into a stripe for storing the data to be stored. The first memory space can be the storage space provided by any kind of memory device, such as a memory module or a memory card. Each first memory space is independent of each other and independent of the second memory space. In this way, when there is a memory space failure or data loss in the memory device where one of the first memory spaces is located or the memory device where the second memory space is located, the faulty data can be recovered through the remaining normal data and the redundant checksum. Taking the first memory space and the second memory space as the memory spaces provided by DIMM (Dual-Inline-Memory-Modules), the data to be stored is divided into several parts, and each part is stored on a DIMM memory module of the remote memory system. The exclusive-or checksum between the several parts of the stored data is calculated and stored on another independent memory module, forming an "n + 1" type redundancy. The n + 1 memory modules are a stripe. Further, the check data of different stripes are evenly staggered in different memory spaces, that is, the check pages of different stripes are evenly staggered in different DIMMs, which can avoid the problem of uneven hardware wear.

[0054] As can be seen from the above, the data storage method of the striping memory redundancy mechanism constructed according to the erasure code principle in this embodiment can be effectively recovered after data loss or errors occur, improving the data reliability of the remote memory system. In addition, by evenly staggering the check data of different stripes in different memory spaces, the problem of uneven hardware wear caused by different access frequencies of the memory data read and write operations to the data cache space and the check space can be avoided.

[0055] Further, in order to achieve address continuity, the present invention also provides a striping memory redundancy mechanism based on page granularity, enabling the operating system to map continuous virtual addresses to physical data addresses and skip the check page addresses, thus maintaining the continuity of virtual addresses. It may include the following content:

[0056] Divide the data to be stored into multiple data blocks to be stored, and store each data block to be stored into the corresponding data cache page; calculate the exclusive-or checksum between the data blocks to be stored, and store the exclusive-or checksum into the check page; combine each data cache page and the check page into a stripe for storing the data to be stored; wherein, each data cache page is independent of each other and independent of the check page.

[0057] Since the mapping between physical memory addresses and virtual addresses is based on pages as the smallest unit, the striping mechanism at the page granularity enables the operating system to map consecutive virtual addresses to physical data addresses and skip the verification page addresses, thereby maintaining the continuity of virtual addresses. If striping is implemented at the cache line granularity, then for every n cache lines of virtual addresses, one cache line is required to store the verification line information. This cache line cannot be used for writing data, and the continuity of the virtual address space cannot be achieved. In this embodiment, a memory device can provide one data cache page or one verification page. The memory device can be a DIMM, that is, one verification page is constructed for the cache pages of every n different DIMMs, forming an information redundancy of n + 1. The data of the n data cache pages in the same stripe are respectively , then the verification page is obtained through exclusive OR operation (XOR) based on the data of each data cache page, that is, the data P for constructing the verification page can be obtained through . When a failure occurs in a certain DIMM resulting in data error or loss, the striping memory redundancy mechanism enables the data cache page in this DIMM to be recovered through calculation using other data pages and the verification page in the same stripe, and the verification page can be recalculated through all the data pages in the same stripe. In addition, the striping memory redundancy mechanism and the memory ECC (Error Checking and Correcting) mechanism are complementary memory fault tolerance mechanisms and can coexist. ECC is oriented towards faults such as random bit flips in a single memory and can detect and recover a small number of data bit errors; while the striping memory redundancy mechanism is oriented towards data faults and losses at the cache page level and can recover one or more data pages.

[0058] As can be seen from the above, the striping memory redundancy mechanism of this embodiment is implemented at the cache page level, which can ensure the continuity of virtual addresses. Through "n + 1" memory redundancy, the faulty data can be recovered, guaranteeing the high reliability of the remote memory system. The verification pages of different stripes are evenly staggered in different DIMMs, which can avoid the problem of uneven hardware wear caused by different access frequencies of data cache pages and verification pages during memory data read and write operations.

[0059] Based on the above embodiments, where the different data writing methods are evaluated for the writing cost based on the average number of reads and writes required for one cache line write operation, this embodiment also provides a more convenient dynamic selection method, which may include the following content:

[0060] Pre-calculate the write cost corresponding to each data write method based on the read operation count, read cost, write operation count, and write cost of each data write method, and determine the write cost of the same check data determination method under different write conditions to determine the target write condition with a low write cost; determine the target check data determination method with a low write cost according to the storage parameter information of the target storage space of the data to be stored in the remote memory system; use the data write method determined by the target check data determination method and the target write condition as the target data write method with the lowest storage cost.

[0061] In this embodiment, the number of data pages associated with the same stripe is n. When the cache space is triggered to write to the remote memory system, the number of other dirty data in the cache space that is associated with the dirty data to be written and belongs to the same stripe but does not include the dirty data itself that triggers the write mechanism, that is, the number of target dirty data to be written is d, and the number of clean data belonging to the same stripe is c. Then the number of other data in the same stripe that is not in the cache space is . Taking the data write methods composed of the stripe checksum calculation method + direct write method as Policy A, defining the data write method composed of the stripe checksum calculation method + group write-back method as Policy B, defining the differential checksum calculation method + direct write method as Policy C, and defining the differential checksum calculation method + group write-back method as Policy D as examples, the determination process of the read and write operation counts of different data write methods includes:

[0062] For Policy A, after initiating a write of dirty data to the remote memory system, except for the clean data already in the cache space, all other data under the same stripe are read from the remote memory system into the cache space, and after the checksum calculation is completed, the dirty data to be written and the updated check data are written to the remote memory system. Therefore, a total of read operations and 2 write operations are required to achieve the write of 1 dirty data. For Policy B, when there is no need to read the existing dirty data and clean data in the cache space, after initiating a write of dirty data to the remote memory system, all other data under the same stripe that is not in the cache space needs to be read from the remote memory system into the cache space, and after the checksum calculation is completed, all the dirty data under the same stripe in the cache space and the updated check data are written to the remote memory system. Therefore, a total of read operations and write operations are required to achieve Writing of one dirty data. For policy C, after initiating a write operation of dirty data to the remote memory system, it is necessary to read the old data on the dirty data bit to be written and the old data of the parity bit corresponding to the same stripe from the remote memory system. After completing the calculation of the check value, the dirty data to be written and the updated check data are written to the remote memory system. Therefore, a total of 2 read operations and 2 write operations are required, achieving the writing of 1 dirty data. For policy D, after initiating a write operation of dirty data to the remote memory system, it is necessary to read the old data on the corresponding bits of all dirty data in the cache space that belong to the same stripe as the dirty data to be written (including the dirty data itself to be written) and the old data of the parity bit corresponding to the same stripe from the remote memory system. After completing the calculation of the check value, all the dirty data in the cache space of this stripe and the updated check data are written to the remote memory system. Therefore, a total of read operations and write operations are required, achieving the writing of dirty data.

[0063] After determining the number of read and write operations for different data writing methods, the read cost corresponding to one read operation from the remote memory system to the cache space is , and the write cost corresponding to one write operation from the cache space to the remote memory system is , and the average write costs of policies A to D are respectively expressed as , representing the read and write costs of writing 1 dirty data to the remote memory system on average. The write costs of various policies can be respectively expressed as:

[0064] ;

[0065] ;

[0066] ;

[0067] .

[0068] Among them, for any , there is , therefore, when other factors are the same, whether it is the strip calculation check value method or the differential calculation check value method, the write condition of the group write-back method is better than that of the direct write-back method. In this embodiment, the target write condition is the group write-back method. When the target write condition is determined, and for the scenario where the remote memory system includes multiple strips and the target storage space is a strip, the target check data determination method can be determined according to the quantitative relationship among the number of associated data pages of the strip, the number of other dirty data associated with the dirty cache line to be written in the cache space, and the number of clean data associated with the dirty cache line to be written. Exemplarily, the number of data cache pages included in the strip to which the dirty data to be written belongs, the first number of target dirty data to be written belonging to the same strip as the dirty data to be written, and the second number of clean data belonging to the same strip as the dirty data to be written can be obtained; by comparing the first number, the second number with the number of data cache pages, the target check data determination method with low write cost is determined.

[0069] In this embodiment, the first number is d, the second number is c, and the number of data cache pages is n. By comparing with n to determine the target check data determination method: when holds, , the strip calculation check value strategy A of the group write-back is better than the differential calculation check value strategy D of the group write-back; when holds, , the differential calculation check value strategy of the group write-back is better than the strip calculation check value strategy of the group write-back; when holds, , the strip calculation check value strategy of the group write-back and the differential calculation check value strategy of the group write-back have the same efficiency.

[0070] As can be seen from the above, in this embodiment, an evaluation model for the cost of the data write method based on the write timing is first established, and then for each data write request, the check value calculation method is selected according to the storage parameter information, so that a more efficient data write method can be selected in real time, thereby realizing the dynamic selection of the data write method of the highly reliable remote memory system and improving the data write efficiency.

[0071] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. The present invention also provides a corresponding device for the data processing method, further making the method more practical. Among them, the device can be described from the perspective of functional modules and the perspective of hardware respectively. The data processing device provided by the present invention will be introduced below. The device is used to implement the data processing method provided by the present invention. For the description of the features in the corresponding embodiments of the data processing device, reference can be made to the relevant descriptions of the corresponding embodiments of the data processing method, and the device embodiments will not be elaborated one by one.

[0072] First, from the perspective of functional modules, please refer to Figure 4 , Figure 4 which is a structural diagram of the data processing device provided in this embodiment in a specific implementation manner. The device may include:

[0073] A write mode determination module 401, configured to determine multiple data write modes for storing the data to be stored in the remote memory system according to the check data determination method corresponding to the data to be stored, and / or the write condition for writing the data to be stored into the remote memory system.

[0074] A cost evaluation module 402, configured to respectively count the number of read and write operations of each data write mode, and determine the storage cost of each data write mode according to the storage parameter information of the target storage space corresponding to the data to be stored in the remote memory system, the number of read and write operations of each data write mode, and the corresponding read and write costs; wherein, the check data determination method and the write condition of different data write modes are at least one different.

[0075] A data write module 403, configured to store the data to be stored in the remote memory system by using the target data write mode with the lowest storage cost; the storage parameter information at least includes the storage capacity of the target storage space and the data volume associated with the data to be stored.

[0076] Exemplarily, in some embodiments of this embodiment, the above write mode determination module 401 may also be configured to: obtain each target associated data belonging to the same stripe as the dirty data to be written; calculate the exclusive OR sum between the dirty data to be written and each target associated data as the check value of the dirty data to be written.

[0077] In an exemplary implementation of the above embodiment, the write method determination module 401 may further be configured to: store the dirty data to be written into a pre-constructed cache space; based on the target memory address in the remote memory system where the dirty data to be written is written, determine, in the remote memory system, each target data belonging to the same stripe as the dirty data to be written, and read each target data into the cache space; calculate the first exclusive OR sum between the dirty data to be written and each target data; write the dirty data to be written to the target memory address, and write the first exclusive OR sum to the first memory check address in the remote memory system.

[0078] In another exemplary implementation of the above embodiment, the write method determination module 401 may further be configured to: store the dirty data to be written into a pre-constructed cache space; when the group write-back condition is met, based on the target memory address in the remote memory system where the dirty data to be written is written, determine, in the remote memory space of the remote memory system, the target data belonging to the same stripe as the dirty data to be written, and read the target data into the cache space; based on the target memory address, determine, in the cache space, the target dirty data to be written and the target clean data belonging to the same stripe as the dirty data to be written; calculate the second exclusive OR sum between the dirty data to be written, the target dirty data to be written, and the target clean data, as the check value of the dirty data to be written and the target dirty data to be written; write the dirty data to be written and the target dirty data to be written to the corresponding memory addresses in the remote memory system respectively, and write the second target exclusive OR sum to the second memory check address in the remote memory system.

[0079] Exemplarily, in some other implementations of this embodiment, the write method determination module 401 may further be configured to: based on the target memory address corresponding to the dirty data to be written in the remote memory system, read, from the remote memory system, the old data stored at the target memory address at the current moment and before, and the old check data of the old data; calculate the difference between the dirty data to be written and the old data, and calculate the exclusive OR value between the difference and the old check data, as the check value of the dirty data to be written.

[0080] Exemplarily, in some other implementations of this embodiment, the apparatus may further include a storage module, which may be configured to: the remote memory system includes multiple stripes, and the process of storing the data to be stored in the target storage space, which is a stripe, includes: dividing the data to be stored into multiple data blocks to be stored, and storing each data block to be stored in the corresponding first memory space respectively; calculating the exclusive OR check sum between each data block to be stored, and storing the exclusive OR check sum in the second memory space; combining each first memory space and the second memory space into a stripe for storing the data to be stored; where each first memory space is independent of each other and independent of the second memory space.

[0081] Exemplarily, in some other embodiments of this embodiment, the above device may further include a storage module, which can be used for: the remote memory system includes multiple stripes, and the process of storing the data to be stored into the corresponding stripes includes: dividing the data to be stored into multiple data blocks to be stored, and storing each data block to be stored into the corresponding data cache page respectively; calculating the exclusive OR checksum between each data block to be stored, and storing the exclusive OR checksum into the checksum page; combining each data cache page and the checksum page into a stripe for storing the data to be stored; wherein, each data cache page is independent of each other and independent of the checksum page.

[0082] Exemplarily, in some other embodiments of this embodiment, the above device may further include a read processing module, which can be used for: when receiving a data read request, determining the target data cache page where the data to be read corresponding to the data read request is located, and determining the target stripe to which the target data cache page belongs; pre-reading and storing the target data cache page and other data cache pages in the target stripe except the target data cache page into a pre-constructed read operation cache space.

[0083] Exemplarily, in some other embodiments of this embodiment, the above device may further include a write processing module, which can be used for: when receiving a data write request, storing the data to be written corresponding to the data write request into a pre-constructed write operation cache space; when the data write condition is satisfied, writing the data in the write operation cache space into the remote memory space; wherein, the write operation cache space and the remote memory space belong to the remote memory system.

[0084] Exemplarily, in some other embodiments of this embodiment, the above cost evaluation module 402 may further be used for: calculating the write cost corresponding to each data write method in advance according to the number of read operations and read cost, and the number of write operations and write cost of each data write method, and determining the target write condition with a low write cost according to the write cost of the same check data determination method under different write conditions; determining the target check data determination method with a low write cost according to the storage parameter information of the target storage space of the data to be stored in the remote memory system; taking the data write method determined by the target check data determination method and the target write condition as the target data write method with the lowest storage cost.

[0085] As an exemplary embodiment of the above embodiment, the above cost evaluation module 402 may further be used for: the remote memory system includes multiple stripes, the target storage space is a stripe, obtaining the number of data cache pages included in the stripe where the dirty data to be written is located, the first number of target dirty data to be written in the same stripe as the dirty data to be written, and the second number of clean data in the same stripe as the dirty data to be written; determining the target check data determination method with a low write cost by comparing the first number, the second number with the number of data cache pages.

[0086] The data processing device mentioned above is described from the perspective of functional modules. Further, the present invention also provides an electronic device, which is described from the perspective of hardware. The electronic device includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned data processing method embodiments.

[0087] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any one of the above-mentioned data processing method embodiments when running.

[0088] In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disc and other media that can store computer programs.

[0089] An embodiment of the present application also provides a computer program product. The above-mentioned computer program product includes a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned data processing method embodiments are implemented.

[0090] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned data processing method embodiments are implemented.

[0091] Finally, the present invention also provides a remote memory system. Please refer to Figure 5 , the remote memory system 12 may at least include a remote memory 123, a buffer 124, a read-write processor 125, and a program memory 126; the remote memory 123 is connected to the buffer 124, and the read-write processor 125 is connected to the buffer 124 and the program memory 126.

[0092] Among them, the remote memory storage 123 includes at least one stripe, and the stripe is used to store the corresponding data to be stored. When storing data, the stripe can refer to the striped memory redundancy method recorded in the above embodiment. The cache 124 is the memory space of the read-write processor 125. It can use any high-speed cache device to realize the memory function when calculating the check value recorded in the above method embodiment, and the function of improving the read and write performance of the remote memory by supporting specific read and write strategies. Further, the cache 124 can be divided into a read operation cache space and / or a write operation cache space, and the occupied capacity of the read operation cache space and the write operation cache space is determined according to the current application requirements and load characteristics. In other words, the cache space provided by the cache 124 can be set to a ratio of read and write space, and a part of the cache space is allocated to the read operation, which is defined as the read operation cache space, and the other part of the cache space is allocated to the write operation, which is defined as the write operation cache space, which is used to support specific read strategies or write strategies respectively. Generally speaking, for applications with frequent read operations, more cache can be allocated to read operations, and for applications with frequent write operations, more cache can be allocated to write operations. The setting of this ratio needs to be determined according to specific application requirements and load characteristics. Among them, the read strategy includes: when reading data, pre-reading data in other memory spaces with high locality with the target cache line space currently being read into the cache space, such as cache lines belonging to the same page and other pages belonging to the same stripe. The write strategy includes: when writing data, temporarily storing the data to be written in the cache space, and when a specific condition is met, such as after a certain clock cycle, or when the amount of data to be written reaches a certain level, then writing the data in the cache space to the remote memory storage 123, multiple dirty data of the same stripe are written to share the update check value operation, and the overhead cost of the check value calculation is amortized. The program memory 126 is used to store a computer program that implements the steps of the data processing method recorded in the above embodiment, which can be a read-only memory. The read-write processor 125, for example, can select a CPU with relatively general performance, which is used to implement the steps of the data processing method recorded in the above embodiment when executing the computer program stored in the program memory 126, such as completing the check value calculation, logical judgment, addressing and other operations of the above-mentioned striped memory redundancy mechanism.

[0093] In order to make those skilled in the art more clearly understand the technical solution of the present invention, the present invention also provides a remote memory system in an exemplary embodiment, such as Figure 6As shown, the remote memory system serves as the remote memory resource of the computing node, and there is data interaction with the computing node through the network. The computing node can be a general-purpose server or an artificial intelligence server, and is used to complete corresponding computing tasks based on the data read from the remote memory system or write data to the remote storage system. The remote memory system 12 may include a remote memory storage 123, a buffer 124, a read-write processor 125, and a program memory 126. The remote memory storage 123 adopts a data storage method of striping memory redundancy mechanism and serves as the memory space for the remote memory system 12 to store data. Among them, the data block marked with "p" represents the redundant data block. The buffer 124 adopts a cache, and the program memory 126 adopts a ROM (Read-Only Memory), and the corresponding read-only space stores computer program codes for implementing redundant data calculation, verification, and data writing. The read-write processor 125 completes the redundant data calculation task, verification task, cost evaluation task, and dynamic selection task of the data writing method of the remote memory system. Based on Figure 6 Taking the strip of the remote memory storage 123 shown, which includes 7 data cache pages (which can be represented as page0 - page6) and 1 parity page (which can be represented as page7) as an example, the size of each page is 4K and they are located on different memory sticks respectively. Then the strip can store 28K of valid data and 4K of redundant data. Indicates the dirty cache line data in the cache space, which is derived from the data to be written sent by the server. Indicates the clean cache line data in the cache space, which is derived from the data read by the server from the remote memory system 12. Indicates the calculated parity data. Indicates the parity data stored in the remote memory system 12. The group write-back condition is that the occupied proportion of the cache space for write operation exceeds 50%. The data storage and read-write implementation process of the remote memory system may include:

[0094] The server writes data to the 3rd cache line of the first data cache page page0 of the remote memory system, that is, the dirty data to be written can be represented as the dirty cache line. The remote memory system stores in the cache space of the buffer 124 and sends an acknowledgment message to the server. When the occupied proportion of the write operation cache space is less than 50%, the buffer 124 does not write to the strip of the remote memory system. During the waiting period, the server writes the dirty cache line and to the remote memory system. The corresponding target memory address is the 3rd cache line of the third data cache page page3. The corresponding target memory address is the 4th cache line of the fourth data cache page, page4. Meanwhile, the server also reads the cache line data of from the remote memory system. The remote memory system reads the content of the entire cache page into the cache space. When the occupancy ratio of the write operation cache space is greater than or equal to 50%, the remote memory system continues to process the and uncompleted write operations. According to the request order, it first processes the write operation. The data associated with in the stripe where is located can be expressed as . For , the first number of the subscript of each letter corresponds to the identifier of the data cache page, and the second number represents the line number in the data cache page where it is located. Among them, exists in the cache space, and the corresponding dirty data to be written also exists in the cache space. Therefore, the number of other dirty data associated with in the cache space is , the number of clean data belonging to the same stripe is , and the number of stripe data pages is . Correspondingly, , so the check value is calculated by the conditional calculation check value method and written using the group write-back method. That is, the currently dynamically selected data write method is the data write method composed of the stripe calculation check value method + the group write-back method. Read the data from the remote memory system into the cache space, calculate the check cache line , and then write the dirty cache line , and the check cache line into the remote memory system. At this time, has been written into the remote memory system, and continue to process the write operation: The data associated with in the stripe where is located can be expressed as . The first number of the subscript of each letter corresponds to the identifier of the data cache page, and the second number represents the line number in the data cache page where it is located. Among them, exists in the cache space. Therefore, at this time, the number of other dirty data associated with in the cache space is , the number of clean data belonging to the same stripe is , and the number of stripe data pages is . , so the method of calculating the check value by difference is adopted. Considering that , the cost of the write-back group mode and the direct write-back mode is the same. Therefore, the currently dynamically selected data write mode is the data write mode composed of the method of calculating the check value by difference + the write-back group mode or the data write mode composed of the method of calculating the check value by difference + the direct mode. Read data from the remote memory system and to the cache space, calculate the check cache line , and then write the dirty cache line and the check cache line into the remote memory system.

[0095] As can be seen from the above, the remote memory system provides multiple data write modes based on the register, and through the dynamic selection of different data write modes, it can achieve the efficient data storage and read / write functions of a highly reliable remote memory system, and can be applied to the computing system scenarios with memory expansion requirements and high reliability requirements.

[0096] The above has introduced in detail a data processing method, an electronic device, a computer-readable storage medium, a computer program product and a remote memory system provided by the present invention. Each embodiment in this specification is described in a progressive manner, and the key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts between each embodiment can be referred to each other. Whether the units and algorithm steps of each example described in the disclosed embodiments are executed in the form of electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A data processing method, characterized in that, Applied to a remote memory system, including: Pre-calculate the write cost corresponding to each data writing method according to the read operation times and read cost, and write operation times and write cost of each data writing method; Determine multiple data writing methods for storing the data to be stored in the remote memory system according to the check data determination method corresponding to the data to be stored, and / or the write condition for writing the data to be stored into the remote memory system; the remote memory system includes multiple stripes; Determine the target write condition with a low write cost according to the write cost of the same check data determination method under different write conditions; Obtain the number of data cache pages included in the stripe to which the dirty data to be written belongs, the first quantity of target dirty data to be written that belongs to the same stripe as the dirty data to be written, and the second quantity of clean data that belongs to the same stripe as the dirty data to be written; determine the target check data determination method with a low write cost by comparing the first quantity, the second quantity with the number of data cache pages; use the data writing method determined by the target check data determination method and the target write condition as the target data writing method with the lowest storage cost; Use the target data writing method with the lowest storage cost to store the data to be stored in the remote memory system; Among them, at least one of the check data determination method and the write condition of different data writing methods is different, and the storage parameter information at least includes the storage capacity of the target storage space and the data volume associated with the data to be stored.

2. The data processing method according to claim 1, wherein According to the check data determination method corresponding to the data to be stored, including: Obtain each target associated data that belongs to the same stripe as the dirty data to be written; Calculate the exclusive OR sum between the dirty data to be written and each target associated data as the check value of the dirty data to be written.

3. The data processing method according to claim 2, wherein Calculating the exclusive OR sum between the dirty data to be written and each target associated data includes: Store the dirty data to be written into a pre-constructed cache space; Based on the target memory address where the dirty data to be written is written into the remote memory system, in the remote memory system, determine each target data that belongs to the same stripe as the dirty data to be written, and read each target data into the cache space; Calculate the first exclusive OR sum between the dirty data to be written and each target data; Write the dirty data to be written to the target memory address, and write the first exclusive OR sum to the first memory check address of the remote memory system.

4. The data processing method according to claim 2, wherein Calculating the exclusive OR sum between the dirty data to be written and each target associated data includes: Store the dirty data to be written into a pre-constructed cache space; When the group write-back condition is satisfied, based on the target memory address where the dirty data to be written is written into the remote memory system, determine the target data that belongs to the same stripe as the dirty data to be written in the remote memory space of the remote memory system, and read the target data into the cache space; Determine the target dirty data to be written and target clean data that belong to the same stripe as the dirty data to be written in the cache space according to the target memory address; Calculate the second exclusive OR sum between the dirty data to be written, the target dirty data to be written, and the target clean data, and use it as the check value of the dirty data to be written and the target dirty data to be written; Write the dirty data to be written and the target dirty data to be written to the corresponding memory addresses of the remote memory system respectively, and write the second target exclusive OR sum to the second memory check address of the remote memory system.

5. The data processing method according to claim 1, wherein According to the check data determination method corresponding to the data to be stored, including: According to the target memory address where the dirty data to be written is correspondingly written to the remote memory system, read the old data stored in the target memory address at the current time and before, and the old check data of the old data, from the remote memory system; Calculate the difference between the dirty data to be written and the old data, and calculate the exclusive OR value of the difference and the old check data, as the check value of the dirty data to be written.

6. The data processing method according to claim 1, characterized in that The remote memory system includes multiple stripes, the target storage space is a stripe, and the process of storing the data to be stored to the corresponding stripe includes: Divide the data to be stored into multiple data blocks to be stored, and store each data block to be stored in the corresponding first memory space respectively; Calculate the exclusive OR checksum between each data block to be stored, and store the exclusive OR checksum in the second memory space; Combine each first memory space and the second memory space into a stripe for storing the data to be stored; Among them, each first memory space is independent of each other, and is independent of the second memory space.

7. The data processing method according to claim 1, wherein The remote memory system includes multiple stripes, and the process of storing the data to be stored to the corresponding stripe includes: Divide the data to be stored into multiple data blocks to be stored, and store each data block to be stored in the corresponding data cache page respectively; Calculate the exclusive OR checksum between each data block to be stored, and store the exclusive OR checksum in the check page; Combine each data cache page and the check page into a stripe for storing the data to be stored; Among them, each data cache page is independent of each other, and is independent of the check page.

8. The data processing method according to claim 1, wherein Further include: When receiving a data read request, determine the target data cache page where the data to be read corresponding to the data read request is located, and determine the target stripe to which the target data cache page belongs; Pre-read and store the target data cache page and other data cache pages in the target stripe except the target data cache page into a pre-constructed read operation cache space.

9. The data processing method according to claim 1, wherein Further include: When receiving a data write request, store the data to be written corresponding to the data write request in a pre-constructed write operation cache space; When the data write condition is met, write the data in the write operation cache space to the remote memory space; wherein, the write operation cache space and the remote memory space belong to the remote memory system.

10. An electronic device, characterized in that, Include: A memory for storing a computer program; A processor for implementing the steps of the data processing method according to any one of claims 1 to 9 when executing the computer program.

11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

13. A remote memory system, characterized in that, It includes at least a remote memory, a buffer, a read / write processor, and a program memory; The remote memory is connected to the buffer, and the read / write processor is connected to the buffer and the program memory; The remote memory includes at least one stripe for storing corresponding data to be stored; the buffer serves as the memory space of the read / write processor and includes a read operation buffer space and / or a write operation buffer space, and the occupied capacities of the read operation buffer space and the write operation buffer space are determined according to current application requirements and load characteristics; the read / write processor is configured to implement the steps of the data processing method according to any one of claims 1 to 9 when executing the computer program stored in the program memory.

Images