Fault-tolerant processing method and device for memory and electronic equipment
By employing distributed data backup and error correction code technology in persistent memory media, the problem of poor fault tolerance of persistent memory media is solved, achieving efficient fault tolerance processing, reducing latency and power consumption, and improving system reliability and performance.
Patent Information
- Application Number
- CN202410961238.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2026-01-27
AI Technical Summary
Persistent memory media have poor fault tolerance during use, resulting in high failure rates, long read latency, increased cost and power consumption, and affecting the QoS of computing systems.
The target data is backed up in a distributed manner across multiple memory modules. By decoding with an error correction code module and designing redundant data, abnormal memory modules are marked, and data is read using memory modules that are not marked as abnormal, thus achieving multiple backups and fault tolerance processing of data.
It improves the fault tolerance of the memory module, reduces read latency and power consumption, reduces media wear, simplifies the design, reduces costs, and ensures QoS controllability and data reliability.
Smart Images
Figure CN121412005A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of memory technology, and in particular to methods, apparatus and electronic devices for fault-tolerant memory processing. Background Technology
[0002] Persistent memory media offers random data access and non-volatility, with access latency significantly lower than flash memory and approaching that of traditional DRAM (Dynamic Random Access Memory). Therefore, persistent memory media has the potential to optimize computing and storage performance.
[0003] To meet the demands of various applications, the capacity and throughput of persistent memory media are continuously improving, along with production yield and media stability. However, compared to DRAM, the failure rate of persistent memory media still poses a challenge in practical applications. This increased failure rate can be attributed to the inherent limitations of persistent memory media in data retention and resistance to environmental factors. Therefore, to apply persistent memory media to cloud services, its fault tolerance needs to be enhanced. However, currently, persistent memory media suffers from poor fault tolerance in use. Summary of the Invention
[0004] This application provides various methods, apparatuses, and electronic devices for handling fault tolerance in memory, in order to solve the problem of poor fault tolerance in current persistent memory media during use.
[0005] The first aspect of this application provides a memory fault tolerance processing method applied to a distributed processing system. The distributed processing system includes a storage cluster, which includes multiple storage nodes. Each storage node includes a memory module and a storage component. The memory fault tolerance processing method includes: reading target data from any one of the multiple memory modules, wherein the target data is stored in all of the multiple memory modules; determining whether the target data was successfully read; if not, marking the memory module currently reading the target data as an abnormal memory module; and reading the target data from any memory module that is not marked as abnormal.
[0006] A second aspect of this application provides a distributed processing system, including: at least one cloud server and a storage cluster, the storage cluster including multiple storage nodes, each storage node including a memory module and a storage component, the cloud server being configured to: read target data from any one of the multiple memory modules, wherein the multiple memory modules all store the target data; determine whether the target data was successfully read; if not, mark the memory module currently reading the target data as an abnormal memory module; and read the target data from any memory module among the multiple memory modules that was not marked as abnormal.
[0007] A third aspect of this application provides a memory fault-tolerant processing device applied to a distributed processing system. The distributed processing system includes a storage cluster, which includes multiple storage nodes. Each storage node includes a memory module and a storage component. The memory fault-tolerant processing device includes:
[0008] The read module is used to read target data from any one of multiple memory modules, wherein the target data is stored in multiple memory modules.
[0009] The determination module is used to determine whether the target data has been successfully read.
[0010] The marking module is used to mark the memory module currently reading the target data as an abnormal memory module if no exception is made.
[0011] The read module is also used to read target data from any memory module that is not marked as abnormal among multiple memory modules.
[0012] A fourth aspect of this application provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the memory fault-tolerant processing method of the first aspect.
[0013] The fifth aspect of this application provides a computer program product storing a computer program. When the computer program is executed by a processor, it causes the processor to implement the memory fault-tolerant processing method as described in the first aspect.
[0014] This application applies to a distributed processing system. The distributed processing system includes a storage cluster, which includes multiple storage nodes. Each storage node includes a memory module and a storage component. The memory fault tolerance method includes: reading target data from any one of the multiple memory modules, wherein the target data is stored in all multiple memory modules; determining whether the target data was successfully read; if not, marking the memory module currently reading the target data as an abnormal memory module; and reading the target data from any memory module that is not marked as abnormal. This application backs up the target data in multiple memory modules, so that if one memory module fails, the target data can be read from other memory modules, thereby improving the fault tolerance capability of the memory modules. Attached Figure Description
[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0016] Figure 1A schematic diagram illustrating the memory usage process for related technologies;
[0017] Figure 2 An application scenario diagram provided for an exemplary embodiment of this application;
[0018] Figure 3 A flowchart illustrating the steps of a memory fault-tolerant processing method provided for an exemplary embodiment of this application;
[0019] Figure 4 A flowchart illustrating the steps of another memory fault-tolerant processing method provided for an exemplary embodiment of this application;
[0020] Figure 5 A schematic diagram of a memory module provided for an exemplary embodiment of this application;
[0021] Figure 6 A schematic diagram of a storage medium provided for an exemplary embodiment of this application;
[0022] Figure 7 A structural block diagram of a memory fault-tolerant processing device provided for an exemplary embodiment of this application;
[0023] Figure 8 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0025] Reference Figure 1 The electronic device 10 provided for the related technology includes a memory fault-tolerant design. The electronic device 10 includes: a CPU (Central Processing Unit) 101, a network interface card (NIC) 102, other devices 103, and a memory module 104. The memory module 104 includes: a persistent memory controller and multiple storage media chips. The persistent memory controller includes at least one ECC (Error Correcting Code) module (e.g., ...). Figure 1 ECC modules e1 and e2) and at least one RAID (redundant array of independent disks) module (such as...) Figure 1(RAID modules r1 and r2). Multiple storage media particles store ECC codewords and RAID redundancy data. Furthermore, due to... Figure 1 The provided technologies employ only one persistent memory controller, resulting in a large number of bytes for ECC codewords (e.g., 256 bytes), exceeding the memory access size (e.g., 64 bytes). This leads to read amplification and write amplification, accelerates media wear, and shortens media lifespan. Furthermore, the large capacity required to store ECC codewords and RAID redundancy increases the cost and power consumption of the memory module. Additionally, in the event of a high failure rate and / or storage media chip failure, the time required to restore data from failed storage media is lengthy, increasing read latency and impacting the QoS (quality of service) of the processing system.
[0026] To address the aforementioned issues, this application employs a distributed approach, backing up the target data across multiple memory modules. This allows the target data to be read from other memory modules in the event of an anomaly in one module, reducing read latency. Furthermore, by distributing the memory modules, this application enhances their fault tolerance.
[0027] In this application embodiment, the execution device of the memory fault tolerance method is not limited. Optionally, the memory fault tolerance method can be implemented by a server, and this application can also implement the overall memory fault tolerance method with the help of a cloud computing system. For example, the memory fault tolerance method can be applied to a cloud server to run various models by taking advantage of the resources on the cloud; instead of applying it to the cloud, the memory fault tolerance method can also be applied to server-side devices such as conventional servers, cloud servers, or server arrays.
[0028] In addition, refer to Figure 2 This is an application scenario diagram of this application. The application scenario diagram includes a distributed storage processing system, which comprises multiple servers (such as...). Figure 2 The system consists of servers a1 to an, where n is a positive integer, and a storage cluster. These servers can be cloud servers. The storage cluster includes multiple storage nodes (e.g., ...). Figure 2 Storage nodes bi range from bi1 to bim, where m is a positive integer. Storage node bi includes memory module bi1 and storage component bi2, where i is an integer from 1 to m.
[0029] Figure 2 This is merely one example of an application scenario exemplified by this application. This application can also be applied to other related scenarios, which are not limited here.
[0030] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0031] Figure 3 This is a flowchart illustrating the steps of a memory fault-tolerance method provided as an exemplary embodiment of this application. This memory fault-tolerance method can be used for... Figure 1 The distributed processing system shown, such as Figure 3 The fault tolerance handling method for this memory, as shown, specifically includes the following steps:
[0032] S301 reads target data from any of the multiple memory modules.
[0033] The target data is stored in multiple memory modules. In this embodiment, the memory modules are persistent memory modules, such as PCM (Phase Change Memory), ReRAM (Resistive Random Access Memory), MRAM (Magneto resistive Random Access Memory), STT-RAM (Spin Transfer Torque-Magnetic Random Access Memory), and FRAM (ferroelectric Random Access Memory).
[0034] Reference Figure 2 A storage cluster corresponds to multiple servers, and any server can read data from the storage cluster or write data to the storage cluster.
[0035] In one embodiment, if writing data, before reading the target data from any of the multiple memory modules, the method further includes: responding to a received write request for the target data and writing the target data into the multiple memory modules respectively. Specifically, when the server receives a write request for the target data, it writes the target data into the memory modules using a distributed backup method. For example, refer to... Figure 2 The target data is written to memory module b11, then to memory module b21, and so on, until the target data is written to memory module bm1.
[0036] In another embodiment, if the data is being read, reading the target data from any one of the multiple memory modules includes: responding to a received read request for the target data and reading the target data from any one of the multiple memory modules. Specifically, when the server receives a read request, it can read the target data from any one of the memory modules. Each memory module stores the target data.
[0037] S302, determine whether the target data was successfully read.
[0038] If the target data is successfully read, it is then stored in multiple storage components.
[0039] Specifically, in the embodiments of this application, the storage component is such as SSD (Solid State Disk) or HDD (Hard Disk Drive).
[0040] In this system, the server pre-stores the target data on each storage component, achieving multiple backups of the target data. For example, refer to... Figure 2 The target data is stored in storage components b12, b22, ..., bm2. This enables the writing or reading of the target data.
[0041] Furthermore, in the case of a write process, after the target data is successfully written to the storage component, the target data in each memory module is deleted, thereby releasing the corresponding space in the memory module.
[0042] If not, proceed with S303.
[0043] Specifically, a CRC (Cyclic redundancy check) is performed on the read target data. If the CRC check passes, it is determined that the target data has been successfully read; if the CRC check fails, it is determined that the target data has not been successfully read.
[0044] In the embodiments of this application, other methods may also be used to determine whether the target data has been successfully read, which are not limited here.
[0045] S303, mark the memory module currently reading target data as an abnormal memory module.
[0046] Among them, reference Figure 3 If in S301, if the target data is read from memory module b11 and the read fails, then memory module b11 is marked as an abnormal memory module.
[0047] S304: Read target data from any memory module that is not marked as abnormal among multiple memory modules.
[0048] Among them, reference Figure 2 If memory modules b11 and b21 are both marked as abnormal, then the target data is read from one of the memory modules b31 to bm1 that are not marked as abnormal, such as b31.
[0049] Furthermore, S302 is executed after S304 to implement a loop for reading the target data until the target data is successfully read.
[0050] Among them, based on Figure 1 After receiving a read request, the CPU's ECC module decodes the raw data. If decoding fails, it reads parameters such as the threshold voltage and read window, and rereads the raw data from each storage medium chip for decoding. This process is repeated multiple times in an attempt to obtain successfully decoded data. If, after multiple repetitions, successfully decoded data is still not obtained, the RAID module is activated. This requires reading multiple copies of data corresponding to a RAID redundant data set from multiple storage medium chips, and then calculating the required data by accumulating the data bit by bit. This method requires repeated data reading, repeated decoding, and reading multiple copies of data, resulting in excessively long operation times and impacting QoS. Furthermore, the multiple copies of data read from multiple storage medium chips all depend on successful decoding of the error correction code. If any one or more copies of the data are corrupted, the RAID redundant data becomes invalid.
[0051] In this embodiment, read requests attempt to read data sequentially across m storage nodes. The time consumed for each read and decode operation on each storage node is determined, and the overall time consumption is ensured to be within a known range. Therefore, each read request has a clear expected return time, making QoS controllable. Furthermore, the probability of data read errors can be controlled within a satisfactory range. For example, if there is only one storage node, the probability of a single storage node being available due to failure is Pa, and the failure probability of a single storage node is Pe = 1 - Pa. If there are m storage nodes, the probability that data cannot be read correctly is PeN = (1 - Pa)N, and the availability of the storage cluster in this application is PaN = 1 - PeN = 1 - (1 - Pa)N. Since PaN is greater than Pe, the distributed processing system can ensure data quality while also providing the advantage of quickly returning the decoding status, avoiding the high latency, high power consumption, and high media wear incurred by traditional memory repeatedly attempting to read data.
[0052] In this embodiment of the application, after it is determined that the target data in a memory module has failed to be read, the backup target data can be read immediately from other available memory modules. This avoids the long-tailed latency caused by repeated rereading and retrying when there is only one memory module, and further avoids the problem of poor fault tolerance of memory modules during use.
[0053] Figure 4 A flowchart illustrating the steps of another memory fault-tolerant processing method provided for an exemplary embodiment of this application. Figure 4 The fault tolerance handling method for this memory, as shown, specifically includes the following steps:
[0054] S401, read raw data from at least one first storage medium particle of any of the plurality of memory modules.
[0055] Among them, reference Figure 5 The memory module includes a memory controller and a storage medium. The memory controller includes an error correction code module, and there can be multiple error correction code modules, such as... Figure 5 The storage medium includes ECC modules c1 and c2. Further, the storage medium includes at least one first storage medium particle; that is, the storage medium may include one or more first storage medium particles. When multiple first storage medium particles are included, such as... Figure 5 The first storage medium particles d1 to dx are in the first storage medium particles, where x is a positive integer.
[0056] In this embodiment of the application, the original data is stored in at least one first storage medium particle.
[0057] S402 uses an error correction code module to decode the original data to obtain the target data.
[0058] In this embodiment of the application, the raw data read from the storage medium needs to be decoded by the error correction code module in order to obtain the target data.
[0059] Furthermore, when storing target data in a storage medium, the error correction code module is required to encode the target data before storage.
[0060] S403, compare the original data and the target data to determine the physical location of the defective region in the first storage medium particle.
[0061] Reference Figure 6 The first storage medium particle includes a nominal capacity area and a redundant capacity area. Further, multiple nominal capacity areas of multiple storage medium particles constitute the nominal capacity presented externally. The nominal capacity area is used to store the original data.
[0062] In this embodiment of the application, after the original data is successfully decoded by the error correction code module, the target data is obtained. The target data and the original data are compared bit by bit at the bit level to determine the bits that do not match. The defective region of the first storage medium particle where the bit is located is determined, and then the physical location of the defective region is determined.
[0063] For example, refer to Figure 6 After performing a bit-by-bit comparison between the original data and the target data, it was determined that the comparison between bytes 85 and 100 was inconsistent. Then, the area where bytes 85 to 100 of the original data were located in the storage medium was determined to be the defect area, and then the physical address of the defect area was determined.
[0064] S404, determine the target error correction codeword where the defect area is located based on its physical location.
[0065] Among them, reference Figure 6 The target error correction codeword is stored in at least a portion of the nominal capacity area, which includes the defect area.
[0066] S405, mark at least a portion of the nominal capacity area containing the target error correction codeword as an abnormal area.
[0067] For example, if an ECC codeword is 80 bytes, then at least a portion of the nominal capacity area occupying 80 bytes is an abnormal area.
[0068] The abnormal region indicates that it is not used for data storage. Specifically, when data needs to be written to the storage medium again, data will not be written to the abnormal region. When data needs to be read from the storage medium again, the corresponding data (such as ECC codewords) will be read from at least a portion of the redundant capacity region that replaces the abnormal region.
[0069] S406, store the target error correction codeword in at least a partially redundant capacity area.
[0070] Specifically, at least a portion of the redundant capacity area is designated within the redundant capacity area for storing the target error correction code word, wherein the storage size of at least a portion of the redundant capacity area matches the byte size of the target error correction code word.
[0071] In the embodiments of this application, at least a portion of the redundant capacity region and the abnormal region belong to the same first storage medium particle.
[0072] S407, determine the physical address of at least a portion of the redundant capacity area.
[0073] In this embodiment, each part of the redundant capacity region has a corresponding physical address.
[0074] S408, update the physical address of the error correction codeword in the preset mapping table.
[0075] In this embodiment, a preset mapping table is used to indicate the target error correction codeword to be read according to the physical address. Then, the next time data is read from the memory module, the target error correction codeword is read from at least a portion of the redundant capacity area according to the physical address in the preset mapping table.
[0076] Specifically, the ECC codewords stored in the abnormal region will be migrated and stored in at least a partially redundant capacity region, as per [reference]. Figure 6 An abnormal region f1 contains an ECC codeword, which is then migrated and stored in at least a partially redundant capacity region g1.
[0077] The size of the target error correction codeword in bytes is the same as the size of a single access to the memory module in bytes. For example, both the size of a single access to the memory module in bytes and the size of the target error correction codeword in bytes are 64 bytes. This avoids the situation in related technologies where the size of a single access to the memory module in bytes (e.g., 64 bytes) is smaller than the size of the target error correction codeword in bytes (e.g., 256 bytes). In such cases, when reading data from the memory module, it would be necessary to read data four times (each time reading 64 bytes) and then concatenate them (to form 256 bytes) before returning, resulting in a delay.
[0078] Furthermore, since the embodiments of this application employ distributed storage with multiple memory modules, the byte size of the target error correction codeword can be the same as the byte size of a single access to the memory module.
[0079] This application defines at least one first storage medium particle (e.g., a target error correction codeword) where a target error correction codeword is located. Figure 6 The first storage medium particle d1 and the second storage medium particle d2 in the image are the defect replacement column, and each row (abnormal area) of the defect replacement column can store a target error correction code word, such as... Figure 6 In the middle, the target error correction code codeword occupies the abnormal regions f1 and f2 of the first storage medium particle. A row is selected from the redundant capacity region in the same defect replacement column as at least a partial redundant capacity region, such as... Figure 6 The system comprises at least a partially redundant capacity region g1 and an execution partially redundant capacity region g2. The target error correction codeword for the abnormal region is written into the partially redundant capacity region. Simultaneously, the physical address of the partially redundant capacity region is updated in a preset mapping table (L2P mapping table), marking the abnormal region so that it will not be used again in subsequent data reads and writes. Based on the updated preset mapping table, subsequent accesses to the logical block address of the target error correction codeword will read data from the updated physical address.
[0080] In the L2P mapping table, the index is LBA (Logical Block Address), and the memory is the physical address of the LBA in the memory module.
[0081] In this embodiment of the application, it is further included: if it is determined that the first storage medium particle has failed, then the first storage medium particle is replaced with a second storage medium particle.
[0082] The memory module further includes a second storage medium chip. In this embodiment, the second storage medium chip is a redundant chip used to replace a failed first storage medium chip. There may be one or more second storage medium chips.
[0083] Furthermore, in S409, if the percentage of abnormal areas in the first storage medium particle exceeds a preset threshold, it can be determined that the first storage medium particle is abnormal, and subsequently, a second storage medium particle can be used to replace the first storage medium particle. For example, in Figure 6 If the first storage medium particle d1 has 80% of its abnormal area, then the first storage medium particle can be replaced by the second storage medium particle.
[0084] In the embodiments of this application, the failure of the first storage medium particle can also be determined by other means, such as an error occurring during read and / or write operations, or the data retention time of the first storage medium particle being lower than a preset time requirement, or the performance (such as latency and throughput) of the first storage medium particle's write and / or read operations not meeting the preset performance requirements.
[0085] In the embodiments of this application, when reading and / or writing data in the memory module, the failure of the first storage medium particle and / or the abnormal area existing in the first storage medium particle can be identified, and then the failure of the first storage medium particle and / or the abnormal area can be replaced to improve the fault tolerance of the memory module and thus improve the performance of the memory module.
[0086] Furthermore, this application significantly reduces the risk of error correction failure by backing up target data multiple times, making QoS expectations controllable and overcoming long-tail latency in read operations. In addition, by setting the byte size of the target error correction codeword to be the same as the byte size of a single access to the memory module, read / write amplification is avoided. Therefore, this application applies persistent memory to multi-backup distributed storage, replacing the inefficient RAID design within a single persistent memory module with the inherent multi-backup functionality of the distributed processing system, simplifying the design of persistent memory components (this application does not require setting...). Figure 1 (Regarding RAID-related content), it saves on persistent memory media usage, reduces read / write amplification interference within the memory module, shortens the long-tail latency of distributed processing systems, and reduces the redundancy space required for fault tolerance. Furthermore, it solves the problem of first-stage storage media failure, reducing the memory module's power consumption and the maintenance time for developers.
[0087] In this application embodiment, a distributed processing system is also provided, with reference to Figure 2The system includes: at least one cloud server and a storage cluster, the storage cluster including multiple storage nodes, each storage node including a memory module and a storage component. The cloud server is used to: read target data from any one of the multiple memory modules, wherein the target data is stored in all of the multiple memory modules; determine whether the target data has been successfully read; if so, store the target data in the multiple storage components; if not, mark the memory module currently reading the target data as an abnormal memory module; read the target data from any of the multiple memory modules that has not been marked as abnormal, and perform the step of determining whether the target data has been successfully read.
[0088] Furthermore, the memory module includes: an error correction code module and at least one first storage medium particle; the first storage medium particle includes: a nominal capacity area and a redundant capacity area.
[0089] In this embodiment, the specific implementation process of the distributed processing system is the same as that described in the above method embodiment, and will not be repeated here.
[0090] In this application embodiment, in addition to providing a memory fault tolerance processing method, a memory fault tolerance processing device 70 is also provided, such as... Figure 7 As shown, the fault-tolerant processing device 70 for the memory includes: a reading module 71, a determining module 72, and a marking module 73, wherein:
[0091] The reading module 71 is used to read target data from any one of the multiple memory modules, wherein the target data is stored in all of the multiple memory modules;
[0092] Module 72 is used to determine whether the target data has been successfully read.
[0093] The marking module 73 is used to mark the memory module currently reading the target data as an abnormal memory module if no;
[0094] The reading module 71 is also used to read target data from any of the multiple memory modules that are not marked as abnormal.
[0095] In an optional embodiment, the memory module includes: an error correction code module and at least one first storage medium particle. The reading module 71 is specifically used to: read raw data from at least one first storage medium particle of any of the multiple memory modules; and decode the raw data using the error correction code module to obtain target data.
[0096] In an optional embodiment, the first storage medium particle includes a nominal capacity area, and the fault-tolerant processing device for the memory further includes a comparison module (not shown) for comparing the original data and the target data to determine the physical location of the defective area in the first storage medium particle; determining the target error correction codeword where the defective area is located based on the physical location; and marking at least a portion of the nominal capacity area where the target error correction codeword is located as an abnormal area, wherein the abnormal area indicates that it is not used for data storage.
[0097] In an optional embodiment, the first storage medium particle further includes a redundant capacity region, and the memory fault tolerance processing method further includes a processing module (not shown) for storing a target error correction code word in at least a portion of the redundant capacity region; determining the physical address of the at least a portion of the redundant capacity region; and updating the physical address of the error correction code word in a preset mapping table, wherein the preset mapping table is used to indicate reading the target error correction code word according to the physical address of the at least a portion of the redundant capacity region.
[0098] In an optional embodiment, the size of the target error correction codeword in bytes is the same as the size of a single access to the memory module in bytes.
[0099] In an optional embodiment, the memory module further includes a second storage medium particle, and the memory fault tolerance processing device further includes a replacement module (not shown) for replacing the first storage medium particle with the second storage medium particle if it is determined that the first storage medium particle has failed.
[0100] In an optional embodiment, a writing module (not shown) is further included, which, in response to receiving a write request for the target data, writes the target data into each of the multiple memory modules before reading the target data in any of the multiple memory modules.
[0101] In an optional embodiment, the reading module 71 is specifically configured to: in response to receiving a read request for target data, read the target data from any one of a plurality of memory modules.
[0102] In an alternative embodiment, a storage module (not shown) is further included for storing the target data in a plurality of storage components if it is determined that the target data has been successfully read.
[0103] In an optional embodiment, the reading module 71 is further configured to, after reading the target data in any of the multiple memory modules that are not marked as abnormal, perform a determination of whether the target data was successfully read.
[0104] In this embodiment, the specific implementation process of the memory fault tolerance processing device is the same as that in the above method embodiment, and will not be repeated here.
[0105] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.
[0106] Figure 8 This is a schematic diagram of an electronic device provided as an exemplary embodiment of this application. The electronic device 80 is used to run the aforementioned memory fault-tolerant processing method. Figure 8 As shown, the electronic device includes a memory 84 and a processor 85.
[0107] Memory 84 is used to store computer programs and can be configured to store various other data to support operation on electronic devices. This memory 84 may be object storage (OSS).
[0108] The memory 84 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0109] The processor 85, coupled to the memory 84, is configured to execute a computer program in the memory 84 for: reading target data from any of a plurality of memory modules, wherein the target data is stored in all of the plurality of memory modules; determining whether the target data was successfully read; if not, marking the memory module currently reading the target data as an abnormal memory module; and reading the target data from any of the plurality of memory modules that is not marked as abnormal.
[0110] In an optional embodiment, the memory module includes an error correction code module and at least one first storage medium particle. When the processor 85 reads target data from any of the multiple memory modules, it specifically performs the following: reads original data from at least one first storage medium particle of any of the multiple memory modules; and decodes the original data using the error correction code module to obtain the target data.
[0111] In an optional embodiment, the first storage medium particle includes a nominal capacity region and a redundant capacity region. The processor 85 is further configured to: compare the original data and the target data to determine the physical location of the defective region in the first storage medium particle; determine the target error correction codeword where the defective region is located based on the physical location; and mark at least a portion of the nominal capacity region where the target error correction codeword is located as an abnormal region, wherein the abnormal region indicates that it is not used for data storage.
[0112] In an optional embodiment, the first storage medium particle further includes a redundant capacity region, and the processor 85 is further configured to: store the target error correction codeword in at least a portion of the redundant capacity region; determine the physical address of the at least a portion of the redundant capacity region; and update the physical address of the error correction codeword in a preset mapping table, the preset mapping table being used to indicate reading the target error correction codeword according to the physical address of the at least a portion of the redundant capacity region.
[0113] In an optional embodiment, the size of the target error correction codeword in bytes is the same as the size of a single access to the memory module in bytes.
[0114] In an optional embodiment, the memory module further includes a second storage medium particle, and the processor 85 is further configured to replace the first storage medium particle with the second storage medium particle if it is determined that the first storage medium particle has failed.
[0115] In an optional embodiment, the processor 85 is further configured to: replace the first storage medium particle with a second storage medium particle if it is determined that the first storage medium particle has failed.
[0116] In an optional embodiment, the processor 85 is further configured to: respond to a received write request for the target data and write the target data into the multiple memory modules respectively before reading the target data in any of the multiple memory modules.
[0117] In an optional embodiment, when the processor 85 reads target data in any of the multiple memory modules, it is specifically configured to: respond to receiving a read request for target data, read the target data in any of the multiple memory modules.
[0118] In an optional embodiment, the processor 85 is further configured to: after determining whether the target data has been successfully read, if the target data has been successfully read, store the target data in a plurality of storage components.
[0119] In an optional embodiment, the processor 85 is further configured to: after reading target data in any of the multiple memory modules that are not marked as abnormal, perform a determination of whether the target data was successfully read.
[0120] Furthermore, such as Figure 8As shown, the electronic device also includes other components such as a firewall 81, a load balancer 82, a communication component 86, and a power supply component 83. Figure 8 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 8 The components shown.
[0121] Accordingly, embodiments of this application also provide a computer program product storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the method described above.
[0122] Accordingly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, cause the processor to implement the steps in the method described above.
[0123] The above Figure 8 The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related text from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0124] The above Figure 8 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.
[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0126] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable text processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable text processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable text processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0128] These computer program instructions can also be loaded onto a computer or other programmable text processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0129] In a typical configuration, a computing device includes one or more processors (CPU and / or GPU), input / output interfaces, network interfaces, and memory.
[0130] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0131] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be used to store text by any method or technology. Text can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store text accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0132] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0133] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A memory fault-tolerance processing method, characterized in that, This method is applied to a distributed processing system, which includes a storage cluster comprising multiple storage nodes, each storage node comprising a memory module and a storage component; the memory fault tolerance processing method includes: Read target data from any one of a plurality of memory modules, wherein the target data is stored in all of the plurality of memory modules; Determine whether the target data was successfully read; If not, mark the memory module currently reading the target data as an abnormal memory module; The target data is read from any memory module among the plurality of memory modules that is not marked as abnormal.
2. The memory fault-tolerance processing method according to claim 1, characterized in that, The memory module includes: an error correction code module and at least one first storage medium particle. Reading target data from any one of the multiple memory modules includes: Read raw data from at least one first storage medium particle of any of the plurality of memory modules; The original data is decoded using the error correction code module to obtain the target data.
3. The memory fault-tolerance processing method according to claim 2, characterized in that, The first storage medium particle includes: a nominal capacity area, and the fault tolerance processing method for the memory further includes: The original data and the target data are compared to determine the physical location of the defect region in the first storage medium particle. The target error correction codeword where the defect area is located is determined based on the physical location; At least a portion of the nominal capacity region containing the target error correction codeword is marked as an anomalous region, wherein the anomalous region indicates that it is not used for data storage.
4. The memory fault-tolerance processing method according to claim 3, characterized in that, The first storage medium particle further includes: a redundant capacity region, and the memory fault tolerance processing method further includes: The target error correction codeword is stored in at least a partially redundant capacity region; Determine the physical address of the at least partial redundant capacity region; The physical address of the target error correction codeword is updated in a preset mapping table, which is used to indicate the reading of the target error correction codeword based on the physical address of the at least partially redundant capacity region.
5. The memory fault-tolerance processing method according to claim 3, characterized in that, The size of the target error correction codeword in bytes is the same as the size of the memory module in bytes per access.
6. The memory fault-tolerance processing method according to claim 2, characterized in that, The memory module further includes: a second storage medium particle, and the fault tolerance processing method for the memory further includes: If it is determined that the first storage medium particle is faulty, the second storage medium particle shall be used to replace the first storage medium particle.
7. The memory fault-tolerant processing method according to any one of claims 1 to 6, characterized in that, Before reading the target data from any of the multiple memory modules, the method further includes: Upon receiving a write request for the target data, the target data is written into each of the plurality of memory modules.
8. The memory fault-tolerant processing method according to any one of claims 1 to 6, characterized in that, The step of reading target data from any one of the multiple memory modules includes: In response to a read request for the target data, the target data is read from any one of the multiple memory modules.
9. The memory fault-tolerant processing method according to any one of claims 1 to 6, characterized in that, After determining whether the target data has been successfully read, the process further includes: If the target data is successfully read, the target data is stored in one of the multiple storage components.
10. The memory fault-tolerant processing method according to any one of claims 1 to 6, characterized in that, Also includes: After reading the target data from any memory module that is not marked as abnormal among the plurality of memory modules, the step of determining whether the target data was successfully read is performed.
11. A distributed processing system, characterized in that, include: At least one cloud server and a storage cluster, the storage cluster comprising multiple storage nodes, each storage node comprising a memory module and a storage component, the cloud server being used for: Read target data from any one of a plurality of memory modules, wherein the target data is stored in all of the plurality of memory modules; Determine whether the target data was successfully read; If not, mark the memory module currently reading the target data as an abnormal memory module; The target data is read from any memory module among the plurality of memory modules that is not marked as abnormal.
12. The distributed processing system according to claim 11, characterized in that, The memory module includes: an error correction code module and at least one first storage medium particle; the first storage medium particle includes: a nominal capacity area and a redundant capacity area.
13. A memory fault-tolerant processing device, characterized in that, This is applied to a distributed processing system, which includes: a storage cluster comprising multiple storage nodes, each storage node comprising: a memory module and a storage component; and a memory fault-tolerant processing device comprising: A reading module is used to read target data from any one of a plurality of memory modules, wherein the target data is stored in all of the plurality of memory modules; The determination module is used to determine whether the target data has been successfully read. The marking module is used to mark the memory module currently reading the target data as an abnormal memory module if no exception is made. The reading module is also used to read the target data from any memory module that is not marked as abnormal among the plurality of memory modules.
14. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the memory fault-tolerant processing method as described in any one of claims 1 to 10.
15. A computer program product, characterized in that, The computer program product stores a computer program that, when executed by a processor, causes the processor to implement the memory fault-tolerant processing method as described in any one of claims 1 to 10.