Data reading and writing optimization method, device, computer and storage medium for domestic platform

Abstract

The present invention proposes a data reading and writing optimization method, device, computer and storage medium suitable for domestic platforms; wherein the method includes: obtaining relevant information of a RAID shared storage pool, wherein the relevant information includes the number of data disks, prime numbers, and check codes; in response to receiving a read request or a write request, determining the number of data blocks to be read or written; according to the number of data disks, prime numbers, and check codes, calculating the distribution of data blocks in the RAID shared storage pool, and reading or writing data according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool stripes. The present invention rearranges the distribution of data blocks in the storage pool stripes so that some check blocks are shared between data blocks, thereby reducing the modification of check blocks, reducing the number of operations, and improving the overall reading and writing performance.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data reading and writing optimization method, device, computer and storage medium suitable for domestic platforms. Background Art

[0002] Under the current international situation, autonomy and controllability have risen to a national strategy. Whether it is the party, government, military or civilian key application scenarios, it has become a consensus to promote the domestic autonomy and controllability of information systems. The development of autonomous and controllable technology plays a core supporting role in the construction of national information security. Data security is a core component of information security and a guarantee for the security and controllability of the information industry system. Storage, as the ultimate carrier of data, is a "bottom line" that must be maintained for information security. On the one hand, due to the layers of technical barriers in the storage system, the core software and hardware have a high technical threshold; on the other hand, due to historical reasons, the core software and hardware technology of the storage system is controlled by a few international storage giants, and the formulation of storage-related standards is also subject to foreign countries.

[0003] The FT2000+ processor integrates 64 high-performance processor cores FTC662 independently developed by Feiteng, adopts a four-issue out-of-order superscalar pipeline, and adopts a PSoCb architecture. It integrates high-efficiency processor cores, a large-scale consistent storage structure based on data affinity, and a hierarchical two-dimensional Mesh interconnection network to optimize storage access latency and provide industry-leading computing performance, access bandwidth, and IO expansion capabilities. The chip is compatible with the 64-bit ARMv8 instruction set and is suitable for high-performance, high-throughput servers, storage, and other fields.

[0004] Therefore, in order to adapt to the above-mentioned FT2000+ processor integrating 64 Feiteng's independently developed high-performance processor cores FTC662 and adopting an out-of-order four-issue superscalar pipeline, a data reading and writing method that can match it is urgently needed. Summary of the invention

[0005] The present invention aims to propose a data reading and writing method suitable for the multi-core processing mode of domestic servers. And the method of processing read and write requests serially by each core is changed to a method of processing read and write requests in parallel by multiple cores to make up for the problem of insufficient computing power of domestic cores. In order to realize multi-core parallel data reading and writing, it is necessary to establish a shared storage pool RAID, and re-plan the new distribution of data blocks in the shared storage pool to ensure the processing speed of multi-core parallel reading and writing. In one aspect of the present invention, a data reading and writing optimization method suitable for domestic platforms is proposed, the method comprising: obtaining relevant information of the RAID shared storage pool, the relevant information including the number of data disks, prime numbers, and check codes; in response to receiving a read request or a write request, determining the number of data blocks to be read or written; according to the number of data disks, prime numbers, and check codes, calculating the distribution of data blocks in the RAID shared storage pool, and reading or writing data according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool strips.

[0006] In one or more embodiments, the method further includes: in response to receiving a read request or a write request in which the amount of data to be read or written is less than the amount of data of a data block, aggregating the read request or the write request in which the amount of data to be read or written is less than the amount of data of a data block with other read requests or write requests, and re-determining the number of data blocks to be read or written by the aggregated read request or write request until the data reading or writing operation is completed.

[0007] In one or more embodiments, the read or write request is executed synchronously by multiple CPU cores. Accordingly, the method further includes: modifying the interface part of the read and write processing protocol, RAID algorithm and electronic disk read and write scheduling to convert the asynchronous call into a synchronous call.

[0008] In one or more embodiments, the method further includes: distributing different data I / Os to different CPU cores and corresponding memories.

[0009] In another aspect of the present invention, a data reading and writing optimization device suitable for domestic platforms is proposed, including: an initialization module, configured to obtain relevant information of a RAID shared storage pool, the relevant information including the number of data disks, prime numbers, and check codes; a selection module: configured to determine the number of data blocks to be read or written in response to receiving a read request or a write request; a data block reorganization and read-write module: configured to calculate the distribution of data blocks in the RAID shared storage pool based on the number of data disks, prime numbers, and check codes, and read or write data according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool strips.

[0010] In one or more embodiments, the data read and write optimization device also includes: an aggregation module, which, in response to receiving a read request or a write request in which the amount of data to be read or written is less than the amount of data of a data block, aggregates the read request or the write request in which the amount of data to be read or written is less than the amount of data of a data block with other read requests or write requests, and re-determines the number of data blocks to be read or written by the aggregated read request or write request until the data reading or writing operation is completed.

[0011] In one or more embodiments, the data block reorganization and read / write module is also configured to: when performing synchronous read or write requests of multiple CPU cores, modify the interface part of the storage data read / write processing protocol, RAID algorithm and electronic disk read / write scheduling, and convert asynchronous calls into synchronous calls.

[0012] In one or more embodiments, the data read and write optimization device further includes: a data I / O allocation module configured to distribute different data I / Os to different CPU cores and corresponding memories.

[0013] In another aspect of the present invention, a computer is provided, wherein the computer includes a data reading and writing optimization device as provided in any one of the above embodiments.

[0014] In another aspect of the present invention, a storage medium is provided, wherein an executable computer program is stored in the storage medium, and when the computer program is executed, the steps of a data reading and writing optimization method as provided in any one of the above embodiments are implemented.

[0015] The beneficial effects of the present invention include: a data read and write optimization method, device, computer and storage medium proposed by the present invention, by reorganizing the distribution of data blocks when the read-write dual-control storage shares the storage pool, sharing data check blocks between each data block, thereby improving the overall read and write performance of the storage pool. This method can effectively reduce the number of data check blocks that need to be modified, thereby improving the parallel data read and write performance of the multi-core system as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A workflow diagram of a data reading and writing optimization method applicable to a domestic platform of the present invention;

[0018] Figure 2This is a schematic diagram of the distribution of general data blocks in the storage pool stripe;

[0019] Figure 3 A schematic diagram of the distribution of data blocks in a storage pool stripe according to the present invention;

[0020] Figure 4 This is a structural schematic diagram of a data reading and writing optimization device applicable to a domestic platform according to the first embodiment of the present invention;

[0021] Figure 5 This is a structural schematic diagram of a data reading and writing optimization device applicable to a domestic platform according to a second embodiment of the present invention;

[0022] Figure 6 The figure is a schematic diagram of the dual-control storage hardware architecture based on a domestic platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0024] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0025] The present invention aims to propose a data reading and writing method suitable for the multi-core processing mode of domestic servers. And the method of processing read and write requests serially by each core is changed to a method of processing read and write requests in parallel by multiple cores to make up for the problem of insufficient computing power of domestic cores. In order to realize multi-core parallel data reading and writing, it is necessary to establish a shared storage pool RAID, and re-plan the new distribution of data blocks in the shared storage pool to ensure the processing speed of multi-core parallel reading and writing. The scheme of the present invention includes:

[0026] Figure 1 FIG. 1 is a flowchart of a data reading and writing optimization method applicable to a domestic platform of the present invention. Figure 1 As shown, the workflow of the data reading and writing optimization method of the present invention includes: step S1, obtaining relevant information of the RAID shared storage pool, the relevant information including the number of data disks, prime numbers, and check codes; step S2, in response to receiving a read request or a write request, determining the number of data blocks to be read or written; step S3, calculating the distribution of data blocks in the RAID shared storage pool according to the number of data disks, prime numbers, and check codes, and reading or writing data according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool strips.

[0027] In the prior art, data blocks are distributed in the storage pool stripe as follows: Figure 2 As shown, Figure 2 The following is a schematic diagram of the distribution of general data blocks in the storage pool stripe. Figure 2 As shown in the figure, five consecutive data blocks A, B, C, D, and E are written to the storage pool stripe according to the traditional EVENODD data encoding method. The row check code is , the column check code is , , , , , the oblique check code is , , , , , because the Adjuster position was modified, so and It also needs to be modified. A total of 1 row check code, 5 column check codes and 7 oblique check codes need to be updated, a total of 13 check codes.

[0028] According to the distribution rules proposed in the above embodiments of the present invention, the distribution of data blocks in the storage pool stripes is as follows: Figure 3 As shown, Figure 3 The figure is a schematic diagram of the distribution of data blocks in the storage pool strip of the present invention. In order to reduce the check code modified when writing data, the present invention arranges the continuously written data blocks in two adjacent storage pool strips in a staggered manner into a continuous "W" shape, so that the data blocks can share some oblique check codes, reducing the amount of check code modification. According to the distribution rule of the present invention, write five consecutive data blocks A, B, C, D, E, and the row check code is , , the column check code is , , , , , the oblique check code is , , , it is necessary to update 2 row check codes, 5 column check codes and 3 oblique check codes, a total of 10 check codes. Compared with the 13 check codes of the traditional EVENODD method, 3 check codes can be updated less, thereby improving the read and write performance of the storage pool.

[0029] Therefore, this embodiment rearranges the distribution of data blocks in the storage pool stripe so that some check blocks are shared between data blocks, thereby reducing the modification of check blocks, reducing the number of operations, and improving the overall read and write performance.

[0030] In a further implementation, the method of the present invention also includes: in response to receiving a read request or a write request in which the amount of data to be read or written is less than the amount of data of a data block, aggregating the read request or the write request in which the amount of data to be read or written is less than the amount of data of a data block with other read requests or write requests, and re-determining the number of data blocks to be read or written by the aggregated read request or write request until the data reading or writing operation is completed.

[0031] This embodiment can further ensure the efficiency of data reading and writing, thereby further optimizing the reading and writing process; and, the method of this embodiment can fully exert the concurrency capability (concurrent read requests or write requests) of a multi-core system, such as FT2000+ multi-core.

[0032] In a further embodiment, the read request or write request is executed synchronously by multiple CPU cores (dual control). Accordingly, the method of the present invention also includes: modifying the interface part of the storage data read and write processing protocol, RAID algorithm and electronic disk read and write scheduling, and converting the asynchronous call into a synchronous call.

[0033] This embodiment converts some asynchronous calls of the interface of storage data read and write processing protocol, RAID algorithm and electronic disk read and write scheduling into synchronous calls, thereby reducing the read and write thread switching delay caused by asynchronous calls during reading and writing, and helping to improve the multi-core parallel reading and writing speed.

[0034] In a further embodiment, the method of the present invention further includes: distributing different data I / Os to different CPU cores and corresponding memories.

[0035] The purpose of this embodiment is to make full use of the advantages of FT2000+ multi-core, distribute different data IO to different CPU cores and corresponding memories, reduce the coupling between CPU cores, and improve the utilization efficiency of CPU multi-cores.

[0036] In a further embodiment, the method of the present invention further comprises: enabling multiple data blocks in the new distribution to share a data check block.

[0037] This embodiment can further optimize the RAID algorithm by combining the embodiment of the distribution of data blocks when reorganizing the read-write dual-control storage shared storage pool, so that the data check blocks are shared between each data block, so as to further give play to the advantages of FT2000+ multi-core, thereby improving the overall read and write performance of the dual-control storage.

[0038] Based on the above embodiments, the present invention also proposes a data reading and writing optimization device, such as Figure 4 As shown, Figure 4The schematic diagram of the structure of a data read and write optimization device suitable for a domestic platform according to the first embodiment of the present invention. The data read and write optimization device of the present invention comprises: an initialization module 10, configured to obtain relevant information of a RAID shared storage pool, wherein the relevant information includes the number of data disks, prime numbers, and check codes; a selection module 20, configured to determine the number of data blocks to be read or written in response to receiving a read request or a write request; a data block reorganization and read and write module 30, configured to calculate the distribution of data blocks in the RAID shared storage pool according to the number of data disks, prime numbers, and check codes, and read or write data according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool strips.

[0039] In a further embodiment, the data read and write optimization device of the present invention further includes: an aggregation module 40, configured to respond to a received read request or write request whose data amount to be read or written is less than the data amount of a data block, aggregate the read request or write request whose data amount to be read or written is less than the data amount of a data block with other read requests or write requests, and return the aggregated read request or write request to the selection module 20, so as to re-determine the number of data blocks to be read or written by the aggregated read request or write request through the selection module 20, until the data reading or writing operation is completed. The structure of the data read and write optimization device of this embodiment is as follows: Figure 5 shown. Figure 5 This is a structural schematic diagram of a data reading and writing optimization device suitable for domestic platforms according to the second embodiment of the present invention.

[0040] In a further embodiment, the data block reorganization and read / write module is further configured to: when performing synchronous read or write requests of multiple CPU cores, modify the interface part of the storage data read / write processing protocol, RAID algorithm and electronic disk read / write scheduling, and convert asynchronous calls into synchronous calls.

[0041] In a further embodiment, the apparatus of the present invention further comprises a data I / O allocation module configured to distribute different data I / Os to different CPU cores and corresponding memories.

[0042] In a further embodiment, the data block reorganization and read / write module is further configured to enable multiple data blocks in the new distribution to share a data check block.

[0043] On the basis of the above embodiments, the present invention further proposes a computer, which includes a data reading and writing optimization device as mentioned in any one of the above embodiments.

[0044] In a specific embodiment, the computer is a FT2000+ dual-control platform, which can apply a data reading and writing optimization method or device in any of the above embodiments of the present invention to realize fast and parallel data reading and writing operations.

[0045] Figure 6 FIG. 1 is a schematic diagram of a dual-control storage hardware architecture based on a domestic platform according to an embodiment of the present invention. Figure 6 The FT2000+ dual-control storage shown is mainly composed of a mechanical structure layer, a circuit hardware layer, a supporting software layer and an integrated application layer.

[0046] 1) Mechanical structure layer

[0047] The mechanical structure layer includes the chassis, heat dissipation module and physical reinforcement parts. The dual-control storage chassis adopts a 19-inch standard rack structure. The physical reinforcement adopts multiple reinforcement technologies such as anti-vibration and shock, thermal design, electromagnetic compatibility protection, and three-proofing to achieve high environmental adaptability and high reliability.

[0048] 2) Circuit hardware layer

[0049] The circuit hardware layer mainly consists of the mainboard module, power module, array module and hard disk module. The mainboard is based on the domestic FT2000+ processor, provides Gigabit and 10 Gigabit network interfaces, and can provide high-performance and highly reliable information processing services; the power supply has dual redundancy function to improve reliability. It can support up to 24 pluggable hard disks to meet the needs of large-capacity data storage.

[0050] 3) Support software layer

[0051] The support software layer includes the underlying BIOS, drivers, kernel, Linux file system, storage management software and system management software. The support layer provides basic services to the application layer, including fault detection, information synchronization, fault takeover and fault recovery functions.

[0052] 4) Integrated application layer

[0053] The integrated application layer mainly provides basic storage management functions, including SMI-S unified storage management software and Web service software. It mainly implements storage management by analyzing storage system fault links and formulating relevant strategies.

[0054] Based on the above embodiments, the present invention further proposes a storage medium, in which an executable computer program is stored. When the computer program is executed, it is used to implement the steps of a data reading and writing optimization method mentioned in any embodiment.

[0055] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0056] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0057] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0058] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A data reading and writing optimization method suitable for domestic platforms, characterized in that: The method comprises: Obtain relevant information of the RAID shared storage pool, the relevant information including the number of data disks, prime numbers, and check codes; In response to receiving a read request or a write request, determining a number of data blocks to be read or written; According to the number of data disks, prime numbers, and check codes, the distribution of data blocks in the RAID shared storage pool is calculated, and data is read or written according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool strips to form a continuous W-shaped distribution.

2. The data reading and writing optimization method applicable to domestic platforms as claimed in claim 1 is characterized in that: The method further comprises: In response to receiving a read request or a write request in which the amount of data to be read or written is less than the amount of data in one data block, the read request or the write request in which the amount of data to be read or written is less than the amount of data in one data block is aggregated with other read requests or write requests, and the number of data blocks to be read or written by the aggregated read request or write request is re-determined until the data read or write operation is completed.

3. The data reading and writing optimization method applicable to domestic platforms as claimed in claim 1 is characterized in that: The read request or write request is executed synchronously by multiple CPU cores, and accordingly, the method further includes: Modify the interface parts of the read-write processing protocol, RAID algorithm, and electronic disk read-write scheduling, and convert asynchronous calls into synchronous calls.

4. The data reading and writing optimization method applicable to domestic platforms as claimed in claim 3 is characterized in that: The method further comprises: Distribute different data I / O to different CPU cores and corresponding memory.

5. A data reading and writing optimization device suitable for domestic platforms, characterized in that: include: An initialization module configured to obtain relevant information of the RAID shared storage pool, the relevant information including the number of data disks, prime numbers, and check codes; A selection module: configured to determine the number of data blocks to be read or written in response to receiving a read request or a write request; Data block reorganization and read / write module: configured to calculate the distribution of data blocks in the RAID shared storage pool according to the number of data disks, prime numbers, and check codes, and read or write data according to a preset distribution rule, wherein the distribution rule includes staggering the data blocks in adjacent storage pool strips to form a continuous W-shaped distribution.

6. A data reading and writing optimization device suitable for a domestic platform as claimed in claim 5, characterized in that: Also includes: An aggregation module is configured to, in response to receiving a read request or a write request in which the amount of data to be read or written is less than the amount of data of one data block, aggregate the read request or the write request in which the amount of data to be read or written is less than the amount of data of one data block with other read requests or write requests, and re-determine the number of data blocks to be read or written by the aggregated read request or write request until the data reading or writing operation is completed.

7. A data reading and writing optimization device suitable for domestic platforms as described in claim 5, characterized in that: The data block reorganization and read / write module is also configured to modify the interface part of the storage data read / write processing protocol, RAID algorithm and electronic disk read / write scheduling when performing synchronous read or write requests of multiple CPU cores, and convert asynchronous calls into synchronous calls.

8. A data reading and writing optimization device suitable for a domestic platform as claimed in claim 7, characterized in that: Also includes: The data I / O distribution module is configured to distribute different data I / Os to different CPU cores and corresponding memories.

9. A computer, characterized in that: The computer includes a data reading and writing optimization device suitable for domestic platforms as described in any one of claims 5-8.

10. A storage medium, characterized in that: The storage medium stores an executable computer program, which, when executed, is used to implement the steps of a data reading and writing optimization method applicable to a domestic platform as described in any one of claims 1 to 4.

Images